2023-11-28T21:18:03.718 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T21:18:03.719 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} --archive /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703 --name vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703/orig.config.yaml 2023-11-28T21:18:03.768 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T21:18:03.850 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703 branch: wip-vshankar-testing-20231127.102914 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} email: vshankar@redhat.com first_in_suite: false job_id: '7469703' kernel: client: branch: testing flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-vshankar-testing-20231127.102914 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - Replacing daemon mds sha1: 43dd94c190073cf16cff1e711dd7f5a7b1cbc883 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 43dd94c190073cf16cff1e711dd7f5a7b1cbc883 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20231127.102914 sha1: 43dd94c190073cf16cff1e711dd7f5a7b1cbc883 owner: scheduled_vshankar@teuthology priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 8377 sha1: 43dd94c190073cf16cff1e711dd7f5a7b1cbc883 sleep_before_teardown: 0 subset: 462/1024 suite: fs suite_branch: wip-vshankar-testing-20231127.102914 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_43dd94c190073cf16cff1e711dd7f5a7b1cbc883/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 43dd94c190073cf16cff1e711dd7f5a7b1cbc883 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUISU2UNRkZ8KvhahKPJ/Cs+rXF3/sD7bxiYbw/RwxUeVWemGwd9BwvJrvWDwo4pvBt+N1nYfzMl5NONDD2Hro= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWCPwqcYt1BY50Y5frCiKMdgs43eqUVjQ57TA8TpCWOuXtYVKXsQiKY6HgEqCtuq0al9oexvbteSJacgpNbMgg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - mds_thrash: null - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_06:29:14 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T21:18:03.850 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_43dd94c190073cf16cff1e711dd7f5a7b1cbc883/qa; will attempt to use it 2023-11-28T21:18:03.851 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_43dd94c190073cf16cff1e711dd7f5a7b1cbc883/qa/tasks 2023-11-28T21:18:03.851 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T21:18:03.855 INFO:teuthology.task.internal:Checking packages... 2023-11-28T21:18:03.880 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '43dd94c190073cf16cff1e711dd7f5a7b1cbc883' 2023-11-28T21:18:03.881 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T21:18:03.881 INFO:teuthology.packaging:ref: None 2023-11-28T21:18:03.881 INFO:teuthology.packaging:tag: None 2023-11-28T21:18:03.881 INFO:teuthology.packaging:branch: wip-vshankar-testing-20231127.102914 2023-11-28T21:18:03.881 INFO:teuthology.packaging:sha1: 43dd94c190073cf16cff1e711dd7f5a7b1cbc883 2023-11-28T21:18:03.881 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-vshankar-testing-20231127.102914 2023-11-28T21:18:04.102 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7520-g43dd94c1-1jammy 2023-11-28T21:18:04.102 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T21:18:04.107 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T21:18:04.108 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T21:18:04.148 INFO:teuthology.task.internal:Saving configuration 2023-11-28T21:18:04.167 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T21:18:04.178 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T21:18:04.202 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 21:12:41.173991', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBt8eKn7dHJGbG6rRa4Pz1foN2Y9+vtJztKy8nLNu3eZ'} 2023-11-28T21:18:04.225 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 21:12:41.172423', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPcilCSQtGzR0gygA/ne+G0lPePZLiF8k2DxxjKQIhq9'} 2023-11-28T21:18:04.225 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T21:18:04.230 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2023-11-28T21:18:04.230 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-11-28T21:18:04.230 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T21:18:04.323 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f121b655040>, signals=[15]) 2023-11-28T21:18:04.324 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T21:18:04.329 INFO:teuthology.task.internal:Opening connections... 2023-11-28T21:18:04.329 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:18:04.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T21:18:04.407 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:18:04.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T21:18:04.481 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T21:18:04.487 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2023-11-28T21:18:04.494 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2023-11-28T21:18:04.494 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2023-11-28T21:18:04.542 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-28T21:18:04.542 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu" 2023-11-28T21:18:04.542 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="22.04" 2023-11-28T21:18:04.542 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-28T21:18:04.542 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=jammy 2023-11-28T21:18:04.543 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu 2023-11-28T21:18:04.543 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian 2023-11-28T21:18:04.543 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-28T21:18:04.543 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-28T21:18:04.543 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-28T21:18:04.543 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-28T21:18:04.543 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=jammy 2023-11-28T21:18:04.544 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-11-28T21:18:04.564 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2023-11-28T21:18:04.570 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2023-11-28T21:18:04.570 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2023-11-28T21:18:04.617 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-28T21:18:04.617 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Ubuntu" 2023-11-28T21:18:04.617 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="22.04" 2023-11-28T21:18:04.617 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-28T21:18:04.617 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_CODENAME=jammy 2023-11-28T21:18:04.618 INFO:teuthology.orchestra.run.smithi144.stdout:ID=ubuntu 2023-11-28T21:18:04.618 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE=debian 2023-11-28T21:18:04.618 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-28T21:18:04.618 INFO:teuthology.orchestra.run.smithi144.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-28T21:18:04.618 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-28T21:18:04.618 INFO:teuthology.orchestra.run.smithi144.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-28T21:18:04.618 INFO:teuthology.orchestra.run.smithi144.stdout:UBUNTU_CODENAME=jammy 2023-11-28T21:18:04.619 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-11-28T21:18:04.639 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T21:18:04.652 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T21:18:04.681 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T21:18:04.682 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2023-11-28T21:18:04.685 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2023-11-28T21:18:04.689 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T21:18:04.711 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T21:18:04.711 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2023-11-28T21:18:04.731 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2023-11-28T21:18:04.749 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T21:18:04.806 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T21:18:04.823 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-28T21:18:04.823 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-11-28T21:18:04.823 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-11-28T21:18:04.823 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T21:18:04.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:18:04.829 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-11-28T21:18:05.146 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T21:18:05.146 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-28T21:18:05.146 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2023-11-28T21:18:05.152 INFO:teuthology.orchestra.run.smithi012.stdout:5.15.0-56-generic 2023-11-28T21:18:05.152 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T21:18:05.152 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-28T21:18:05.153 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-11-28T21:18:05.153 DEBUG:teuthology.orchestra.run.smithi012:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-28T21:18:05.221 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-11-28T21:18:05.337 INFO:teuthology.orchestra.run.smithi012.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-11-28T21:18:05.337 INFO:teuthology.orchestra.run.smithi012.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-11-28T21:18:05.337 INFO:teuthology.orchestra.run.smithi012.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-11-28T21:18:05.337 INFO:teuthology.orchestra.run.smithi012.stderr:hint: 2023-11-28T21:18:05.337 INFO:teuthology.orchestra.run.smithi012.stderr:hint: git config --global init.defaultBranch 2023-11-28T21:18:05.337 INFO:teuthology.orchestra.run.smithi012.stderr:hint: 2023-11-28T21:18:05.337 INFO:teuthology.orchestra.run.smithi012.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-11-28T21:18:05.337 INFO:teuthology.orchestra.run.smithi012.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-11-28T21:18:05.337 INFO:teuthology.orchestra.run.smithi012.stderr:hint: 2023-11-28T21:18:05.337 INFO:teuthology.orchestra.run.smithi012.stderr:hint: git branch -m 2023-11-28T21:18:05.338 INFO:teuthology.orchestra.run.smithi012.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-28T21:18:05.340 DEBUG:teuthology.orchestra.run.smithi012:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-28T21:18:05.406 DEBUG:teuthology.orchestra.run.smithi012:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-28T21:18:05.465 DEBUG:teuthology.orchestra.run.smithi012:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-28T21:19:00.783 INFO:teuthology.orchestra.run.smithi012.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-28T21:19:00.783 INFO:teuthology.orchestra.run.smithi012.stderr: * [new branch] main -> origin/main 2023-11-28T21:19:00.816 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190312 -> 20190312 2023-11-28T21:19:00.817 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190416 -> 20190416 2023-11-28T21:19:00.817 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190514 -> 20190514 2023-11-28T21:19:00.817 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190618 -> 20190618 2023-11-28T21:19:00.817 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190717 -> 20190717 2023-11-28T21:19:00.817 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190815 -> 20190815 2023-11-28T21:19:00.817 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20190923 -> 20190923 2023-11-28T21:19:00.817 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20191022 -> 20191022 2023-11-28T21:19:00.817 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20191215 -> 20191215 2023-11-28T21:19:00.817 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200122 -> 20200122 2023-11-28T21:19:00.817 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200316 -> 20200316 2023-11-28T21:19:00.818 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200421 -> 20200421 2023-11-28T21:19:00.818 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200519 -> 20200519 2023-11-28T21:19:00.818 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200619 -> 20200619 2023-11-28T21:19:00.821 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200721 -> 20200721 2023-11-28T21:19:00.821 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200817 -> 20200817 2023-11-28T21:19:00.821 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20200918 -> 20200918 2023-11-28T21:19:00.821 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20201022 -> 20201022 2023-11-28T21:19:00.821 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20201118 -> 20201118 2023-11-28T21:19:00.821 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20201218 -> 20201218 2023-11-28T21:19:00.821 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210208 -> 20210208 2023-11-28T21:19:00.821 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210315 -> 20210315 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210511 -> 20210511 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210716 -> 20210716 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210818 -> 20210818 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20210919 -> 20210919 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20211027 -> 20211027 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20211216 -> 20211216 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220209 -> 20220209 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220310 -> 20220310 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220411 -> 20220411 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220509 -> 20220509 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220610 -> 20220610 2023-11-28T21:19:00.822 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220708 -> 20220708 2023-11-28T21:19:00.823 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220815 -> 20220815 2023-11-28T21:19:00.823 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20220913 -> 20220913 2023-11-28T21:19:00.823 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20221012 -> 20221012 2023-11-28T21:19:00.823 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20221109 -> 20221109 2023-11-28T21:19:00.823 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20221214 -> 20221214 2023-11-28T21:19:00.823 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230117 -> 20230117 2023-11-28T21:19:00.823 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230210 -> 20230210 2023-11-28T21:19:00.823 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230310 -> 20230310 2023-11-28T21:19:00.823 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230404 -> 20230404 2023-11-28T21:19:00.823 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230515 -> 20230515 2023-11-28T21:19:00.824 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230625 -> 20230625 2023-11-28T21:19:00.824 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230804 -> 20230804 2023-11-28T21:19:00.824 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20230919 -> 20230919 2023-11-28T21:19:00.824 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20231030 -> 20231030 2023-11-28T21:19:00.824 INFO:teuthology.orchestra.run.smithi012.stderr: * [new tag] 20231111 -> 20231111 2023-11-28T21:19:05.037 INFO:teuthology.orchestra.run.smithi012.stderr:Updating files: 48% (1588/3250) Updating files: 49% (1593/3250) Updating files: 50% (1625/3250) Updating files: 51% (1658/3250) Updating files: 52% (1690/3250) Updating files: 53% (1723/3250) Updating files: 54% (1755/3250) Updating files: 55% (1788/3250) Updating files: 56% (1820/3250) Updating files: 57% (1853/3250) Updating files: 58% (1885/3250) Updating files: 59% (1918/3250) Updating files: 60% (1950/3250) Updating files: 61% (1983/3250) Updating files: 62% (2015/3250) Updating files: 63% (2048/3250) Updating files: 63% (2072/3250) Updating files: 64% (2080/3250) Updating files: 65% (2113/3250) Updating files: 66% (2145/3250) Updating files: 67% (2178/3250) Updating files: 68% (2210/3250) Updating files: 69% (2243/3250) Updating files: 70% (2275/3250) Updating files: 71% (2308/3250) Updating files: 72% (2340/3250) Updating files: 73% (2373/3250) Updating files: 74% (2405/3250) Updating files: 75% (2438/3250) Updating files: 76% (2470/3250) Updating files: 77% (2503/3250) Updating files: 78% (2535/3250) Updating files: 78% (2562/3250) Updating files: 79% (2568/3250) Updating files: 80% (2600/3250) Updating files: 81% (2633/3250) Updating files: 82% (2665/3250) Updating files: 83% (2698/3250) Updating files: 84% (2730/3250) Updating files: 84% (2738/3250) Updating files: 85% (2763/3250) Updating files: 86% (2795/3250) Updating files: 87% (2828/3250) Updating files: 88% (2860/3250) Updating files: 89% (2893/3250) Updating files: 90% (2925/3250) Updating files: 91% (2958/3250) Updating files: 92% (2990/3250) Updating files: 93% (3023/3250) Updating files: 94% (3055/3250) Updating files: 95% (3088/3250) Updating files: 96% (3120/3250) Updating files: 97% (3153/3250) Updating files: 98% (3185/3250) Updating files: 99% (3218/3250) Updating files: 100% (3250/3250) Updating files: 100% (3250/3250), done. 2023-11-28T21:19:05.042 INFO:teuthology.orchestra.run.smithi012.stdout:HEAD is now at aae60524 Merge branch 'make-opts' into 'main' 2023-11-28T21:19:05.064 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-28T21:19:05.064 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-28T21:19:05.291 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ 2023-11-28T21:19:05.291 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ --input-file=- 2023-11-28T21:19:05.294 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi012.front.sepia.ceph.com to complete... 2023-11-28T21:19:09.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28 21:19:09 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb [38938726/38938726] -> "/tmp/linux-image.deb" [1] 2023-11-28T21:19:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:FINISHED --2023-11-28 21:19:09-- 2023-11-28T21:19:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:Total wall clock time: 3.9s 2023-11-28T21:19:09.234 INFO:teuthology.orchestra.run.smithi012.stderr:Downloaded: 1 files, 37M in 3.6s (10.3 MB/s) 2023-11-28T21:19:09.236 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-28T21:19:09.236 DEBUG:teuthology.orchestra.run.smithi012:> sudo dpkg -i /tmp/linux-image.deb 2023-11-28T21:19:09.464 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-image-6.6.0-g8247bbb5b470. 2023-11-28T21:19:11.550 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-11-28T21:19:11.555 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-28T21:19:11.724 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-28T21:19:15.318 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-28T21:19:15.448 INFO:teuthology.orchestra.run.smithi012.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-28T21:19:36.980 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2023-11-28T21:19:37.001 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-28T21:19:37.022 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2023-11-28T21:19:37.382 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T21:19:37.402 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-28T21:19:37.750 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-28T21:19:37.762 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-28T21:19:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-28T21:19:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-28T21:19:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-28T21:19:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-28T21:19:37.999 INFO:teuthology.orchestra.run.smithi012.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-28T21:19:38.042 INFO:teuthology.orchestra.run.smithi012.stderr:done 2023-11-28T21:19:38.247 DEBUG:teuthology.orchestra.run.smithi012:> dpkg-deb -c /tmp/linux-image.deb 2023-11-28T21:19:38.254 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./ 2023-11-28T21:19:38.254 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./boot/ 2023-11-28T21:19:38.255 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 4415617 2023-11-23 03:47 ./boot/System.map-6.6.0-g8247bbb5b470 2023-11-28T21:19:38.267 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 169214 2023-11-23 03:47 ./boot/config-6.6.0-g8247bbb5b470 2023-11-28T21:19:38.268 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11352832 2023-11-23 03:47 ./boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T21:19:38.287 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/ 2023-11-28T21:19:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/ 2023-11-28T21:19:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postinst.d/ 2023-11-28T21:19:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postrm.d/ 2023-11-28T21:19:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/preinst.d/ 2023-11-28T21:19:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/prerm.d/ 2023-11-28T21:19:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/ 2023-11-28T21:19:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/ 2023-11-28T21:19:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/ 2023-11-28T21:19:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/ 2023-11-28T21:19:38.288 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/ 2023-11-28T21:19:38.289 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/ 2023-11-28T21:19:38.289 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ 2023-11-28T21:19:38.289 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 490233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-28T21:19:38.291 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-28T21:19:38.291 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-28T21:19:38.291 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-28T21:19:38.291 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 93113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-28T21:19:38.291 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 106729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 152241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 83889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-28T21:19:38.292 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/ 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/ 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/ 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpuid.ko 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/msr.ko 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/ 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 354433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-28T21:19:38.293 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 881625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-28T21:19:38.294 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 2275865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-28T21:19:38.309 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ 2023-11-28T21:19:38.309 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/af_alg.ko 2023-11-28T21:19:38.309 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_hash.ko 2023-11-28T21:19:38.309 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_skcipher.ko 2023-11-28T21:19:38.309 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-28T21:19:38.309 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-28T21:19:38.309 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/ 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/ 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-28T21:19:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cryptd.ko 2023-11-28T21:19:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_simd.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lrw.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-28T21:19:38.312 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 149193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-28T21:19:38.313 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xts.ko 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_extlog.ko 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/ 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/einj.ko 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-28T21:19:38.314 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-28T21:19:38.315 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-28T21:19:38.315 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-28T21:19:38.315 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 103025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-28T21:19:38.315 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ 2023-11-28T21:19:38.315 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-28T21:19:38.315 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 88865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci.ko 2023-11-28T21:19:38.315 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-28T21:19:38.315 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 88289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-28T21:19:38.316 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_legacy.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-28T21:19:38.317 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-28T21:19:38.318 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-28T21:19:38.318 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-28T21:19:38.318 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_radisys.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rz1000.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-28T21:19:38.320 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 83345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-28T21:19:38.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/ 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 124473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/ 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/ 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 113665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/ 2023-11-28T21:19:38.322 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 965585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-28T21:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 173225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-28T21:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/ 2023-11-28T21:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 94297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-28T21:19:38.323 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 115793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-28T21:19:38.369 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/ 2023-11-28T21:19:38.369 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 110385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-28T21:19:38.369 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 98193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-28T21:19:38.369 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 263769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-28T21:19:38.369 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/ 2023-11-28T21:19:38.369 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 98145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-28T21:19:38.369 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/ 2023-11-28T21:19:38.369 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/applicom.ko 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/ 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-28T21:19:38.370 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 148601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 139313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/ 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 84097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/nvram.ko 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/ 2023-11-28T21:19:38.371 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/ 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/ 2023-11-28T21:19:38.372 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-palmas.ko 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-si5351.ko 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-twl6040.ko 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-wm831x.ko 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/ 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 243025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/ 2023-11-28T21:19:38.373 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/ 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 438761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/ 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/ 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/ 2023-11-28T21:19:38.374 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 122569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 79953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 63617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-28T21:19:38.375 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 56825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/ 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-gpio.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max14577.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max77693.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max8997.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-palmas.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-28T21:19:38.376 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/ 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/ 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/ 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-arizona.ko 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9052.ko 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9055.ko 2023-11-28T21:19:38.377 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-dln2.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 77849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-generic.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-kempld.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max7300.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max730x.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max732x.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-28T21:19:38.378 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-vx855.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/ 2023-11-28T21:19:38.379 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-a4tech.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-apple.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-belkin.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cherry.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-chicony.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cp2112.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cypress.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-28T21:19:38.380 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ezkey.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-generic.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kensington.ko 2023-11-28T21:19:38.381 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lg-g15.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 80993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-magicmouse.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-microsoft.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-monterey.ko 2023-11-28T21:19:38.382 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ntrig.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-28T21:19:38.383 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-28T21:19:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 71073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 143473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-28T21:19:38.385 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 267825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid.ko 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/ 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 217873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/ 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 64201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 305465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ 2023-11-28T21:19:38.386 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 97977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-28T21:19:38.387 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 74433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-28T21:19:38.388 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 99601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc2103.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 94457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-28T21:19:38.389 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hih6130.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-28T21:19:38.390 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 130449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/jc42.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-28T21:19:38.391 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 108841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-28T21:19:38.392 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-28T21:19:38.393 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 119817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 82337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ 2023-11-28T21:19:38.394 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 82577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-28T21:19:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-28T21:19:38.396 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 71345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 63225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 71321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-28T21:19:38.397 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 119505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/ 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/ 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/ 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-28T21:19:38.398 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-28T21:19:38.399 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-28T21:19:38.400 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-28T21:19:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/ 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 273609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 879969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 69273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 324217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-28T21:19:38.402 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 97985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 282273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 493073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/ 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 419737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/ 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 739025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 296625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-28T21:19:38.403 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 153697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/ 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 168969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 263081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 100241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 111425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ 2023-11-28T21:19:38.404 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 128753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 129529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/evbug.ko 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ff-memless.ko 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/ 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/ 2023-11-28T21:19:38.405 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8323.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8333.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-28T21:19:38.406 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt1070.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt2160.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/stowaway.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-28T21:19:38.407 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/matrix-keymap.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/ 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 123681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 335929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/psmouse.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/ 2023-11-28T21:19:38.408 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 109873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ct82c710.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/parkbd.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/pcips2.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ps2mult.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serport.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-28T21:19:38.409 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/ 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/ 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/ 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-88pm860x.ko 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-adp5520.ko 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-bd2802.ko 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da903x.ko 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da9052.ko 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-gpio.ko 2023-11-28T21:19:38.410 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-28T21:19:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3533.ko 2023-11-28T21:19:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm355x.ko 2023-11-28T21:19:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3642.ko 2023-11-28T21:19:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-28T21:19:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp8788.ko 2023-11-28T21:19:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-max8997.ko 2023-11-28T21:19:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-mc13783.ko 2023-11-28T21:19:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-28T21:19:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca9532.ko 2023-11-28T21:19:38.411 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca955x.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca963x.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-tca6507.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm8350.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-28T21:19:38.412 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/ 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/ 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 654969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bufio.ko 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 156009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 106897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-28T21:19:38.413 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-era.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-mirror.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-region-hash.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 109529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-snapshot.ko 2023-11-28T21:19:38.414 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 181865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-zero.ko 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/ 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 199937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 113073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-28T21:19:38.415 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 158681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-28T21:19:38.416 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 395721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-28T21:19:38.452 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/ 2023-11-28T21:19:38.452 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/ 2023-11-28T21:19:38.452 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 178137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-28T21:19:38.452 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 63385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-28T21:19:38.452 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-28T21:19:38.452 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-28T21:19:38.452 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 123321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-28T21:19:38.452 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 79041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm800.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm805.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm80x.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona-i2c.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 123721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/bcm590xx.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/dln2.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/janz-cmodio.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/kempld-core.ko 2023-11-28T21:19:38.453 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-core.ko 2023-11-28T21:19:38.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-28T21:19:38.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lp3943.ko 2023-11-28T21:19:38.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-28T21:19:38.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-28T21:19:38.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/max8907.ko 2023-11-28T21:19:38.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-28T21:19:38.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-28T21:19:38.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/menf21bmc.ko 2023-11-28T21:19:38.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-28T21:19:38.454 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/retu-mfd.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 80425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/si476x-core.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps65010.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps6507x.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-28T21:19:38.455 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wm8994.ko 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot.ko 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/ 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/ 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-28T21:19:38.456 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/core.ko 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/ 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ds1682.ko 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/dummy-irq.ko 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/ 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/echo.ko 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/ 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-28T21:19:38.457 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/ 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 152201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hmc6352.ko 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ics932s401.ko 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-28T21:19:38.458 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/ 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/ 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 56521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 310113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/phantom.ko 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/ 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 74265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-28T21:19:38.459 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/ 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 183609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/ 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 490129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/ 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/ 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 106033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-28T21:19:38.460 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 453737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/ 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 130553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 68849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-28T21:19:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 99049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/ 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/ 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 69137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/ 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/ 2023-11-28T21:19:38.462 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/ 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 87297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/ 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 97337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 104673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 90257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-28T21:19:38.463 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 81569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 69729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/ 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 176761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1421497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 128569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 132089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 312081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/ 2023-11-28T21:19:38.464 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 323513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/ 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 193497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 347417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1044497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 166193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-28T21:19:38.465 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/ 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 204345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/ 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 69449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 141225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-28T21:19:38.466 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/ 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/ 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 281137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/ 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ 2023-11-28T21:19:38.467 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 90465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 295577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 584497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 246209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 925825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 302937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/ 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 514641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-28T21:19:38.468 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 93593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 734305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 151769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 86921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/ 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 102289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 122649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/ 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 727601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-28T21:19:38.469 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 278937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1113681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 95665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/ 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 108649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-28T21:19:38.470 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/ 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 150905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/ 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 136721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/ 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/ 2023-11-28T21:19:38.471 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 227897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 87081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 672049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/ 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/ 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 83985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 189801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-28T21:19:38.472 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/ 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 104241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/ 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1052497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/ 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/ 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 63337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/ 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-28T21:19:38.473 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 68177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/ 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 487025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/ 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 103337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 166097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-28T21:19:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/ 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/ 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 86529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/ 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 80273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 83633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/ 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/ 2023-11-28T21:19:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/ 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/defxx.ko 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/ 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 216985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 77345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/ 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 235521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/ 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-28T21:19:38.476 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 83769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/ 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-28T21:19:38.477 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/ 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-28T21:19:38.478 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 82049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 76113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 97857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/ 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/plip.ko 2023-11-28T21:19:38.479 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sb1000.ko 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/ 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-28T21:19:38.480 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 64449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/ 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 106169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 87017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/ 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 144297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-28T21:19:38.481 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/ 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 297329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/ 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 144585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/ 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/ 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 265121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 102209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/ 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 133841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-28T21:19:38.482 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 84465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/ 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/ 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pci-stub.ko 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/ 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-28T21:19:38.483 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 145809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia.ko 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 112889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/ 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/ 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/ 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/ 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/ 2023-11-28T21:19:38.484 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/ 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/ 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-28T21:19:38.485 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 127441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-28T21:19:38.486 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/ 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-28T21:19:38.487 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/ 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9030_battery.ko 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9052-battery.ko 2023-11-28T21:19:38.488 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/gpio-charger.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max14577_charger.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17040_battery.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17042_battery.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8903_charger.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8925_power.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-28T21:19:38.489 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/sbs-battery.ko 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/test_power.ko 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_power.ko 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm8350_power.ko 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/ 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/ 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/ 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-28T21:19:38.490 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/ 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9052.ko 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9055.ko 2023-11-28T21:19:38.491 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9063.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 68577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-28T21:19:38.492 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8907.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8925.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8997.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8998.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-28T21:19:38.493 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-palmas.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-28T21:19:38.494 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 77617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-28T21:19:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 135185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/ 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 272305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 100617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/ 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 356201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 295801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/ 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 189185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-28T21:19:38.496 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/ 2023-11-28T21:19:38.497 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 106881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-28T21:19:38.497 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-28T21:19:38.497 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/ 2023-11-28T21:19:38.497 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 246985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-28T21:19:38.530 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/ 2023-11-28T21:19:38.530 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1171945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-28T21:19:38.530 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/ 2023-11-28T21:19:38.530 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 237273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-28T21:19:38.530 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/ 2023-11-28T21:19:38.530 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 115369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-28T21:19:38.530 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-28T21:19:38.530 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/ 2023-11-28T21:19:38.530 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 308921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/ 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 76145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 146689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 75553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/ 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 212425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/ 2023-11-28T21:19:38.531 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 158505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fdomain.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/ 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 263489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 201817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/imm.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 266537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-28T21:19:38.532 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/ 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 317153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/ 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 314865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 145585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/ 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 216665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/ 2023-11-28T21:19:38.533 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 2197249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/ 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 310617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 84721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/ 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 659825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/ 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 119561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/ 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 81625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-28T21:19:38.534 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/ 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 364105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 123617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ppa.ko 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/ 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1876689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 85289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/ 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 559033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-28T21:19:38.535 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qlogicfas408.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 238761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 173937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 279313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 97729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 115889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 162121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-28T21:19:38.536 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/xen-scsifront.ko 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 129441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/ 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/ 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 736601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/ 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-28T21:19:38.537 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 930257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 122521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/ 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 75593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/ 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/ 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-28T21:19:38.538 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-28T21:19:38.539 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/moxa.ko 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 50953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/mxser.ko 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 108601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/ 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/ 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-28T21:19:38.540 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_uart.ko 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/ 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 103073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/rp2.ko 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/ 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 531265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-28T21:19:38.541 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/ 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_mf624.ko 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_netx.ko 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/ 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/ 2023-11-28T21:19:38.542 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 144377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/ 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 87089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ulpi.ko 2023-11-28T21:19:38.543 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/ 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/ 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 448505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/ 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/ 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-28T21:19:38.544 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 92489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 122081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 85609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-28T21:19:38.545 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/ 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-28T21:19:38.546 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 62185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 161225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/ 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 157681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 69705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 56033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-28T21:19:38.547 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 76057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 75073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 73169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 65809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 81505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 156073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 80113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 87353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-28T21:19:38.548 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811_cs.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/ 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cytherm.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ehset.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-28T21:19:38.549 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/lvstest.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/ 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 64841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-28T21:19:38.550 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-28T21:19:38.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-28T21:19:38.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbtest.ko 2023-11-28T21:19:38.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-28T21:19:38.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-28T21:19:38.551 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/ 2023-11-28T21:19:38.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/usbmon.ko 2023-11-28T21:19:38.551 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/ 2023-11-28T21:19:38.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 301697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-28T21:19:38.551 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/ 2023-11-28T21:19:38.551 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-28T21:19:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/f81232.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 175137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 137081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 124833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 107521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-28T21:19:38.553 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/metro-usb.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 78217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 56337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mxuport.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 191137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-28T21:19:38.554 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 51625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-28T21:19:38.555 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usbserial.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 52913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-28T21:19:38.556 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 179969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/ 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-28T21:19:38.557 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 107001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/ 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/ 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 154297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 108737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 77609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/ 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 113785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 58089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-28T21:19:38.558 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/ 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/ 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/bd6107.ko 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-28T21:19:38.559 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-28T21:19:38.560 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/ 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/ 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-28T21:19:38.561 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 56425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 102673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/acquirewdt.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/advantechwdt.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-28T21:19:38.562 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/dw_wdt.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-28T21:19:38.563 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/retu_wdt.ko 2023-11-28T21:19:38.564 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-28T21:19:38.565 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/ 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/ 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 176377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 71249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-scsiback.ko 2023-11-28T21:19:38.566 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/ 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/ 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 166425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/ 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/adfs.ko 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/ 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 163593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/ 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 932817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/kafs.ko 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/ 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/autofs4.ko 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/ 2023-11-28T21:19:38.567 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 64545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-28T21:19:38.568 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/ 2023-11-28T21:19:38.568 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/bfs.ko 2023-11-28T21:19:38.568 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-28T21:19:38.568 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/ 2023-11-28T21:19:38.568 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 3402729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-28T21:19:38.568 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/ 2023-11-28T21:19:38.568 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 306009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-28T21:19:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ 2023-11-28T21:19:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1333017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-28T21:19:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/ 2023-11-28T21:19:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 100801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-28T21:19:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/ 2023-11-28T21:19:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 108393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/configfs.ko 2023-11-28T21:19:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/ 2023-11-28T21:19:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 29625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-28T21:19:38.577 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/ 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 642513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/ 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/efs.ko 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/ 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1505025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/ 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/ 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/freevxfs.ko 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/ 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 226481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-28T21:19:38.578 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/ 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/ 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1231529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/ 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 141209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/ 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 222817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/ 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 192329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/hpfs.ko 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/ 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 74065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/ 2023-11-28T21:19:38.579 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 419497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/ 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 280313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/ 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/ 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 124649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/ 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/ 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 81449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/ 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 59449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/ 2023-11-28T21:19:38.580 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 117465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1055721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv2.ko 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 120177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 2042137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/ 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/ 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1516873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/ 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 592833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/ 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-28T21:19:38.581 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ascii.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-28T21:19:38.582 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-28T21:19:38.583 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 94137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 142673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 165745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 118121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-28T21:19:38.584 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 219417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ntfs.ko 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ 2023-11-28T21:19:38.585 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 244409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 440737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 2592433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/ 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/omfs.ko 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/ 2023-11-28T21:19:38.586 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 352161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/ 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/qnx4.ko 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/ 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 46689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/qnx6.ko 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/ 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_tree.ko 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v1.ko 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v2.ko 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/ 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 532673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-28T21:19:38.587 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/ 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/ 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/ 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 2668401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/ 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/ 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 134641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/ 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 90721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/ 2023-11-28T21:19:38.588 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 247737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-28T21:19:38.589 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ 2023-11-28T21:19:38.589 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 162497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-28T21:19:38.589 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/ 2023-11-28T21:19:38.589 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 4011017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-28T21:19:38.594 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/ 2023-11-28T21:19:38.594 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/ 2023-11-28T21:19:38.594 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 75489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/locktorture.ko 2023-11-28T21:19:38.594 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/ 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/test_udelay.ko 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc7.ko 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/ 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/interval_tree_test.ko 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-28T21:19:38.595 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/ 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/ 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/memory-notifier-error-inject.ko 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/notifier-error-inject.ko 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/percpu_test.ko 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/pm-notifier-error-inject.ko 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/ 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 143433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/rbtree_test.ko 2023-11-28T21:19:38.596 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/ 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-kstrtox.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-string_helpers.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 4760425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_bpf.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_firmware.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_module.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_user_copy.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/ 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/xz_dec_test.ko 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/ 2023-11-28T21:19:38.597 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/ 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 6985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/ 2023-11-28T21:19:38.598 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/ 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 81737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/ 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 198337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/ 2023-11-28T21:19:38.599 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 649297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/ 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 63809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 777689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-28T21:19:38.600 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 55929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-28T21:19:38.601 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-28T21:19:38.602 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-28T21:19:38.602 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/ 2023-11-28T21:19:38.602 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1046833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-28T21:19:38.624 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/ 2023-11-28T21:19:38.624 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-28T21:19:38.624 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/ 2023-11-28T21:19:38.624 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 194209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-28T21:19:38.624 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-28T21:19:38.624 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 57881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-28T21:19:38.624 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-28T21:19:38.624 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/ 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 390473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/ 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 99729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/hsr.ko 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 45233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 254625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-28T21:19:38.625 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-28T21:19:38.626 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-28T21:19:38.627 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-28T21:19:38.628 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-28T21:19:38.629 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 41665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-28T21:19:38.630 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 80897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_gre.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 93577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/mip6.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-28T21:19:38.631 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-28T21:19:38.632 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 74073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/ 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 82833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-28T21:19:38.633 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/ 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 25953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 53241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/ 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 47977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/lapb.ko 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/ 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 267913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc2.ko 2023-11-28T21:19:38.634 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/ 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 290417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/ 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 119585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 100297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 101441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 102817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-28T21:19:38.635 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 103417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 112849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 60329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 109593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 113129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 112585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 111585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 114465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 411729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-28T21:19:38.636 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 408257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-28T21:19:38.637 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 118145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 107233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 37129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-28T21:19:38.638 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 95353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 44065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 615649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 42561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 39785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-28T21:19:38.639 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 40721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_osf.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-28T21:19:38.640 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_synproxy.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tproxy.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 89313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/x_tables.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 18169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-28T21:19:38.641 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-28T21:19:38.642 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 14337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-28T21:19:38.643 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-28T21:19:38.644 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-28T21:19:38.645 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 16089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-28T21:19:38.646 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 8553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpudp.ko 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/ 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/ 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/ 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 380033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 11793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-28T21:19:38.647 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/ 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/ 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 110201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/phonet.ko 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/pn_pep.ko 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/ 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 251481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 164937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 67377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/ 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 819657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/rxrpc.ko 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/ 2023-11-28T21:19:38.648 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 43449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_cgroup.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-28T21:19:38.649 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 38425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 61409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 13409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 21649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 34633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 33609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-28T21:19:38.650 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq_codel.ko 2023-11-28T21:19:38.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 35433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-28T21:19:38.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-28T21:19:38.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-28T21:19:38.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 66393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-28T21:19:38.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-28T21:19:38.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 30273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-28T21:19:38.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-28T21:19:38.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-28T21:19:38.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 36137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-28T21:19:38.651 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 27497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-28T21:19:38.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-28T21:19:38.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-28T21:19:38.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 48729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-28T21:19:38.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 32905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-28T21:19:38.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 26881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-28T21:19:38.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 31353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-28T21:19:38.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-28T21:19:38.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 28105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-28T21:19:38.652 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/ 2023-11-28T21:19:38.652 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 929745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-28T21:19:38.656 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 19033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-28T21:19:38.656 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/ 2023-11-28T21:19:38.656 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/ 2023-11-28T21:19:38.656 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 336529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-28T21:19:38.656 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 54737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-28T21:19:38.656 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 1520097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/ 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 707657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/ 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 862665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/ 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/ 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 85409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 68049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 96193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-28T21:19:38.657 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 10705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 17521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/ 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 125177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/x25.ko 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/ 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 22657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_algo.ko 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 20649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 102257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_user.ko 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/ 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/ 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 15537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 640081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-28T21:19:38.658 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 624107 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 7663 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 23428 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 9821 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 70004 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 146313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 232548 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 210 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 72606 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 879 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 254438 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 309363 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/ 2023-11-28T21:19:38.659 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/ 2023-11-28T21:19:38.660 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/ 2023-11-28T21:19:38.660 INFO:teuthology.orchestra.run.smithi012.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/ 2023-11-28T21:19:38.660 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 172 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/changelog.Debian.gz 2023-11-28T21:19:38.660 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r-- root/root 691 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/copyright 2023-11-28T21:19:38.661 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-28T21:19:38.661 INFO:teuthology.task.kernel:searching for kernel 6.6.0-g8247bbb5b470 2023-11-28T21:19:38.661 DEBUG:teuthology.orchestra.run.smithi012:> egrep '(submenu|menuentry.*6.6.0-g8247bbb5b470).*{' /boot/grub/grub.cfg 2023-11-28T21:19:38.704 INFO:teuthology.orchestra.run.smithi012.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-28T21:19:38.704 INFO:teuthology.orchestra.run.smithi012.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-28T21:19:38.704 INFO:teuthology.orchestra.run.smithi012.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-28T21:19:38.704 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-28T21:19:38.704 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-g8247bbb5b470 2023-11-28T21:19:38.705 DEBUG:teuthology.orchestra.run.smithi012:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-28T21:19:38.747 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi012.front.sepia.ceph.com to complete... 2023-11-28T21:19:39.193 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2023-11-28T21:19:39.194 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-28T21:19:39.220 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2023-11-28T21:19:39.460 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-28T21:19:39.467 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-28T21:19:39.751 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-28T21:19:39.757 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-28T21:19:39.801 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-28T21:19:39.809 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-28T21:19:39.932 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-28T21:19:39.932 INFO:teuthology.orchestra.run.smithi012.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-28T21:19:39.932 INFO:teuthology.orchestra.run.smithi012.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-28T21:19:39.946 INFO:teuthology.orchestra.run.smithi012.stderr:done 2023-11-28T21:20:11.079 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-28T21:20:11.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-11-28T21:20:11.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T21:20:29.542 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-11-28T21:20:38.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-11-28T21:20:38.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T21:21:38.547 DEBUG:teuthology.orchestra.remote:timed out 2023-11-28T21:21:50.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-11-28T21:21:50.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T21:21:50.758 DEBUG:teuthology.orchestra.run.smithi012:> true 2023-11-28T21:21:51.398 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-11-28T21:21:51.399 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g8247bbb5b470"... 2023-11-28T21:21:51.399 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2023-11-28T21:21:51.448 INFO:teuthology.orchestra.run.smithi012.stdout:6.6.0-g8247bbb5b470 2023-11-28T21:21:51.449 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-28T21:21:51.449 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T21:21:51.449 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-28T21:21:52.452 DEBUG:teuthology.parallel:result is None 2023-11-28T21:21:52.453 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T21:21:52.463 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T21:21:52.463 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T21:21:52.467 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T21:21:52.474 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T21:21:52.539 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T21:21:52.622 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T21:21:52.622 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T21:21:52.625 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T21:21:52.653 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T21:21:52.662 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T21:21:52.662 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T21:21:52.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:21:52.687 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T21:21:52.691 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T21:21:52.754 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T21:21:52.765 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T21:21:52.770 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T21:21:52.795 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T21:21:52.797 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T21:21:52.805 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T21:21:52.805 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T21:21:52.809 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T21:21:52.828 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T21:21:52.859 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T21:21:52.859 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T21:21:52.872 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T21:21:52.878 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T21:21:52.930 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T21:21:52.981 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:21:52.981 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T21:21:53.060 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T21:21:53.066 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T21:21:53.112 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:21:53.112 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T21:21:53.178 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2023-11-28T21:21:53.182 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2023-11-28T21:21:53.279 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T21:21:53.286 INFO:teuthology.task.internal:Starting timer... 2023-11-28T21:21:53.286 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T21:21:53.339 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T21:21:53.373 DEBUG:teuthology.task.selinux:Excluding smithi012: OS 'ubuntu' does not support SELinux 2023-11-28T21:21:53.373 DEBUG:teuthology.task.selinux:Excluding smithi144: OS 'ubuntu' does not support SELinux 2023-11-28T21:21:53.373 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T21:21:53.373 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-11-28T21:21:53.373 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T21:21:53.373 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T21:21:53.446 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-28T21:21:53.454 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-28T21:21:53.618 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T21:21:53.636 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T21:21:53.637 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T21:30:39.747 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2023-11-28T21:30:39.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-11-28T21:30:39.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T21:30:39.851 DEBUG:teuthology.orchestra.run.smithi012:> true 2023-11-28T21:30:39.961 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-11-28T21:30:39.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-11-28T21:30:39.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T21:30:40.083 DEBUG:teuthology.orchestra.run.smithi144:> true 2023-11-28T21:30:40.206 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-11-28T21:30:40.207 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T21:30:40.219 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T21:30:40.219 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T21:30:40.220 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T21:30:40.223 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T21:30:40.224 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T21:30:40.261 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-28T21:30:40.261 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: Command line: ntpd -gq 2023-11-28T21:30:40.262 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: ---------------------------------------------------- 2023-11-28T21:30:40.262 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: ntp-4 is maintained by Network Time Foundation, 2023-11-28T21:30:40.262 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-28T21:30:40.262 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: corporation. Support and training for ntp-4 are 2023-11-28T21:30:40.262 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: available at https://www.nwtime.org/support 2023-11-28T21:30:40.262 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: ---------------------------------------------------- 2023-11-28T21:30:40.263 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: proto: precision = 0.043 usec (-24) 2023-11-28T21:30:40.263 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: basedate set to 2022-02-04 2023-11-28T21:30:40.263 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: gps base set to 2022-02-06 (week 2196) 2023-11-28T21:30:40.263 INFO:teuthology.orchestra.run.smithi012.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-28T21:30:40.263 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-28T21:30:40.264 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: restrict ::: KOD does nothing without LIMITED. 2023-11-28T21:30:40.264 INFO:teuthology.orchestra.run.smithi012.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-28T21:30:40.265 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: Listen and drop on 0 v6wildcard [::]:123 2023-11-28T21:30:40.265 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-28T21:30:40.265 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: Listen normally on 2 lo 127.0.0.1:123 2023-11-28T21:30:40.265 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: Listen normally on 3 ens1f0 172.21.15.12:123 2023-11-28T21:30:40.265 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: Listen normally on 4 lo [::1]:123 2023-11-28T21:30:40.265 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25ea%4]:123 2023-11-28T21:30:40.265 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:40 ntpd[27862]: Listening on routing socket on fd #22 for interface updates 2023-11-28T21:30:40.275 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-28T21:30:40.275 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: Command line: ntpd -gq 2023-11-28T21:30:40.275 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: ---------------------------------------------------- 2023-11-28T21:30:40.275 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: ntp-4 is maintained by Network Time Foundation, 2023-11-28T21:30:40.276 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-28T21:30:40.276 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: corporation. Support and training for ntp-4 are 2023-11-28T21:30:40.276 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: available at https://www.nwtime.org/support 2023-11-28T21:30:40.276 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: ---------------------------------------------------- 2023-11-28T21:30:40.276 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: proto: precision = 0.058 usec (-24) 2023-11-28T21:30:40.277 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: basedate set to 2022-02-04 2023-11-28T21:30:40.277 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: gps base set to 2022-02-06 (week 2196) 2023-11-28T21:30:40.277 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-28T21:30:40.277 INFO:teuthology.orchestra.run.smithi144.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-28T21:30:40.277 INFO:teuthology.orchestra.run.smithi144.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-28T21:30:40.278 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: restrict ::: KOD does nothing without LIMITED. 2023-11-28T21:30:40.278 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: Listen and drop on 0 v6wildcard [::]:123 2023-11-28T21:30:40.278 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-28T21:30:40.279 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: Listen normally on 2 lo 127.0.0.1:123 2023-11-28T21:30:40.279 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: Listen normally on 3 enp3s0f1 172.21.15.144:123 2023-11-28T21:30:40.279 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: Listen normally on 4 lo [::1]:123 2023-11-28T21:30:40.279 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f5f%5]:123 2023-11-28T21:30:40.279 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:40 ntpd[28443]: Listening on routing socket on fd #22 for interface updates 2023-11-28T21:30:41.262 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:41 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:41.262 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:41 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:41.262 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:41 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:41.262 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:41 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:41.262 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:41 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:41.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:41 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:41.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:41 ntpd[27862]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:41.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:41 ntpd[27862]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:41.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:41.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:41.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:41.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:41 ntpd[28443]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:43.262 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:43 ntpd[27862]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:43.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:43.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:43.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:43.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:43.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:43.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:43.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:43.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:43.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:43.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:43.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:43.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:43 ntpd[28443]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:44.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:44 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:44.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:44 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:44.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:44 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:44.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:44 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:44.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:44 ntpd[28443]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:44.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:44 ntpd[28443]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:45 ntpd[27862]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:45.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:45.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:45.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:45.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:45.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:45.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:45.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:45.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:45.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:45.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:45.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:45.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:45 ntpd[28443]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:46.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:46 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:46.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:46 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:46.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:46 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:46.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:46 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:46.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:46 ntpd[28443]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:46.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:46 ntpd[28443]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:47.262 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:47.263 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:47.264 INFO:teuthology.orchestra.run.smithi012.stdout:28 Nov 21:30:47 ntpd[27862]: ntpd: time slew +0.000353 s 2023-11-28T21:30:47.264 INFO:teuthology.orchestra.run.smithi012.stdout:ntpd: time slew +0.000353s 2023-11-28T21:30:47.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-28T21:30:47.264 INFO:teuthology.orchestra.run.smithi012.stderr:28 Nov 21:30:47 ntpd[27862]: can't open /var/log/ntpstats/loopstats.20231128: Permission denied 2023-11-28T21:30:47.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:47 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:47.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:47 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:47.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:47 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:47.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:47 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:47.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:47 ntpd[28443]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-28T21:30:47.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:47 ntpd[28443]: can't open /var/log/ntpstats/rawstats.20231128: Permission denied 2023-11-28T21:30:47.276 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:47 ntpd[28443]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-28T21:30:47.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:47 ntpd[28443]: can't open /var/log/ntpstats/peerstats.20231128: Permission denied 2023-11-28T21:30:47.277 INFO:teuthology.orchestra.run.smithi144.stdout:28 Nov 21:30:47 ntpd[28443]: ntpd: time slew -0.004324 s 2023-11-28T21:30:47.277 INFO:teuthology.orchestra.run.smithi144.stdout:ntpd: time slew -0.004324s 2023-11-28T21:30:47.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:47 ntpd[28443]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-28T21:30:47.277 INFO:teuthology.orchestra.run.smithi144.stderr:28 Nov 21:30:47 ntpd[28443]: can't open /var/log/ntpstats/loopstats.20231128: Permission denied 2023-11-28T21:30:47.356 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2023-11-28T21:30:47.356 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2023-11-28T21:30:47.356 INFO:teuthology.orchestra.run.smithi144.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T21:30:47.356 INFO:teuthology.orchestra.run.smithi144.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T21:30:47.356 INFO:teuthology.orchestra.run.smithi144.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T21:30:47.357 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T21:30:47.357 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2023-11-28T21:30:47.357 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2023-11-28T21:30:47.357 INFO:teuthology.orchestra.run.smithi012.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T21:30:47.357 INFO:teuthology.orchestra.run.smithi012.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T21:30:47.357 INFO:teuthology.orchestra.run.smithi012.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T21:30:47.357 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-28T21:30:47.358 INFO:teuthology.run_tasks:Running task install... 2023-11-28T21:30:47.368 DEBUG:teuthology.task.install:project ceph 2023-11-28T21:30:47.369 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '43dd94c190073cf16cff1e711dd7f5a7b1cbc883'}} 2023-11-28T21:30:47.369 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '43dd94c190073cf16cff1e711dd7f5a7b1cbc883'} 2023-11-28T21:30:47.369 INFO:teuthology.task.install:Using flavor: default 2023-11-28T21:30:47.375 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T21:30:47.375 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T21:30:47.375 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-key list | grep Ceph 2023-11-28T21:30:47.376 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-key list | grep Ceph 2023-11-28T21:30:47.454 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-28T21:30:47.481 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-28T21:30:47.481 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph.com (release key) 2023-11-28T21:30:47.482 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-28T21:30:47.482 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-28T21:30:47.482 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 2023-11-28T21:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-28T21:30:47.568 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-28T21:30:47.568 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph.com (release key) 2023-11-28T21:30:47.569 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-28T21:30:47.569 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-28T21:30:47.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 2023-11-28T21:30:47.619 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default/ 2023-11-28T21:30:47.619 INFO:teuthology.task.install.deb:Package version is 18.0.0-7520-g43dd94c1-1jammy 2023-11-28T21:30:47.705 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default/ 2023-11-28T21:30:47.706 INFO:teuthology.task.install.deb:Package version is 18.0.0-7520-g43dd94c1-1jammy 2023-11-28T21:30:47.794 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:30:47.795 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-28T21:30:47.809 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2023-11-28T21:30:47.880 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:30:47.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-28T21:30:47.901 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2023-11-28T21:30:48.062 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-28T21:30:48.087 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy InRelease 2023-11-28T21:30:48.093 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-28T21:30:48.095 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-28T21:30:48.102 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-28T21:30:48.121 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-28T21:30:48.124 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2023-11-28T21:30:48.160 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-28T21:30:48.176 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-28T21:30:48.197 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-11-28T21:30:48.201 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy InRelease 2023-11-28T21:30:48.209 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-28T21:30:48.237 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2023-11-28T21:30:48.238 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2023-11-28T21:30:48.258 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-28T21:30:48.274 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-28T21:30:48.311 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-11-28T21:30:48.359 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2023-11-28T21:30:48.424 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 27.9 kB in 0s (63.5 kB/s) 2023-11-28T21:30:48.722 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 27.9 kB in 1s (47.0 kB/s) 2023-11-28T21:30:49.301 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2023-11-28T21:30:49.318 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7520-g43dd94c1-1jammy cephadm=18.0.0-7520-g43dd94c1-1jammy ceph-mds=18.0.0-7520-g43dd94c1-1jammy ceph-mgr=18.0.0-7520-g43dd94c1-1jammy ceph-common=18.0.0-7520-g43dd94c1-1jammy ceph-fuse=18.0.0-7520-g43dd94c1-1jammy ceph-test=18.0.0-7520-g43dd94c1-1jammy ceph-volume=18.0.0-7520-g43dd94c1-1jammy radosgw=18.0.0-7520-g43dd94c1-1jammy python3-rados=18.0.0-7520-g43dd94c1-1jammy python3-rgw=18.0.0-7520-g43dd94c1-1jammy python3-cephfs=18.0.0-7520-g43dd94c1-1jammy python3-rbd=18.0.0-7520-g43dd94c1-1jammy libcephfs2=18.0.0-7520-g43dd94c1-1jammy libcephfs-dev=18.0.0-7520-g43dd94c1-1jammy librados2=18.0.0-7520-g43dd94c1-1jammy librbd1=18.0.0-7520-g43dd94c1-1jammy rbd-fuse=18.0.0-7520-g43dd94c1-1jammy python3-cephfs=18.0.0-7520-g43dd94c1-1jammy cephfs-shell=18.0.0-7520-g43dd94c1-1jammy cephfs-top=18.0.0-7520-g43dd94c1-1jammy cephfs-mirror=18.0.0-7520-g43dd94c1-1jammy 2023-11-28T21:30:49.379 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2023-11-28T21:30:49.566 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2023-11-28T21:30:49.566 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2023-11-28T21:30:49.681 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-11-28T21:30:49.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7520-g43dd94c1-1jammy cephadm=18.0.0-7520-g43dd94c1-1jammy ceph-mds=18.0.0-7520-g43dd94c1-1jammy ceph-mgr=18.0.0-7520-g43dd94c1-1jammy ceph-common=18.0.0-7520-g43dd94c1-1jammy ceph-fuse=18.0.0-7520-g43dd94c1-1jammy ceph-test=18.0.0-7520-g43dd94c1-1jammy ceph-volume=18.0.0-7520-g43dd94c1-1jammy radosgw=18.0.0-7520-g43dd94c1-1jammy python3-rados=18.0.0-7520-g43dd94c1-1jammy python3-rgw=18.0.0-7520-g43dd94c1-1jammy python3-cephfs=18.0.0-7520-g43dd94c1-1jammy python3-rbd=18.0.0-7520-g43dd94c1-1jammy libcephfs2=18.0.0-7520-g43dd94c1-1jammy libcephfs-dev=18.0.0-7520-g43dd94c1-1jammy librados2=18.0.0-7520-g43dd94c1-1jammy librbd1=18.0.0-7520-g43dd94c1-1jammy rbd-fuse=18.0.0-7520-g43dd94c1-1jammy python3-cephfs=18.0.0-7520-g43dd94c1-1jammy cephfs-shell=18.0.0-7520-g43dd94c1-1jammy cephfs-top=18.0.0-7520-g43dd94c1-1jammy cephfs-mirror=18.0.0-7520-g43dd94c1-1jammy 2023-11-28T21:30:49.733 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2023-11-28T21:30:49.734 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-28T21:30:49.734 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-28T21:30:49.734 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-28T21:30:49.734 INFO:teuthology.orchestra.run.smithi144.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-11-28T21:30:49.734 INFO:teuthology.orchestra.run.smithi144.stdout: python2.7-minimal 2023-11-28T21:30:49.735 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-28T21:30:49.737 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2023-11-28T21:30:49.738 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-28T21:30:49.738 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-28T21:30:49.738 INFO:teuthology.orchestra.run.smithi144.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-11-28T21:30:49.738 INFO:teuthology.orchestra.run.smithi144.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-11-28T21:30:49.738 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2023-11-28T21:30:49.739 INFO:teuthology.orchestra.run.smithi144.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2023-11-28T21:30:49.739 INFO:teuthology.orchestra.run.smithi144.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-28T21:30:49.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-28T21:30:49.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-11-28T21:30:49.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-11-28T21:30:49.739 INFO:teuthology.orchestra.run.smithi144.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-11-28T21:30:49.740 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-28T21:30:49.740 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-28T21:30:49.740 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-28T21:30:49.740 INFO:teuthology.orchestra.run.smithi144.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-28T21:30:49.740 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-28T21:30:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-28T21:30:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-11-28T21:30:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-11-28T21:30:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-11-28T21:30:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-11-28T21:30:49.741 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-11-28T21:30:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-11-28T21:30:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2023-11-28T21:30:49.742 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2023-11-28T21:30:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2023-11-28T21:30:49.742 INFO:teuthology.orchestra.run.smithi144.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2023-11-28T21:30:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2023-11-28T21:30:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2023-11-28T21:30:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2023-11-28T21:30:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-11-28T21:30:49.743 INFO:teuthology.orchestra.run.smithi144.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2023-11-28T21:30:49.744 INFO:teuthology.orchestra.run.smithi144.stdout:Recommended packages: 2023-11-28T21:30:49.744 INFO:teuthology.orchestra.run.smithi144.stdout: btrfs-tools 2023-11-28T21:30:49.773 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-11-28T21:30:49.841 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2023-11-28T21:30:49.841 INFO:teuthology.orchestra.run.smithi144.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-28T21:30:49.841 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-28T21:30:49.841 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-28T21:30:49.841 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-28T21:30:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-11-28T21:30:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-11-28T21:30:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2023-11-28T21:30:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2023-11-28T21:30:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2023-11-28T21:30:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-11-28T21:30:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-11-28T21:30:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-11-28T21:30:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-11-28T21:30:49.843 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-11-28T21:30:49.844 INFO:teuthology.orchestra.run.smithi144.stdout: socat unzip xmlstarlet zip 2023-11-28T21:30:49.899 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 124 newly installed, 0 to remove and 264 not upgraded. 2023-11-28T21:30:49.899 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 187 MB of archives. 2023-11-28T21:30:49.899 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 835 MB of additional disk space will be used. 2023-11-28T21:30:49.899 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-28T21:30:49.916 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-28T21:30:49.926 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-28T21:30:49.943 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2023-11-28T21:30:49.944 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2023-11-28T21:30:49.966 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-28T21:30:49.983 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7520-g43dd94c1-1jammy [3,469 kB] 2023-11-28T21:30:49.987 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-28T21:30:49.990 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-28T21:30:50.001 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-28T21:30:50.045 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-28T21:30:50.049 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-28T21:30:50.061 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-28T21:30:50.064 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-28T21:30:50.065 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-28T21:30:50.073 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2023-11-28T21:30:50.075 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2023-11-28T21:30:50.077 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2023-11-28T21:30:50.078 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2023-11-28T21:30:50.079 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2023-11-28T21:30:50.083 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] 2023-11-28T21:30:50.085 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2023-11-28T21:30:50.098 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-28T21:30:50.099 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-28T21:30:50.099 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-28T21:30:50.100 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-28T21:30:50.100 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-28T21:30:50.101 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-28T21:30:50.101 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-28T21:30:50.102 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-28T21:30:50.108 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2023-11-28T21:30:50.108 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-28T21:30:50.108 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-28T21:30:50.108 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-28T21:30:50.108 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-28T21:30:50.109 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-28T21:30:50.109 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-28T21:30:50.110 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2023-11-28T21:30:50.110 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-28T21:30:50.110 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-28T21:30:50.110 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-28T21:30:50.111 INFO:teuthology.orchestra.run.smithi012.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-28T21:30:50.112 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-28T21:30:50.112 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-28T21:30:50.112 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-11-28T21:30:50.112 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-11-28T21:30:50.112 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-11-28T21:30:50.112 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-11-28T21:30:50.112 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-11-28T21:30:50.112 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-11-28T21:30:50.112 INFO:teuthology.orchestra.run.smithi012.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2023-11-28T21:30:50.113 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2023-11-28T21:30:50.113 INFO:teuthology.orchestra.run.smithi012.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2023-11-28T21:30:50.113 INFO:teuthology.orchestra.run.smithi012.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2023-11-28T21:30:50.113 INFO:teuthology.orchestra.run.smithi012.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2023-11-28T21:30:50.113 INFO:teuthology.orchestra.run.smithi012.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2023-11-28T21:30:50.113 INFO:teuthology.orchestra.run.smithi012.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2023-11-28T21:30:50.113 INFO:teuthology.orchestra.run.smithi012.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-11-28T21:30:50.113 INFO:teuthology.orchestra.run.smithi012.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2023-11-28T21:30:50.113 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2023-11-28T21:30:50.113 INFO:teuthology.orchestra.run.smithi012.stdout: btrfs-tools 2023-11-28T21:30:50.116 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-28T21:30:50.117 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-28T21:30:50.133 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-28T21:30:50.134 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-28T21:30:50.134 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-28T21:30:50.135 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-28T21:30:50.136 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-28T21:30:50.136 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-28T21:30:50.136 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-28T21:30:50.138 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-28T21:30:50.151 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-28T21:30:50.151 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-28T21:30:50.169 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-28T21:30:50.169 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-28T21:30:50.170 INFO:teuthology.orchestra.run.smithi144.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-28T21:30:50.170 INFO:teuthology.orchestra.run.smithi144.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-28T21:30:50.172 INFO:teuthology.orchestra.run.smithi144.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-28T21:30:50.173 INFO:teuthology.orchestra.run.smithi144.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-28T21:30:50.174 INFO:teuthology.orchestra.run.smithi144.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-11-28T21:30:50.175 INFO:teuthology.orchestra.run.smithi144.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-28T21:30:50.185 INFO:teuthology.orchestra.run.smithi144.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-28T21:30:50.188 INFO:teuthology.orchestra.run.smithi144.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-28T21:30:50.202 INFO:teuthology.orchestra.run.smithi144.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-28T21:30:50.203 INFO:teuthology.orchestra.run.smithi144.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-28T21:30:50.205 INFO:teuthology.orchestra.run.smithi144.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-28T21:30:50.254 INFO:teuthology.orchestra.run.smithi144.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-28T21:30:50.255 INFO:teuthology.orchestra.run.smithi144.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-28T21:30:50.256 INFO:teuthology.orchestra.run.smithi144.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-28T21:30:50.256 INFO:teuthology.orchestra.run.smithi144.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-28T21:30:50.257 INFO:teuthology.orchestra.run.smithi144.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-28T21:30:50.263 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2023-11-28T21:30:50.263 INFO:teuthology.orchestra.run.smithi012.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-28T21:30:50.263 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-28T21:30:50.263 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-28T21:30:50.265 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-28T21:30:50.265 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-11-28T21:30:50.265 INFO:teuthology.orchestra.run.smithi012.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-11-28T21:30:50.265 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2023-11-28T21:30:50.265 INFO:teuthology.orchestra.run.smithi012.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2023-11-28T21:30:50.265 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2023-11-28T21:30:50.265 INFO:teuthology.orchestra.run.smithi012.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-11-28T21:30:50.265 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-11-28T21:30:50.265 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-11-28T21:30:50.266 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-11-28T21:30:50.267 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-11-28T21:30:50.267 INFO:teuthology.orchestra.run.smithi012.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-11-28T21:30:50.267 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-11-28T21:30:50.267 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-11-28T21:30:50.267 INFO:teuthology.orchestra.run.smithi012.stdout: socat unzip xmlstarlet zip 2023-11-28T21:30:50.288 INFO:teuthology.orchestra.run.smithi144.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-28T21:30:50.290 INFO:teuthology.orchestra.run.smithi144.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-28T21:30:50.291 INFO:teuthology.orchestra.run.smithi144.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-28T21:30:50.305 INFO:teuthology.orchestra.run.smithi144.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-28T21:30:50.306 INFO:teuthology.orchestra.run.smithi144.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-28T21:30:50.306 INFO:teuthology.orchestra.run.smithi144.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-28T21:30:50.307 INFO:teuthology.orchestra.run.smithi144.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-28T21:30:50.307 INFO:teuthology.orchestra.run.smithi144.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-28T21:30:50.308 INFO:teuthology.orchestra.run.smithi144.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-28T21:30:50.308 INFO:teuthology.orchestra.run.smithi144.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-28T21:30:50.311 INFO:teuthology.orchestra.run.smithi144.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7520-g43dd94c1-1jammy [3,111 kB] 2023-11-28T21:30:50.324 INFO:teuthology.orchestra.run.smithi144.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-28T21:30:50.327 INFO:teuthology.orchestra.run.smithi144.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-28T21:30:50.329 INFO:teuthology.orchestra.run.smithi144.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-28T21:30:50.330 INFO:teuthology.orchestra.run.smithi144.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-28T21:30:50.341 INFO:teuthology.orchestra.run.smithi144.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-28T21:30:50.344 INFO:teuthology.orchestra.run.smithi144.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-28T21:30:50.347 INFO:teuthology.orchestra.run.smithi144.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-28T21:30:50.347 INFO:teuthology.orchestra.run.smithi144.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2023-11-28T21:30:50.349 INFO:teuthology.orchestra.run.smithi144.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2023-11-28T21:30:50.349 INFO:teuthology.orchestra.run.smithi144.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-28T21:30:50.358 INFO:teuthology.orchestra.run.smithi144.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2023-11-28T21:30:50.359 INFO:teuthology.orchestra.run.smithi144.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-28T21:30:50.361 INFO:teuthology.orchestra.run.smithi144.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-28T21:30:50.362 INFO:teuthology.orchestra.run.smithi144.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-28T21:30:50.375 INFO:teuthology.orchestra.run.smithi144.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-28T21:30:50.375 INFO:teuthology.orchestra.run.smithi144.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-28T21:30:50.377 INFO:teuthology.orchestra.run.smithi144.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-28T21:30:50.377 INFO:teuthology.orchestra.run.smithi144.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-28T21:30:50.383 INFO:teuthology.orchestra.run.smithi144.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-28T21:30:50.384 INFO:teuthology.orchestra.run.smithi144.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-28T21:30:50.392 INFO:teuthology.orchestra.run.smithi144.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-28T21:30:50.394 INFO:teuthology.orchestra.run.smithi144.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-28T21:30:50.394 INFO:teuthology.orchestra.run.smithi144.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-28T21:30:50.442 INFO:teuthology.orchestra.run.smithi144.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7520-g43dd94c1-1jammy [691 kB] 2023-11-28T21:30:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 124 newly installed, 0 to remove and 264 not upgraded. 2023-11-28T21:30:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 187 MB of archives. 2023-11-28T21:30:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 835 MB of additional disk space will be used. 2023-11-28T21:30:50.462 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-28T21:30:50.464 INFO:teuthology.orchestra.run.smithi144.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7520-g43dd94c1-1jammy [356 kB] 2023-11-28T21:30:50.481 INFO:teuthology.orchestra.run.smithi144.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7520-g43dd94c1-1jammy [32.8 kB] 2023-11-28T21:30:50.482 INFO:teuthology.orchestra.run.smithi144.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7520-g43dd94c1-1jammy [180 kB] 2023-11-28T21:30:50.486 INFO:teuthology.orchestra.run.smithi144.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7520-g43dd94c1-1jammy [63.5 kB] 2023-11-28T21:30:50.487 INFO:teuthology.orchestra.run.smithi144.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7520-g43dd94c1-1jammy [330 kB] 2023-11-28T21:30:50.491 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7520-g43dd94c1-1jammy [3,469 kB] 2023-11-28T21:30:50.501 INFO:teuthology.orchestra.run.smithi144.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7520-g43dd94c1-1jammy [6,233 kB] 2023-11-28T21:30:50.544 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-28T21:30:50.590 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-28T21:30:50.730 INFO:teuthology.orchestra.run.smithi144.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7520-g43dd94c1-1jammy [112 kB] 2023-11-28T21:30:50.734 INFO:teuthology.orchestra.run.smithi144.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7520-g43dd94c1-1jammy [439 kB] 2023-11-28T21:30:50.752 INFO:teuthology.orchestra.run.smithi144.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7520-g43dd94c1-1jammy [22.6 MB] 2023-11-28T21:30:50.791 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-28T21:30:50.885 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-28T21:30:50.899 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-28T21:30:50.950 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-28T21:30:51.123 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-28T21:30:51.133 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-28T21:30:51.160 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-28T21:30:51.168 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-28T21:30:51.262 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7520-g43dd94c1-1jammy [3,111 kB] 2023-11-28T21:30:51.332 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-28T21:30:51.783 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7520-g43dd94c1-1jammy [691 kB] 2023-11-28T21:30:51.809 INFO:teuthology.orchestra.run.smithi144.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7520-g43dd94c1-1jammy [6,900 kB] 2023-11-28T21:30:51.853 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7520-g43dd94c1-1jammy [356 kB] 2023-11-28T21:30:51.914 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2023-11-28T21:30:51.914 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7520-g43dd94c1-1jammy [32.8 kB] 2023-11-28T21:30:51.917 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7520-g43dd94c1-1jammy [180 kB] 2023-11-28T21:30:51.928 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2023-11-28T21:30:51.944 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2023-11-28T21:30:51.947 INFO:teuthology.orchestra.run.smithi144.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7520-g43dd94c1-1jammy [238 kB] 2023-11-28T21:30:51.948 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7520-g43dd94c1-1jammy [63.5 kB] 2023-11-28T21:30:51.953 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2023-11-28T21:30:51.953 INFO:teuthology.orchestra.run.smithi144.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7520-g43dd94c1-1jammy [124 kB] 2023-11-28T21:30:51.954 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2023-11-28T21:30:51.955 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7520-g43dd94c1-1jammy [330 kB] 2023-11-28T21:30:51.957 INFO:teuthology.orchestra.run.smithi144.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7520-g43dd94c1-1jammy [1,496 kB] 2023-11-28T21:30:51.970 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] 2023-11-28T21:30:52.007 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2023-11-28T21:30:52.010 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7520-g43dd94c1-1jammy [6,233 kB] 2023-11-28T21:30:52.019 INFO:teuthology.orchestra.run.smithi144.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7520-g43dd94c1-1jammy [9,956 kB] 2023-11-28T21:30:52.020 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-28T21:30:52.024 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-28T21:30:52.025 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-28T21:30:52.195 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-28T21:30:52.277 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-28T21:30:52.284 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-28T21:30:52.358 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-28T21:30:52.387 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-28T21:30:52.397 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-28T21:30:52.427 INFO:teuthology.orchestra.run.smithi144.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7520-g43dd94c1-1jammy [31.9 MB] 2023-11-28T21:30:52.439 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-28T21:30:52.617 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-28T21:30:52.632 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-28T21:30:52.638 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-28T21:30:52.643 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-28T21:30:52.827 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-28T21:30:52.910 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-28T21:30:52.924 INFO:teuthology.orchestra.run.smithi012.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-28T21:30:53.051 INFO:teuthology.orchestra.run.smithi012.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7520-g43dd94c1-1jammy [112 kB] 2023-11-28T21:30:53.068 INFO:teuthology.orchestra.run.smithi012.stdout:Get:47 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7520-g43dd94c1-1jammy [439 kB] 2023-11-28T21:30:53.100 INFO:teuthology.orchestra.run.smithi012.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-28T21:30:53.141 INFO:teuthology.orchestra.run.smithi012.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7520-g43dd94c1-1jammy [22.6 MB] 2023-11-28T21:30:53.148 INFO:teuthology.orchestra.run.smithi012.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-28T21:30:53.162 INFO:teuthology.orchestra.run.smithi012.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-28T21:30:53.320 INFO:teuthology.orchestra.run.smithi012.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-28T21:30:53.320 INFO:teuthology.orchestra.run.smithi012.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-28T21:30:53.325 INFO:teuthology.orchestra.run.smithi012.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-28T21:30:53.330 INFO:teuthology.orchestra.run.smithi012.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-28T21:30:53.345 INFO:teuthology.orchestra.run.smithi012.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-28T21:30:53.529 INFO:teuthology.orchestra.run.smithi012.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-28T21:30:53.782 INFO:teuthology.orchestra.run.smithi012.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-11-28T21:30:53.891 INFO:teuthology.orchestra.run.smithi144.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7520-g43dd94c1-1jammy [14.1 kB] 2023-11-28T21:30:53.892 INFO:teuthology.orchestra.run.smithi144.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7520-g43dd94c1-1jammy [839 kB] 2023-11-28T21:30:53.909 INFO:teuthology.orchestra.run.smithi144.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7520-g43dd94c1-1jammy [2,148 kB] 2023-11-28T21:30:53.915 INFO:teuthology.orchestra.run.smithi012.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-28T21:30:53.948 INFO:teuthology.orchestra.run.smithi144.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7520-g43dd94c1-1jammy [801 kB] 2023-11-28T21:30:53.958 INFO:teuthology.orchestra.run.smithi144.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7520-g43dd94c1-1jammy [139 kB] 2023-11-28T21:30:53.959 INFO:teuthology.orchestra.run.smithi012.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-28T21:30:53.961 INFO:teuthology.orchestra.run.smithi144.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7520-g43dd94c1-1jammy [1,900 kB] 2023-11-28T21:30:53.989 INFO:teuthology.orchestra.run.smithi144.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7520-g43dd94c1-1jammy [8,625 kB] 2023-11-28T21:30:54.049 INFO:teuthology.orchestra.run.smithi012.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-28T21:30:54.058 INFO:teuthology.orchestra.run.smithi012.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-28T21:30:54.071 INFO:teuthology.orchestra.run.smithi012.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-28T21:30:54.114 INFO:teuthology.orchestra.run.smithi012.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-28T21:30:54.343 INFO:teuthology.orchestra.run.smithi144.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7520-g43dd94c1-1jammy [14.2 kB] 2023-11-28T21:30:54.343 INFO:teuthology.orchestra.run.smithi144.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7520-g43dd94c1-1jammy [48.7 MB] 2023-11-28T21:30:54.376 INFO:teuthology.orchestra.run.smithi012.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-28T21:30:54.377 INFO:teuthology.orchestra.run.smithi012.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-28T21:30:54.379 INFO:teuthology.orchestra.run.smithi012.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-28T21:30:54.557 INFO:teuthology.orchestra.run.smithi012.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-28T21:30:54.900 INFO:teuthology.orchestra.run.smithi012.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-28T21:30:55.308 INFO:teuthology.orchestra.run.smithi012.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-28T21:30:55.317 INFO:teuthology.orchestra.run.smithi012.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-28T21:30:55.318 INFO:teuthology.orchestra.run.smithi012.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-28T21:30:55.388 INFO:teuthology.orchestra.run.smithi012.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-28T21:30:55.389 INFO:teuthology.orchestra.run.smithi012.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-28T21:30:55.390 INFO:teuthology.orchestra.run.smithi012.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-28T21:30:55.391 INFO:teuthology.orchestra.run.smithi012.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-28T21:30:55.391 INFO:teuthology.orchestra.run.smithi012.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-28T21:30:55.392 INFO:teuthology.orchestra.run.smithi012.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-28T21:30:55.556 INFO:teuthology.orchestra.run.smithi012.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-28T21:30:55.760 INFO:teuthology.orchestra.run.smithi012.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-28T21:30:56.083 INFO:teuthology.orchestra.run.smithi012.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-28T21:30:56.129 INFO:teuthology.orchestra.run.smithi012.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-28T21:30:56.156 INFO:teuthology.orchestra.run.smithi012.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-28T21:30:56.165 INFO:teuthology.orchestra.run.smithi012.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-28T21:30:56.210 INFO:teuthology.orchestra.run.smithi012.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-28T21:30:56.236 INFO:teuthology.orchestra.run.smithi012.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-28T21:30:56.238 INFO:teuthology.orchestra.run.smithi012.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2023-11-28T21:30:56.244 INFO:teuthology.orchestra.run.smithi012.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2023-11-28T21:30:56.247 INFO:teuthology.orchestra.run.smithi012.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-28T21:30:56.444 INFO:teuthology.orchestra.run.smithi012.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2023-11-28T21:30:56.582 INFO:teuthology.orchestra.run.smithi144.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7520-g43dd94c1-1jammy [121 kB] 2023-11-28T21:30:56.584 INFO:teuthology.orchestra.run.smithi144.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-7520-g43dd94c1-1jammy [225 kB] 2023-11-28T21:30:56.589 INFO:teuthology.orchestra.run.smithi144.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-7520-g43dd94c1-1jammy [34.0 kB] 2023-11-28T21:30:56.589 INFO:teuthology.orchestra.run.smithi144.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-7520-g43dd94c1-1jammy [23.4 kB] 2023-11-28T21:30:56.590 INFO:teuthology.orchestra.run.smithi144.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7520-g43dd94c1-1jammy [38.4 kB] 2023-11-28T21:30:56.591 INFO:teuthology.orchestra.run.smithi144.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7520-g43dd94c1-1jammy [12.3 MB] 2023-11-28T21:30:56.636 INFO:teuthology.orchestra.run.smithi012.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7520-g43dd94c1-1jammy [6,900 kB] 2023-11-28T21:30:56.701 INFO:teuthology.orchestra.run.smithi012.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-28T21:30:56.953 INFO:teuthology.orchestra.run.smithi144.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7520-g43dd94c1-1jammy [91.2 kB] 2023-11-28T21:30:56.967 INFO:teuthology.orchestra.run.smithi012.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-28T21:30:56.969 INFO:teuthology.orchestra.run.smithi012.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-28T21:30:56.985 INFO:teuthology.orchestra.run.smithi012.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-28T21:30:56.991 INFO:teuthology.orchestra.run.smithi012.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-28T21:30:57.030 INFO:teuthology.orchestra.run.smithi012.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-28T21:30:57.044 INFO:teuthology.orchestra.run.smithi012.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-28T21:30:57.134 INFO:teuthology.orchestra.run.smithi012.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-28T21:30:57.136 INFO:teuthology.orchestra.run.smithi012.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-28T21:30:57.137 INFO:teuthology.orchestra.run.smithi012.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-28T21:30:57.299 INFO:teuthology.orchestra.run.smithi012.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7520-g43dd94c1-1jammy [238 kB] 2023-11-28T21:30:57.304 INFO:teuthology.orchestra.run.smithi012.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7520-g43dd94c1-1jammy [124 kB] 2023-11-28T21:30:57.317 INFO:teuthology.orchestra.run.smithi012.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-28T21:30:57.319 INFO:teuthology.orchestra.run.smithi012.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7520-g43dd94c1-1jammy [1,496 kB] 2023-11-28T21:30:57.414 INFO:teuthology.orchestra.run.smithi012.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7520-g43dd94c1-1jammy [9,956 kB] 2023-11-28T21:30:57.520 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 187 MB in 7s (26.3 MB/s) 2023-11-28T21:30:57.580 INFO:teuthology.orchestra.run.smithi012.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-28T21:30:57.645 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-28T21:30:57.727 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 130504 files and directories currently installed.) 2023-11-28T21:30:57.731 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-28T21:30:57.793 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-28T21:30:58.021 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-28T21:30:58.031 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-28T21:30:58.113 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-28T21:30:58.115 INFO:teuthology.orchestra.run.smithi012.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7520-g43dd94c1-1jammy [31.9 MB] 2023-11-28T21:30:58.474 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cmd2. 2023-11-28T21:30:58.494 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-28T21:30:58.532 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-28T21:30:58.910 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-28T21:30:58.930 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-28T21:30:59.043 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-28T21:30:59.430 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-28T21:30:59.451 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-28T21:30:59.488 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-28T21:30:59.783 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-28T21:30:59.804 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-28T21:30:59.867 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-28T21:31:00.330 INFO:teuthology.orchestra.run.smithi012.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7520-g43dd94c1-1jammy [14.1 kB] 2023-11-28T21:31:00.330 INFO:teuthology.orchestra.run.smithi012.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7520-g43dd94c1-1jammy [839 kB] 2023-11-28T21:31:00.354 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-28T21:31:00.374 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T21:31:00.379 INFO:teuthology.orchestra.run.smithi012.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7520-g43dd94c1-1jammy [2,148 kB] 2023-11-28T21:31:00.453 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:31:00.569 INFO:teuthology.orchestra.run.smithi012.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7520-g43dd94c1-1jammy [801 kB] 2023-11-28T21:31:00.657 INFO:teuthology.orchestra.run.smithi012.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7520-g43dd94c1-1jammy [139 kB] 2023-11-28T21:31:00.665 INFO:teuthology.orchestra.run.smithi012.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7520-g43dd94c1-1jammy [1,900 kB] 2023-11-28T21:31:00.863 INFO:teuthology.orchestra.run.smithi012.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7520-g43dd94c1-1jammy [8,625 kB] 2023-11-28T21:31:00.873 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-28T21:31:00.893 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T21:31:00.931 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:31:01.335 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-28T21:31:01.355 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T21:31:01.393 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:31:01.469 INFO:teuthology.orchestra.run.smithi012.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7520-g43dd94c1-1jammy [14.2 kB] 2023-11-28T21:31:01.469 INFO:teuthology.orchestra.run.smithi012.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7520-g43dd94c1-1jammy [48.7 MB] 2023-11-28T21:31:01.788 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-28T21:31:01.808 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-28T21:31:01.846 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-28T21:31:02.224 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librados2. 2023-11-28T21:31:02.244 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../010-librados2_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:02.282 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librados2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:02.701 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librbd1. 2023-11-28T21:31:02.713 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../011-librbd1_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:02.785 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librbd1 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:03.265 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs2. 2023-11-28T21:31:03.285 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:03.348 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:03.667 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rados. 2023-11-28T21:31:03.688 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../013-python3-rados_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:03.725 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rados (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:04.095 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-28T21:31:04.115 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:04.153 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-argparse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:04.431 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cephfs. 2023-11-28T21:31:04.452 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:04.514 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cephfs (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:04.925 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-28T21:31:04.946 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:04.983 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-common (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:05.373 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-prettytable. 2023-11-28T21:31:05.394 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2023-11-28T21:31:05.445 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-28T21:31:05.582 INFO:teuthology.orchestra.run.smithi012.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7520-g43dd94c1-1jammy [121 kB] 2023-11-28T21:31:05.583 INFO:teuthology.orchestra.run.smithi012.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-7520-g43dd94c1-1jammy [225 kB] 2023-11-28T21:31:05.588 INFO:teuthology.orchestra.run.smithi012.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-7520-g43dd94c1-1jammy [34.0 kB] 2023-11-28T21:31:05.594 INFO:teuthology.orchestra.run.smithi012.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-7520-g43dd94c1-1jammy [23.4 kB] 2023-11-28T21:31:05.595 INFO:teuthology.orchestra.run.smithi012.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7520-g43dd94c1-1jammy [38.4 kB] 2023-11-28T21:31:05.597 INFO:teuthology.orchestra.run.smithi012.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7520-g43dd94c1-1jammy [12.3 MB] 2023-11-28T21:31:05.748 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rbd. 2023-11-28T21:31:05.769 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:05.806 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rbd (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-28T21:31:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-28T21:31:06.267 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-28T21:31:06.497 INFO:teuthology.orchestra.run.smithi012.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231127.102914/43dd94c190073cf16cff1e711dd7f5a7b1cbc883/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7520-g43dd94c1-1jammy [91.2 kB] 2023-11-28T21:31:06.879 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libreadline-dev:amd64. 2023-11-28T21:31:06.899 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2023-11-28T21:31:06.937 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2023-11-28T21:31:07.163 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 187 MB in 16s (11.5 MB/s) 2023-11-28T21:31:07.273 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2023-11-28T21:31:07.294 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2023-11-28T21:31:07.332 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-11-28T21:31:07.338 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-28T21:31:07.411 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-11-28T21:31:07.414 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-28T21:31:07.454 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-28T21:31:07.724 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-28T21:31:07.738 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-28T21:31:07.751 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua5.1. 2023-11-28T21:31:07.772 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2023-11-28T21:31:07.773 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-28T21:31:07.810 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2023-11-28T21:31:08.076 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cmd2. 2023-11-28T21:31:08.091 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-28T21:31:08.146 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua-any. 2023-11-28T21:31:08.151 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-28T21:31:08.167 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2023-11-28T21:31:08.205 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua-any (27ubuntu1) ... 2023-11-28T21:31:08.474 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package zip. 2023-11-28T21:31:08.486 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2023-11-28T21:31:08.524 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking zip (3.0-12build2) ... 2023-11-28T21:31:08.613 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-28T21:31:08.628 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-28T21:31:08.713 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-28T21:31:08.902 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package unzip. 2023-11-28T21:31:08.923 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.1_amd64.deb ... 2023-11-28T21:31:08.960 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking unzip (6.0-26ubuntu3.1) ... 2023-11-28T21:31:09.083 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-28T21:31:09.099 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-28T21:31:09.142 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-28T21:31:09.356 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package luarocks. 2023-11-28T21:31:09.377 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2023-11-28T21:31:09.414 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2023-11-28T21:31:09.478 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-28T21:31:09.493 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-28T21:31:09.536 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-28T21:31:09.865 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librgw2. 2023-11-28T21:31:09.885 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../027-librgw2_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:09.922 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librgw2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:09.957 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-28T21:31:09.972 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T21:31:10.023 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:31:10.410 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-28T21:31:10.425 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T21:31:10.468 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:31:10.483 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rgw. 2023-11-28T21:31:10.495 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../028-python3-rgw_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:10.533 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rgw (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:10.905 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-28T21:31:10.911 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-28T21:31:10.920 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-28T21:31:10.931 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-28T21:31:10.963 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:31:10.969 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-28T21:31:11.301 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-28T21:31:11.305 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libradosstriper1. 2023-11-28T21:31:11.314 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-28T21:31:11.326 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../030-libradosstriper1_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:11.358 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-28T21:31:11.363 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libradosstriper1 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:11.719 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librados2. 2023-11-28T21:31:11.734 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../010-librados2_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:11.777 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librados2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:11.867 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-common. 2023-11-28T21:31:11.887 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../031-ceph-common_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:11.925 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-common (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:12.246 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librbd1. 2023-11-28T21:31:12.262 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../011-librbd1_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:12.297 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librbd1 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:12.758 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs2. 2023-11-28T21:31:12.773 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:12.816 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:13.094 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rados. 2023-11-28T21:31:13.109 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../013-python3-rados_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:13.152 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rados (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:13.281 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-base. 2023-11-28T21:31:13.301 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../032-ceph-base_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:13.348 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-base (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:13.471 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-28T21:31:13.487 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:13.521 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-argparse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:13.757 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cephfs. 2023-11-28T21:31:13.772 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:13.816 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cephfs (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:14.085 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-28T21:31:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-28T21:31:14.126 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-28T21:31:14.142 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:14.167 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-28T21:31:14.210 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-common (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:14.505 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cheroot. 2023-11-28T21:31:14.513 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-prettytable. 2023-11-28T21:31:14.526 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-28T21:31:14.528 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2023-11-28T21:31:14.563 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-28T21:31:14.571 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-28T21:31:14.799 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rbd. 2023-11-28T21:31:14.814 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:14.849 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rbd (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:14.916 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-28T21:31:14.937 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-28T21:31:14.974 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-28T21:31:15.193 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-28T21:31:15.209 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-28T21:31:15.252 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-28T21:31:15.269 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-28T21:31:15.289 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-28T21:31:15.352 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-28T21:31:15.521 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libreadline-dev:amd64. 2023-11-28T21:31:15.537 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2023-11-28T21:31:15.579 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2023-11-28T21:31:15.671 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-28T21:31:15.692 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-28T21:31:15.738 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-28T21:31:15.866 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2023-11-28T21:31:15.881 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2023-11-28T21:31:15.949 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-11-28T21:31:16.066 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempora. 2023-11-28T21:31:16.087 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2023-11-28T21:31:16.124 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-28T21:31:16.228 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua5.1. 2023-11-28T21:31:16.243 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2023-11-28T21:31:16.286 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2023-11-28T21:31:16.436 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-portend. 2023-11-28T21:31:16.456 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2023-11-28T21:31:16.511 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-28T21:31:16.588 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-any. 2023-11-28T21:31:16.604 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2023-11-28T21:31:16.647 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-any (27ubuntu1) ... 2023-11-28T21:31:16.855 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-28T21:31:16.876 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-28T21:31:16.913 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-28T21:31:16.925 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package zip. 2023-11-28T21:31:16.940 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2023-11-28T21:31:16.983 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking zip (3.0-12build2) ... 2023-11-28T21:31:17.386 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package unzip. 2023-11-28T21:31:22.443 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.1_amd64.deb ... 2023-11-28T21:31:22.443 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking unzip (6.0-26ubuntu3.1) ... 2023-11-28T21:31:22.444 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package luarocks. 2023-11-28T21:31:22.444 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2023-11-28T21:31:22.444 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2023-11-28T21:31:22.444 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librgw2. 2023-11-28T21:31:22.444 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../027-librgw2_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:22.444 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librgw2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rgw. 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../028-python3-rgw_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rgw (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libradosstriper1. 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../030-libradosstriper1_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libradosstriper1 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-common. 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../031-ceph-common_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-common (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:22.445 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-base. 2023-11-28T21:31:22.470 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../032-ceph-base_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:22.470 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-base (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:22.470 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-28T21:31:22.470 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-28T21:31:22.470 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-28T21:31:22.470 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cheroot. 2023-11-28T21:31:22.470 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-natsort. 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-logutils. 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-28T21:31:22.472 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-mako. 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webob. 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-28T21:31:22.473 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-waitress. 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempita. 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-paste. 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-28T21:31:22.474 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-28T21:31:22.475 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-28T21:31:22.498 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-28T21:31:22.798 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webtest. 2023-11-28T21:31:22.818 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2023-11-28T21:31:22.859 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-28T21:31:22.875 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-28T21:31:22.889 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-28T21:31:22.917 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-28T21:31:23.229 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-28T21:31:23.244 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-28T21:31:23.287 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-28T21:31:24.116 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pecan. 2023-11-28T21:31:24.137 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-28T21:31:24.184 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-28T21:31:24.215 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-28T21:31:24.231 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-28T21:31:24.265 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-28T21:31:24.526 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempora. 2023-11-28T21:31:24.542 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2023-11-28T21:31:24.584 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-28T21:31:24.598 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-certifi. 2023-11-28T21:31:24.618 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2023-11-28T21:31:24.656 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-28T21:31:24.912 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-portend. 2023-11-28T21:31:24.928 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2023-11-28T21:31:24.970 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-28T21:31:25.042 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-idna. 2023-11-28T21:31:25.063 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2023-11-28T21:31:25.100 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-28T21:31:25.240 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-28T21:31:25.256 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-28T21:31:25.290 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-28T21:31:25.445 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-urllib3. 2023-11-28T21:31:25.466 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-11-28T21:31:25.495 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-28T21:31:25.634 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-28T21:31:25.650 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-28T21:31:25.693 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-28T21:31:25.823 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-requests. 2023-11-28T21:31:25.844 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-28T21:31:25.915 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-28T21:31:25.996 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-natsort. 2023-11-28T21:31:26.011 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2023-11-28T21:31:26.046 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-28T21:31:26.234 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-28T21:31:26.255 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-28T21:31:26.293 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-28T21:31:26.340 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-logutils. 2023-11-28T21:31:26.356 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2023-11-28T21:31:26.399 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-28T21:31:26.604 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-28T21:31:26.616 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:26.654 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:26.668 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-28T21:31:26.684 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-28T21:31:26.718 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-28T21:31:26.996 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-mako. 2023-11-28T21:31:27.013 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-28T21:31:27.054 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-28T21:31:27.117 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-28T21:31:27.141 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:27.184 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:27.424 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-28T21:31:27.440 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-28T21:31:27.482 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-28T21:31:27.778 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-28T21:31:27.793 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-28T21:31:27.820 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-28T21:31:27.835 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-28T21:31:27.841 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-28T21:31:27.887 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-28T21:31:28.121 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webob. 2023-11-28T21:31:28.137 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-28T21:31:28.165 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-openssl. 2023-11-28T21:31:28.186 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2023-11-28T21:31:28.213 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-28T21:31:28.214 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-28T21:31:28.541 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-waitress. 2023-11-28T21:31:28.556 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-28T21:31:28.602 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-28T21:31:28.626 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr. 2023-11-28T21:31:28.647 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../065-ceph-mgr_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:28.692 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:28.919 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempita. 2023-11-28T21:31:28.934 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-28T21:31:28.977 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-28T21:31:29.179 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mon. 2023-11-28T21:31:29.200 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../066-ceph-mon_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:29.283 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mon (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:29.296 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-paste. 2023-11-28T21:31:29.312 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-28T21:31:29.354 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-28T21:31:29.658 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-28T21:31:29.673 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-28T21:31:29.715 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-28T21:31:30.052 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-28T21:31:30.067 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-28T21:31:30.110 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-28T21:31:30.258 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-osd. 2023-11-28T21:31:30.279 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../067-ceph-osd_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:30.315 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-osd (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:30.413 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webtest. 2023-11-28T21:31:30.429 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2023-11-28T21:31:30.471 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-28T21:31:30.807 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pecan. 2023-11-28T21:31:30.823 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-28T21:31:30.865 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-28T21:31:31.221 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-certifi. 2023-11-28T21:31:31.237 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2023-11-28T21:31:31.279 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-28T21:31:31.582 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-idna. 2023-11-28T21:31:31.597 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2023-11-28T21:31:31.677 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-28T21:31:32.001 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-urllib3. 2023-11-28T21:31:32.017 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-11-28T21:31:32.059 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-28T21:31:32.252 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph. 2023-11-28T21:31:32.264 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../068-ceph_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:32.293 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:32.388 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests. 2023-11-28T21:31:32.403 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-28T21:31:32.445 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-28T21:31:32.681 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-fuse. 2023-11-28T21:31:32.701 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../069-ceph-fuse_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:32.738 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-fuse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:32.948 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-28T21:31:32.964 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-28T21:31:33.007 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-28T21:31:33.100 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mds. 2023-11-28T21:31:33.121 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../070-ceph-mds_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:33.158 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mds (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:33.293 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-28T21:31:33.309 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:33.352 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:33.586 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephadm. 2023-11-28T21:31:33.607 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../071-cephadm_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:33.652 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephadm (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:33.748 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-28T21:31:33.763 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:33.806 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:34.014 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-28T21:31:34.034 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-28T21:31:34.108 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-28T21:31:34.109 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-28T21:31:34.125 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-28T21:31:34.175 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-28T21:31:34.442 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-28T21:31:34.462 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-28T21:31:34.491 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-28T21:31:34.495 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-openssl. 2023-11-28T21:31:34.510 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2023-11-28T21:31:34.545 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-28T21:31:34.865 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr. 2023-11-28T21:31:34.880 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../065-ceph-mgr_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:34.923 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:35.343 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mon. 2023-11-28T21:31:35.358 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../066-ceph-mon_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:35.401 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mon (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:35.663 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-babel. 2023-11-28T21:31:35.685 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-28T21:31:35.721 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-28T21:31:36.083 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jinja2. 2023-11-28T21:31:36.105 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1_all.deb ... 2023-11-28T21:31:36.158 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-28T21:31:36.221 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-osd. 2023-11-28T21:31:36.237 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../067-ceph-osd_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:36.274 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-osd (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:36.528 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-28T21:31:36.549 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:36.594 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:37.023 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-28T21:31:37.045 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2023-11-28T21:31:37.090 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-28T21:31:37.393 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-routes. 2023-11-28T21:31:37.414 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-28T21:31:37.459 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-28T21:31:37.465 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph. 2023-11-28T21:31:37.481 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../068-ceph_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:37.524 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:37.837 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-28T21:31:37.859 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:37.868 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-fuse. 2023-11-28T21:31:37.884 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../069-ceph-fuse_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:37.926 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-fuse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:37.929 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:38.296 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mds. 2023-11-28T21:31:38.311 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../070-ceph-mds_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:38.354 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mds (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:38.582 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-28T21:31:38.595 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-28T21:31:38.646 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-28T21:31:38.748 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephadm. 2023-11-28T21:31:38.764 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../071-cephadm_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:38.807 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephadm (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:39.093 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-28T21:31:39.109 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-28T21:31:39.143 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-28T21:31:39.161 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-joblib. 2023-11-28T21:31:39.182 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-28T21:31:39.261 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-28T21:31:39.556 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-28T21:31:39.572 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-28T21:31:39.620 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-28T21:31:39.665 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-28T21:31:39.686 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-28T21:31:39.858 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-28T21:31:40.211 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn. 2023-11-28T21:31:40.232 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-28T21:31:40.269 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-28T21:31:40.584 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-babel. 2023-11-28T21:31:40.598 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-28T21:31:40.634 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-28T21:31:40.936 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jinja2. 2023-11-28T21:31:40.951 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1_all.deb ... 2023-11-28T21:31:40.995 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-28T21:31:41.120 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-28T21:31:41.142 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:41.186 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:41.448 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-28T21:31:41.462 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:41.506 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:41.818 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-28T21:31:41.833 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2023-11-28T21:31:41.876 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-28T21:31:42.198 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-routes. 2023-11-28T21:31:42.211 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-28T21:31:42.223 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cachetools. 2023-11-28T21:31:42.244 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2023-11-28T21:31:42.253 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-28T21:31:42.298 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-28T21:31:42.573 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-28T21:31:42.589 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:42.631 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:42.867 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-28T21:31:42.867 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-28T21:31:42.867 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-28T21:31:43.079 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-28T21:31:43.100 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-28T21:31:43.170 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-28T21:31:43.276 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-28T21:31:43.292 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-28T21:31:43.334 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-28T21:31:43.515 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rsa. 2023-11-28T21:31:43.537 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2023-11-28T21:31:43.582 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-28T21:31:43.830 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-joblib. 2023-11-28T21:31:43.845 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-28T21:31:43.921 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-28T21:31:43.927 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-google-auth. 2023-11-28T21:31:43.949 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2023-11-28T21:31:44.027 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-28T21:31:44.266 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-28T21:31:44.281 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-28T21:31:44.380 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-28T21:31:44.401 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-28T21:31:44.402 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-28T21:31:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-28T21:31:44.779 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn. 2023-11-28T21:31:44.795 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-28T21:31:44.837 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-28T21:31:44.927 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-websocket. 2023-11-28T21:31:44.948 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2023-11-28T21:31:44.977 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-28T21:31:45.347 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-28T21:31:45.368 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-28T21:31:45.413 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-28T21:31:45.530 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-28T21:31:45.545 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:45.622 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:46.161 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-28T21:31:46.183 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:46.227 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:46.534 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cachetools. 2023-11-28T21:31:46.550 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2023-11-28T21:31:46.580 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-28T21:31:46.583 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-28T21:31:46.602 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-28T21:31:46.646 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-28T21:31:46.869 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-28T21:31:46.886 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-28T21:31:46.927 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-28T21:31:47.017 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-28T21:31:47.038 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-28T21:31:47.066 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-28T21:31:47.248 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-28T21:31:47.264 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-28T21:31:47.306 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-28T21:31:47.361 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package jq. 2023-11-28T21:31:47.382 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-28T21:31:47.427 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-28T21:31:47.633 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rsa. 2023-11-28T21:31:47.650 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2023-11-28T21:31:47.692 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-28T21:31:47.722 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package socat. 2023-11-28T21:31:47.743 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-28T21:31:47.780 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-28T21:31:47.996 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-google-auth. 2023-11-28T21:31:48.013 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2023-11-28T21:31:48.054 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-28T21:31:48.134 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package xmlstarlet. 2023-11-28T21:31:48.155 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-28T21:31:48.192 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-28T21:31:48.383 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-28T21:31:48.399 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-28T21:31:48.432 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-28T21:31:48.613 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-test. 2023-11-28T21:31:48.634 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../099-ceph-test_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:48.712 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-test (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:48.743 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-websocket. 2023-11-28T21:31:48.762 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2023-11-28T21:31:48.801 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-28T21:31:49.130 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-28T21:31:49.146 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-28T21:31:49.189 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-28T21:31:49.893 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-28T21:31:49.908 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:49.968 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:50.262 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-28T21:31:50.277 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-28T21:31:50.320 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-28T21:31:50.657 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-28T21:31:50.672 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-28T21:31:50.715 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-28T21:31:51.018 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package jq. 2023-11-28T21:31:51.034 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-28T21:31:51.076 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-28T21:31:51.117 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-volume. 2023-11-28T21:31:51.138 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../100-ceph-volume_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:51.175 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-volume (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:51.287 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package socat. 2023-11-28T21:31:51.303 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-28T21:31:51.337 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-28T21:31:51.599 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package xmlstarlet. 2023-11-28T21:31:51.612 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-28T21:31:51.615 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-28T21:31:51.634 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../101-cephfs-mirror_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:51.679 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephfs-mirror (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:51.690 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-28T21:31:52.028 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-test. 2023-11-28T21:31:52.032 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-colorama. 2023-11-28T21:31:52.043 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../099-ceph-test_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:52.053 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2023-11-28T21:31:52.077 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-test (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:52.132 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-28T21:31:52.418 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephfs-shell. 2023-11-28T21:31:52.440 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../103-cephfs-shell_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:52.477 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephfs-shell (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:52.779 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephfs-top. 2023-11-28T21:31:52.791 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../104-cephfs-top_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:52.829 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephfs-top (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:53.115 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-28T21:31:53.136 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../105-libcephfs-dev_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:53.181 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs-dev (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:53.835 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua-socket:amd64. 2023-11-28T21:31:53.856 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2023-11-28T21:31:53.934 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-11-28T21:31:54.073 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-volume. 2023-11-28T21:31:54.089 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../100-ceph-volume_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:54.131 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-volume (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:54.306 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package lua-sec:amd64. 2023-11-28T21:31:54.327 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2023-11-28T21:31:54.372 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2023-11-28T21:31:54.494 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-28T21:31:54.510 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../101-cephfs-mirror_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:54.552 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephfs-mirror (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:54.735 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package nvme-cli. 2023-11-28T21:31:54.757 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-28T21:31:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-28T21:31:54.888 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-colorama. 2023-11-28T21:31:54.905 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2023-11-28T21:31:54.946 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-28T21:31:55.207 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephfs-shell. 2023-11-28T21:31:55.223 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../103-cephfs-shell_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:55.265 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephfs-shell (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:55.265 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package pkg-config. 2023-11-28T21:31:55.287 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2023-11-28T21:31:55.357 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2023-11-28T21:31:55.543 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephfs-top. 2023-11-28T21:31:55.559 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../104-cephfs-top_18.0.0-7520-g43dd94c1-1jammy_all.deb ... 2023-11-28T21:31:55.601 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephfs-top (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:55.644 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-28T21:31:55.665 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-28T21:31:55.702 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-28T21:31:55.795 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-28T21:31:55.811 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../105-libcephfs-dev_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:31:55.853 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs-dev (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:31:56.164 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-28T21:31:56.164 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-socket:amd64. 2023-11-28T21:31:56.180 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2023-11-28T21:31:56.185 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-28T21:31:56.222 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-11-28T21:31:56.255 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-28T21:31:56.585 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-sec:amd64. 2023-11-28T21:31:56.600 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastescript. 2023-11-28T21:31:56.601 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2023-11-28T21:31:56.621 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2023-11-28T21:31:56.643 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2023-11-28T21:31:56.666 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-28T21:31:56.981 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package nvme-cli. 2023-11-28T21:31:56.997 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-28T21:31:57.003 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pluggy. 2023-11-28T21:31:57.025 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-28T21:31:57.039 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-28T21:31:57.061 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-28T21:31:57.398 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-psutil. 2023-11-28T21:31:57.419 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-28T21:31:57.454 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package pkg-config. 2023-11-28T21:31:57.470 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2023-11-28T21:31:57.473 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-28T21:31:57.512 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2023-11-28T21:31:57.765 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-28T21:31:57.782 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-28T21:31:57.824 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-28T21:31:57.851 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-py. 2023-11-28T21:31:57.872 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2023-11-28T21:31:57.917 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-28T21:31:58.277 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-28T21:31:58.279 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pygments. 2023-11-28T21:31:58.293 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-28T21:31:58.301 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-28T21:31:58.334 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-28T21:31:58.370 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-28T21:31:58.638 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastescript. 2023-11-28T21:31:58.654 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2023-11-28T21:31:58.696 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-28T21:31:58.917 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-28T21:31:58.939 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-28T21:31:58.983 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-28T21:31:59.016 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pluggy. 2023-11-28T21:31:59.032 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-28T21:31:59.074 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-28T21:31:59.319 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-toml. 2023-11-28T21:31:59.341 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2023-11-28T21:31:59.377 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-28T21:31:59.385 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-psutil. 2023-11-28T21:31:59.401 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-28T21:31:59.452 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-28T21:31:59.688 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pytest. 2023-11-28T21:31:59.710 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-28T21:31:59.771 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-py. 2023-11-28T21:31:59.788 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2023-11-28T21:31:59.796 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-28T21:31:59.854 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-28T21:32:00.159 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplejson. 2023-11-28T21:32:00.181 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-28T21:32:00.191 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pygments. 2023-11-28T21:32:00.207 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-28T21:32:00.217 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-28T21:32:00.249 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-28T21:32:00.494 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-28T21:32:00.507 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-28T21:32:00.536 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-28T21:32:00.720 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-28T21:32:00.736 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-28T21:32:00.778 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-28T21:32:01.090 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-toml. 2023-11-28T21:32:01.105 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2023-11-28T21:32:01.147 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-28T21:32:01.415 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package radosgw. 2023-11-28T21:32:01.416 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pytest. 2023-11-28T21:32:01.433 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-28T21:32:01.437 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../122-radosgw_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:32:01.474 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking radosgw (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:01.474 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-28T21:32:01.853 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplejson. 2023-11-28T21:32:01.869 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-28T21:32:01.912 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-28T21:32:02.189 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-28T21:32:02.206 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-28T21:32:02.247 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-28T21:32:02.278 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package rbd-fuse. 2023-11-28T21:32:02.289 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../123-rbd-fuse_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:32:02.327 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking rbd-fuse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:02.663 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-28T21:32:02.982 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-28T21:32:03.043 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package radosgw. 2023-11-28T21:32:03.058 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../122-radosgw_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:32:03.100 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-28T21:32:03.101 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking radosgw (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:03.394 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-28T21:32:03.704 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package rbd-fuse. 2023-11-28T21:32:03.719 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../123-rbd-fuse_18.0.0-7520-g43dd94c1-1jammy_amd64.deb ... 2023-11-28T21:32:03.762 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking rbd-fuse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:03.768 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-28T21:32:04.040 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-28T21:32:04.265 INFO:teuthology.orchestra.run.smithi144.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-28T21:32:04.309 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-28T21:32:04.360 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephadm (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:04.418 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-28T21:32:04.689 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-28T21:32:04.992 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user cephadm....done 2023-11-28T21:32:05.088 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-28T21:32:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-28T21:32:05.459 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-28T21:32:05.721 INFO:teuthology.orchestra.run.smithi012.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-28T21:32:05.735 INFO:teuthology.orchestra.run.smithi012.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-11-28T21:32:05.744 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-28T21:32:05.820 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephadm (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:05.903 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-28T21:32:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-28T21:32:06.464 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-28T21:32:06.535 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user cephadm....done 2023-11-28T21:32:06.582 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-28T21:32:06.892 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-28T21:32:07.028 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-28T21:32:07.290 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-28T21:32:07.429 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-28T21:32:07.612 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-28T21:32:07.697 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-28T21:32:07.848 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-28T21:32:07.888 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-28T21:32:08.115 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-28T21:32:08.181 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up unzip (6.0-26ubuntu3.1) ... 2023-11-28T21:32:08.383 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-28T21:32:08.484 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-28T21:32:08.522 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-28T21:32:08.825 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-28T21:32:08.978 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-28T21:32:09.085 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-28T21:32:09.363 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-28T21:32:09.378 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-argparse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:09.631 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-28T21:32:09.662 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-28T21:32:09.788 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-11-28T21:32:09.899 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-28T21:32:09.939 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-28T21:32:10.192 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up unzip (6.0-26ubuntu3.1) ... 2023-11-28T21:32:10.249 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2023-11-28T21:32:10.375 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-28T21:32:10.452 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-28T21:32:10.662 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2023-11-28T21:32:10.720 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-28T21:32:10.996 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-28T21:32:11.024 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2023-11-28T21:32:11.107 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2023-11-28T21:32:11.163 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-28T21:32:11.271 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-argparse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:11.272 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-28T21:32:11.548 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-28T21:32:11.665 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-28T21:32:11.673 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-11-28T21:32:11.791 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-28T21:32:11.799 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-28T21:32:12.068 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-28T21:32:12.084 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2023-11-28T21:32:12.218 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-28T21:32:12.335 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-28T21:32:12.486 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2023-11-28T21:32:12.653 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up zip (3.0-12build2) ... 2023-11-28T21:32:12.813 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-28T21:32:12.861 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2023-11-28T21:32:12.892 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2023-11-28T21:32:12.972 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-28T21:32:13.098 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-28T21:32:13.483 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-28T21:32:13.506 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-28T21:32:13.584 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-28T21:32:13.757 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-28T21:32:13.877 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-28T21:32:13.884 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-28T21:32:14.043 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-28T21:32:14.153 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-28T21:32:14.336 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-28T21:32:14.471 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up zip (3.0-12build2) ... 2023-11-28T21:32:14.597 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-28T21:32:14.729 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-28T21:32:15.038 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-28T21:32:15.348 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-28T21:32:15.465 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-28T21:32:15.625 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-28T21:32:15.750 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-28T21:32:15.750 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-28T21:32:15.868 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-28T21:32:16.068 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-28T21:32:16.186 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-28T21:32:16.419 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-28T21:32:16.596 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-28T21:32:16.687 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-28T21:32:16.847 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-common (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:16.855 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-28T21:32:17.148 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-idna (3.3-1) ... 2023-11-28T21:32:17.290 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-28T21:32:17.466 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-28T21:32:17.591 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-28T21:32:17.901 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-28T21:32:18.261 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-28T21:32:18.528 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-28T21:32:18.597 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2023-11-28T21:32:18.654 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-common (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:18.874 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:32:18.999 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-idna (3.3-1) ... 2023-11-28T21:32:19.009 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-28T21:32:19.276 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-28T21:32:19.365 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-28T21:32:19.403 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-28T21:32:19.562 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-28T21:32:19.988 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-28T21:32:20.281 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-28T21:32:20.616 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:32:20.637 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2023-11-28T21:32:20.742 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-28T21:32:20.914 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:32:21.040 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-28T21:32:21.093 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-28T21:32:21.316 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-28T21:32:21.378 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-28T21:32:21.450 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-28T21:32:21.594 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-28T21:32:21.679 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-28T21:32:21.894 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-28T21:32:21.964 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-28T21:32:22.171 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-28T21:32:22.265 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-28T21:32:22.463 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:32:22.566 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-28T21:32:22.589 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-28T21:32:22.725 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-28T21:32:22.907 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-28T21:32:23.010 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-28T21:32:23.136 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-28T21:32:23.185 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-28T21:32:23.460 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-28T21:32:23.481 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-28T21:32:23.727 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-28T21:32:23.797 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-28T21:32:23.995 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-28T21:32:24.073 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-11-28T21:32:24.199 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2023-11-28T21:32:24.279 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-28T21:32:24.359 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-28T21:32:24.405 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-28T21:32:24.447 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-28T21:32:24.698 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-28T21:32:24.719 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-28T21:32:24.824 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-28T21:32:24.845 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-28T21:32:25.134 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-28T21:32:25.263 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-28T21:32:25.451 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-28T21:32:25.531 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up lua-any (27ubuntu1) ... 2023-11-28T21:32:25.666 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-28T21:32:25.702 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-11-28T21:32:25.828 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2023-11-28T21:32:25.917 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:32:25.988 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-28T21:32:26.068 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-28T21:32:26.076 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-28T21:32:26.323 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-28T21:32:26.428 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-28T21:32:26.449 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-28T21:32:26.737 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-28T21:32:26.867 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-28T21:32:26.997 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-28T21:32:27.126 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-any (27ubuntu1) ... 2023-11-28T21:32:27.227 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-28T21:32:27.259 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-28T21:32:27.495 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-28T21:32:27.551 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-28T21:32:27.646 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-28T21:32:27.836 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-28T21:32:27.980 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-28T21:32:28.112 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-28T21:32:28.298 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-28T21:32:28.473 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-28T21:32:28.591 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-28T21:32:28.783 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-28T21:32:28.794 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-28T21:32:29.079 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-28T21:32:29.127 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-28T21:32:29.286 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librados2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:29.397 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-28T21:32:29.412 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:29.572 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2023-11-28T21:32:29.673 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-28T21:32:29.925 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:30.067 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-28T21:32:30.067 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libradosstriper1 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:30.235 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-28T21:32:30.368 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-28T21:32:30.520 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librbd1 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:30.688 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-modules-core (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:30.712 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-28T21:32:30.838 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librados2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:30.839 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-fuse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:30.980 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:31.081 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2023-11-28T21:32:31.091 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-28T21:32:31.091 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-28T21:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:31.552 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libradosstriper1 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:31.632 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs-dev (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:31.678 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-28T21:32:31.759 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rados (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:31.910 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librgw2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:32.005 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librbd1 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:32.036 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephfs-top (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:32.114 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-modules-core (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:32.195 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-28T21:32:32.223 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-fuse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-28T21:32:32.441 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-28T21:32:33.100 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs-dev (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:33.218 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rados (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:33.326 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rbd (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:33.327 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librgw2 (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:33.453 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephfs-top (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:33.469 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up rbd-fuse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:33.579 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-28T21:32:33.645 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rgw (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:33.779 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cephfs (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:33.938 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephfs-shell (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:34.064 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-common (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:34.683 INFO:teuthology.orchestra.run.smithi144.stdout:Adding group ceph....done 2023-11-28T21:32:34.916 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rbd (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:35.034 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up rbd-fuse (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:35.135 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rgw (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:35.234 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user ceph....done 2023-11-28T21:32:35.256 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cephfs (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:35.378 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephfs-shell (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:35.470 INFO:teuthology.orchestra.run.smithi144.stdout:Setting system user ceph properties....done 2023-11-28T21:32:35.483 INFO:teuthology.orchestra.run.smithi144.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-28T21:32:35.596 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-common (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:35.622 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-28T21:32:35.970 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-28T21:32:36.159 INFO:teuthology.orchestra.run.smithi012.stdout:Adding group ceph....done 2023-11-28T21:32:36.491 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephfs-mirror (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:36.545 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user ceph....done 2023-11-28T21:32:36.702 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-28T21:32:36.702 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-28T21:32:36.729 INFO:teuthology.orchestra.run.smithi012.stdout:Setting system user ceph properties....done 2023-11-28T21:32:36.751 INFO:teuthology.orchestra.run.smithi012.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-28T21:32:36.904 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-28T21:32:37.209 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-test (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:37.300 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-28T21:32:37.377 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up radosgw (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:37.865 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-28T21:32:37.865 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-28T21:32:37.972 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephfs-mirror (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:38.213 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-28T21:32:38.213 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-28T21:32:38.387 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-base (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:38.810 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-28T21:32:38.823 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-test (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:39.008 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up radosgw (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:39.322 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mds (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:39.554 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-28T21:32:39.554 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-28T21:32:39.592 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-28T21:32:39.592 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-28T21:32:40.107 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:40.230 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-base (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:40.472 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-28T21:32:40.473 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-28T21:32:40.603 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-28T21:32:40.993 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-osd (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:41.249 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mds (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:41.344 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-28T21:32:41.345 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-28T21:32:41.505 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-28T21:32:41.505 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-28T21:32:41.903 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:42.054 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:42.142 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:42.205 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mon (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:42.346 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-28T21:32:42.346 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-28T21:32:42.442 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-28T21:32:42.442 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-28T21:32:42.906 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-cephadm (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:42.968 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-osd (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:43.041 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:43.209 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-dashboard (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:43.302 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-28T21:32:43.303 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-28T21:32:43.360 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-volume (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:43.746 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2023-11-28T21:32:43.865 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-28T21:32:43.920 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:44.046 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:44.066 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-28T21:32:44.222 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mon (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:44.441 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-28T21:32:44.441 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-28T21:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-cephadm (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:45.208 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:45.250 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-28T21:32:45.334 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-dashboard (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-volume (18.0.0-7520-g43dd94c1-1jammy) ... 2023-11-28T21:32:45.854 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2023-11-28T21:32:45.963 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-28T21:32:46.104 INFO:teuthology.orchestra.run.smithi144.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-28T21:32:46.108 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-28T21:32:46.124 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-28T21:32:46.162 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2023-11-28T21:32:46.356 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2023-11-28T21:32:46.357 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2023-11-28T21:32:46.506 INFO:teuthology.orchestra.run.smithi144.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-28T21:32:46.506 INFO:teuthology.orchestra.run.smithi144.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-28T21:32:46.507 INFO:teuthology.orchestra.run.smithi144.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-28T21:32:46.507 INFO:teuthology.orchestra.run.smithi144.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-28T21:32:46.507 INFO:teuthology.orchestra.run.smithi144.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-28T21:32:46.507 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages were automatically installed and are no longer required: 2023-11-28T21:32:46.507 INFO:teuthology.orchestra.run.smithi144.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-28T21:32:46.507 INFO:teuthology.orchestra.run.smithi144.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-28T21:32:46.508 INFO:teuthology.orchestra.run.smithi144.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-28T21:32:46.508 INFO:teuthology.orchestra.run.smithi144.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-11-28T21:32:46.508 INFO:teuthology.orchestra.run.smithi144.stdout: python2.7-minimal 2023-11-28T21:32:46.508 INFO:teuthology.orchestra.run.smithi144.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-28T21:32:46.508 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2023-11-28T21:32:46.508 INFO:teuthology.orchestra.run.smithi144.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-28T21:32:46.508 INFO:teuthology.orchestra.run.smithi144.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-28T21:32:46.509 INFO:teuthology.orchestra.run.smithi144.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-28T21:32:46.509 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-28T21:32:46.509 INFO:teuthology.orchestra.run.smithi144.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-28T21:32:46.509 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2023-11-28T21:32:46.510 INFO:teuthology.orchestra.run.smithi144.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-28T21:32:46.510 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-28T21:32:46.510 INFO:teuthology.orchestra.run.smithi144.stdout: libengine-pkcs11-openssl 2023-11-28T21:32:46.545 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2023-11-28T21:32:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-28T21:32:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-28T21:32:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-28T21:32:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-28T21:32:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-28T21:32:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-28T21:32:46.547 INFO:teuthology.orchestra.run.smithi144.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-28T21:32:46.548 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages will be upgraded: 2023-11-28T21:32:46.548 INFO:teuthology.orchestra.run.smithi144.stdout: libssl3 2023-11-28T21:32:46.728 INFO:teuthology.orchestra.run.smithi144.stdout:1 upgraded, 39 newly installed, 0 to remove and 263 not upgraded. 2023-11-28T21:32:46.728 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 54.6 MB of archives. 2023-11-28T21:32:46.728 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-28T21:32:46.728 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-28T21:32:47.216 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-28T21:32:47.249 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-28T21:32:47.473 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-28T21:32:47.508 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-28T21:32:47.510 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-28T21:32:47.515 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-28T21:32:47.518 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-28T21:32:47.519 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-28T21:32:47.522 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-28T21:32:47.524 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-28T21:32:47.525 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-28T21:32:47.689 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-28T21:32:47.944 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-28T21:32:47.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-28T21:32:48.002 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-28T21:32:48.026 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2023-11-28T21:32:48.196 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2023-11-28T21:32:48.196 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2023-11-28T21:32:48.351 INFO:teuthology.orchestra.run.smithi012.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-28T21:32:48.351 INFO:teuthology.orchestra.run.smithi012.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-28T21:32:48.352 INFO:teuthology.orchestra.run.smithi012.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-28T21:32:48.352 INFO:teuthology.orchestra.run.smithi012.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-28T21:32:48.352 INFO:teuthology.orchestra.run.smithi012.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-28T21:32:48.352 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2023-11-28T21:32:48.352 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-28T21:32:48.352 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-28T21:32:48.352 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-28T21:32:48.352 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-28T21:32:48.353 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-28T21:32:48.353 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-28T21:32:48.353 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2023-11-28T21:32:48.353 INFO:teuthology.orchestra.run.smithi012.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-28T21:32:48.353 INFO:teuthology.orchestra.run.smithi012.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-28T21:32:48.354 INFO:teuthology.orchestra.run.smithi012.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-28T21:32:48.354 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-28T21:32:48.354 INFO:teuthology.orchestra.run.smithi012.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-28T21:32:48.354 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2023-11-28T21:32:48.354 INFO:teuthology.orchestra.run.smithi012.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-28T21:32:48.355 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-28T21:32:48.355 INFO:teuthology.orchestra.run.smithi012.stdout: libengine-pkcs11-openssl 2023-11-28T21:32:48.406 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2023-11-28T21:32:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-28T21:32:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-28T21:32:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-28T21:32:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-28T21:32:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-28T21:32:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-28T21:32:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-28T21:32:48.408 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2023-11-28T21:32:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: libssl3 2023-11-28T21:32:48.466 INFO:teuthology.orchestra.run.smithi012.stdout:1 upgraded, 39 newly installed, 0 to remove and 263 not upgraded. 2023-11-28T21:32:48.466 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 54.6 MB of archives. 2023-11-28T21:32:48.466 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-28T21:32:48.466 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-28T21:32:48.568 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-28T21:32:48.623 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-28T21:32:48.631 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-28T21:32:48.632 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-28T21:32:48.634 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-28T21:32:48.635 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-28T21:32:48.635 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-28T21:32:48.636 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-28T21:32:48.637 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-28T21:32:48.638 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-28T21:32:48.642 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-28T21:32:48.656 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-28T21:32:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-28T21:32:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-28T21:32:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-28T21:32:48.901 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-28T21:32:48.901 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-28T21:32:48.906 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-28T21:32:48.907 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-28T21:32:48.915 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-28T21:32:48.915 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-28T21:32:48.940 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-28T21:32:48.941 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-28T21:32:48.952 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-28T21:32:48.975 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-28T21:32:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-28T21:32:48.981 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-28T21:32:48.982 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-28T21:32:48.983 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-28T21:32:48.995 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-28T21:32:48.997 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-28T21:32:49.162 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-28T21:32:49.162 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-28T21:32:49.162 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-28T21:32:49.162 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-28T21:32:49.163 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-28T21:32:49.163 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-28T21:32:49.176 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-28T21:32:49.176 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-28T21:32:49.339 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-28T21:32:49.572 INFO:teuthology.orchestra.run.smithi012.stdout:Preconfiguring packages ... 2023-11-28T21:32:49.715 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 54.6 MB in 1s (71.7 MB/s) 2023-11-28T21:32:49.766 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-28T21:32:49.849 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-28T21:32:49.856 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package flex. 2023-11-28T21:32:49.931 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140565 files and directories currently installed.) 2023-11-28T21:32:49.934 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-28T21:32:49.974 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-28T21:32:50.028 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-28T21:32:50.049 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-28T21:32:50.160 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-28T21:32:50.184 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-28T21:32:50.229 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-28T21:32:50.249 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-28T21:32:50.320 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-28T21:32:50.339 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-28T21:32:50.384 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-28T21:32:50.449 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-28T21:32:50.476 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-28T21:32:50.497 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-28T21:32:50.518 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-28T21:32:50.533 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-28T21:32:50.540 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-28T21:32:50.586 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-28T21:32:50.610 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-28T21:32:50.710 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-28T21:32:50.891 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-28T21:32:50.912 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-28T21:32:50.934 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-28T21:32:50.956 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-28T21:32:50.977 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-28T21:32:51.058 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-28T21:32:51.130 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-28T21:32:51.136 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T21:32:51.211 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-28T21:32:51.615 INFO:teuthology.orchestra.run.smithi144.stdout:Preconfiguring packages ... 2023-11-28T21:32:51.686 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package bison. 2023-11-28T21:32:51.761 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140638 files and directories currently installed.) 2023-11-28T21:32:51.765 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-28T21:32:51.813 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-28T21:32:51.816 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 54.6 MB in 5s (11.7 MB/s) 2023-11-28T21:32:51.968 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package flex. 2023-11-28T21:32:52.055 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138139 files and directories currently installed.) 2023-11-28T21:32:52.061 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-28T21:32:52.108 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-28T21:32:52.239 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-28T21:32:52.253 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-28T21:32:52.297 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-28T21:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package dump. 2023-11-28T21:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-28T21:32:52.684 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking dump (0.4b47-3) ... 2023-11-28T21:32:52.760 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-28T21:32:52.980 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-28T21:32:52.997 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package indent. 2023-11-28T21:32:53.010 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:32:53.045 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-28T21:32:53.264 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-28T21:32:53.280 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-28T21:32:53.322 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-28T21:32:53.456 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T21:32:53.678 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-28T21:32:53.694 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-28T21:32:53.736 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-28T21:32:53.997 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-28T21:32:53.998 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package bison. 2023-11-28T21:32:54.013 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-28T21:32:54.055 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-28T21:32:54.082 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138212 files and directories currently installed.) 2023-11-28T21:32:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-28T21:32:54.167 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-28T21:32:54.384 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-28T21:32:54.400 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-28T21:32:54.442 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-28T21:32:54.576 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-28T21:32:54.598 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-28T21:32:54.643 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-28T21:32:54.705 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-28T21:32:54.719 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-28T21:32:54.761 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-28T21:32:55.039 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjson-perl. 2023-11-28T21:32:55.054 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package dump. 2023-11-28T21:32:55.055 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-28T21:32:55.076 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-28T21:32:55.097 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-28T21:32:55.121 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking dump (0.4b47-3) ... 2023-11-28T21:32:55.409 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-28T21:32:55.425 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-28T21:32:55.450 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package indent. 2023-11-28T21:32:55.459 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-28T21:32:55.462 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:32:55.500 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-28T21:32:55.810 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-28T21:32:55.832 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-28T21:32:55.902 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-28T21:32:56.264 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-28T21:32:56.286 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-28T21:32:56.356 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-28T21:32:56.651 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-28T21:32:56.663 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-28T21:32:56.701 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-28T21:32:56.836 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-28T21:32:56.853 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-28T21:32:56.894 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T21:32:57.104 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-28T21:32:57.126 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-28T21:32:57.139 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-28T21:32:57.155 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-28T21:32:57.171 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-28T21:32:57.197 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T21:32:57.465 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-28T21:32:57.487 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-28T21:32:57.531 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-28T21:32:57.533 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-28T21:32:57.550 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-28T21:32:57.591 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T21:32:57.793 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libjson-perl. 2023-11-28T21:32:57.815 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-28T21:32:57.851 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-28T21:32:57.919 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-28T21:32:57.935 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-28T21:32:57.977 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-28T21:32:58.254 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-28T21:32:58.276 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-28T21:32:58.321 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-28T21:32:58.338 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-28T21:32:58.355 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-28T21:32:58.397 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-28T21:32:58.750 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-28T21:32:58.766 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:32:58.810 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:32:59.144 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-28T21:32:59.161 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:32:59.202 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T21:32:59.497 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-28T21:32:59.513 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:32:59.555 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T21:32:59.790 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-28T21:32:59.806 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-28T21:32:59.849 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T21:33:00.118 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-28T21:33:00.137 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-28T21:33:00.215 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T21:33:00.361 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-28T21:33:00.377 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-28T21:33:00.419 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-28T21:33:00.510 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-28T21:33:00.532 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-28T21:33:00.576 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T21:33:00.747 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package modemmanager. 2023-11-28T21:33:00.763 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-28T21:33:00.805 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T21:33:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-28T21:33:00.942 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-28T21:33:00.987 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T21:33:01.428 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package network-manager. 2023-11-28T21:33:01.445 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-28T21:33:01.474 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-28T21:33:01.495 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-28T21:33:01.495 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-28T21:33:01.550 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-28T21:33:01.886 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-28T21:33:01.908 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-28T21:33:01.977 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-28T21:33:02.086 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ppp. 2023-11-28T21:33:02.102 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-28T21:33:02.364 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-28T21:33:02.386 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:33:02.430 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:02.440 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-28T21:33:02.773 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package pptp-linux. 2023-11-28T21:33:02.788 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-28T21:33:02.825 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-28T21:33:02.831 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-28T21:33:02.847 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:33:02.892 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T21:33:03.093 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-28T21:33:03.108 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-28T21:33:03.151 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-28T21:33:03.236 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-28T21:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:33:03.294 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T21:33:03.429 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-28T21:33:03.444 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-28T21:33:03.504 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-client-common (238) ... 2023-11-28T21:33:03.638 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-28T21:33:03.651 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-28T21:33:03.730 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T21:33:03.808 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-28T21:33:03.823 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:33:03.866 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:04.359 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-28T21:33:04.381 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-28T21:33:04.425 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-28T21:33:04.471 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-common. 2023-11-28T21:33:04.486 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-28T21:33:04.549 INFO:teuthology.orchestra.run.smithi012.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-28T21:33:04.570 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-common (238) ... 2023-11-28T21:33:04.753 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package modemmanager. 2023-11-28T21:33:04.776 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-28T21:33:04.820 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T21:33:05.029 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-14. 2023-11-28T21:33:05.045 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:33:05.090 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:05.568 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package network-manager. 2023-11-28T21:33:05.591 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-28T21:33:05.643 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-28T21:33:06.240 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql. 2023-11-28T21:33:06.255 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-28T21:33:06.298 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql (14+238) ... 2023-11-28T21:33:06.326 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ppp. 2023-11-28T21:33:06.348 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-28T21:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-client. 2023-11-28T21:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-28T21:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-client (14+238) ... 2023-11-28T21:33:06.682 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-28T21:33:06.920 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-28T21:33:06.935 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-28T21:33:06.986 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-28T21:33:07.106 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package pptp-linux. 2023-11-28T21:33:07.128 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-28T21:33:07.173 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-28T21:33:07.272 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package tcl8.6. 2023-11-28T21:33:07.288 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-28T21:33:07.331 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-28T21:33:07.476 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-28T21:33:07.498 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-28T21:33:07.576 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-28T21:33:07.600 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package tcl. 2023-11-28T21:33:07.617 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-28T21:33:07.664 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-28T21:33:07.879 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-28T21:33:13.330 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package wpasupplicant. 2023-11-28T21:33:13.330 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-28T21:33:13.330 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-28T21:33:13.330 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-28T21:33:13.330 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-28T21:33:13.330 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-28T21:33:13.330 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-client-common (238) ... 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-28T21:33:13.331 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-28T21:33:13.332 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-28T21:33:13.332 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T21:33:13.332 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-28T21:33:13.332 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-28T21:33:13.332 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-28T21:33:13.332 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-28T21:33:13.332 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-28T21:33:13.332 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-28T21:33:13.332 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-client-common (238) ... 2023-11-28T21:33:13.332 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-common. 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-common (238) ... 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-14. 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql. 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql (14+238) ... 2023-11-28T21:33:13.333 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-client. 2023-11-28T21:33:13.334 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-28T21:33:13.334 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-client (14+238) ... 2023-11-28T21:33:13.334 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-28T21:33:13.334 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-28T21:33:13.334 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-28T21:33:13.334 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package tcl8.6. 2023-11-28T21:33:13.334 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-28T21:33:13.334 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-28T21:33:13.334 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package tcl. 2023-11-28T21:33:13.334 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-28T21:33:13.335 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-28T21:33:13.335 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package wpasupplicant. 2023-11-28T21:33:13.335 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-28T21:33:13.403 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-28T21:33:13.710 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-28T21:33:13.807 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-28T21:33:13.827 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up dump (0.4b47-3) ... 2023-11-28T21:33:13.829 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-28T21:33:13.874 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-28T21:33:13.956 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-28T21:33:14.021 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T21:33:14.147 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-28T21:33:14.273 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-28T21:33:14.319 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-28T21:33:14.341 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-28T21:33:14.394 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-28T21:33:14.398 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-28T21:33:14.525 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-28T21:33:14.598 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-28T21:33:14.660 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-28T21:33:14.720 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-client-common (238) ... 2023-11-28T21:33:14.811 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-28T21:33:14.966 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-28T21:33:14.987 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-28T21:33:15.092 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-28T21:33:15.113 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T21:33:15.239 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-28T21:33:15.243 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:15.369 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-28T21:33:15.407 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-client (14+238) ... 2023-11-28T21:33:15.520 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-28T21:33:15.542 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-28T21:33:15.638 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:16.103 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-28T21:33:16.120 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:16.120 INFO:teuthology.orchestra.run.smithi012.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-28T21:33:16.120 INFO:teuthology.orchestra.run.smithi012.stdout:which means they are currently configured by ifupdown: 2023-11-28T21:33:16.120 INFO:teuthology.orchestra.run.smithi012.stdout:- ens1f0 2023-11-28T21:33:16.121 INFO:teuthology.orchestra.run.smithi012.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-28T21:33:16.121 INFO:teuthology.orchestra.run.smithi012.stdout:remove their configuration from /etc/network/interfaces. 2023-11-28T21:33:16.121 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-28T21:33:16.261 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-28T21:33:16.603 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-28T21:33:16.603 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-28T21:33:16.668 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-28T21:33:17.077 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-28T21:33:17.313 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-28T21:33:17.430 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-28T21:33:17.589 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-28T21:33:17.716 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-28T21:33:17.778 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T21:33:17.876 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-28T21:33:17.921 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-28T21:33:18.010 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-28T21:33:18.046 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-28T21:33:18.136 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-28T21:33:18.198 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-common (238) ... 2023-11-28T21:33:18.850 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-28T21:33:18.976 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up dump (0.4b47-3) ... 2023-11-28T21:33:19.103 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-28T21:33:19.161 INFO:teuthology.orchestra.run.smithi012.stdout:Adding user postgres to group ssl-cert 2023-11-28T21:33:19.169 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T21:33:19.320 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-28T21:33:19.430 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-28T21:33:19.450 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:19.450 INFO:teuthology.orchestra.run.smithi012.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-28T21:33:19.539 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-28T21:33:19.698 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-28T21:33:19.716 INFO:teuthology.orchestra.run.smithi012.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-28T21:33:19.716 INFO:teuthology.orchestra.run.smithi012.stdout:Removing obsolete dictionary files: 2023-11-28T21:33:19.788 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-28T21:33:19.895 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-28T21:33:20.026 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-28T21:33:20.228 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-28T21:33:20.354 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T21:33:20.447 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-28T21:33:20.505 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-28T21:33:20.656 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-client (14+238) ... 2023-11-28T21:33:20.824 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-28T21:33:20.908 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:21.245 INFO:teuthology.orchestra.run.smithi012.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-28T21:33:21.267 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-28T21:33:21.307 INFO:teuthology.orchestra.run.smithi012.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-28T21:33:21.307 INFO:teuthology.orchestra.run.smithi012.stdout:This user must also own the server process. 2023-11-28T21:33:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:21.308 INFO:teuthology.orchestra.run.smithi012.stdout:The database cluster will be initialized with locale "en_US". 2023-11-28T21:33:21.308 INFO:teuthology.orchestra.run.smithi012.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-28T21:33:21.308 INFO:teuthology.orchestra.run.smithi012.stdout:The default text search configuration will be set to "english". 2023-11-28T21:33:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:21.308 INFO:teuthology.orchestra.run.smithi012.stdout:Data page checksums are disabled. 2023-11-28T21:33:21.308 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:21.308 INFO:teuthology.orchestra.run.smithi012.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-28T21:33:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:creating subdirectories ... ok 2023-11-28T21:33:21.309 INFO:teuthology.orchestra.run.smithi012.stdout:selecting dynamic shared memory implementation ... posix 2023-11-28T21:33:21.334 INFO:teuthology.orchestra.run.smithi012.stdout:selecting default max_connections ... 100 2023-11-28T21:33:21.361 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:21.361 INFO:teuthology.orchestra.run.smithi144.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-28T21:33:21.361 INFO:teuthology.orchestra.run.smithi144.stdout:which means they are currently configured by ifupdown: 2023-11-28T21:33:21.361 INFO:teuthology.orchestra.run.smithi144.stdout:- enp3s0f1 2023-11-28T21:33:21.361 INFO:teuthology.orchestra.run.smithi144.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-28T21:33:21.361 INFO:teuthology.orchestra.run.smithi144.stdout:remove their configuration from /etc/network/interfaces. 2023-11-28T21:33:21.361 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:21.371 INFO:teuthology.orchestra.run.smithi012.stdout:selecting default shared_buffers ... 128MB 2023-11-28T21:33:21.385 INFO:teuthology.orchestra.run.smithi012.stdout:selecting default time zone ... Etc/UTC 2023-11-28T21:33:21.385 INFO:teuthology.orchestra.run.smithi012.stdout:creating configuration files ... ok 2023-11-28T21:33:21.473 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-28T21:33:21.564 INFO:teuthology.orchestra.run.smithi012.stdout:running bootstrap script ... ok 2023-11-28T21:33:21.832 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-28T21:33:22.183 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-28T21:33:22.201 INFO:teuthology.orchestra.run.smithi012.stdout:performing post-bootstrap initialization ... ok 2023-11-28T21:33:22.750 INFO:teuthology.orchestra.run.smithi012.stdout:syncing data to disk ... ok 2023-11-28T21:33:22.769 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-28T21:33:22.953 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-28T21:33:23.113 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-28T21:33:23.214 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-common (238) ... 2023-11-28T21:33:23.296 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-28T21:33:24.156 INFO:teuthology.orchestra.run.smithi144.stdout:Adding user postgres to group ssl-cert 2023-11-28T21:33:24.369 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:24.369 INFO:teuthology.orchestra.run.smithi144.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-28T21:33:24.567 INFO:teuthology.orchestra.run.smithi144.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-28T21:33:24.567 INFO:teuthology.orchestra.run.smithi144.stdout:Removing obsolete dictionary files: 2023-11-28T21:33:25.190 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-28T21:33:25.576 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-28T21:33:25.922 INFO:teuthology.orchestra.run.smithi144.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-28T21:33:25.945 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-28T21:33:25.981 INFO:teuthology.orchestra.run.smithi144.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-28T21:33:25.981 INFO:teuthology.orchestra.run.smithi144.stdout:This user must also own the server process. 2023-11-28T21:33:25.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:25.982 INFO:teuthology.orchestra.run.smithi144.stdout:The database cluster will be initialized with locale "en_US". 2023-11-28T21:33:25.982 INFO:teuthology.orchestra.run.smithi144.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-28T21:33:25.982 INFO:teuthology.orchestra.run.smithi144.stdout:The default text search configuration will be set to "english". 2023-11-28T21:33:25.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:25.982 INFO:teuthology.orchestra.run.smithi144.stdout:Data page checksums are disabled. 2023-11-28T21:33:25.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:25.982 INFO:teuthology.orchestra.run.smithi144.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-28T21:33:25.982 INFO:teuthology.orchestra.run.smithi144.stdout:creating subdirectories ... ok 2023-11-28T21:33:25.982 INFO:teuthology.orchestra.run.smithi144.stdout:selecting dynamic shared memory implementation ... posix 2023-11-28T21:33:25.994 INFO:teuthology.orchestra.run.smithi144.stdout:selecting default max_connections ... 100 2023-11-28T21:33:26.011 INFO:teuthology.orchestra.run.smithi144.stdout:selecting default shared_buffers ... 128MB 2023-11-28T21:33:26.021 INFO:teuthology.orchestra.run.smithi144.stdout:selecting default time zone ... Etc/UTC 2023-11-28T21:33:26.022 INFO:teuthology.orchestra.run.smithi144.stdout:creating configuration files ... ok 2023-11-28T21:33:26.133 INFO:teuthology.orchestra.run.smithi144.stdout:running bootstrap script ... ok 2023-11-28T21:33:26.691 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T21:33:26.746 INFO:teuthology.orchestra.run.smithi144.stdout:performing post-bootstrap initialization ... ok 2023-11-28T21:33:26.817 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-28T21:33:26.968 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up postgresql (14+238) ... 2023-11-28T21:33:27.094 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T21:33:27.203 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T21:33:27.455 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-28T21:33:27.455 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-28T21:33:27.609 INFO:teuthology.orchestra.run.smithi144.stdout:syncing data to disk ... ok 2023-11-28T21:33:28.017 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-28T21:33:28.189 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-28T21:33:28.483 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-28T21:33:28.651 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-28T21:33:30.154 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-28T21:33:30.237 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-28T21:33:30.768 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-28T21:33:30.773 DEBUG:teuthology.parallel:result is None 2023-11-28T21:33:30.849 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T21:33:30.975 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-28T21:33:31.126 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up postgresql (14+238) ... 2023-11-28T21:33:31.252 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-28T21:33:31.395 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-28T21:33:31.638 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-28T21:33:31.638 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-28T21:33:32.264 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-28T21:33:32.625 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-28T21:33:32.818 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-28T21:33:34.221 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-28T21:33:34.312 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-28T21:33:34.848 INFO:teuthology.orchestra.run.smithi144.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-28T21:33:34.851 DEBUG:teuthology.parallel:result is None 2023-11-28T21:33:34.852 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 2023-11-28T21:33:34.992 DEBUG:teuthology.orchestra.run.smithi012:> dpkg-query -W -f '${Version}' ceph 2023-11-28T21:33:35.017 INFO:teuthology.orchestra.run.smithi012.stdout:18.0.0-7520-g43dd94c1-1jammy 2023-11-28T21:33:35.018 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7520-g43dd94c1-1jammy 2023-11-28T21:33:35.018 INFO:teuthology.task.install:The correct ceph version 18.0.0-7520-g43dd94c1-1jammy is installed. 2023-11-28T21:33:35.018 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 2023-11-28T21:33:35.236 DEBUG:teuthology.orchestra.run.smithi144:> dpkg-query -W -f '${Version}' ceph 2023-11-28T21:33:35.256 INFO:teuthology.orchestra.run.smithi144.stdout:18.0.0-7520-g43dd94c1-1jammy 2023-11-28T21:33:35.256 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7520-g43dd94c1-1jammy 2023-11-28T21:33:35.257 INFO:teuthology.task.install:The correct ceph version 18.0.0-7520-g43dd94c1-1jammy is installed. 2023-11-28T21:33:35.257 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T21:33:35.257 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:35.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T21:33:35.281 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:33:35.281 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T21:33:35.313 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T21:33:35.314 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:35.314 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T21:33:35.346 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T21:33:35.410 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:33:35.410 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T21:33:35.424 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T21:33:35.480 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T21:33:35.480 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:35.480 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T21:33:35.503 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T21:33:35.566 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:33:35.566 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T21:33:35.579 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T21:33:35.632 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T21:33:35.632 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:35.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T21:33:35.655 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T21:33:35.718 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:33:35.718 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T21:33:35.732 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T21:33:35.788 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T21:33:35.864 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T21:33:35.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /var/log/ceph 2023-11-28T21:33:35.868 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2023-11-28T21:33:35.886 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T21:33:35.886 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T21:33:35.931 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T21:33:35.950 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T21:33:35.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T21:33:35.995 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T21:33:36.021 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T21:33:36.021 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T21:33:36.022 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T21:33:36.022 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:36.022 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T21:33:36.066 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T21:33:36.165 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:33:36.165 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T21:33:36.171 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T21:33:36.250 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T21:33:36.250 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T21:33:36.250 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703', 'branch': 'wip-vshankar-testing-20231127.102914', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7469703', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20231127.102914'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'sha1': '43dd94c190073cf16cff1e711dd7f5a7b1cbc883'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '43dd94c190073cf16cff1e711dd7f5a7b1cbc883'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20231127.102914', 'sha1': '43dd94c190073cf16cff1e711dd7f5a7b1cbc883'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 150, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 8377, 'sha1': '43dd94c190073cf16cff1e711dd7f5a7b1cbc883', 'sleep_before_teardown': 0, 'subset': '462/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20231127.102914', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_43dd94c190073cf16cff1e711dd7f5a7b1cbc883/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '43dd94c190073cf16cff1e711dd7f5a7b1cbc883', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi012.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUISU2UNRkZ8KvhahKPJ/Cs+rXF3/sD7bxiYbw/RwxUeVWemGwd9BwvJrvWDwo4pvBt+N1nYfzMl5NONDD2Hro=', 'smithi144.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWCPwqcYt1BY50Y5frCiKMdgs43eqUVjQ57TA8TpCWOuXtYVKXsQiKY6HgEqCtuq0al9oexvbteSJacgpNbMgg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '43dd94c190073cf16cff1e711dd7f5a7b1cbc883'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'sha1': '43dd94c190073cf16cff1e711dd7f5a7b1cbc883', 'cluster': 'ceph'}}, {'kclient': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_06:29:14', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T21:33:36.251 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T21:33:36.254 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T21:33:36.293 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T21:33:36.308 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T21:33:36.353 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:36.353 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T21:33:36.377 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T21:33:36.378 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2023-11-28T21:33:36.433 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T21:33:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T21:33:36.434 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 890 Links: 1 2023-11-28T21:33:36.434 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T21:33:36.434 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-11-28 21:27:02.580628543 +0000 2023-11-28T21:33:36.434 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-11-28 21:27:02.164632259 +0000 2023-11-28T21:33:36.434 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-11-28 21:27:02.164632259 +0000 2023-11-28T21:33:36.434 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2023-11-28 21:27:02.164632259 +0000 2023-11-28T21:33:36.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T21:33:36.500 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-11-28T21:33:36.501 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-11-28T21:33:36.501 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000407363 s, 1.3 MB/s 2023-11-28T21:33:36.502 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T21:33:36.555 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2023-11-28T21:33:36.605 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T21:33:36.606 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T21:33:36.606 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 908 Links: 1 2023-11-28T21:33:36.606 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T21:33:36.606 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-11-28 21:27:03.284622254 +0000 2023-11-28T21:33:36.606 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-11-28 21:27:02.864626006 +0000 2023-11-28T21:33:36.606 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-11-28 21:27:02.864626006 +0000 2023-11-28T21:33:36.606 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2023-11-28 21:27:02.864626006 +0000 2023-11-28T21:33:36.606 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T21:33:36.672 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-11-28T21:33:36.672 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-11-28T21:33:36.672 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000374347 s, 1.4 MB/s 2023-11-28T21:33:36.674 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T21:33:36.727 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2023-11-28T21:33:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T21:33:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T21:33:36.782 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 924 Links: 1 2023-11-28T21:33:36.782 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T21:33:36.782 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-11-28 21:27:03.540619968 +0000 2023-11-28T21:33:36.782 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-11-28 21:27:03.536620004 +0000 2023-11-28T21:33:36.782 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-11-28 21:27:03.536620004 +0000 2023-11-28T21:33:36.782 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2023-11-28 21:27:03.536620004 +0000 2023-11-28T21:33:36.783 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T21:33:36.845 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-11-28T21:33:36.845 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-11-28T21:33:36.845 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000358355 s, 1.4 MB/s 2023-11-28T21:33:36.846 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T21:33:36.903 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2023-11-28T21:33:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T21:33:36.957 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T21:33:36.958 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 940 Links: 1 2023-11-28T21:33:36.958 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T21:33:36.958 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2023-11-28 21:27:04.640610143 +0000 2023-11-28T21:33:36.958 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-11-28 21:27:04.212613966 +0000 2023-11-28T21:33:36.958 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-11-28 21:27:04.212613966 +0000 2023-11-28T21:33:36.958 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2023-11-28 21:27:04.212613966 +0000 2023-11-28T21:33:36.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T21:33:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2023-11-28T21:33:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2023-11-28T21:33:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000401184 s, 1.3 MB/s 2023-11-28T21:33:37.022 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T21:33:37.075 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T21:33:37.075 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:33:37.075 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T21:33:37.081 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T21:33:37.081 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2023-11-28T21:33:37.129 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T21:33:37.129 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T21:33:37.129 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 876 Links: 1 2023-11-28T21:33:37.129 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T21:33:37.129 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-11-28 21:26:37.388483583 +0000 2023-11-28T21:33:37.129 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-11-28 21:26:37.052486585 +0000 2023-11-28T21:33:37.130 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-11-28 21:26:37.052486585 +0000 2023-11-28T21:33:37.130 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-11-28T21:33:37.130 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T21:33:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-11-28T21:33:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-11-28T21:33:37.183 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000490127 s, 1.0 MB/s 2023-11-28T21:33:37.185 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T21:33:37.234 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2023-11-28T21:33:37.281 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T21:33:37.281 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T21:33:37.281 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 895 Links: 1 2023-11-28T21:33:37.282 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T21:33:37.282 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-11-28 21:26:37.984478261 +0000 2023-11-28T21:33:37.282 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-11-28 21:26:37.636481368 +0000 2023-11-28T21:33:37.282 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-11-28 21:26:37.636481368 +0000 2023-11-28T21:33:37.282 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-11-28T21:33:37.283 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T21:33:37.335 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-11-28T21:33:37.335 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-11-28T21:33:37.335 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000472478 s, 1.1 MB/s 2023-11-28T21:33:37.336 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T21:33:37.385 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2023-11-28T21:33:37.433 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T21:33:37.433 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T21:33:37.433 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 914 Links: 1 2023-11-28T21:33:37.433 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T21:33:37.433 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-11-28 21:26:38.552473187 +0000 2023-11-28T21:33:37.434 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-11-28 21:26:38.212476224 +0000 2023-11-28T21:33:37.434 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-11-28 21:26:38.212476224 +0000 2023-11-28T21:33:37.434 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-11-28T21:33:37.434 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T21:33:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-11-28T21:33:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-11-28T21:33:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000478725 s, 1.1 MB/s 2023-11-28T21:33:37.488 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T21:33:37.538 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2023-11-28T21:33:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T21:33:37.589 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T21:33:37.589 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 932 Links: 1 2023-11-28T21:33:37.589 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T21:33:37.590 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-11-28 21:26:39.148467863 +0000 2023-11-28T21:33:37.590 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-11-28 21:26:38.804470936 +0000 2023-11-28T21:33:37.590 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-11-28 21:26:38.804470936 +0000 2023-11-28T21:33:37.590 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2023-11-28T21:33:37.591 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T21:33:37.642 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-11-28T21:33:37.643 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-11-28T21:33:37.643 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000505733 s, 1.0 MB/s 2023-11-28T21:33:37.645 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T21:33:37.694 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T21:33:37.694 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T21:33:37.695 INFO:tasks.ceph:Generating config... 2023-11-28T21:33:37.697 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T21:33:37.697 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2023-11-28T21:33:37.697 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T21:33:37.697 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T21:33:37.698 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T21:33:37.698 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T21:33:37.698 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-28T21:33:37.698 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T21:33:37.699 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T21:33:37.699 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T21:33:37.699 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2023-11-28T21:33:37.699 INFO:tasks.ceph:[mds] mds bal merge size = 5 2023-11-28T21:33:37.699 INFO:tasks.ceph:[mds] mds bal split bits = 3 2023-11-28T21:33:37.699 INFO:tasks.ceph:[mds] mds bal split size = 100 2023-11-28T21:33:37.700 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T21:33:37.700 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T21:33:37.700 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2023-11-28T21:33:37.700 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T21:33:37.700 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T21:33:37.700 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2023-11-28T21:33:37.701 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2023-11-28T21:33:37.701 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2023-11-28T21:33:37.701 INFO:tasks.ceph:[mds] ms inject delay max = 1 2023-11-28T21:33:37.701 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2023-11-28T21:33:37.701 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2023-11-28T21:33:37.701 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2023-11-28T21:33:37.702 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T21:33:37.702 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T21:33:37.702 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T21:33:37.705 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T21:33:37.706 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T21:33:37.706 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T21:33:37.707 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T21:33:37.707 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T21:33:37.707 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T21:33:37.707 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T21:33:37.707 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-11-28T21:33:37.707 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T21:33:37.708 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-28T21:33:37.708 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-28T21:33:37.708 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T21:33:37.708 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T21:33:37.708 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-28T21:33:37.708 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2023-11-28T21:33:37.709 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T21:33:37.709 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T21:33:37.709 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T21:33:37.709 INFO:tasks.ceph:[osd] ms inject delay max = 1 2023-11-28T21:33:37.709 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2023-11-28T21:33:37.709 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2023-11-28T21:33:37.710 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2023-11-28T21:33:37.710 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-11-28T21:33:37.710 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T21:33:37.710 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-11-28T21:33:37.710 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T21:33:37.710 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T21:33:37.711 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T21:33:37.711 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T21:33:37.761 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T21:33:37.778 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T21:33:37.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T21:33:37.851 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.12'), ('mon.b', '172.21.15.144'), ('mon.c', '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]')] 2023-11-28T21:33:37.851 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.12,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T21:33:37.852 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:37.852 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T21:33:37.902 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.12 --add b 172.21.15.144 --addv c '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:33:37.993 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:33:37.993 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: generated fsid 2975d7d0-8668-45d5-addd-c7b2ef978be7 2023-11-28T21:33:37.993 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = pacific 2023-11-28T21:33:37.994 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 0 2023-11-28T21:33:37.994 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 2975d7d0-8668-45d5-addd-c7b2ef978be7 2023-11-28T21:33:37.994 INFO:teuthology.orchestra.run.smithi012.stdout:last_changed 2023-11-28T21:33:37.993570+0000 2023-11-28T21:33:37.994 INFO:teuthology.orchestra.run.smithi012.stdout:created 2023-11-28T21:33:37.993570+0000 2023-11-28T21:33:37.994 INFO:teuthology.orchestra.run.smithi012.stdout:min_mon_release 16 (pacific) 2023-11-28T21:33:37.994 INFO:teuthology.orchestra.run.smithi012.stdout:election_strategy: 1 2023-11-28T21:33:37.995 INFO:teuthology.orchestra.run.smithi012.stdout:0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2023-11-28T21:33:37.995 INFO:teuthology.orchestra.run.smithi012.stdout:1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2023-11-28T21:33:37.995 INFO:teuthology.orchestra.run.smithi012.stdout:2: [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] mon.c 2023-11-28T21:33:37.995 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T21:33:37.997 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T21:33:38.007 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2975d7d0-8668-45d5-addd-c7b2ef978be7... 2023-11-28T21:33:38.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T21:33:38.051 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T21:33:38.085 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2023-11-28T21:33:38.086 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2023-11-28T21:33:38.086 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T21:33:38.086 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2023-11-28T21:33:38.086 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.086 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2023-11-28T21:33:38.087 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.087 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2023-11-28T21:33:38.087 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.087 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2023-11-28T21:33:38.087 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2023-11-28T21:33:38.087 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.088 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2023-11-28T21:33:38.088 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2023-11-28T21:33:38.088 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.088 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T21:33:38.088 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2023-11-28T21:33:38.088 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.089 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T21:33:38.089 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T21:33:38.089 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.089 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2023-11-28T21:33:38.089 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.089 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2023-11-28T21:33:38.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2023-11-28T21:33:38.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2023-11-28T21:33:38.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2023-11-28T21:33:38.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2023-11-28T21:33:38.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2023-11-28T21:33:38.090 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2023-11-28T21:33:38.091 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T21:33:38.091 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T21:33:38.091 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2023-11-28T21:33:38.091 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.091 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T21:33:38.092 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.092 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2023-11-28T21:33:38.092 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.092 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2023-11-28T21:33:38.092 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.092 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2023-11-28T21:33:38.093 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2023-11-28T21:33:38.093 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2023-11-28T21:33:38.093 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.12,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2023-11-28T21:33:38.093 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on skipped message = False 2023-11-28T21:33:38.093 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 2975d7d0-8668-45d5-addd-c7b2ef978be7 2023-11-28T21:33:38.093 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.094 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2023-11-28T21:33:38.094 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2023-11-28T21:33:38.094 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.094 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2023-11-28T21:33:38.094 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2023-11-28T21:33:38.094 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2023-11-28T21:33:38.095 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.095 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2023-11-28T21:33:38.095 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2023-11-28T21:33:38.095 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.095 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2023-11-28T21:33:38.095 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2023-11-28T21:33:38.096 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2023-11-28T21:33:38.096 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2023-11-28T21:33:38.096 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.096 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2023-11-28T21:33:38.096 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2023-11-28T21:33:38.097 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.097 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2023-11-28T21:33:38.097 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.097 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2023-11-28T21:33:38.097 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.097 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2023-11-28T21:33:38.098 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.098 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2023-11-28T21:33:38.098 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2023-11-28T21:33:38.098 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.098 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2023-11-28T21:33:38.098 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2023-11-28T21:33:38.099 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2023-11-28T21:33:38.099 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore compression mode = aggressive 2023-11-28T21:33:38.099 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2023-11-28T21:33:38.099 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 20 2023-11-28T21:33:38.099 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 20 2023-11-28T21:33:38.099 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-11-28T21:33:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2023-11-28T21:33:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 10 2023-11-28T21:33:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: mon client directed command retry = 5 2023-11-28T21:33:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T21:33:38.100 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2023-11-28T21:33:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2023-11-28T21:33:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject delay max = 1 2023-11-28T21:33:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject delay probability = 0.005 2023-11-28T21:33:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject delay type = client mds 2023-11-28T21:33:38.102 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject socket failures = 2500 2023-11-28T21:33:38.102 INFO:teuthology.orchestra.run.smithi144.stdout: osd blocked scrub grace period = 3600 2023-11-28T21:33:38.102 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2023-11-28T21:33:38.102 INFO:teuthology.orchestra.run.smithi144.stdout: osd max markdown count = 1000 2023-11-28T21:33:38.102 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2023-11-28T21:33:38.102 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2023-11-28T21:33:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.103 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2023-11-28T21:33:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-11-28T21:33:38.103 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2023-11-28T21:33:38.104 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2023-11-28T21:33:38.104 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2023-11-28T21:33:38.104 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2023-11-28T21:33:38.104 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2023-11-28T21:33:38.104 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2023-11-28T21:33:38.104 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.105 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2023-11-28T21:33:38.105 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-11-28T21:33:38.105 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2023-11-28T21:33:38.105 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2023-11-28T21:33:38.106 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2023-11-28T21:33:38.106 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2023-11-28T21:33:38.106 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2023-11-28T21:33:38.106 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2023-11-28T21:33:38.106 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2023-11-28T21:33:38.107 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2023-11-28T21:33:38.107 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2023-11-28T21:33:38.107 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.107 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T21:33:38.107 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T21:33:38.108 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T21:33:38.108 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.108 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T21:33:38.108 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T21:33:38.108 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T21:33:38.108 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.109 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2023-11-28T21:33:38.109 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2023-11-28T21:33:38.109 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.109 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2023-11-28T21:33:38.109 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2023-11-28T21:33:38.109 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-11-28T21:33:38.110 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2023-11-28T21:33:38.110 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2023-11-28T21:33:38.110 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2023-11-28T21:33:38.111 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2023-11-28T21:33:38.111 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T21:33:38.111 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T21:33:38.111 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2023-11-28T21:33:38.113 INFO:teuthology.orchestra.run.smithi144.stdout: client_shutdown_timeout = 120 2023-11-28T21:33:38.113 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2023-11-28T21:33:38.113 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-11-28T21:33:38.114 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2023-11-28T21:33:38.114 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2023-11-28T21:33:38.114 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2023-11-28T21:33:38.114 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2023-11-28T21:33:38.115 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2023-11-28T21:33:38.115 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2023-11-28T21:33:38.115 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2023-11-28T21:33:38.116 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.e] 2023-11-28T21:33:38.116 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2023-11-28T21:33:38.116 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2023-11-28T21:33:38.116 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.f] 2023-11-28T21:33:38.117 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2023-11-28T21:33:38.117 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2023-11-28T21:33:38.117 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2023-11-28T21:33:38.117 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-11-28T21:33:38.118 INFO:teuthology.orchestra.run.smithi144.stdout: mds bal fragment size max = 10000 2023-11-28T21:33:38.118 INFO:teuthology.orchestra.run.smithi144.stdout: mds bal merge size = 5 2023-11-28T21:33:38.118 INFO:teuthology.orchestra.run.smithi144.stdout: mds bal split bits = 3 2023-11-28T21:33:38.119 INFO:teuthology.orchestra.run.smithi144.stdout: mds bal split size = 100 2023-11-28T21:33:38.119 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2023-11-28T21:33:38.119 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = True 2023-11-28T21:33:38.119 INFO:teuthology.orchestra.run.smithi144.stdout: mds oft prefetch dirfrags = True 2023-11-28T21:33:38.119 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2023-11-28T21:33:38.120 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = True 2023-11-28T21:33:38.120 INFO:teuthology.orchestra.run.smithi144.stdout: mds_dir_prefetch = True 2023-11-28T21:33:38.120 INFO:teuthology.orchestra.run.smithi144.stdout: mds_sleep_rank_change = 5000000.0 2023-11-28T21:33:38.120 INFO:teuthology.orchestra.run.smithi144.stdout: mon client directed command retry = 5 2023-11-28T21:33:38.120 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject delay max = 1 2023-11-28T21:33:38.120 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject delay probability = 0.005 2023-11-28T21:33:38.121 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject delay type = client mds osd 2023-11-28T21:33:38.121 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject socket failures = 2500 2023-11-28T21:33:38.121 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2023-11-28T21:33:38.121 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2023-11-28T21:33:38.121 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2023-11-28T21:33:38.124 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2023-11-28T21:33:38.124 INFO:teuthology.orchestra.run.smithi012.stdout: chdir = "" 2023-11-28T21:33:38.124 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T21:33:38.124 INFO:teuthology.orchestra.run.smithi012.stdout: auth supported = cephx 2023-11-28T21:33:38.124 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.125 INFO:teuthology.orchestra.run.smithi012.stdout: filestore xattr use omap = true 2023-11-28T21:33:38.125 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.125 INFO:teuthology.orchestra.run.smithi012.stdout: mon clock drift allowed = 1.000 2023-11-28T21:33:38.125 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.125 INFO:teuthology.orchestra.run.smithi012.stdout: osd crush chooseleaf type = 0 2023-11-28T21:33:38.125 INFO:teuthology.orchestra.run.smithi012.stdout: auth debug = true 2023-11-28T21:33:38.126 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.126 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on old message = true 2023-11-28T21:33:38.126 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on bug = true 2023-11-28T21:33:38.126 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.127 INFO:teuthology.orchestra.run.smithi012.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T21:33:38.127 INFO:teuthology.orchestra.run.smithi012.stdout: mon pg warn max object skew = 0 2023-11-28T21:33:38.127 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.127 INFO:teuthology.orchestra.run.smithi012.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T21:33:38.127 INFO:teuthology.orchestra.run.smithi012.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T21:33:38.127 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.128 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default size = 2 2023-11-28T21:33:38.128 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.128 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow primary affinity = true 2023-11-28T21:33:38.128 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow pg remap = true 2023-11-28T21:33:38.128 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on legacy crush tunables = false 2023-11-28T21:33:38.128 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on crush straw calc version zero = false 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on no sortbitwise = false 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on osd down out interval zero = false 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on too few osds = false 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: mon_allow_pool_size_one = true 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: osd default data pool replay window = 5 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: mon allow pool delete = true 2023-11-28T21:33:38.129 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: mon cluster log file level = debug 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: debug asserts on shutdown = true 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: mon health detail to clog = false 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: mon host = "172.21.15.12,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on skipped message = False 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 2975d7d0-8668-45d5-addd-c7b2ef978be7 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout:[osd] 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: osd journal size = 100 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub load threshold = 5.0 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub max interval = 600 2023-11-28T21:33:38.130 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock profile = high_recovery_ops 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: osd recover clone overlap = true 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: osd recovery max chunk = 1048576 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug shutdown = true 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug op order = true 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify stray on activate = true 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug trim objects = true 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: osd open classes on start = true 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug pg log writeout = true 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.131 INFO:teuthology.orchestra.run.smithi012.stdout: osd deep scrub update digest min age = 30 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: osd map max advance = 10 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: journal zero on create = true 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: filestore ondisk finisher threads = 3 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: filestore apply finisher threads = 3 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: bdev debug aio = true 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug misdirected ops = true 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore block size = 96636764160 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore compression mode = aggressive 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore fsck on mount = True 2023-11-28T21:33:38.132 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluefs = 20 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluestore = 20 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: debug osd = 20 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: debug rocksdb = 10 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: mon client directed command retry = 5 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd full ratio = 0.9 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd nearfull ratio = 0.8 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay max = 1 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay probability = 0.005 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay type = client mds 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject socket failures = 2500 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: osd blocked scrub grace period = 3600 2023-11-28T21:33:38.133 INFO:teuthology.orchestra.run.smithi012.stdout: osd failsafe full ratio = 0.95 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: osd max markdown count = 1000 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: osd objectstore = bluestore 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: osd op complaint time = 180 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr] 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: debug mgr = 20 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: mgr/telemetry/nag = false 2023-11-28T21:33:38.134 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout:[mon] 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: debug paxos = 20 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: mon data avail warn = 5 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: mon mgr mkfs grace = 240 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd reporter subtree level = osd 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd prime pg temp = true 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T21:33:38.135 INFO:teuthology.orchestra.run.smithi012.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: # 1m isn't quite enough 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: mon_down_mkfs_grace = 2m 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_filestore_osds = false 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: mon op complaint time = 120 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout:[client] 2023-11-28T21:33:38.136 INFO:teuthology.orchestra.run.smithi012.stdout: rgw cache enabled = true 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable ops log = true 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable usage log = true 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout: client mount timeout = 600 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout: client_shutdown_timeout = 120 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout: debug client = 20 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout: rados mon op timeout = 900 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout: rados osd op timeout = 900 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.a] 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.b] 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.c] 2023-11-28T21:33:38.137 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a] 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c] 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e] 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b] 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d] 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f] 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout:[mds] 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout: debug mds = 20 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout: debug mds balancer = 20 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout: mds bal fragment size max = 10000 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout: mds bal merge size = 5 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout: mds bal split bits = 3 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout: mds bal split size = 100 2023-11-28T21:33:38.138 INFO:teuthology.orchestra.run.smithi012.stdout: mds debug frag = True 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: mds debug scatterstat = True 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: mds oft prefetch dirfrags = True 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: mds op complaint time = 180 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: mds verify scatter = True 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: mds_dir_prefetch = True 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: mds_sleep_rank_change = 5000000.0 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: mon client directed command retry = 5 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay max = 1 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay probability = 0.005 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject delay type = client mds osd 2023-11-28T21:33:38.139 INFO:teuthology.orchestra.run.smithi012.stdout: ms inject socket failures = 2500 2023-11-28T21:33:38.140 INFO:teuthology.orchestra.run.smithi012.stdout: osd op complaint time = 180 2023-11-28T21:33:38.140 INFO:teuthology.orchestra.run.smithi012.stdout: rados mon op timeout = 900 2023-11-28T21:33:38.140 INFO:teuthology.orchestra.run.smithi012.stdout: rados osd op timeout = 900 2023-11-28T21:33:38.140 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T21:33:38.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T21:33:38.226 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T21:33:38.227 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:38.227 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T21:33:38.236 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:38.236 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T21:33:38.288 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:33:38.288 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:38.288 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T21:33:38.288 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T21:33:38.366 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:33:38.367 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:33:38.417 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:33:38.417 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:33:38.418 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T21:33:38.418 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T21:33:38.441 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:33:38.441 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:33:38.493 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T21:33:38.493 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T21:33:38.493 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T21:33:38.552 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T21:33:38.557 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T21:33:38.599 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T21:33:38.616 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T21:33:38.616 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T21:33:38.674 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T21:33:38.678 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T21:33:38.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T21:33:38.801 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T21:33:38.805 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T21:33:38.826 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2023-11-28T21:33:38.929 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2023-11-28T21:33:38.933 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2023-11-28T21:33:38.954 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T21:33:38.995 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T21:33:38.998 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T21:33:39.052 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T21:33:39.133 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T21:33:39.136 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T21:33:39.190 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2023-11-28T21:33:39.269 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2023-11-28T21:33:39.271 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2023-11-28T21:33:39.324 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T21:33:39.325 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T21:33:39.370 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T21:33:39.388 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T21:33:39.388 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T21:33:39.388 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T21:33:39.452 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T21:33:39.453 INFO:tasks.ceph:role: osd.0 2023-11-28T21:33:39.453 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:33:39.453 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T21:33:39.525 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T21:33:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T21:33:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T21:33:39.526 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T21:33:39.527 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T21:33:39.527 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-11-28T21:33:39.527 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T21:33:39.527 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T21:33:39.528 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T21:33:39.528 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T21:33:39.998 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-11-28T21:33:40.001 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-11-28T21:33:40.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T21:33:40.154 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T21:33:40.173 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T21:33:40.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:33:40.174 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T21:33:40.238 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T21:33:40.238 INFO:tasks.ceph:role: osd.1 2023-11-28T21:33:40.238 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:33:40.238 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T21:33:40.309 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T21:33:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T21:33:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T21:33:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T21:33:40.310 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T21:33:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-11-28T21:33:40.311 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T21:33:40.311 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T21:33:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T21:33:40.311 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T21:33:40.752 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-11-28T21:33:40.755 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-11-28T21:33:40.756 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T21:33:40.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T21:33:40.878 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T21:33:40.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:33:40.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T21:33:40.946 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T21:33:40.946 INFO:tasks.ceph:role: osd.2 2023-11-28T21:33:40.946 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:33:40.947 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T21:33:41.017 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T21:33:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T21:33:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T21:33:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T21:33:41.018 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T21:33:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-11-28T21:33:41.018 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T21:33:41.018 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T21:33:41.019 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T21:33:41.019 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T21:33:41.485 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-11-28T21:33:41.487 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-11-28T21:33:41.488 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T21:33:41.641 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T21:33:41.660 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T21:33:41.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:33:41.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T21:33:41.726 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T21:33:41.726 INFO:tasks.ceph:role: osd.3 2023-11-28T21:33:41.726 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:33:41.726 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T21:33:41.799 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T21:33:41.800 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T21:33:41.800 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T21:33:41.800 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T21:33:41.800 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T21:33:41.800 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2023-11-28T21:33:41.800 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T21:33:41.800 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T21:33:41.800 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T21:33:41.800 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T21:33:42.291 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2023-11-28T21:33:42.294 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2023-11-28T21:33:42.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T21:33:42.455 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T21:33:42.474 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T21:33:42.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:33:42.475 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:33:42.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:42.629+0000 7f880fb82640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T21:33:42.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:42.633+0000 7f880fb82640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T21:33:42.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:42.633+0000 7f880fb82640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T21:33:42.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:42.633+0000 7f880fb82640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T21:33:42.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:42.633+0000 7f880fb82640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T21:33:42.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:42.637+0000 7f880fb82640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T21:33:45.625 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T21:33:45.648 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:33:45.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:45.745+0000 7f85aefda640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T21:33:45.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:45.745+0000 7f85aefda640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T21:33:45.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:45.745+0000 7f85aefda640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T21:33:45.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:45.745+0000 7f85aefda640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T21:33:45.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:45.745+0000 7f85aefda640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T21:33:45.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:45.749+0000 7f85aefda640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T21:33:48.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T21:33:48.782 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:33:48.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:48.877+0000 7f141102b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T21:33:48.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:48.877+0000 7f141102b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T21:33:48.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:48.877+0000 7f141102b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T21:33:48.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:48.877+0000 7f141102b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T21:33:48.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:48.877+0000 7f141102b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T21:33:48.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:48.881+0000 7f141102b640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T21:33:51.867 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T21:33:51.890 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:33:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:51.985+0000 7f16374cb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T21:33:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:51.985+0000 7f16374cb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T21:33:51.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:51.985+0000 7f16374cb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T21:33:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:51.985+0000 7f16374cb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T21:33:51.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:51.985+0000 7f16374cb640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T21:33:51.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:33:51.985+0000 7f16374cb640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T21:33:54.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T21:33:54.998 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T21:33:55.013 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T21:33:55.013 INFO:tasks.ceph:role: osd.4 2023-11-28T21:33:55.013 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:33:55.013 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T21:33:55.076 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T21:33:55.076 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T21:33:55.077 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T21:33:55.077 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T21:33:55.077 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T21:33:55.077 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-11-28T21:33:55.077 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T21:33:55.077 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T21:33:55.077 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T21:33:55.077 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T21:33:55.538 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-11-28T21:33:55.540 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-11-28T21:33:55.540 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T21:33:55.633 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T21:33:55.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:33:55.643 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T21:33:55.643 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T21:33:55.694 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T21:33:55.694 INFO:tasks.ceph:role: osd.5 2023-11-28T21:33:55.694 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:33:55.694 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T21:33:55.757 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T21:33:55.757 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T21:33:55.758 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T21:33:55.758 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T21:33:55.758 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T21:33:55.758 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-11-28T21:33:55.758 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T21:33:55.758 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T21:33:55.758 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T21:33:55.758 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T21:33:56.221 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-11-28T21:33:56.223 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-11-28T21:33:56.223 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T21:33:56.354 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T21:33:56.365 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T21:33:56.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:33:56.366 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T21:33:56.424 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T21:33:56.424 INFO:tasks.ceph:role: osd.6 2023-11-28T21:33:56.424 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:33:56.424 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T21:33:56.488 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T21:33:56.488 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T21:33:56.489 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T21:33:56.489 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T21:33:56.489 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T21:33:56.489 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-11-28T21:33:56.489 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T21:33:56.489 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T21:33:56.489 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T21:33:56.489 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T21:33:56.969 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-11-28T21:33:56.971 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-11-28T21:33:56.971 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T21:33:57.115 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T21:33:57.125 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T21:33:57.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:33:57.126 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T21:33:57.180 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T21:33:57.180 INFO:tasks.ceph:role: osd.7 2023-11-28T21:33:57.180 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:33:57.180 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T21:33:57.243 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T21:33:57.243 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T21:33:57.243 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T21:33:57.243 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T21:33:57.243 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T21:33:57.243 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-11-28T21:33:57.243 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T21:33:57.243 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T21:33:57.244 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T21:33:57.244 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T21:33:57.660 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-11-28T21:33:57.662 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-11-28T21:33:57.662 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T21:33:57.775 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T21:33:57.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:33:57.784 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2023-11-28T21:33:57.785 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:33:57.895 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:33:57.888+0000 7fa977aee640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T21:33:57.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:33:57.892+0000 7fa977aee640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T21:33:57.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:33:57.892+0000 7fa977aee640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T21:33:57.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:33:57.892+0000 7fa977aee640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T21:33:57.897 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:33:57.892+0000 7fa977aee640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T21:33:57.898 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:33:57.892+0000 7fa977aee640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T21:34:00.929 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T21:34:00.943 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:34:01.022 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:01.016+0000 7f648426a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T21:34:01.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:01.016+0000 7f648426a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T21:34:01.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:01.016+0000 7f648426a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T21:34:01.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:01.016+0000 7f648426a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T21:34:01.024 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:01.016+0000 7f648426a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T21:34:01.025 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:01.020+0000 7f648426a640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T21:34:04.040 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T21:34:04.097 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:34:04.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:04.172+0000 7f005974e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T21:34:04.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:04.172+0000 7f005974e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T21:34:04.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:04.172+0000 7f005974e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T21:34:04.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:04.172+0000 7f005974e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T21:34:04.179 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:04.172+0000 7f005974e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T21:34:04.180 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:04.176+0000 7f005974e640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T21:34:06.252 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:34:06.256 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:34:07.200 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T21:34:07.213 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:34:07.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:07.288+0000 7fc5bc602640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T21:34:07.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:07.288+0000 7fc5bc602640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T21:34:07.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:07.288+0000 7fc5bc602640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T21:34:07.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:07.288+0000 7fc5bc602640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T21:34:07.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:07.288+0000 7fc5bc602640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T21:34:07.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:07.288+0000 7fc5bc602640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T21:34:10.308 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T21:34:10.365 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T21:34:10.365 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:10.365 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T21:34:10.389 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:10.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T21:34:10.455 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:10.455 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T21:34:10.518 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:10.518 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2023-11-28T21:34:10.586 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:10.586 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T21:34:10.650 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:10.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T21:34:10.715 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:10.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T21:34:10.778 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:10.778 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T21:34:10.842 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:10.842 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T21:34:10.856 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:10.856 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T21:34:10.912 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:10.912 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T21:34:10.968 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:10.968 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2023-11-28T21:34:11.024 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:11.024 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T21:34:11.079 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:11.079 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T21:34:11.133 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:11.133 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T21:34:11.188 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:11.188 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T21:34:11.244 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:11.244 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T21:34:11.254 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T21:34:11.254 DEBUG:teuthology.orchestra.run.smithi012:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T21:34:11.304 DEBUG:teuthology.orchestra.run.smithi144:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T21:34:11.316 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.y] 2023-11-28T21:34:11.316 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCyXGZlYhztIBAAAyVPVgVpInoL6SE21F4c/w== 2023-11-28T21:34:11.316 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2023-11-28T21:34:11.317 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCyXGZlPlU+KBAAFhXKx0p81FpAik+O9ZbD4Q== 2023-11-28T21:34:11.317 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2023-11-28T21:34:11.317 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCyXGZlXrjKLxAA+RYGxhSJgM9Jt/7LottRMA== 2023-11-28T21:34:11.317 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.e] 2023-11-28T21:34:11.317 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCyXGZlS6ZoNxAAjRNbem2ntxYIRJAOzCRcZQ== 2023-11-28T21:34:11.317 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2023-11-28T21:34:11.317 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQC2XGZlGaLaJRAAEyIExij4XgmOKBbWZ0Tkmw== 2023-11-28T21:34:11.317 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2023-11-28T21:34:11.317 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQC5XGZlFmXMLBAAdsIVG5fUkJP857CRkRwSvA== 2023-11-28T21:34:11.318 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2023-11-28T21:34:11.318 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQC8XGZlBfKfNBAAUHmsIuzeH3XFjjTr5iYBKg== 2023-11-28T21:34:11.318 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.3] 2023-11-28T21:34:11.319 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQC/XGZlGjwLOxAA9FHLqKdnkM05zM3qtRASog== 2023-11-28T21:34:11.319 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.x] 2023-11-28T21:34:11.319 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCyXGZlyni7IxAALsJPntiCUUI1OH0Hz6S6Og== 2023-11-28T21:34:11.319 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2023-11-28T21:34:11.320 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCyXGZlQ2daOxAAdiiuKOT+1SasBjzN8s/6Hg== 2023-11-28T21:34:11.320 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2023-11-28T21:34:11.320 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCzXGZlj973BxAAaQM7kNVZ4wDLfHYrAxTzIw== 2023-11-28T21:34:11.320 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.f] 2023-11-28T21:34:11.321 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCzXGZleqQJEBAAErOPB/VKcgNNm2JWlALM9Q== 2023-11-28T21:34:11.321 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.4] 2023-11-28T21:34:11.321 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDFXGZlx35eNRAAan4up9xQZByoQ5tqmyXumg== 2023-11-28T21:34:11.322 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.5] 2023-11-28T21:34:11.322 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDJXGZlojFcARAAystyiecnHs8tcmK2cZ9bpQ== 2023-11-28T21:34:11.322 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.6] 2023-11-28T21:34:11.322 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDMXGZlTeGnChAA28KZQO6/At3WiTR2O4hTOg== 2023-11-28T21:34:11.322 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.7] 2023-11-28T21:34:11.323 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPXGZl5Q2RERAA/6TyM/MLwqL1lURzemqOOQ== 2023-11-28T21:34:11.323 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2023-11-28T21:34:11.323 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCzXGZl9igiFhAAzDLjXsTvm2GemM1+fOMupQ== 2023-11-28T21:34:11.325 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.y] 2023-11-28T21:34:11.325 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCyXGZlYhztIBAAAyVPVgVpInoL6SE21F4c/w== 2023-11-28T21:34:11.326 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a] 2023-11-28T21:34:11.326 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCyXGZlPlU+KBAAFhXKx0p81FpAik+O9ZbD4Q== 2023-11-28T21:34:11.326 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c] 2023-11-28T21:34:11.326 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCyXGZlXrjKLxAA+RYGxhSJgM9Jt/7LottRMA== 2023-11-28T21:34:11.326 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e] 2023-11-28T21:34:11.326 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCyXGZlS6ZoNxAAjRNbem2ntxYIRJAOzCRcZQ== 2023-11-28T21:34:11.327 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.0] 2023-11-28T21:34:11.327 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC2XGZlGaLaJRAAEyIExij4XgmOKBbWZ0Tkmw== 2023-11-28T21:34:11.327 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.1] 2023-11-28T21:34:11.327 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC5XGZlFmXMLBAAdsIVG5fUkJP857CRkRwSvA== 2023-11-28T21:34:11.327 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.2] 2023-11-28T21:34:11.327 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC8XGZlBfKfNBAAUHmsIuzeH3XFjjTr5iYBKg== 2023-11-28T21:34:11.327 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.3] 2023-11-28T21:34:11.327 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQC/XGZlGjwLOxAA9FHLqKdnkM05zM3qtRASog== 2023-11-28T21:34:11.328 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.x] 2023-11-28T21:34:11.328 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCyXGZlyni7IxAALsJPntiCUUI1OH0Hz6S6Og== 2023-11-28T21:34:11.328 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b] 2023-11-28T21:34:11.328 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCyXGZlQ2daOxAAdiiuKOT+1SasBjzN8s/6Hg== 2023-11-28T21:34:11.328 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d] 2023-11-28T21:34:11.328 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCzXGZlj973BxAAaQM7kNVZ4wDLfHYrAxTzIw== 2023-11-28T21:34:11.328 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f] 2023-11-28T21:34:11.329 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCzXGZleqQJEBAAErOPB/VKcgNNm2JWlALM9Q== 2023-11-28T21:34:11.329 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.4] 2023-11-28T21:34:11.329 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDFXGZlx35eNRAAan4up9xQZByoQ5tqmyXumg== 2023-11-28T21:34:11.329 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.5] 2023-11-28T21:34:11.329 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDJXGZlojFcARAAystyiecnHs8tcmK2cZ9bpQ== 2023-11-28T21:34:11.329 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.6] 2023-11-28T21:34:11.329 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDMXGZlTeGnChAA28KZQO6/At3WiTR2O4hTOg== 2023-11-28T21:34:11.329 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.7] 2023-11-28T21:34:11.330 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDPXGZl5Q2RERAA/6TyM/MLwqL1lURzemqOOQ== 2023-11-28T21:34:11.330 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2023-11-28T21:34:11.330 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCzXGZl9igiFhAAzDLjXsTvm2GemM1+fOMupQ== 2023-11-28T21:34:11.330 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T21:34:11.372 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T21:34:11.429 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:11.433 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:11.490 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:11.494 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:11.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:11.573 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:11.627 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:11.631 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:11.698 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:11.702 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:11.784 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:11.788 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:11.873 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:11.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:11.928 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T21:34:11.932 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T21:34:11.999 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:12.003 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:12.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:12.094 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:12.174 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:12.178 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T21:34:12.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:12.270 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:12.322 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:12.326 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:12.398 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:12.402 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:12.484 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:12.488 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T21:34:12.568 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T21:34:12.572 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T21:34:12.653 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T21:34:12.654 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T21:34:12.674 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T21:34:12.793 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T21:34:12.814 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T21:34:12.828 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T21:34:12.933 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T21:34:12.946 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T21:34:12.999 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T21:34:13.098 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T21:34:13.110 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:34:13.116 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T21:34:13.159 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T21:34:13.159 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T21:34:13.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T21:34:13.168 INFO:tasks.ceph.mon.a:Started 2023-11-28T21:34:13.169 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T21:34:13.169 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T21:34:13.207 INFO:tasks.ceph.mon.b:Started 2023-11-28T21:34:13.207 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T21:34:13.208 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T21:34:13.211 INFO:tasks.ceph.mon.c:Started 2023-11-28T21:34:13.211 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T21:34:13.211 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T21:34:13.211 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T21:34:13.217 INFO:tasks.ceph.mgr.y:Started 2023-11-28T21:34:13.218 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T21:34:13.219 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T21:34:13.223 INFO:tasks.ceph.mgr.x:Started 2023-11-28T21:34:13.223 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T21:34:13.223 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T21:34:13.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.361+0000 7f5877832640 1 Processor -- start 2023-11-28T21:34:13.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.361+0000 7f5877832640 1 -- start start 2023-11-28T21:34:13.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f5877832640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:13.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f5877832640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f5877832640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58700c6350 con 0x7f587006a5b0 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f5877832640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58700c6b90 con 0x7f5870153ee0 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f5877832640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5870072410 con 0x7f5870070bd0 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f58755ce640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f58755ce640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33020/0 (socket says 172.21.15.12:33020) 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 learned_addr learned my addr 172.21.15.12:0/427144635 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 msgr2=0x7f587015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 msgr2=0x7f587015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T21:34:13.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f5875dcf640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 msgr2=0x7f5870071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T21:34:13.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f5875dcf640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 msgr2=0x7f5870071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:13.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:13.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.365+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T21:34:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.569+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.569+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 msgr2=0x7f587015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T21:34:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.569+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 msgr2=0x7f587015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:13.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.569+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:13.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.569+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T21:34:13.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.569+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:13.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.569+0000 7f5875dcf640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 msgr2=0x7f5870071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T21:34:13.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.569+0000 7f5875dcf640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 msgr2=0x7f5870071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:13.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.569+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:13.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.569+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T21:34:13.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.973+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:13.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.973+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 msgr2=0x7f587015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T21:34:13.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.973+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 msgr2=0x7f587015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:13.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.973+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:13.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.973+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T21:34:13.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.973+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:13.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.973+0000 7f5875dcf640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 msgr2=0x7f5870071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T21:34:13.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.973+0000 7f5875dcf640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 msgr2=0x7f5870071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:13.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.973+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:13.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:13.973+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T21:34:14.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:14.777+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:14.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:14.777+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 msgr2=0x7f587015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T21:34:14.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:14.777+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 msgr2=0x7f587015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:14.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:14.777+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:14.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:14.777+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:14.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:14.777+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T21:34:14.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:14.777+0000 7f5875dcf640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 msgr2=0x7f5870071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T21:34:14.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:14.777+0000 7f5875dcf640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 msgr2=0x7f5870071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:14.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:14.777+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:14.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:14.777+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T21:34:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5866ffd640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 msgr2=0x7f5870071030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T21:34:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5866ffd640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.144:3300/0 conn(0x7f5870070bd0 0x7f5870071030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5866ffd640 1 -- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 msgr2=0x7f587015a680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T21:34:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5866ffd640 1 --2- 172.21.15.12:0/427144635 >> v2:172.21.15.12:3300/0 conn(0x7f5870153ee0 0x7f587015a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5866ffd640 1 -- 172.21.15.12:0/427144635 >> v1:172.21.15.144:6789/0 conn(0x7f587006a5b0 legacy=0x7f58700c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5866ffd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5866ffd640 1 -- 172.21.15.12:0/427144635 --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58700c65d0 con 0x7f58540036c0 2023-11-28T21:34:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5866ffd640 1 -- 172.21.15.12:0/427144635 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58700c6b90 con 0x7f5854007360 2023-11-28T21:34:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5874dcd640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 msgr2=0x7f5854009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T21:34:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5874dcd640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 msgr2=0x7f5854009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:16.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.365+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T21:34:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.569+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.569+0000 7f5874dcd640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 msgr2=0x7f5854009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T21:34:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.569+0000 7f5874dcd640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 msgr2=0x7f5854009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.569+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:16.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.569+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-28T21:34:16.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.973+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:16.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.973+0000 7f5874dcd640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 msgr2=0x7f5854009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T21:34:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.973+0000 7f5874dcd640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 msgr2=0x7f5854009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.973+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:16.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:16.973+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-28T21:34:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:17.777+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:17.777+0000 7f5874dcd640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 msgr2=0x7f5854009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T21:34:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:17.777+0000 7f5874dcd640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 msgr2=0x7f5854009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:17.777+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:17.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:17.777+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-28T21:34:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428150044 0 0) 0x7f58700c65d0 con 0x7f58540036c0 2023-11-28T21:34:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f585c003530 con 0x7f58540036c0 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2599146463 0 0) 0x7f585c003530 con 0x7f58540036c0 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 msgr2=0x7f5854009820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f5854009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585400b040 con 0x7f58540036c0 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f586c00c570 con 0x7f58540036c0 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 >> v1:172.21.15.12:6789/0 conn(0x7f58540036c0 legacy=0x7f5854005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5875dcf640 1 -- 172.21.15.12:0/427144635 reap_dead start 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f585c0039f0 0x7f585c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f585c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f585c004e30 0x7f585c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58700c6b90 con 0x7f585c0039f0 2023-11-28T21:34:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0055f0 con 0x7f585c004e30 2023-11-28T21:34:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c00bed0 con 0x7f5854007360 2023-11-28T21:34:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f585c0039f0 0x7f585c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5875dcf640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f585c004e30 0x7f585c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f5874dcd640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f585c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 msgr2=0x7f585c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f585c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f585c004e30 msgr2=0x7f585c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f585c004e30 0x7f585c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.617+0000 7f58755ce640 1 -- 172.21.15.12:0/427144635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c003530 con 0x7f585c0039f0 2023-11-28T21:34:18.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f58755ce640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f585c0039f0 0x7f585c003df0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f586c00cfc0 tx=0x7f586c01d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff26d816a1ea702 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c0a6840 con 0x7f585c0039f0 2023-11-28T21:34:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c0aa070 con 0x7f585c0039f0 2023-11-28T21:34:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5867fff640 1 -- 172.21.15.12:0/427144635 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c0a6c00 con 0x7f585c0039f0 2023-11-28T21:34:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5877832640 1 -- 172.21.15.12:0/427144635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f585c0039f0 msgr2=0x7f585c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5877832640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f585c0039f0 0x7f585c003df0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f586c00cfc0 tx=0x7f586c01d990 comp rx=0 tx=0).stop 2023-11-28T21:34:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5877832640 1 -- 172.21.15.12:0/427144635 shutdown_connections 2023-11-28T21:34:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5877832640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5854007360 0x7f585c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5877832640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f585c0039f0 0x7f585c003df0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5877832640 1 --2- 172.21.15.12:0/427144635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f585c004e30 0x7f585c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5877832640 1 -- 172.21.15.12:0/427144635 >> 172.21.15.12:0/427144635 conn(0x7f58700c8450 msgr2=0x7f58700c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5877832640 1 -- 172.21.15.12:0/427144635 shutdown_connections 2023-11-28T21:34:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.621+0000 7f5877832640 1 -- 172.21.15.12:0/427144635 wait complete. 2023-11-28T21:34:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5877832640 1 Processor -- start 2023-11-28T21:34:18.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5877832640 1 -- start start 2023-11-28T21:34:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5877832640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5854007360 0x7f58700c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:18.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5877832640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f585c0039f0 0x7f58700c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5877832640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f585c004e30 0x7f587014c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5877832640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5870150fe0 con 0x7f5854007360 2023-11-28T21:34:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5877832640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58701f0a40 con 0x7f585c0039f0 2023-11-28T21:34:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5877832640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58700c2cd0 con 0x7f585c004e30 2023-11-28T21:34:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f58755ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5854007360 0x7f58700c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f58755ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5854007360 0x7f58700c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37568/0 (socket says 172.21.15.12:37568) 2023-11-28T21:34:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f58755ce640 1 -- 172.21.15.12:0/3715545457 learned_addr learned my addr 172.21.15.12:0/3715545457 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5874dcd640 1 --2- 172.21.15.12:0/3715545457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f585c0039f0 0x7f58700c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:18.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5875dcf640 1 --2- 172.21.15.12:0/3715545457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f585c004e30 0x7f587014c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f58755ce640 1 -- 172.21.15.12:0/3715545457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f585c004e30 msgr2=0x7f587014c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f58755ce640 1 --2- 172.21.15.12:0/3715545457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f585c004e30 0x7f587014c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f58755ce640 1 -- 172.21.15.12:0/3715545457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f585c0039f0 msgr2=0x7f58700c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f58755ce640 1 --2- 172.21.15.12:0/3715545457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f585c0039f0 0x7f58700c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f58755ce640 1 -- 172.21.15.12:0/3715545457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587014ddc0 con 0x7f5854007360 2023-11-28T21:34:18.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.625+0000 7f5875dcf640 1 --2- 172.21.15.12:0/3715545457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f585c004e30 0x7f587014c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:18.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.629+0000 7f58755ce640 1 --2- 172.21.15.12:0/3715545457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5854007360 0x7f58700c5250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f586c01de70 tx=0x7f585c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:18.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.629+0000 7f5865ffb640 1 -- 172.21.15.12:0/3715545457 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c0a6980 con 0x7f5854007360 2023-11-28T21:34:18.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.629+0000 7f5865ffb640 1 -- 172.21.15.12:0/3715545457 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c00c040 con 0x7f5854007360 2023-11-28T21:34:18.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.629+0000 7f5865ffb640 1 -- 172.21.15.12:0/3715545457 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c0a6ce0 con 0x7f5854007360 2023-11-28T21:34:18.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.629+0000 7f5877832640 1 -- 172.21.15.12:0/3715545457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58701f0780 con 0x7f5854007360 2023-11-28T21:34:18.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.629+0000 7f5877832640 1 -- 172.21.15.12:0/3715545457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58700c2f20 con 0x7f5854007360 2023-11-28T21:34:18.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.629+0000 7f5865ffb640 1 -- 172.21.15.12:0/3715545457 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f586c0bb770 con 0x7f5854007360 2023-11-28T21:34:18.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.629+0000 7f5865ffb640 1 -- 172.21.15.12:0/3715545457 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f586c0c4540 con 0x7f5854007360 2023-11-28T21:34:18.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.629+0000 7f5877832640 1 -- 172.21.15.12:0/3715545457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58700704c0 con 0x7f5854007360 2023-11-28T21:34:18.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.649+0000 7f5865ffb640 1 -- 172.21.15.12:0/3715545457 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f586c0bf020 con 0x7f5854007360 2023-11-28T21:34:18.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.713+0000 7f5877832640 1 -- 172.21.15.12:0/3715545457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5870074f60 con 0x7f5854007360 2023-11-28T21:34:18.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.717+0000 7f5865ffb640 1 -- 172.21.15.12:0/3715545457 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f586c0ac070 con 0x7f5854007360 2023-11-28T21:34:18.721 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2023-11-28T21:34:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.717+0000 7f5877832640 1 -- 172.21.15.12:0/3715545457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5854007360 msgr2=0x7f58700c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.717+0000 7f5877832640 1 --2- 172.21.15.12:0/3715545457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5854007360 0x7f58700c5250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f586c01de70 tx=0x7f585c005800 comp rx=0 tx=0).stop 2023-11-28T21:34:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.717+0000 7f5877832640 1 -- 172.21.15.12:0/3715545457 shutdown_connections 2023-11-28T21:34:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.717+0000 7f5877832640 1 --2- 172.21.15.12:0/3715545457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f585c004e30 0x7f587014c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.717+0000 7f5877832640 1 --2- 172.21.15.12:0/3715545457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f585c0039f0 0x7f58700c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.717+0000 7f5877832640 1 --2- 172.21.15.12:0/3715545457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5854007360 0x7f58700c5250 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.717+0000 7f5877832640 1 -- 172.21.15.12:0/3715545457 >> 172.21.15.12:0/3715545457 conn(0x7f58700c8450 msgr2=0x7f5870070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:18.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.717+0000 7f5877832640 1 -- 172.21.15.12:0/3715545457 shutdown_connections 2023-11-28T21:34:18.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:18.717+0000 7f5877832640 1 -- 172.21.15.12:0/3715545457 wait complete. 2023-11-28T21:34:18.742 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T21:34:18.743 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T21:34:18.743 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:18.743 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T21:34:18.766 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:18.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T21:34:18.834 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:18.835 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T21:34:18.841 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:18.837+0000 7f17cf036180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T21:34:18.865 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:18.865 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T21:34:18.935 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:18.935 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T21:34:18.949 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:18.950 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T21:34:19.003 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:19.004 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T21:34:19.060 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-11-28T21:34:19.061 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T21:34:19.116 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 62cf2d46-0ca9-441b-9e65-70b06b579b20 0 2023-11-28T21:34:19.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb09e60a640 1 Processor -- start 2023-11-28T21:34:19.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb09e60a640 1 -- start start 2023-11-28T21:34:19.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb09e60a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb098151ed0 0x7fb0980c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb09e60a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09806e300 0x7fb098072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb09e60a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb098157b70 con 0x7fb098151af0 2023-11-28T21:34:19.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb09e60a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0980707f0 con 0x7fb098151ed0 2023-11-28T21:34:19.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb09e60a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0980c7b90 con 0x7fb09806e300 2023-11-28T21:34:19.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb097fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb098151af0 0x7fb0980c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:49286/0 (socket says 172.21.15.144:49286) 2023-11-28T21:34:19.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb097fff640 1 -- 172.21.15.144:0/1167305302 learned_addr learned my addr 172.21.15.144:0/1167305302 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T21:34:19.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb0977fe640 1 --2- 172.21.15.144:0/1167305302 >> v2:172.21.15.144:3300/0 conn(0x7fb098151ed0 0x7fb0980c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb09cba7640 1 --2- 172.21.15.144:0/1167305302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09806e300 0x7fb098072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.288 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb096ffd640 1 -- 172.21.15.144:0/1167305302 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227465295 0 0) 0x7fb098157b70 con 0x7fb098151af0 2023-11-28T21:34:19.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.284+0000 7fb096ffd640 1 -- 172.21.15.144:0/1167305302 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb080003530 con 0x7fb098151af0 2023-11-28T21:34:19.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb0977fe640 1 -- 172.21.15.144:0/1167305302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09806e300 msgr2=0x7fb098072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:19.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb0977fe640 1 --2- 172.21.15.144:0/1167305302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09806e300 0x7fb098072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb0977fe640 1 -- 172.21.15.144:0/1167305302 >> v1:172.21.15.144:6789/0 conn(0x7fb098151af0 legacy=0x7fb0980c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb0977fe640 1 -- 172.21.15.144:0/1167305302 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb098157e50 con 0x7fb098151ed0 2023-11-28T21:34:19.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb09cba7640 1 --2- 172.21.15.144:0/1167305302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09806e300 0x7fb098072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:34:19.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb0977fe640 1 --2- 172.21.15.144:0/1167305302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb098151ed0 0x7fb0980c7650 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0097c0 tx=0x7fb08c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b78ed89d283ac51 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:19.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb096ffd640 1 -- 172.21.15.144:0/1167305302 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0ab600 con 0x7fb098151ed0 2023-11-28T21:34:19.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb096ffd640 1 -- 172.21.15.144:0/1167305302 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c0a7050 con 0x7fb098151ed0 2023-11-28T21:34:19.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb096ffd640 1 -- 172.21.15.144:0/1167305302 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0ab980 con 0x7fb098151ed0 2023-11-28T21:34:19.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb09e60a640 1 -- 172.21.15.144:0/1167305302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb098151ed0 msgr2=0x7fb0980c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb09e60a640 1 --2- 172.21.15.144:0/1167305302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb098151ed0 0x7fb0980c7650 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0097c0 tx=0x7fb08c01c480 comp rx=0 tx=0).stop 2023-11-28T21:34:19.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb09e60a640 1 -- 172.21.15.144:0/1167305302 shutdown_connections 2023-11-28T21:34:19.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb09e60a640 1 --2- 172.21.15.144:0/1167305302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb09806e300 0x7fb098072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb09e60a640 1 --2- 172.21.15.144:0/1167305302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb098151ed0 0x7fb0980c7650 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb09e60a640 1 -- 172.21.15.144:0/1167305302 >> 172.21.15.144:0/1167305302 conn(0x7fb098066360 msgr2=0x7fb098066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:19.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb09e60a640 1 -- 172.21.15.144:0/1167305302 shutdown_connections 2023-11-28T21:34:19.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.288+0000 7fb09e60a640 1 -- 172.21.15.144:0/1167305302 wait complete. 2023-11-28T21:34:19.292 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 Processor -- start 2023-11-28T21:34:19.292 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 -- start start 2023-11-28T21:34:19.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb09806e300 0x7fb0981408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb098151af0 0x7fb098155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0981525a0 0x7fb098155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981f6dc0 con 0x7fb0981525a0 2023-11-28T21:34:19.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098156e30 con 0x7fb09806e300 2023-11-28T21:34:19.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0981f61b0 con 0x7fb098151af0 2023-11-28T21:34:19.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb097fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb09806e300 0x7fb0981408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb0977fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb098151af0 0x7fb098155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb097fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb09806e300 0x7fb0981408d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55042/0 (socket says 172.21.15.144:55042) 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb097fff640 1 -- 172.21.15.144:0/3234157667 learned_addr learned my addr 172.21.15.144:0/3234157667 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09cba7640 1 --2- 172.21.15.144:0/3234157667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0981525a0 0x7fb098155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb097fff640 1 -- 172.21.15.144:0/3234157667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb098151af0 msgr2=0x7fb098155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb097fff640 1 --2- 172.21.15.144:0/3234157667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb098151af0 0x7fb098155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb097fff640 1 -- 172.21.15.144:0/3234157667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0981525a0 msgr2=0x7fb098155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb097fff640 1 --2- 172.21.15.144:0/3234157667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0981525a0 0x7fb098155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb097fff640 1 -- 172.21.15.144:0/3234157667 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090009a70 con 0x7fb09806e300 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09cba7640 1 --2- 172.21.15.144:0/3234157667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0981525a0 0x7fb098155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb097fff640 1 --2- 172.21.15.144:0/3234157667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb09806e300 0x7fb0981408d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb08800c500 tx=0x7fb08800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb094ff9640 1 -- 172.21.15.144:0/3234157667 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088005cf0 con 0x7fb09806e300 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb094ff9640 1 -- 172.21.15.144:0/3234157667 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb088005e50 con 0x7fb09806e300 2023-11-28T21:34:19.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 -- 172.21.15.144:0/3234157667 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08c009420 con 0x7fb09806e300 2023-11-28T21:34:19.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb094ff9640 1 -- 172.21.15.144:0/3234157667 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0880072f0 con 0x7fb09806e300 2023-11-28T21:34:19.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 -- 172.21.15.144:0/3234157667 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0981f6700 con 0x7fb09806e300 2023-11-28T21:34:19.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb094ff9640 1 -- 172.21.15.144:0/3234157667 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb08801c440 con 0x7fb09806e300 2023-11-28T21:34:19.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb094ff9640 1 -- 172.21.15.144:0/3234157667 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb088012780 con 0x7fb09806e300 2023-11-28T21:34:19.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.292+0000 7fb09e60a640 1 -- 172.21.15.144:0/3234157667 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0981f6f00 con 0x7fb09806e300 2023-11-28T21:34:19.309 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.308+0000 7fb094ff9640 1 -- 172.21.15.144:0/3234157667 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb0981f6f00 con 0x7fb09806e300 2023-11-28T21:34:19.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.376+0000 7fb09e60a640 1 -- 172.21.15.144:0/3234157667 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "62cf2d46-0ca9-441b-9e65-70b06b579b20", "id": 0} v 0) v1 -- 0x7fb098074f60 con 0x7fb09806e300 2023-11-28T21:34:19.383 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.380+0000 7fb094ff9640 1 -- 172.21.15.144:0/3234157667 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "62cf2d46-0ca9-441b-9e65-70b06b579b20", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb088021060 con 0x7fb09806e300 2023-11-28T21:34:19.383 INFO:teuthology.orchestra.run.smithi144.stdout:0 2023-11-28T21:34:19.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.384+0000 7fb09e60a640 1 -- 172.21.15.144:0/3234157667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb09806e300 msgr2=0x7fb0981408d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.384+0000 7fb09e60a640 1 --2- 172.21.15.144:0/3234157667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb09806e300 0x7fb0981408d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb08800c500 tx=0x7fb08800c9c0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.384 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.384+0000 7fb09e60a640 1 -- 172.21.15.144:0/3234157667 shutdown_connections 2023-11-28T21:34:19.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.384+0000 7fb09e60a640 1 --2- 172.21.15.144:0/3234157667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0981525a0 0x7fb098155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.384+0000 7fb09e60a640 1 --2- 172.21.15.144:0/3234157667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb098151af0 0x7fb098155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.384+0000 7fb09e60a640 1 --2- 172.21.15.144:0/3234157667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb09806e300 0x7fb0981408d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.384+0000 7fb09e60a640 1 -- 172.21.15.144:0/3234157667 >> 172.21.15.144:0/3234157667 conn(0x7fb098066360 msgr2=0x7fb0980c71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:19.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.384+0000 7fb09e60a640 1 -- 172.21.15.144:0/3234157667 shutdown_connections 2023-11-28T21:34:19.385 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.384+0000 7fb09e60a640 1 -- 172.21.15.144:0/3234157667 wait complete. 2023-11-28T21:34:19.405 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:19.401+0000 7f17cf036180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T21:34:19.406 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new df411803-82fc-4bb1-b849-419f56db865d 1 2023-11-28T21:34:19.509 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:19.508+0000 7febf02d9180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T21:34:19.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.520+0000 7fe315b2f640 1 Processor -- start 2023-11-28T21:34:19.523 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.520+0000 7fe315b2f640 1 -- start start 2023-11-28T21:34:19.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe315b2f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 0x7fe310072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe315b2f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe3100c8540 0x7fe31006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe315b2f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3100c9160 con 0x7fe3100702d0 2023-11-28T21:34:19.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe315b2f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3100c9d60 con 0x7fe3100c8540 2023-11-28T21:34:19.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe315b2f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100c8aa0 con 0x7fe310153e20 2023-11-28T21:34:19.524 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30effd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe3100c8540 0x7fe31006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30f7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 0x7fe310072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30ffff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe3100702d0 0x7fe310159c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:49298/0 (socket says 172.21.15.144:49298) 2023-11-28T21:34:19.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30ffff640 1 -- 172.21.15.144:0/159295389 learned_addr learned my addr 172.21.15.144:0/159295389 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T21:34:19.525 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30f7fe640 1 -- 172.21.15.144:0/159295389 >> v2:172.21.15.144:3300/0 conn(0x7fe3100c8540 msgr2=0x7fe31006b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:19.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30f7fe640 1 --2- 172.21.15.144:0/159295389 >> v2:172.21.15.144:3300/0 conn(0x7fe3100c8540 0x7fe31006b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30f7fe640 1 -- 172.21.15.144:0/159295389 >> v1:172.21.15.144:6789/0 conn(0x7fe3100702d0 legacy=0x7fe310159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30e7fc640 1 -- 172.21.15.144:0/159295389 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744171383 0 0) 0x7fe3100c9160 con 0x7fe3100702d0 2023-11-28T21:34:19.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30f7fe640 1 -- 172.21.15.144:0/159295389 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100c8e30 con 0x7fe310153e20 2023-11-28T21:34:19.526 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30effd640 1 --2- 172.21.15.144:0/159295389 >> v2:172.21.15.144:3300/0 conn(0x7fe3100c8540 0x7fe31006b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:34:19.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30f7fe640 1 --2- 172.21.15.144:0/159295389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 0x7fe310072610 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3000097f0 tx=0x7fe30001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5798158d24ff03da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:19.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30e7fc640 1 -- 172.21.15.144:0/159295389 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000a8970 con 0x7fe310153e20 2023-11-28T21:34:19.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30e7fc640 1 -- 172.21.15.144:0/159295389 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3000a8ad0 con 0x7fe310153e20 2023-11-28T21:34:19.527 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe30e7fc640 1 -- 172.21.15.144:0/159295389 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000a8e60 con 0x7fe310153e20 2023-11-28T21:34:19.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe315b2f640 1 -- 172.21.15.144:0/159295389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 msgr2=0x7fe310072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.524+0000 7fe315b2f640 1 --2- 172.21.15.144:0/159295389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 0x7fe310072610 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3000097f0 tx=0x7fe30001ec70 comp rx=0 tx=0).stop 2023-11-28T21:34:19.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 -- 172.21.15.144:0/159295389 shutdown_connections 2023-11-28T21:34:19.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 --2- 172.21.15.144:0/159295389 >> v2:172.21.15.144:3300/0 conn(0x7fe3100c8540 0x7fe31006b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 --2- 172.21.15.144:0/159295389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 0x7fe310072610 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 -- 172.21.15.144:0/159295389 >> 172.21.15.144:0/159295389 conn(0x7fe3100cd3e0 msgr2=0x7fe3100cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:19.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 -- 172.21.15.144:0/159295389 shutdown_connections 2023-11-28T21:34:19.528 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 -- 172.21.15.144:0/159295389 wait complete. 2023-11-28T21:34:19.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 Processor -- start 2023-11-28T21:34:19.529 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 -- start start 2023-11-28T21:34:19.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3100702d0 0x7fe31014e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100c8540 0x7fe31015d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.531 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 0x7fe310148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe310158f30 con 0x7fe3100702d0 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3101583a0 con 0x7fe3100c8540 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31015bde0 con 0x7fe310153e20 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100c8540 0x7fe31015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30ffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 0x7fe310148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100c8540 0x7fe31015d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55092/0 (socket says 172.21.15.144:55092) 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30effd640 1 -- 172.21.15.144:0/2471868873 learned_addr learned my addr 172.21.15.144:0/2471868873 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30f7fe640 1 --2- 172.21.15.144:0/2471868873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3100702d0 0x7fe31014e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30effd640 1 -- 172.21.15.144:0/2471868873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 msgr2=0x7fe310148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30effd640 1 --2- 172.21.15.144:0/2471868873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 0x7fe310148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.532 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30effd640 1 -- 172.21.15.144:0/2471868873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3100702d0 msgr2=0x7fe31014e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30effd640 1 --2- 172.21.15.144:0/2471868873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3100702d0 0x7fe31014e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30effd640 1 -- 172.21.15.144:0/2471868873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe304009420 con 0x7fe3100c8540 2023-11-28T21:34:19.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30ffff640 1 --2- 172.21.15.144:0/2471868873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 0x7fe310148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:34:19.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30f7fe640 1 --2- 172.21.15.144:0/2471868873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3100702d0 0x7fe31014e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:19.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe30effd640 1 --2- 172.21.15.144:0/2471868873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100c8540 0x7fe31015d980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe30400ebe0 tx=0x7fe30400c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:19.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe2f7fff640 1 -- 172.21.15.144:0/2471868873 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30400eee0 con 0x7fe3100c8540 2023-11-28T21:34:19.533 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe2f7fff640 1 -- 172.21.15.144:0/2471868873 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30400cc30 con 0x7fe3100c8540 2023-11-28T21:34:19.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe2f7fff640 1 -- 172.21.15.144:0/2471868873 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304013470 con 0x7fe3100c8540 2023-11-28T21:34:19.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 -- 172.21.15.144:0/2471868873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe300009420 con 0x7fe3100c8540 2023-11-28T21:34:19.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe315b2f640 1 -- 172.21.15.144:0/2471868873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31015c360 con 0x7fe3100c8540 2023-11-28T21:34:19.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe2f7fff640 1 -- 172.21.15.144:0/2471868873 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe304005ca0 con 0x7fe3100c8540 2023-11-28T21:34:19.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.528+0000 7fe2f7fff640 1 -- 172.21.15.144:0/2471868873 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fe304013980 con 0x7fe3100c8540 2023-11-28T21:34:19.534 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.532+0000 7fe315b2f640 1 -- 172.21.15.144:0/2471868873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe310074f60 con 0x7fe3100c8540 2023-11-28T21:34:19.535 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.532+0000 7fe2f7fff640 1 -- 172.21.15.144:0/2471868873 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fe304021060 con 0x7fe3100c8540 2023-11-28T21:34:19.603 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.600+0000 7fe315b2f640 1 -- 172.21.15.144:0/2471868873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "df411803-82fc-4bb1-b849-419f56db865d", "id": 1} v 0) v1 -- 0x7fe310072d50 con 0x7fe3100c8540 2023-11-28T21:34:19.611 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.608+0000 7fe2f7fff640 1 -- 172.21.15.144:0/2471868873 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df411803-82fc-4bb1-b849-419f56db865d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe304017070 con 0x7fe3100c8540 2023-11-28T21:34:19.611 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-11-28T21:34:19.612 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.612+0000 7fe315b2f640 1 -- 172.21.15.144:0/2471868873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100c8540 msgr2=0x7fe31015d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.612+0000 7fe315b2f640 1 --2- 172.21.15.144:0/2471868873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100c8540 0x7fe31015d980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe30400ebe0 tx=0x7fe30400c520 comp rx=0 tx=0).stop 2023-11-28T21:34:19.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.612+0000 7fe315b2f640 1 -- 172.21.15.144:0/2471868873 shutdown_connections 2023-11-28T21:34:19.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.612+0000 7fe315b2f640 1 --2- 172.21.15.144:0/2471868873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310153e20 0x7fe310148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.612+0000 7fe315b2f640 1 --2- 172.21.15.144:0/2471868873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3100c8540 0x7fe31015d980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.612+0000 7fe315b2f640 1 --2- 172.21.15.144:0/2471868873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3100702d0 0x7fe31014e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.612+0000 7fe315b2f640 1 -- 172.21.15.144:0/2471868873 >> 172.21.15.144:0/2471868873 conn(0x7fe3100cd3e0 msgr2=0x7fe310073c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:19.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.612+0000 7fe315b2f640 1 -- 172.21.15.144:0/2471868873 shutdown_connections 2023-11-28T21:34:19.613 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.612+0000 7fe315b2f640 1 -- 172.21.15.144:0/2471868873 wait complete. 2023-11-28T21:34:19.613 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:19.609+0000 7f17cf036180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T21:34:19.629 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 5a9299bc-9381-48fe-bf4b-5f105fffd914 2 2023-11-28T21:34:19.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6438884640 1 Processor -- start 2023-11-28T21:34:19.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6438884640 1 -- start start 2023-11-28T21:34:19.744 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6438884640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64340c7fd0 0x7f64340724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6438884640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64340c8480 con 0x7f6434152230 2023-11-28T21:34:19.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6438884640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64340c0e70 con 0x7f6434157cd0 2023-11-28T21:34:19.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6438884640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643415b770 con 0x7f64340c7fd0 2023-11-28T21:34:19.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f642bfff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f6434157cd0 0x7f64340b1910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:49306/0 (socket says 172.21.15.144:49306) 2023-11-28T21:34:19.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6432d9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64340c7fd0 0x7f64340724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.745 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f642bfff640 1 -- 172.21.15.144:0/3038474247 learned_addr learned my addr 172.21.15.144:0/3038474247 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T21:34:19.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6432d9d640 1 -- 172.21.15.144:0/3038474247 >> v1:172.21.15.144:6789/0 conn(0x7f6434157cd0 legacy=0x7f64340b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6432d9d640 1 -- 172.21.15.144:0/3038474247 >> v1:172.21.15.12:6789/0 conn(0x7f6434152230 legacy=0x7f643415bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.746 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6432d9d640 1 -- 172.21.15.144:0/3038474247 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64340c1680 con 0x7f64340c7fd0 2023-11-28T21:34:19.747 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6432d9d640 1 --2- 172.21.15.144:0/3038474247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64340c7fd0 0x7f64340724e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f642c009dc0 tx=0x7f642c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ef1d8cdf3f70ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:19.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.744+0000 7f6431d9b640 1 -- 172.21.15.144:0/3038474247 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0a58b0 con 0x7f64340c7fd0 2023-11-28T21:34:19.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6431d9b640 1 -- 172.21.15.144:0/3038474247 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c0a5a10 con 0x7f64340c7fd0 2023-11-28T21:34:19.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6431d9b640 1 -- 172.21.15.144:0/3038474247 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0a5da0 con 0x7f64340c7fd0 2023-11-28T21:34:19.748 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- 172.21.15.144:0/3038474247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64340c7fd0 msgr2=0x7f64340724e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 --2- 172.21.15.144:0/3038474247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64340c7fd0 0x7f64340724e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f642c009dc0 tx=0x7f642c01c8e0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- 172.21.15.144:0/3038474247 shutdown_connections 2023-11-28T21:34:19.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 --2- 172.21.15.144:0/3038474247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64340c7fd0 0x7f64340724e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- 172.21.15.144:0/3038474247 >> 172.21.15.144:0/3038474247 conn(0x7f64340cbb80 msgr2=0x7f64340cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:19.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- 172.21.15.144:0/3038474247 shutdown_connections 2023-11-28T21:34:19.749 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- 172.21.15.144:0/3038474247 wait complete. 2023-11-28T21:34:19.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 Processor -- start 2023-11-28T21:34:19.750 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- start start 2023-11-28T21:34:19.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64340c7fd0 0x7f64341f1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6434152230 0x7f643414c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6434157cd0 0x7f6434155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643414b170 con 0x7f6434152230 2023-11-28T21:34:19.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643414b3a0 con 0x7f64340c7fd0 2023-11-28T21:34:19.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643414b640 con 0x7f6434157cd0 2023-11-28T21:34:19.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6432d9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6434157cd0 0x7f6434155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.752 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f643259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64340c7fd0 0x7f64341f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6432d9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6434157cd0 0x7f6434155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:51876/0 (socket says 172.21.15.144:51876) 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6432d9d640 1 -- 172.21.15.144:0/635785778 learned_addr learned my addr 172.21.15.144:0/635785778 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f642bfff640 1 --2- 172.21.15.144:0/635785778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6434152230 0x7f643414c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6432d9d640 1 -- 172.21.15.144:0/635785778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64340c7fd0 msgr2=0x7f64341f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6432d9d640 1 --2- 172.21.15.144:0/635785778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64340c7fd0 0x7f64341f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6432d9d640 1 -- 172.21.15.144:0/635785778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6434152230 msgr2=0x7f643414c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6432d9d640 1 --2- 172.21.15.144:0/635785778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6434152230 0x7f643414c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6432d9d640 1 -- 172.21.15.144:0/635785778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c009a70 con 0x7f6434157cd0 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6432d9d640 1 --2- 172.21.15.144:0/635785778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6434157cd0 0x7f6434155300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f642c01cdc0 tx=0x7f642c019680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f642affd640 1 -- 172.21.15.144:0/635785778 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0b9860 con 0x7f6434157cd0 2023-11-28T21:34:19.753 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f642affd640 1 -- 172.21.15.144:0/635785778 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c0b99e0 con 0x7f6434157cd0 2023-11-28T21:34:19.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- 172.21.15.144:0/635785778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643414cbe0 con 0x7f6434157cd0 2023-11-28T21:34:19.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f642affd640 1 -- 172.21.15.144:0/635785778 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0b9d90 con 0x7f6434157cd0 2023-11-28T21:34:19.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f6438884640 1 -- 172.21.15.144:0/635785778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643414bbc0 con 0x7f6434157cd0 2023-11-28T21:34:19.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.748+0000 7f642affd640 1 -- 172.21.15.144:0/635785778 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f642c0ad740 con 0x7f6434157cd0 2023-11-28T21:34:19.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.752+0000 7f642affd640 1 -- 172.21.15.144:0/635785778 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f642c0c2b40 con 0x7f6434157cd0 2023-11-28T21:34:19.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.752+0000 7f6438884640 1 -- 172.21.15.144:0/635785778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6434074f60 con 0x7f6434157cd0 2023-11-28T21:34:19.754 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.752+0000 7f642affd640 1 -- 172.21.15.144:0/635785778 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f642c0bd020 con 0x7f6434157cd0 2023-11-28T21:34:19.818 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.816+0000 7f6438884640 1 -- 172.21.15.144:0/635785778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5a9299bc-9381-48fe-bf4b-5f105fffd914", "id": 2} v 0) v1 -- 0x7f64340cb2c0 con 0x7f6434157cd0 2023-11-28T21:34:19.825 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.824+0000 7f642affd640 1 -- 172.21.15.144:0/635785778 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a9299bc-9381-48fe-bf4b-5f105fffd914", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f642c0b2070 con 0x7f6434157cd0 2023-11-28T21:34:19.826 INFO:teuthology.orchestra.run.smithi144.stdout:2 2023-11-28T21:34:19.827 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.824+0000 7f6438884640 1 -- 172.21.15.144:0/635785778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6434157cd0 msgr2=0x7f6434155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.827 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.824+0000 7f6438884640 1 --2- 172.21.15.144:0/635785778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6434157cd0 0x7f6434155300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f642c01cdc0 tx=0x7f642c019680 comp rx=0 tx=0).stop 2023-11-28T21:34:19.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.824+0000 7f6438884640 1 -- 172.21.15.144:0/635785778 shutdown_connections 2023-11-28T21:34:19.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.824+0000 7f6438884640 1 --2- 172.21.15.144:0/635785778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6434157cd0 0x7f6434155300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.824+0000 7f6438884640 1 --2- 172.21.15.144:0/635785778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6434152230 0x7f643414c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.828 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.824+0000 7f6438884640 1 --2- 172.21.15.144:0/635785778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64340c7fd0 0x7f64341f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.824+0000 7f6438884640 1 -- 172.21.15.144:0/635785778 >> 172.21.15.144:0/635785778 conn(0x7f64340cbb80 msgr2=0x7f64340ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:19.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.824+0000 7f6438884640 1 -- 172.21.15.144:0/635785778 shutdown_connections 2023-11-28T21:34:19.829 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.824+0000 7f6438884640 1 -- 172.21.15.144:0/635785778 wait complete. 2023-11-28T21:34:19.847 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new ad45d502-66d6-4abb-b218-a53b0518dc04 3 2023-11-28T21:34:19.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61d12c6640 1 Processor -- start 2023-11-28T21:34:19.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61d12c6640 1 -- start start 2023-11-28T21:34:19.967 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61d12c6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc151de0 0x7f61cc072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61d12c6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f61cc0c4ff0 0x7f61cc15b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61d12c6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61cc1589a0 con 0x7f61cc0c9690 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61d12c6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61cc1521c0 con 0x7f61cc0c4ff0 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61d12c6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc0c9c10 con 0x7f61cc151de0 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61cb7fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f61cc0c4ff0 0x7f61cc15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61ca7fc640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f61cc0c9690 0x7f61cc0c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:49322/0 (socket says 172.21.15.144:49322) 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61ca7fc640 1 -- 172.21.15.144:0/3108171296 learned_addr learned my addr 172.21.15.144:0/3108171296 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.964+0000 7f61caffd640 1 --2- 172.21.15.144:0/3108171296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc151de0 0x7f61cc072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61cb7fe640 1 -- 172.21.15.144:0/3108171296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc151de0 msgr2=0x7f61cc072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61cb7fe640 1 --2- 172.21.15.144:0/3108171296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc151de0 0x7f61cc072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.968 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61cb7fe640 1 -- 172.21.15.144:0/3108171296 >> v1:172.21.15.144:6789/0 conn(0x7f61cc0c9690 legacy=0x7f61cc0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61cb7fe640 1 -- 172.21.15.144:0/3108171296 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc157d90 con 0x7f61cc0c4ff0 2023-11-28T21:34:19.969 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61caffd640 1 --2- 172.21.15.144:0/3108171296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc151de0 0x7f61cc072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:34:19.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61cb7fe640 1 --2- 172.21.15.144:0/3108171296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc0c4ff0 0x7f61cc15b310 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f61b40097c0 tx=0x7f61b401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae4619ca3faff67 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:19.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61c9ffb640 1 -- 172.21.15.144:0/3108171296 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40a8970 con 0x7f61cc0c4ff0 2023-11-28T21:34:19.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61c9ffb640 1 -- 172.21.15.144:0/3108171296 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b40a8ad0 con 0x7f61cc0c4ff0 2023-11-28T21:34:19.970 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61c9ffb640 1 -- 172.21.15.144:0/3108171296 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40a8e60 con 0x7f61cc0c4ff0 2023-11-28T21:34:19.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61d12c6640 1 -- 172.21.15.144:0/3108171296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc0c4ff0 msgr2=0x7f61cc15b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61d12c6640 1 --2- 172.21.15.144:0/3108171296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc0c4ff0 0x7f61cc15b310 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f61b40097c0 tx=0x7f61b401ec70 comp rx=0 tx=0).stop 2023-11-28T21:34:19.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61d12c6640 1 -- 172.21.15.144:0/3108171296 shutdown_connections 2023-11-28T21:34:19.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61d12c6640 1 --2- 172.21.15.144:0/3108171296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc0c4ff0 0x7f61cc15b310 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61d12c6640 1 --2- 172.21.15.144:0/3108171296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc151de0 0x7f61cc072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61d12c6640 1 -- 172.21.15.144:0/3108171296 >> 172.21.15.144:0/3108171296 conn(0x7f61cc0cd3e0 msgr2=0x7f61cc0cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:19.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61d12c6640 1 -- 172.21.15.144:0/3108171296 shutdown_connections 2023-11-28T21:34:19.971 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.968+0000 7f61d12c6640 1 -- 172.21.15.144:0/3108171296 wait complete. 2023-11-28T21:34:19.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61d12c6640 1 Processor -- start 2023-11-28T21:34:19.973 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61d12c6640 1 -- start start 2023-11-28T21:34:19.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61d12c6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61cc0c4ff0 0x7f61cc150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61d12c6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc0c9690 0x7f61cc1ed920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61d12c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc151de0 0x7f61cc0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61d12c6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc14caa0 con 0x7f61cc0c4ff0 2023-11-28T21:34:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61d12c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc150500 con 0x7f61cc151de0 2023-11-28T21:34:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61d12c6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc14ff20 con 0x7f61cc0c9690 2023-11-28T21:34:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61ca7fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc0c9690 0x7f61cc1ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61cb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc151de0 0x7f61cc0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61cb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc151de0 0x7f61cc0cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55140/0 (socket says 172.21.15.144:55140) 2023-11-28T21:34:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61cb7fe640 1 -- 172.21.15.144:0/1406843868 learned_addr learned my addr 172.21.15.144:0/1406843868 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T21:34:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61ca7fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc0c9690 0x7f61cc1ed920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:51886/0 (socket says 172.21.15.144:51886) 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61caffd640 1 --2- 172.21.15.144:0/1406843868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61cc0c4ff0 0x7f61cc150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61cb7fe640 1 -- 172.21.15.144:0/1406843868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc0c9690 msgr2=0x7f61cc1ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61cb7fe640 1 --2- 172.21.15.144:0/1406843868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc0c9690 0x7f61cc1ed920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61cb7fe640 1 -- 172.21.15.144:0/1406843868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61cc0c4ff0 msgr2=0x7f61cc150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61cb7fe640 1 --2- 172.21.15.144:0/1406843868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61cc0c4ff0 0x7f61cc150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61cb7fe640 1 -- 172.21.15.144:0/1406843868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4009a70 con 0x7f61cc151de0 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61ca7fc640 1 --2- 172.21.15.144:0/1406843868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc0c9690 0x7f61cc1ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61cb7fe640 1 --2- 172.21.15.144:0/1406843868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc151de0 0x7f61cc0cc400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61b401c8a0 tx=0x7f61b401ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61c37fe640 1 -- 172.21.15.144:0/1406843868 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40b0d70 con 0x7f61cc151de0 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61c37fe640 1 -- 172.21.15.144:0/1406843868 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b40b9430 con 0x7f61cc151de0 2023-11-28T21:34:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61d12c6640 1 -- 172.21.15.144:0/1406843868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b4009420 con 0x7f61cc151de0 2023-11-28T21:34:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61c37fe640 1 -- 172.21.15.144:0/1406843868 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40b97e0 con 0x7f61cc151de0 2023-11-28T21:34:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61d12c6640 1 -- 172.21.15.144:0/1406843868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61cc1ede60 con 0x7f61cc151de0 2023-11-28T21:34:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61c37fe640 1 -- 172.21.15.144:0/1406843868 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f61b40b7040 con 0x7f61cc151de0 2023-11-28T21:34:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.972+0000 7f61c37fe640 1 -- 172.21.15.144:0/1406843868 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f61b40c3b70 con 0x7f61cc151de0 2023-11-28T21:34:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.976+0000 7f61d12c6640 1 -- 172.21.15.144:0/1406843868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61cc074f60 con 0x7f61cc151de0 2023-11-28T21:34:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:19.976+0000 7f61c37fe640 1 -- 172.21.15.144:0/1406843868 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f61b40b5070 con 0x7f61cc151de0 2023-11-28T21:34:20.049 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.048+0000 7f61d12c6640 1 -- 172.21.15.144:0/1406843868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ad45d502-66d6-4abb-b218-a53b0518dc04", "id": 3} v 0) v1 -- 0x7f61cc14e670 con 0x7f61cc151de0 2023-11-28T21:34:20.055 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.052+0000 7f61c37fe640 1 -- 172.21.15.144:0/1406843868 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad45d502-66d6-4abb-b218-a53b0518dc04", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f61b40c36e0 con 0x7f61cc151de0 2023-11-28T21:34:20.055 INFO:teuthology.orchestra.run.smithi144.stdout:3 2023-11-28T21:34:20.056 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.056+0000 7f61d12c6640 1 -- 172.21.15.144:0/1406843868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc151de0 msgr2=0x7f61cc0cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.056+0000 7f61d12c6640 1 --2- 172.21.15.144:0/1406843868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc151de0 0x7f61cc0cc400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61b401c8a0 tx=0x7f61b401ba10 comp rx=0 tx=0).stop 2023-11-28T21:34:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.056+0000 7f61d12c6640 1 -- 172.21.15.144:0/1406843868 shutdown_connections 2023-11-28T21:34:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.056+0000 7f61d12c6640 1 --2- 172.21.15.144:0/1406843868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61cc151de0 0x7f61cc0cc400 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.056+0000 7f61d12c6640 1 --2- 172.21.15.144:0/1406843868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61cc0c9690 0x7f61cc1ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.056+0000 7f61d12c6640 1 --2- 172.21.15.144:0/1406843868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61cc0c4ff0 0x7f61cc150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.056+0000 7f61d12c6640 1 -- 172.21.15.144:0/1406843868 >> 172.21.15.144:0/1406843868 conn(0x7f61cc0cd3e0 msgr2=0x7f61cc0c4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:20.057 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.056+0000 7f61d12c6640 1 -- 172.21.15.144:0/1406843868 shutdown_connections 2023-11-28T21:34:20.058 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.056+0000 7f61d12c6640 1 -- 172.21.15.144:0/1406843868 wait complete. 2023-11-28T21:34:20.069 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:20.068+0000 7febf02d9180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T21:34:20.073 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 3a48db2f-807a-4b49-b01f-6e23b5074a97 4 2023-11-28T21:34:20.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f33bd640 1 Processor -- start 2023-11-28T21:34:20.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f33bd640 1 -- start start 2023-11-28T21:34:20.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f33bd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec0c4ff0 0x7f70ec072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f33bd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70ec0c9690 0x7f70ec15b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f33bd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ec1589a0 con 0x7f70ec151de0 2023-11-28T21:34:20.192 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f33bd640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ec1521c0 con 0x7f70ec0c9690 2023-11-28T21:34:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f33bd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0c9c10 con 0x7f70ec0c4ff0 2023-11-28T21:34:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f0958640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec0c4ff0 0x7f70ec072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f0958640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec0c4ff0 0x7f70ec072610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:51900/0 (socket says 172.21.15.144:51900) 2023-11-28T21:34:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f0958640 1 -- 172.21.15.144:0/3556287158 learned_addr learned my addr 172.21.15.144:0/3556287158 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T21:34:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f195a640 1 --2- 172.21.15.144:0/3556287158 >> v2:172.21.15.12:3300/0 conn(0x7f70ec0c9690 0x7f70ec15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f0958640 1 -- 172.21.15.144:0/3556287158 >> v2:172.21.15.12:3300/0 conn(0x7f70ec0c9690 msgr2=0x7f70ec15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f0958640 1 --2- 172.21.15.144:0/3556287158 >> v2:172.21.15.12:3300/0 conn(0x7f70ec0c9690 0x7f70ec15b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f0958640 1 -- 172.21.15.144:0/3556287158 >> v1:172.21.15.12:6789/0 conn(0x7f70ec151de0 legacy=0x7f70ec0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.188+0000 7f70f0958640 1 -- 172.21.15.144:0/3556287158 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec157d90 con 0x7f70ec0c4ff0 2023-11-28T21:34:20.194 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70f0958640 1 --2- 172.21.15.144:0/3556287158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec0c4ff0 0x7f70ec072610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009870 tx=0x7f70dc01ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ebc3b7f34e1fa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:20.194 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70e3fff640 1 -- 172.21.15.144:0/3556287158 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0a89b0 con 0x7f70ec0c4ff0 2023-11-28T21:34:20.194 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70e3fff640 1 -- 172.21.15.144:0/3556287158 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc0a8b10 con 0x7f70ec0c4ff0 2023-11-28T21:34:20.194 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70e3fff640 1 -- 172.21.15.144:0/3556287158 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0a8ea0 con 0x7f70ec0c4ff0 2023-11-28T21:34:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70f33bd640 1 -- 172.21.15.144:0/3556287158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec0c4ff0 msgr2=0x7f70ec072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70f33bd640 1 --2- 172.21.15.144:0/3556287158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec0c4ff0 0x7f70ec072610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009870 tx=0x7f70dc01ecf0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70f33bd640 1 -- 172.21.15.144:0/3556287158 shutdown_connections 2023-11-28T21:34:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70f33bd640 1 --2- 172.21.15.144:0/3556287158 >> v2:172.21.15.12:3300/0 conn(0x7f70ec0c9690 0x7f70ec15b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70f33bd640 1 --2- 172.21.15.144:0/3556287158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec0c4ff0 0x7f70ec072610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70f33bd640 1 -- 172.21.15.144:0/3556287158 >> 172.21.15.144:0/3556287158 conn(0x7f70ec0cd3e0 msgr2=0x7f70ec0cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70f33bd640 1 -- 172.21.15.144:0/3556287158 shutdown_connections 2023-11-28T21:34:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.192+0000 7f70f33bd640 1 -- 172.21.15.144:0/3556287158 wait complete. 2023-11-28T21:34:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 Processor -- start 2023-11-28T21:34:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 -- start start 2023-11-28T21:34:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ec0c4ff0 0x7f70ec14ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70ec0c9690 0x7f70ec1f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec151de0 0x7f70ec06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec1573b0 con 0x7f70ec0c4ff0 2023-11-28T21:34:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec1effb0 con 0x7f70ec0c9690 2023-11-28T21:34:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec1ef6e0 con 0x7f70ec151de0 2023-11-28T21:34:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f195a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec151de0 0x7f70ec06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f0958640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70ec0c9690 0x7f70ec1f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f195a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec151de0 0x7f70ec06c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:51916/0 (socket says 172.21.15.144:51916) 2023-11-28T21:34:20.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f195a640 1 -- 172.21.15.144:0/1752835751 learned_addr learned my addr 172.21.15.144:0/1752835751 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T21:34:20.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f0958640 1 --2- 172.21.15.144:0/1752835751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70ec0c9690 0x7f70ec1f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55142/0 (socket says 172.21.15.144:55142) 2023-11-28T21:34:20.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f1159640 1 --2- 172.21.15.144:0/1752835751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ec0c4ff0 0x7f70ec14ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f0958640 1 -- 172.21.15.144:0/1752835751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec151de0 msgr2=0x7f70ec06c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f0958640 1 --2- 172.21.15.144:0/1752835751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec151de0 0x7f70ec06c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f0958640 1 -- 172.21.15.144:0/1752835751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ec0c4ff0 msgr2=0x7f70ec14ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f0958640 1 --2- 172.21.15.144:0/1752835751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ec0c4ff0 0x7f70ec14ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.199 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f0958640 1 -- 172.21.15.144:0/1752835751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0094d0 con 0x7f70ec0c9690 2023-11-28T21:34:20.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f195a640 1 --2- 172.21.15.144:0/1752835751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec151de0 0x7f70ec06c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:34:20.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f0958640 1 --2- 172.21.15.144:0/1752835751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70ec0c9690 0x7f70ec1f3660 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0af000 tx=0x7f70dc01ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:20.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70e1ffb640 1 -- 172.21.15.144:0/1752835751 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0be070 con 0x7f70ec0c9690 2023-11-28T21:34:20.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70e1ffb640 1 -- 172.21.15.144:0/1752835751 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc0ba400 con 0x7f70ec0c9690 2023-11-28T21:34:20.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 -- 172.21.15.144:0/1752835751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec1f3ba0 con 0x7f70ec0c9690 2023-11-28T21:34:20.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70e1ffb640 1 -- 172.21.15.144:0/1752835751 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0ba7d0 con 0x7f70ec0c9690 2023-11-28T21:34:20.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 -- 172.21.15.144:0/1752835751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ec15af80 con 0x7f70ec0c9690 2023-11-28T21:34:20.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70e1ffb640 1 -- 172.21.15.144:0/1752835751 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f70dc0bac30 con 0x7f70ec0c9690 2023-11-28T21:34:20.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70e1ffb640 1 -- 172.21.15.144:0/1752835751 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f70dc0c23c0 con 0x7f70ec0c9690 2023-11-28T21:34:20.200 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.196+0000 7f70f33bd640 1 -- 172.21.15.144:0/1752835751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ec074f60 con 0x7f70ec0c9690 2023-11-28T21:34:20.201 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.200+0000 7f70e1ffb640 1 -- 172.21.15.144:0/1752835751 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f70ec1f3ba0 con 0x7f70ec0c9690 2023-11-28T21:34:20.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.268+0000 7f70f33bd640 1 -- 172.21.15.144:0/1752835751 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3a48db2f-807a-4b49-b01f-6e23b5074a97", "id": 4} v 0) v1 -- 0x7f70ec072d50 con 0x7f70ec0c9690 2023-11-28T21:34:20.275 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.272+0000 7f70e1ffb640 1 -- 172.21.15.144:0/1752835751 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3a48db2f-807a-4b49-b01f-6e23b5074a97", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f70dc0d10b0 con 0x7f70ec0c9690 2023-11-28T21:34:20.275 INFO:teuthology.orchestra.run.smithi144.stdout:4 2023-11-28T21:34:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.276+0000 7f70f33bd640 1 -- 172.21.15.144:0/1752835751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70ec0c9690 msgr2=0x7f70ec1f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.276+0000 7f70f33bd640 1 --2- 172.21.15.144:0/1752835751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70ec0c9690 0x7f70ec1f3660 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0af000 tx=0x7f70dc01ba90 comp rx=0 tx=0).stop 2023-11-28T21:34:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.276+0000 7f70f33bd640 1 -- 172.21.15.144:0/1752835751 shutdown_connections 2023-11-28T21:34:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.276+0000 7f70f33bd640 1 --2- 172.21.15.144:0/1752835751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70ec151de0 0x7f70ec06c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.276+0000 7f70f33bd640 1 --2- 172.21.15.144:0/1752835751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70ec0c9690 0x7f70ec1f3660 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.276+0000 7f70f33bd640 1 --2- 172.21.15.144:0/1752835751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70ec0c4ff0 0x7f70ec14ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.276+0000 7f70f33bd640 1 -- 172.21.15.144:0/1752835751 >> 172.21.15.144:0/1752835751 conn(0x7f70ec0cd3e0 msgr2=0x7f70ec0c8690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:20.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.276+0000 7f70f33bd640 1 -- 172.21.15.144:0/1752835751 shutdown_connections 2023-11-28T21:34:20.278 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.276+0000 7f70f33bd640 1 -- 172.21.15.144:0/1752835751 wait complete. 2023-11-28T21:34:20.280 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:20.273+0000 7f17cf036180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T21:34:20.283 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:20.280+0000 7febf02d9180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T21:34:20.293 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 338dba1f-d51c-4886-8760-b1f1356b2f15 5 2023-11-28T21:34:20.346 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:20.341+0000 7f17cf036180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T21:34:20.409 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:20.405+0000 7f17cf036180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T21:34:20.410 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f24665b3640 1 Processor -- start 2023-11-28T21:34:20.410 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f24665b3640 1 -- start start 2023-11-28T21:34:20.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f24665b3640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2460150420 0x7f2460156c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f24665b3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f24600c7920 0x7f246006b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f24665b3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24600c8160 con 0x7f24600c6b80 2023-11-28T21:34:20.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f24665b3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246006b360 con 0x7f24600c7920 2023-11-28T21:34:20.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f24665b3640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24600cc8b0 con 0x7f2460150420 2023-11-28T21:34:20.413 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f245ffff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2460150420 0x7f2460156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f245ffff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2460150420 0x7f2460156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55154/0 (socket says 172.21.15.144:55154) 2023-11-28T21:34:20.414 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f245ffff640 1 -- 172.21.15.144:0/2236058529 learned_addr learned my addr 172.21.15.144:0/2236058529 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T21:34:20.415 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.408+0000 7f245f7fe640 1 --2- 172.21.15.144:0/2236058529 >> v2:172.21.15.12:3300/0 conn(0x7f24600c7920 0x7f246006b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.416 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f245ffff640 1 -- 172.21.15.144:0/2236058529 >> v2:172.21.15.12:3300/0 conn(0x7f24600c7920 msgr2=0x7f246006b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:20.416 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f245ffff640 1 --2- 172.21.15.144:0/2236058529 >> v2:172.21.15.12:3300/0 conn(0x7f24600c7920 0x7f246006b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.416 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f245ffff640 1 -- 172.21.15.144:0/2236058529 >> v1:172.21.15.12:6789/0 conn(0x7f24600c6b80 legacy=0x7f24601597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.416 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f245ffff640 1 -- 172.21.15.144:0/2236058529 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24601571b0 con 0x7f2460150420 2023-11-28T21:34:20.419 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f245ffff640 1 --2- 172.21.15.144:0/2236058529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2460150420 0x7f2460156c70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f24600748a0 tx=0x7f245801da10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f0b0603fd46276d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f245effd640 1 -- 172.21.15.144:0/2236058529 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245800a2d0 con 0x7f2460150420 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f245effd640 1 -- 172.21.15.144:0/2236058529 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24580a6850 con 0x7f2460150420 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f245effd640 1 -- 172.21.15.144:0/2236058529 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24580a6be0 con 0x7f2460150420 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f24665b3640 1 -- 172.21.15.144:0/2236058529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2460150420 msgr2=0x7f2460156c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f24665b3640 1 --2- 172.21.15.144:0/2236058529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2460150420 0x7f2460156c70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f24600748a0 tx=0x7f245801da10 comp rx=0 tx=0).stop 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f24665b3640 1 -- 172.21.15.144:0/2236058529 shutdown_connections 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f24665b3640 1 --2- 172.21.15.144:0/2236058529 >> v2:172.21.15.12:3300/0 conn(0x7f24600c7920 0x7f246006b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f24665b3640 1 --2- 172.21.15.144:0/2236058529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2460150420 0x7f2460156c70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f24665b3640 1 -- 172.21.15.144:0/2236058529 >> 172.21.15.144:0/2236058529 conn(0x7f24600cbb80 msgr2=0x7f24600cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f24665b3640 1 -- 172.21.15.144:0/2236058529 shutdown_connections 2023-11-28T21:34:20.420 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.412+0000 7f24665b3640 1 -- 172.21.15.144:0/2236058529 wait complete. 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 Processor -- start 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 -- start start 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24600c6b80 0x7f24600c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24600c7920 0x7f2460152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2460150420 0x7f24601534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24601415a0 con 0x7f24600c6b80 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24601f3820 con 0x7f24600c7920 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24601542e0 con 0x7f2460150420 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24600c7920 0x7f2460152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24600c7920 0x7f2460152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55168/0 (socket says 172.21.15.144:55168) 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f2464b50640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2460150420 0x7f24601534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.421 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245f7fe640 1 -- 172.21.15.144:0/1564458065 learned_addr learned my addr 172.21.15.144:0/1564458065 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245ffff640 1 --2- 172.21.15.144:0/1564458065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24600c6b80 0x7f24600c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245f7fe640 1 -- 172.21.15.144:0/1564458065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2460150420 msgr2=0x7f24601534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245f7fe640 1 --2- 172.21.15.144:0/1564458065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2460150420 0x7f24601534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245f7fe640 1 -- 172.21.15.144:0/1564458065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24600c6b80 msgr2=0x7f24600c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245f7fe640 1 --2- 172.21.15.144:0/1564458065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24600c6b80 0x7f24600c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245f7fe640 1 -- 172.21.15.144:0/1564458065 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245800ac40 con 0x7f24600c7920 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245f7fe640 1 --2- 172.21.15.144:0/1564458065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24600c7920 0x7f2460152f90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f245400d5f0 tx=0x7f245400dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245cff9640 1 -- 172.21.15.144:0/1564458065 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24540125f0 con 0x7f24600c7920 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245cff9640 1 -- 172.21.15.144:0/1564458065 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2454012750 con 0x7f24600c7920 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 -- 172.21.15.144:0/1564458065 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24601545c0 con 0x7f24600c7920 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245cff9640 1 -- 172.21.15.144:0/1564458065 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2454012ae0 con 0x7f24600c7920 2023-11-28T21:34:20.422 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 -- 172.21.15.144:0/1564458065 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2460154b80 con 0x7f24600c7920 2023-11-28T21:34:20.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245cff9640 1 -- 172.21.15.144:0/1564458065 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f245401e460 con 0x7f24600c7920 2023-11-28T21:34:20.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f245cff9640 1 -- 172.21.15.144:0/1564458065 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f2454014a50 con 0x7f24600c7920 2023-11-28T21:34:20.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.416+0000 7f24665b3640 1 -- 172.21.15.144:0/1564458065 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2460074f60 con 0x7f24600c7920 2023-11-28T21:34:20.423 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.420+0000 7f245cff9640 1 -- 172.21.15.144:0/1564458065 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2454014cf0 con 0x7f24600c7920 2023-11-28T21:34:20.484 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.484+0000 7f24665b3640 1 -- 172.21.15.144:0/1564458065 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "338dba1f-d51c-4886-8760-b1f1356b2f15", "id": 5} v 0) v1 -- 0x7f24600b19a0 con 0x7f24600c7920 2023-11-28T21:34:20.490 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.488+0000 7f245cff9640 1 -- 172.21.15.144:0/1564458065 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "338dba1f-d51c-4886-8760-b1f1356b2f15", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f24540144d0 con 0x7f24600c7920 2023-11-28T21:34:20.491 INFO:teuthology.orchestra.run.smithi144.stdout:5 2023-11-28T21:34:20.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.488+0000 7f24665b3640 1 -- 172.21.15.144:0/1564458065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24600c7920 msgr2=0x7f2460152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.488+0000 7f24665b3640 1 --2- 172.21.15.144:0/1564458065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24600c7920 0x7f2460152f90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f245400d5f0 tx=0x7f245400dab0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.492+0000 7f24665b3640 1 -- 172.21.15.144:0/1564458065 shutdown_connections 2023-11-28T21:34:20.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.492+0000 7f24665b3640 1 --2- 172.21.15.144:0/1564458065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2460150420 0x7f24601534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.492+0000 7f24665b3640 1 --2- 172.21.15.144:0/1564458065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24600c7920 0x7f2460152f90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.492+0000 7f24665b3640 1 --2- 172.21.15.144:0/1564458065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24600c6b80 0x7f24600c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.492 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.492+0000 7f24665b3640 1 -- 172.21.15.144:0/1564458065 >> 172.21.15.144:0/1564458065 conn(0x7f24600cbb80 msgr2=0x7f2460144180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:20.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.492+0000 7f24665b3640 1 -- 172.21.15.144:0/1564458065 shutdown_connections 2023-11-28T21:34:20.493 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.492+0000 7f24665b3640 1 -- 172.21.15.144:0/1564458065 wait complete. 2023-11-28T21:34:20.503 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:20.497+0000 7f17cf036180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T21:34:20.507 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 3baa1580-096d-4c3c-8fbe-4335a0a12b2a 6 2023-11-28T21:34:20.566 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:20.561+0000 7f17cf036180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T21:34:20.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.648+0000 7faa3ee17640 1 Processor -- start 2023-11-28T21:34:20.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.648+0000 7faa3ee17640 1 -- start start 2023-11-28T21:34:20.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.648+0000 7faa3ee17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa3815ba20 0x7faa38072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.648+0000 7faa3ee17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa381589a0 con 0x7faa380c4ff0 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.648+0000 7faa3ee17640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa381521c0 con 0x7faa38151de0 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.648+0000 7faa3ee17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa380c9c10 con 0x7faa3815ba20 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.648+0000 7faa3cbb3640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7faa38151de0 0x7faa380c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:48342/0 (socket says 172.21.15.144:48342) 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.648+0000 7faa3cbb3640 1 -- 172.21.15.144:0/191529943 learned_addr learned my addr 172.21.15.144:0/191529943 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.648+0000 7faa3d3b4640 1 --2- 172.21.15.144:0/191529943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa3815ba20 0x7faa38072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa377fe640 1 -- 172.21.15.144:0/191529943 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165418272 0 0) 0x7faa381521c0 con 0x7faa38151de0 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa377fe640 1 -- 172.21.15.144:0/191529943 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa28003530 con 0x7faa38151de0 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3d3b4640 1 -- 172.21.15.144:0/191529943 >> v1:172.21.15.144:6789/0 conn(0x7faa38151de0 legacy=0x7faa380c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3d3b4640 1 -- 172.21.15.144:0/191529943 >> v1:172.21.15.12:6789/0 conn(0x7faa380c4ff0 legacy=0x7faa3815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3d3b4640 1 -- 172.21.15.144:0/191529943 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38157d90 con 0x7faa3815ba20 2023-11-28T21:34:20.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3d3b4640 1 --2- 172.21.15.144:0/191529943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa3815ba20 0x7faa38072610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa30009dc0 tx=0x7faa3001f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3cde20c74331673 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:20.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa377fe640 1 -- 172.21.15.144:0/191529943 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300af600 con 0x7faa3815ba20 2023-11-28T21:34:20.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa377fe640 1 -- 172.21.15.144:0/191529943 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa300ab050 con 0x7faa3815ba20 2023-11-28T21:34:20.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa377fe640 1 -- 172.21.15.144:0/191529943 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300af980 con 0x7faa3815ba20 2023-11-28T21:34:20.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3ee17640 1 -- 172.21.15.144:0/191529943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa3815ba20 msgr2=0x7faa38072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3ee17640 1 --2- 172.21.15.144:0/191529943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa3815ba20 0x7faa38072610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa30009dc0 tx=0x7faa3001f150 comp rx=0 tx=0).stop 2023-11-28T21:34:20.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3ee17640 1 -- 172.21.15.144:0/191529943 shutdown_connections 2023-11-28T21:34:20.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3ee17640 1 --2- 172.21.15.144:0/191529943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa3815ba20 0x7faa38072610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3ee17640 1 -- 172.21.15.144:0/191529943 >> 172.21.15.144:0/191529943 conn(0x7faa380cd3e0 msgr2=0x7faa380cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:20.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3ee17640 1 -- 172.21.15.144:0/191529943 shutdown_connections 2023-11-28T21:34:20.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.652+0000 7faa3ee17640 1 -- 172.21.15.144:0/191529943 wait complete. 2023-11-28T21:34:20.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 Processor -- start 2023-11-28T21:34:20.656 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 -- start start 2023-11-28T21:34:20.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa380c4ff0 0x7faa381f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa38151de0 0x7faa38158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa3815ba20 0x7faa38140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3814e610 con 0x7faa38151de0 2023-11-28T21:34:20.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa381ef4d0 con 0x7faa3815ba20 2023-11-28T21:34:20.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa381f04a0 con 0x7faa380c4ff0 2023-11-28T21:34:20.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3cbb3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa380c4ff0 0x7faa381f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3cbb3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa380c4ff0 0x7faa381f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:47398/0 (socket says 172.21.15.144:47398) 2023-11-28T21:34:20.659 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3cbb3640 1 -- 172.21.15.144:0/3624622026 learned_addr learned my addr 172.21.15.144:0/3624622026 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T21:34:20.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3d3b4640 1 --2- 172.21.15.144:0/3624622026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa3815ba20 0x7faa38140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa37fff640 1 --2- 172.21.15.144:0/3624622026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa38151de0 0x7faa38158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3cbb3640 1 -- 172.21.15.144:0/3624622026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa3815ba20 msgr2=0x7faa38140740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3cbb3640 1 --2- 172.21.15.144:0/3624622026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa3815ba20 0x7faa38140740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3cbb3640 1 -- 172.21.15.144:0/3624622026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa38151de0 msgr2=0x7faa38158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3cbb3640 1 --2- 172.21.15.144:0/3624622026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa38151de0 0x7faa38158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3cbb3640 1 -- 172.21.15.144:0/3624622026 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa30009a70 con 0x7faa380c4ff0 2023-11-28T21:34:20.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3cbb3640 1 --2- 172.21.15.144:0/3624622026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa380c4ff0 0x7faa381f3450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00c500 tx=0x7faa2c00c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:20.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa357fa640 1 -- 172.21.15.144:0/3624622026 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c005cf0 con 0x7faa380c4ff0 2023-11-28T21:34:20.660 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa357fa640 1 -- 172.21.15.144:0/3624622026 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c005e50 con 0x7faa380c4ff0 2023-11-28T21:34:20.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa357fa640 1 -- 172.21.15.144:0/3624622026 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0072f0 con 0x7faa380c4ff0 2023-11-28T21:34:20.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 -- 172.21.15.144:0/3624622026 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa381f3990 con 0x7faa380c4ff0 2023-11-28T21:34:20.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 -- 172.21.15.144:0/3624622026 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa381f0010 con 0x7faa380c4ff0 2023-11-28T21:34:20.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa357fa640 1 -- 172.21.15.144:0/3624622026 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faa2c01c440 con 0x7faa380c4ff0 2023-11-28T21:34:20.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa357fa640 1 -- 172.21.15.144:0/3624622026 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7faa2c0129f0 con 0x7faa380c4ff0 2023-11-28T21:34:20.661 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.656+0000 7faa3ee17640 1 -- 172.21.15.144:0/3624622026 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa38074f60 con 0x7faa380c4ff0 2023-11-28T21:34:20.662 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:20.653+0000 7f17cf036180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T21:34:20.662 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.660+0000 7faa357fa640 1 -- 172.21.15.144:0/3624622026 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7faa2c021060 con 0x7faa380c4ff0 2023-11-28T21:34:20.723 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:20.717+0000 7f17cf036180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T21:34:20.729 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.728+0000 7faa3ee17640 1 -- 172.21.15.144:0/3624622026 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3baa1580-096d-4c3c-8fbe-4335a0a12b2a", "id": 6} v 0) v1 -- 0x7faa38072d50 con 0x7faa380c4ff0 2023-11-28T21:34:20.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.736+0000 7faa357fa640 1 -- 172.21.15.144:0/3624622026 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3baa1580-096d-4c3c-8fbe-4335a0a12b2a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7faa381f3990 con 0x7faa380c4ff0 2023-11-28T21:34:20.739 INFO:teuthology.orchestra.run.smithi144.stdout:6 2023-11-28T21:34:20.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.740+0000 7faa3ee17640 1 -- 172.21.15.144:0/3624622026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa380c4ff0 msgr2=0x7faa381f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.740+0000 7faa3ee17640 1 --2- 172.21.15.144:0/3624622026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa380c4ff0 0x7faa381f3450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00c500 tx=0x7faa2c00c9c0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.740+0000 7faa3ee17640 1 -- 172.21.15.144:0/3624622026 shutdown_connections 2023-11-28T21:34:20.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.740+0000 7faa3ee17640 1 --2- 172.21.15.144:0/3624622026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa3815ba20 0x7faa38140740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.740+0000 7faa3ee17640 1 --2- 172.21.15.144:0/3624622026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa38151de0 0x7faa38158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.740+0000 7faa3ee17640 1 --2- 172.21.15.144:0/3624622026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa380c4ff0 0x7faa381f3450 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.740+0000 7faa3ee17640 1 -- 172.21.15.144:0/3624622026 >> 172.21.15.144:0/3624622026 conn(0x7faa380cd3e0 msgr2=0x7faa381573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:20.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.740+0000 7faa3ee17640 1 -- 172.21.15.144:0/3624622026 shutdown_connections 2023-11-28T21:34:20.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.740+0000 7faa3ee17640 1 -- 172.21.15.144:0/3624622026 wait complete. 2023-11-28T21:34:20.756 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27 7 2023-11-28T21:34:20.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe855d6640 1 Processor -- start 2023-11-28T21:34:20.868 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe855d6640 1 -- start start 2023-11-28T21:34:20.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe855d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbe80151de0 0x7fbe80072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.869 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe855d6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe800c4ff0 0x7fbe8015b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe855d6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe801589a0 con 0x7fbe800c9690 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe855d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe801521c0 con 0x7fbe80151de0 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe855d6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe800c9c10 con 0x7fbe800c4ff0 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7f7fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe800c4ff0 0x7fbe8015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7e7fc640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fbe800c9690 0x7fbe800c48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:48350/0 (socket says 172.21.15.144:48350) 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7e7fc640 1 -- 172.21.15.144:0/3992989793 learned_addr learned my addr 172.21.15.144:0/3992989793 (peer_addr_for_me v1:172.21.15.144:0/0) 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7effd640 1 --2- 172.21.15.144:0/3992989793 >> v2:172.21.15.12:3300/0 conn(0x7fbe80151de0 0x7fbe80072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7f7fe640 1 -- 172.21.15.144:0/3992989793 >> v2:172.21.15.12:3300/0 conn(0x7fbe80151de0 msgr2=0x7fbe80072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7f7fe640 1 --2- 172.21.15.144:0/3992989793 >> v2:172.21.15.12:3300/0 conn(0x7fbe80151de0 0x7fbe80072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7f7fe640 1 -- 172.21.15.144:0/3992989793 >> v1:172.21.15.144:6789/0 conn(0x7fbe800c9690 legacy=0x7fbe800c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.870 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7f7fe640 1 -- 172.21.15.144:0/3992989793 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80157d90 con 0x7fbe800c4ff0 2023-11-28T21:34:20.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7dffb640 1 -- 172.21.15.144:0/3992989793 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174220416 0 0) 0x7fbe801589a0 con 0x7fbe800c9690 2023-11-28T21:34:20.871 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7f7fe640 1 --2- 172.21.15.144:0/3992989793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 0x7fbe8015b310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe800c66a0 tx=0x7fbe7801f0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88af5aa13e14b8e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:20.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.868+0000 7fbe7dffb640 1 -- 172.21.15.144:0/3992989793 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780b0440 con 0x7fbe800c4ff0 2023-11-28T21:34:20.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7dffb640 1 -- 172.21.15.144:0/3992989793 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe780b05a0 con 0x7fbe800c4ff0 2023-11-28T21:34:20.872 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7dffb640 1 -- 172.21.15.144:0/3992989793 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780b0930 con 0x7fbe800c4ff0 2023-11-28T21:34:20.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- 172.21.15.144:0/3992989793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 msgr2=0x7fbe8015b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 --2- 172.21.15.144:0/3992989793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 0x7fbe8015b310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe800c66a0 tx=0x7fbe7801f0f0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- 172.21.15.144:0/3992989793 shutdown_connections 2023-11-28T21:34:20.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 --2- 172.21.15.144:0/3992989793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 0x7fbe8015b310 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 --2- 172.21.15.144:0/3992989793 >> v2:172.21.15.12:3300/0 conn(0x7fbe80151de0 0x7fbe80072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- 172.21.15.144:0/3992989793 >> 172.21.15.144:0/3992989793 conn(0x7fbe800cd3e0 msgr2=0x7fbe800cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:20.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- 172.21.15.144:0/3992989793 shutdown_connections 2023-11-28T21:34:20.873 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- 172.21.15.144:0/3992989793 wait complete. 2023-11-28T21:34:20.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 Processor -- start 2023-11-28T21:34:20.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- start start 2023-11-28T21:34:20.875 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 0x7fbe800cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe800c9690 0x7fbe801f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80151de0 0x7fbe80143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:20.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8015a890 con 0x7fbe800c9690 2023-11-28T21:34:20.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8014fff0 con 0x7fbe800c4ff0 2023-11-28T21:34:20.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801f0c10 con 0x7fbe80151de0 2023-11-28T21:34:20.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 0x7fbe800cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 0x7fbe800cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48892/0 (socket says 172.21.15.144:48892) 2023-11-28T21:34:20.876 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7f7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80151de0 0x7fbe80143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7effd640 1 -- 172.21.15.144:0/1657871938 learned_addr learned my addr 172.21.15.144:0/1657871938 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7e7fc640 1 --2- 172.21.15.144:0/1657871938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe800c9690 0x7fbe801f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7effd640 1 -- 172.21.15.144:0/1657871938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80151de0 msgr2=0x7fbe80143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7effd640 1 --2- 172.21.15.144:0/1657871938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80151de0 0x7fbe80143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7effd640 1 -- 172.21.15.144:0/1657871938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe800c9690 msgr2=0x7fbe801f1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7effd640 1 --2- 172.21.15.144:0/1657871938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe800c9690 0x7fbe801f1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7effd640 1 -- 172.21.15.144:0/1657871938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78009a70 con 0x7fbe800c4ff0 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7f7fe640 1 --2- 172.21.15.144:0/1657871938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80151de0 0x7fbe80143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe7effd640 1 --2- 172.21.15.144:0/1657871938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 0x7fbe800cc490 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000d800 tx=0x7fbe7000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe677fe640 1 -- 172.21.15.144:0/1657871938 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000b6c0 con 0x7fbe800c4ff0 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe677fe640 1 -- 172.21.15.144:0/1657871938 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7000b820 con 0x7fbe800c4ff0 2023-11-28T21:34:20.877 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- 172.21.15.144:0/1657871938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe80158c00 con 0x7fbe800c4ff0 2023-11-28T21:34:20.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe677fe640 1 -- 172.21.15.144:0/1657871938 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000bb90 con 0x7fbe800c4ff0 2023-11-28T21:34:20.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe855d6640 1 -- 172.21.15.144:0/1657871938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8015aa70 con 0x7fbe800c4ff0 2023-11-28T21:34:20.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe677fe640 1 -- 172.21.15.144:0/1657871938 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbe7001e460 con 0x7fbe800c4ff0 2023-11-28T21:34:20.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.872+0000 7fbe677fe640 1 -- 172.21.15.144:0/1657871938 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fbe70014c20 con 0x7fbe800c4ff0 2023-11-28T21:34:20.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.876+0000 7fbe855d6640 1 -- 172.21.15.144:0/1657871938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe80074f60 con 0x7fbe800c4ff0 2023-11-28T21:34:20.878 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.876+0000 7fbe677fe640 1 -- 172.21.15.144:0/1657871938 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fbe700145c0 con 0x7fbe800c4ff0 2023-11-28T21:34:20.889 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:20.885+0000 7f17cf036180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T21:34:20.925 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:20.924+0000 7febf02d9180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T21:34:20.941 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.940+0000 7fbe855d6640 1 -- 172.21.15.144:0/1657871938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27", "id": 7} v 0) v1 -- 0x7fbe80072d50 con 0x7fbe800c4ff0 2023-11-28T21:34:20.950 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.948+0000 7fbe677fe640 1 -- 172.21.15.144:0/1657871938 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbe70014830 con 0x7fbe800c4ff0 2023-11-28T21:34:20.950 INFO:teuthology.orchestra.run.smithi144.stdout:7 2023-11-28T21:34:20.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.948+0000 7fbe855d6640 1 -- 172.21.15.144:0/1657871938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 msgr2=0x7fbe800cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:20.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.948+0000 7fbe855d6640 1 --2- 172.21.15.144:0/1657871938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 0x7fbe800cc490 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000d800 tx=0x7fbe7000dcc0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.952+0000 7fbe855d6640 1 -- 172.21.15.144:0/1657871938 shutdown_connections 2023-11-28T21:34:20.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.952+0000 7fbe855d6640 1 --2- 172.21.15.144:0/1657871938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe80151de0 0x7fbe80143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.952+0000 7fbe855d6640 1 --2- 172.21.15.144:0/1657871938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe800c9690 0x7fbe801f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.952+0000 7fbe855d6640 1 --2- 172.21.15.144:0/1657871938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe800c4ff0 0x7fbe800cc490 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:20.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.952+0000 7fbe855d6640 1 -- 172.21.15.144:0/1657871938 >> 172.21.15.144:0/1657871938 conn(0x7fbe800cd3e0 msgr2=0x7fbe800c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:20.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.952+0000 7fbe855d6640 1 -- 172.21.15.144:0/1657871938 shutdown_connections 2023-11-28T21:34:20.952 INFO:teuthology.orchestra.run.smithi144.stderr:2023-11-28T21:34:20.952+0000 7fbe855d6640 1 -- 172.21.15.144:0/1657871938 wait complete. 2023-11-28T21:34:20.961 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:20.957+0000 7f17cf036180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T21:34:20.967 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T21:34:20.967 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T21:34:20.972 INFO:tasks.ceph.osd.0:Started 2023-11-28T21:34:20.972 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T21:34:20.972 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T21:34:20.977 INFO:tasks.ceph.osd.1:Started 2023-11-28T21:34:20.978 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T21:34:20.978 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T21:34:20.983 INFO:tasks.ceph.osd.2:Started 2023-11-28T21:34:20.983 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T21:34:20.983 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T21:34:20.984 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:20.980+0000 7febf02d9180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T21:34:20.987 INFO:tasks.ceph.osd.3:Started 2023-11-28T21:34:20.988 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T21:34:20.988 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T21:34:20.990 INFO:tasks.ceph.osd.4:Started 2023-11-28T21:34:20.990 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T21:34:20.990 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T21:34:20.992 INFO:tasks.ceph.osd.5:Started 2023-11-28T21:34:20.992 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T21:34:20.992 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T21:34:20.994 INFO:tasks.ceph.osd.6:Started 2023-11-28T21:34:20.994 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T21:34:20.994 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T21:34:20.996 INFO:tasks.ceph.osd.7:Started 2023-11-28T21:34:20.997 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:34:21.062 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:21.060+0000 7febf02d9180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T21:34:21.142 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:21.140+0000 7febf02d9180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T21:34:21.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d37e9e640 1 Processor -- start 2023-11-28T21:34:21.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d37e9e640 1 -- start start 2023-11-28T21:34:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d37e9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d300cac40 0x7f0d3006b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d37e9e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0d300cb020 0x7f0d30159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d37e9e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d300c4da0 con 0x7f0d300c4980 2023-11-28T21:34:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d37e9e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d300cb4b0 con 0x7f0d300cb020 2023-11-28T21:34:21.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d37e9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3015a5e0 con 0x7f0d300cac40 2023-11-28T21:34:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d3643b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0d300cb020 0x7f0d30159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d3643b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0d300cb020 0x7f0d30159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37618/0 (socket says 172.21.15.12:37618) 2023-11-28T21:34:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d3643b640 1 -- 172.21.15.12:0/1175517669 learned_addr learned my addr 172.21.15.12:0/1175517669 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d35c3a640 1 --2- 172.21.15.12:0/1175517669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d300cac40 0x7f0d3006b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d3643b640 1 -- 172.21.15.12:0/1175517669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d300cac40 msgr2=0x7f0d3006b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d3643b640 1 --2- 172.21.15.12:0/1175517669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d300cac40 0x7f0d3006b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d3643b640 1 -- 172.21.15.12:0/1175517669 >> v1:172.21.15.12:6789/0 conn(0x7f0d300c4980 legacy=0x7f0d300ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:21.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.185+0000 7f0d3643b640 1 -- 172.21.15.12:0/1175517669 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3015a8c0 con 0x7f0d300cb020 2023-11-28T21:34:21.193 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:21.192+0000 7febf02d9180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T21:34:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d3643b640 1 --2- 172.21.15.12:0/1175517669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300cb020 0x7f0d30159d50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009e50 tx=0x7f0d2801f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90977e46d16b3562 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d34c38640 1 -- 172.21.15.12:0/1175517669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d280af400 con 0x7f0d300cb020 2023-11-28T21:34:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d34c38640 1 -- 172.21.15.12:0/1175517669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d280ab050 con 0x7f0d300cb020 2023-11-28T21:34:21.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d34c38640 1 -- 172.21.15.12:0/1175517669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d280af780 con 0x7f0d300cb020 2023-11-28T21:34:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d37e9e640 1 -- 172.21.15.12:0/1175517669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300cb020 msgr2=0x7f0d30159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d37e9e640 1 --2- 172.21.15.12:0/1175517669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300cb020 0x7f0d30159d50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009e50 tx=0x7f0d2801f1f0 comp rx=0 tx=0).stop 2023-11-28T21:34:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d37e9e640 1 -- 172.21.15.12:0/1175517669 shutdown_connections 2023-11-28T21:34:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d37e9e640 1 --2- 172.21.15.12:0/1175517669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300cb020 0x7f0d30159d50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d37e9e640 1 --2- 172.21.15.12:0/1175517669 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d300cac40 0x7f0d3006b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d37e9e640 1 -- 172.21.15.12:0/1175517669 >> 172.21.15.12:0/1175517669 conn(0x7f0d300720e0 msgr2=0x7f0d300724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d37e9e640 1 -- 172.21.15.12:0/1175517669 shutdown_connections 2023-11-28T21:34:21.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.189+0000 7f0d37e9e640 1 -- 172.21.15.12:0/1175517669 wait complete. 2023-11-28T21:34:21.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d37e9e640 1 Processor -- start 2023-11-28T21:34:21.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d37e9e640 1 -- start start 2023-11-28T21:34:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d37e9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300c4980 0x7f0d300b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d37e9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d300cac40 0x7f0d301f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d37e9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d300cb020 0x7f0d301f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d37e9e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3015c060 con 0x7f0d300c4980 2023-11-28T21:34:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d37e9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3015baa0 con 0x7f0d300cb020 2023-11-28T21:34:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d37e9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3015bd80 con 0x7f0d300cac40 2023-11-28T21:34:21.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d35c3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300c4980 0x7f0d300b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d35c3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300c4980 0x7f0d300b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37626/0 (socket says 172.21.15.12:37626) 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d35c3a640 1 -- 172.21.15.12:0/3166778616 learned_addr learned my addr 172.21.15.12:0/3166778616 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d3643b640 1 --2- 172.21.15.12:0/3166778616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d300cb020 0x7f0d301f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d35439640 1 --2- 172.21.15.12:0/3166778616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d300cac40 0x7f0d301f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d35c3a640 1 -- 172.21.15.12:0/3166778616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d300cac40 msgr2=0x7f0d301f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d35c3a640 1 --2- 172.21.15.12:0/3166778616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d300cac40 0x7f0d301f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d35c3a640 1 -- 172.21.15.12:0/3166778616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d300cb020 msgr2=0x7f0d301f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d35c3a640 1 --2- 172.21.15.12:0/3166778616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d300cb020 0x7f0d301f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d35c3a640 1 -- 172.21.15.12:0/3166778616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d28009b30 con 0x7f0d300c4980 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d3643b640 1 --2- 172.21.15.12:0/3166778616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d300cb020 0x7f0d301f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d35c3a640 1 --2- 172.21.15.12:0/3166778616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300c4980 0x7f0d300b4280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00e930 tx=0x7f0d2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d267fc640 1 -- 172.21.15.12:0/3166778616 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c009690 con 0x7f0d300c4980 2023-11-28T21:34:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d267fc640 1 -- 172.21.15.12:0/3166778616 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c00ca00 con 0x7f0d300c4980 2023-11-28T21:34:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d267fc640 1 -- 172.21.15.12:0/3166778616 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00cd70 con 0x7f0d300c4980 2023-11-28T21:34:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d37e9e640 1 -- 172.21.15.12:0/3166778616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3015b0b0 con 0x7f0d300c4980 2023-11-28T21:34:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d37e9e640 1 -- 172.21.15.12:0/3166778616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3015b670 con 0x7f0d300c4980 2023-11-28T21:34:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d267fc640 1 -- 172.21.15.12:0/3166778616 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0d2c005ca0 con 0x7f0d300c4980 2023-11-28T21:34:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.193+0000 7f0d267fc640 1 -- 172.21.15.12:0/3166778616 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0d2c013470 con 0x7f0d300c4980 2023-11-28T21:34:21.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.197+0000 7f0d37e9e640 1 -- 172.21.15.12:0/3166778616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d30067c70 con 0x7f0d300c4980 2023-11-28T21:34:21.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.197+0000 7f0d267fc640 1 -- 172.21.15.12:0/3166778616 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f0d2c021060 con 0x7f0d300c4980 2023-11-28T21:34:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d37e9e640 1 -- 172.21.15.12:0/3166778616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d3015af10 con 0x7f0d300c4980 2023-11-28T21:34:21.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d267fc640 1 -- 172.21.15.12:0/3166778616 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0d3015af10 con 0x7f0d300c4980 2023-11-28T21:34:21.263 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:21.263 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:20.942681+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:34:21.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d37e9e640 1 -- 172.21.15.12:0/3166778616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300c4980 msgr2=0x7f0d300b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:21.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d37e9e640 1 --2- 172.21.15.12:0/3166778616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300c4980 0x7f0d300b4280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00e930 tx=0x7f0d2c00edf0 comp rx=0 tx=0).stop 2023-11-28T21:34:21.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d37e9e640 1 -- 172.21.15.12:0/3166778616 shutdown_connections 2023-11-28T21:34:21.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d37e9e640 1 --2- 172.21.15.12:0/3166778616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d300cb020 0x7f0d301f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:21.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d37e9e640 1 --2- 172.21.15.12:0/3166778616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0d300cac40 0x7f0d301f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:21.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d37e9e640 1 --2- 172.21.15.12:0/3166778616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0d300c4980 0x7f0d300b4280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d37e9e640 1 -- 172.21.15.12:0/3166778616 >> 172.21.15.12:0/3166778616 conn(0x7f0d300720e0 msgr2=0x7f0d300cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d37e9e640 1 -- 172.21.15.12:0/3166778616 shutdown_connections 2023-11-28T21:34:21.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:21.257+0000 7f0d37e9e640 1 -- 172.21.15.12:0/3166778616 wait complete. 2023-11-28T21:34:21.270 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:21.268+0000 7febf02d9180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T21:34:21.282 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T21:34:21.282 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T21:34:21.325 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:21.324+0000 7febf02d9180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T21:34:21.463 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:21.459+0000 7febf02d9180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T21:34:21.522 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:21.519+0000 7febf02d9180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T21:34:21.605 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-11-28T21:34:21.603+0000 7fd9e7bbd640 -1 Falling back to public interface 2023-11-28T21:34:21.609 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-11-28T21:34:21.607+0000 7fefdb5d1640 -1 Falling back to public interface 2023-11-28T21:34:21.628 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-11-28T21:34:21.627+0000 7fdfa6e8f640 -1 Falling back to public interface 2023-11-28T21:34:21.636 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-11-28T21:34:21.635+0000 7f04bd6a8640 -1 Falling back to public interface 2023-11-28T21:34:21.654 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-11-28T21:34:21.649+0000 7f5828f19640 -1 Falling back to public interface 2023-11-28T21:34:21.655 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-11-28T21:34:21.649+0000 7f6a4652d640 -1 Falling back to public interface 2023-11-28T21:34:21.655 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-11-28T21:34:21.649+0000 7f6a454ef640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-11-28T21:34:21.662 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-11-28T21:34:21.657+0000 7f30da4ac640 -1 Falling back to public interface 2023-11-28T21:34:21.678 INFO:tasks.ceph.mgr.y.smithi012.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-28T21:34:21.678 INFO:tasks.ceph.mgr.y.smithi012.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-28T21:34:21.679 INFO:tasks.ceph.mgr.y.smithi012.stderr: from numpy import show_config as show_numpy_config 2023-11-28T21:34:21.698 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-11-28T21:34:21.693+0000 7fd165ea7640 -1 Falling back to public interface 2023-11-28T21:34:21.732 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:21.725+0000 7f17cf036180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T21:34:21.874 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:21.869+0000 7f17cf036180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T21:34:22.055 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:22.049+0000 7f17cf036180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T21:34:22.089 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T21:34:22.134 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:22.129+0000 7f17cf036180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T21:34:22.212 INFO:tasks.ceph.mgr.x.smithi144.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-28T21:34:22.212 INFO:tasks.ceph.mgr.x.smithi144.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-28T21:34:22.213 INFO:tasks.ceph.mgr.x.smithi144.stderr: from numpy import show_config as show_numpy_config 2023-11-28T21:34:22.223 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a64d2a640 1 Processor -- start 2023-11-28T21:34:22.223 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a64d2a640 1 -- start start 2023-11-28T21:34:22.225 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a64d2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a60151af0 0x7f1a600c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:22.225 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a64d2a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1a60151ed0 0x7f1a600c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:22.225 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a64d2a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a60157b70 con 0x7f1a6006e300 2023-11-28T21:34:22.225 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a64d2a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a600707f0 con 0x7f1a60151ed0 2023-11-28T21:34:22.225 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a64d2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600c7b90 con 0x7f1a60151af0 2023-11-28T21:34:22.226 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a57fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1a60151ed0 0x7f1a600c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:22.226 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a57fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1a60151ed0 0x7f1a600c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37642/0 (socket says 172.21.15.12:37642) 2023-11-28T21:34:22.226 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a57fff640 1 -- 172.21.15.12:0/2512416327 learned_addr learned my addr 172.21.15.12:0/2512416327 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:22.226 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.217+0000 7f1a5e59c640 1 --2- 172.21.15.12:0/2512416327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a60151af0 0x7f1a600c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:22.226 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.221+0000 7f1a57fff640 1 -- 172.21.15.12:0/2512416327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a60151af0 msgr2=0x7f1a600c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:22.226 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.221+0000 7f1a57fff640 1 --2- 172.21.15.12:0/2512416327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a60151af0 0x7f1a600c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:22.226 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.221+0000 7f1a57fff640 1 -- 172.21.15.12:0/2512416327 >> v1:172.21.15.144:6789/0 conn(0x7f1a6006e300 legacy=0x7f1a60072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:22.227 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.221+0000 7f1a57fff640 1 -- 172.21.15.12:0/2512416327 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60157e50 con 0x7f1a60151ed0 2023-11-28T21:34:22.227 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.221+0000 7f1a57fff640 1 --2- 172.21.15.12:0/2512416327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151ed0 0x7f1a600c49d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009870 tx=0x7f1a4801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406f113b75ed74be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:22.228 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.221+0000 7f1a5dd9b640 1 -- 172.21.15.12:0/2512416327 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480ab600 con 0x7f1a60151ed0 2023-11-28T21:34:22.228 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.221+0000 7f1a5dd9b640 1 -- 172.21.15.12:0/2512416327 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a480a7050 con 0x7f1a60151ed0 2023-11-28T21:34:22.228 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.221+0000 7f1a5dd9b640 1 -- 172.21.15.12:0/2512416327 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480ab980 con 0x7f1a60151ed0 2023-11-28T21:34:22.229 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 -- 172.21.15.12:0/2512416327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151ed0 msgr2=0x7f1a600c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:22.229 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 --2- 172.21.15.12:0/2512416327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151ed0 0x7f1a600c49d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009870 tx=0x7f1a4801c540 comp rx=0 tx=0).stop 2023-11-28T21:34:22.230 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 -- 172.21.15.12:0/2512416327 shutdown_connections 2023-11-28T21:34:22.230 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 --2- 172.21.15.12:0/2512416327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151ed0 0x7f1a600c49d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:22.230 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 --2- 172.21.15.12:0/2512416327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a60151af0 0x7f1a600c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:22.230 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 -- 172.21.15.12:0/2512416327 >> 172.21.15.12:0/2512416327 conn(0x7f1a60066360 msgr2=0x7f1a60066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:22.230 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 -- 172.21.15.12:0/2512416327 shutdown_connections 2023-11-28T21:34:22.230 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 -- 172.21.15.12:0/2512416327 wait complete. 2023-11-28T21:34:22.231 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 Processor -- start 2023-11-28T21:34:22.231 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 -- start start 2023-11-28T21:34:22.233 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a6006e300 0x7f1a6014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:22.233 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151af0 0x7f1a6013ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:22.234 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6014db40 0x7f1a601f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:22.234 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a601439b0 con 0x7f1a60151af0 2023-11-28T21:34:22.234 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6014d3b0 con 0x7f1a6006e300 2023-11-28T21:34:22.234 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a64d2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a60143080 con 0x7f1a6014db40 2023-11-28T21:34:22.234 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a57fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151af0 0x7f1a6013ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:22.235 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a5e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a6006e300 0x7f1a6014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:22.235 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a5e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a6006e300 0x7f1a6014cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:55710/0 (socket says 172.21.15.12:55710) 2023-11-28T21:34:22.235 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a57fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151af0 0x7f1a6013ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37644/0 (socket says 172.21.15.12:37644) 2023-11-28T21:34:22.235 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a5e59c640 1 -- 172.21.15.12:0/3649615623 learned_addr learned my addr 172.21.15.12:0/3649615623 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:22.235 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a5ed9d640 1 --2- 172.21.15.12:0/3649615623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6014db40 0x7f1a601f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:22.235 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a57fff640 1 -- 172.21.15.12:0/3649615623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6014db40 msgr2=0x7f1a601f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:22.236 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a57fff640 1 --2- 172.21.15.12:0/3649615623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6014db40 0x7f1a601f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:22.236 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a57fff640 1 -- 172.21.15.12:0/3649615623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a6006e300 msgr2=0x7f1a6014cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:22.236 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a57fff640 1 --2- 172.21.15.12:0/3649615623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a6006e300 0x7f1a6014cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:22.236 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.225+0000 7f1a57fff640 1 -- 172.21.15.12:0/3649615623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a480094d0 con 0x7f1a60151af0 2023-11-28T21:34:22.236 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.229+0000 7f1a57fff640 1 --2- 172.21.15.12:0/3649615623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151af0 0x7f1a6013ea50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1a600c55a0 tx=0x7f1a480192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:22.236 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.229+0000 7f1a56ffd640 1 -- 172.21.15.12:0/3649615623 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480b4620 con 0x7f1a60151af0 2023-11-28T21:34:22.236 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.229+0000 7f1a56ffd640 1 -- 172.21.15.12:0/3649615623 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a480b47a0 con 0x7f1a60151af0 2023-11-28T21:34:22.237 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.229+0000 7f1a64d2a640 1 -- 172.21.15.12:0/3649615623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a60143360 con 0x7f1a60151af0 2023-11-28T21:34:22.237 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.229+0000 7f1a56ffd640 1 -- 172.21.15.12:0/3649615623 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480b4b50 con 0x7f1a60151af0 2023-11-28T21:34:22.237 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.229+0000 7f1a64d2a640 1 -- 172.21.15.12:0/3649615623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6014d670 con 0x7f1a60151af0 2023-11-28T21:34:22.237 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.229+0000 7f1a56ffd640 1 -- 172.21.15.12:0/3649615623 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1a480ba020 con 0x7f1a60151af0 2023-11-28T21:34:22.237 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.229+0000 7f1a56ffd640 1 -- 172.21.15.12:0/3649615623 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f1a480bcaa0 con 0x7f1a60151af0 2023-11-28T21:34:22.238 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.229+0000 7f1a64d2a640 1 -- 172.21.15.12:0/3649615623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a60143810 con 0x7f1a60151af0 2023-11-28T21:34:22.238 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.233+0000 7f1a56ffd640 1 -- 172.21.15.12:0/3649615623 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f1a60143810 con 0x7f1a60151af0 2023-11-28T21:34:22.254 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:22.251+0000 7febf02d9180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T21:34:22.284 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:22.277+0000 7f17cf036180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T21:34:22.295 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.289+0000 7f1a64d2a640 1 -- 172.21.15.12:0/3649615623 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a60143b60 con 0x7f1a60151af0 2023-11-28T21:34:22.296 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.289+0000 7f1a56ffd640 1 -- 172.21.15.12:0/3649615623 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1a60143b60 con 0x7f1a60151af0 2023-11-28T21:34:22.296 INFO:teuthology.misc.health.smithi012.stdout: 2023-11-28T21:34:22.296 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":10,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:20.942681+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:34:22.298 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.293+0000 7f1a64d2a640 1 -- 172.21.15.12:0/3649615623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151af0 msgr2=0x7f1a6013ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:22.298 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.293+0000 7f1a64d2a640 1 --2- 172.21.15.12:0/3649615623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151af0 0x7f1a6013ea50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1a600c55a0 tx=0x7f1a480192e0 comp rx=0 tx=0).stop 2023-11-28T21:34:22.298 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.293+0000 7f1a64d2a640 1 -- 172.21.15.12:0/3649615623 shutdown_connections 2023-11-28T21:34:22.298 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.293+0000 7f1a64d2a640 1 --2- 172.21.15.12:0/3649615623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a6014db40 0x7f1a601f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:22.298 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.293+0000 7f1a64d2a640 1 --2- 172.21.15.12:0/3649615623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a60151af0 0x7f1a6013ea50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:22.298 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.293+0000 7f1a64d2a640 1 --2- 172.21.15.12:0/3649615623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a6006e300 0x7f1a6014cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:22.299 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.293+0000 7f1a64d2a640 1 -- 172.21.15.12:0/3649615623 >> 172.21.15.12:0/3649615623 conn(0x7f1a60066360 msgr2=0x7f1a60073a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:22.299 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.293+0000 7f1a64d2a640 1 -- 172.21.15.12:0/3649615623 shutdown_connections 2023-11-28T21:34:22.299 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:22.293+0000 7f1a64d2a640 1 -- 172.21.15.12:0/3649615623 wait complete. 2023-11-28T21:34:22.315 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T21:34:22.385 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:22.383+0000 7febf02d9180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T21:34:22.545 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:22.543+0000 7febf02d9180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T21:34:22.612 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:22.611+0000 7febf02d9180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T21:34:22.736 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:22.735+0000 7febf02d9180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T21:34:22.999 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:22.993+0000 7f17cf036180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T21:34:23.090 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:23.085+0000 7f17cf036180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T21:34:23.292 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:23.285+0000 7f17cf036180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T21:34:23.346 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:23.343+0000 7febf02d9180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T21:34:23.396 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:23.377+0000 7f17cf036180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T21:34:23.421 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:23.419+0000 7febf02d9180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T21:34:23.591 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:23.587+0000 7febf02d9180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T21:34:23.663 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:23.659+0000 7febf02d9180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T21:34:23.728 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:23.721+0000 7f17cf036180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T21:34:23.800 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:23.793+0000 7f17cf036180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T21:34:23.955 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:23.951+0000 7febf02d9180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T21:34:24.014 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-11-28T21:34:24.011+0000 7febf02d9180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T21:34:24.818 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:24.813+0000 7f17768ad640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T21:34:24.818 INFO:tasks.ceph.mgr.y.smithi012.stderr:2023-11-28T21:34:24.813+0000 7f17768ad640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T21:34:25.426 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-11-28T21:34:25.423+0000 7fd9e7bbd640 -1 osd.4 0 log_to_monitors true 2023-11-28T21:34:25.469 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-11-28T21:34:25.467+0000 7fefdb5d1640 -1 osd.7 0 log_to_monitors true 2023-11-28T21:34:25.521 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-11-28T21:34:25.519+0000 7fdfa6e8f640 -1 osd.5 0 log_to_monitors true 2023-11-28T21:34:25.578 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-11-28T21:34:25.575+0000 7f04bd6a8640 -1 osd.6 0 log_to_monitors true 2023-11-28T21:34:25.584 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-11-28T21:34:25.577+0000 7f5828f19640 -1 osd.2 0 log_to_monitors true 2023-11-28T21:34:25.664 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-11-28T21:34:25.657+0000 7f6a4652d640 -1 osd.1 0 log_to_monitors true 2023-11-28T21:34:25.717 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-11-28T21:34:25.713+0000 7f30da4ac640 -1 osd.0 0 log_to_monitors true 2023-11-28T21:34:25.724 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-11-28T21:34:25.717+0000 7fd165ea7640 -1 osd.3 0 log_to_monitors true 2023-11-28T21:34:28.011 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-11-28T21:34:28.007+0000 7f04b9e63640 -1 osd.6 0 waiting for initial osdmap 2023-11-28T21:34:28.032 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-11-28T21:34:28.031+0000 7f04b148c640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T21:34:28.049 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-11-28T21:34:28.047+0000 7fefd757a640 -1 osd.7 0 waiting for initial osdmap 2023-11-28T21:34:28.068 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-11-28T21:34:28.067+0000 7fd9e4378640 -1 osd.4 0 waiting for initial osdmap 2023-11-28T21:34:28.073 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-11-28T21:34:28.071+0000 7fefcf3b5640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T21:34:28.089 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-11-28T21:34:28.087+0000 7fdfa2e38640 -1 osd.5 0 waiting for initial osdmap 2023-11-28T21:34:28.092 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-11-28T21:34:28.091+0000 7fd9db9a1640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T21:34:28.124 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-11-28T21:34:28.123+0000 7fdf9ac73640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T21:34:28.491 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-11-28T21:34:28.489+0000 7f6a424d8640 -1 osd.1 0 waiting for initial osdmap 2023-11-28T21:34:28.499 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-11-28T21:34:28.497+0000 7f5824ec4640 -1 osd.2 0 waiting for initial osdmap 2023-11-28T21:34:28.519 INFO:tasks.ceph.osd.1.smithi012.stderr:2023-11-28T21:34:28.517+0000 7f6a3a313640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T21:34:28.521 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-11-28T21:34:28.521+0000 7f30d6455640 -1 osd.0 0 waiting for initial osdmap 2023-11-28T21:34:28.530 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-11-28T21:34:28.529+0000 7fd161e50640 -1 osd.3 0 waiting for initial osdmap 2023-11-28T21:34:28.534 INFO:tasks.ceph.osd.2.smithi012.stderr:2023-11-28T21:34:28.533+0000 7f581ccff640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T21:34:28.570 INFO:tasks.ceph.osd.3.smithi012.stderr:2023-11-28T21:34:28.569+0000 7fd159c8b640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T21:34:28.570 INFO:tasks.ceph.osd.0.smithi012.stderr:2023-11-28T21:34:28.569+0000 7f30ce290640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T21:34:29.122 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T21:34:29.265 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b42632640 1 Processor -- start 2023-11-28T21:34:29.265 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b42632640 1 -- start start 2023-11-28T21:34:29.265 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b42632640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5b3c154060 0x7f5b3c0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.265 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b42632640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 0x7f5b3c066360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.266 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b42632640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3c15ac70 con 0x7f5b3c066f10 2023-11-28T21:34:29.266 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b42632640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3c154440 con 0x7f5b3c154060 2023-11-28T21:34:29.267 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b42632640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c0cb1d0 con 0x7f5b3c0ca2e0 2023-11-28T21:34:29.267 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b40bcf640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5b3c066f10 0x7f5b3c06a890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54296/0 (socket says 172.21.15.12:54296) 2023-11-28T21:34:29.267 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b40bcf640 1 -- 172.21.15.12:0/2852988219 learned_addr learned my addr 172.21.15.12:0/2852988219 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:29.267 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b3bfff640 1 --2- 172.21.15.12:0/2852988219 >> v2:172.21.15.144:3300/0 conn(0x7f5b3c154060 0x7f5b3c0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.267 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b3b7fe640 1 --2- 172.21.15.12:0/2852988219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 0x7f5b3c066360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.267 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b3affd640 1 -- 172.21.15.12:0/2852988219 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630138900 0 0) 0x7f5b3c15ac70 con 0x7f5b3c066f10 2023-11-28T21:34:29.268 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b3affd640 1 -- 172.21.15.12:0/2852988219 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b24003530 con 0x7f5b3c066f10 2023-11-28T21:34:29.268 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b3affd640 1 -- 172.21.15.12:0/2852988219 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1217671613 0 0) 0x7f5b24003530 con 0x7f5b3c066f10 2023-11-28T21:34:29.268 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b3bfff640 1 -- 172.21.15.12:0/2852988219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 msgr2=0x7f5b3c066360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:29.268 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b3bfff640 1 --2- 172.21.15.12:0/2852988219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 0x7f5b3c066360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.268 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b3bfff640 1 -- 172.21.15.12:0/2852988219 >> v1:172.21.15.12:6789/0 conn(0x7f5b3c066f10 legacy=0x7f5b3c06a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.268 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b3bfff640 1 -- 172.21.15.12:0/2852988219 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c159da0 con 0x7f5b3c154060 2023-11-28T21:34:29.268 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.261+0000 7f5b3b7fe640 1 --2- 172.21.15.12:0/2852988219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 0x7f5b3c066360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:34:29.272 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b3bfff640 1 --2- 172.21.15.12:0/2852988219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c154060 0x7f5b3c0cac90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0097c0 tx=0x7f5b2c01c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3094e3543b5e6769 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:29.272 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b3affd640 1 -- 172.21.15.12:0/2852988219 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34001a00 con 0x7f5b3c154060 2023-11-28T21:34:29.272 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b3affd640 1 -- 172.21.15.12:0/2852988219 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c0a7050 con 0x7f5b3c154060 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b3affd640 1 -- 172.21.15.12:0/2852988219 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0ab600 con 0x7f5b3c154060 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b42632640 1 -- 172.21.15.12:0/2852988219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c154060 msgr2=0x7f5b3c0cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b42632640 1 --2- 172.21.15.12:0/2852988219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c154060 0x7f5b3c0cac90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0097c0 tx=0x7f5b2c01c310 comp rx=0 tx=0).stop 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b42632640 1 -- 172.21.15.12:0/2852988219 shutdown_connections 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b42632640 1 --2- 172.21.15.12:0/2852988219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 0x7f5b3c066360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b42632640 1 --2- 172.21.15.12:0/2852988219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c154060 0x7f5b3c0cac90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b42632640 1 -- 172.21.15.12:0/2852988219 >> 172.21.15.12:0/2852988219 conn(0x7f5b3c0cf5b0 msgr2=0x7f5b3c0cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b42632640 1 -- 172.21.15.12:0/2852988219 shutdown_connections 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.265+0000 7f5b42632640 1 -- 172.21.15.12:0/2852988219 wait complete. 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b42632640 1 Processor -- start 2023-11-28T21:34:29.273 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b42632640 1 -- start start 2023-11-28T21:34:29.274 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b42632640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c066f10 0x7f5b3c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.274 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b42632640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 0x7f5b3c0cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.274 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b42632640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c154060 0x7f5b3c0c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.274 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b42632640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c0c6840 con 0x7f5b3c066f10 2023-11-28T21:34:29.274 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b42632640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c15c0b0 con 0x7f5b3c154060 2023-11-28T21:34:29.274 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b42632640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c15c670 con 0x7f5b3c0ca2e0 2023-11-28T21:34:29.274 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c066f10 0x7f5b3c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.274 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c066f10 0x7f5b3c15cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59076/0 (socket says 172.21.15.12:59076) 2023-11-28T21:34:29.274 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b3bfff640 1 -- 172.21.15.12:0/1841799035 learned_addr learned my addr 172.21.15.12:0/1841799035 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:29.274 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b40bcf640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c154060 0x7f5b3c0c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b3b7fe640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 0x7f5b3c0cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b3bfff640 1 -- 172.21.15.12:0/1841799035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 msgr2=0x7f5b3c0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b3bfff640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 0x7f5b3c0cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b3bfff640 1 -- 172.21.15.12:0/1841799035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c154060 msgr2=0x7f5b3c0c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b3bfff640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c154060 0x7f5b3c0c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b3bfff640 1 -- 172.21.15.12:0/1841799035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b30009420 con 0x7f5b3c066f10 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b3bfff640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c066f10 0x7f5b3c15cb00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009710 tx=0x7f5b2c0190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b38ff9640 1 -- 172.21.15.12:0/1841799035 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c003a30 con 0x7f5b3c066f10 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b38ff9640 1 -- 172.21.15.12:0/1841799035 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c003b90 con 0x7f5b3c066f10 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b38ff9640 1 -- 172.21.15.12:0/1841799035 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0ab400 con 0x7f5b3c066f10 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b42632640 1 -- 172.21.15.12:0/1841799035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c009420 con 0x7f5b3c066f10 2023-11-28T21:34:29.275 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.269+0000 7f5b42632640 1 -- 172.21.15.12:0/1841799035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c0c6d70 con 0x7f5b3c066f10 2023-11-28T21:34:29.277 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.273+0000 7f5b42632640 1 -- 172.21.15.12:0/1841799035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3c074f60 con 0x7f5b3c066f10 2023-11-28T21:34:29.278 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.273+0000 7f5b38ff9640 1 -- 172.21.15.12:0/1841799035 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 85796+0+0 (secure 0 0 0) 0x7f5b2c0b2040 con 0x7f5b3c066f10 2023-11-28T21:34:29.278 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.273+0000 7f5b38ff9640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b280663e0 0x7f5b280688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.278 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.273+0000 7f5b38ff9640 1 -- 172.21.15.12:0/1841799035 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f5b2c11d710 con 0x7f5b3c066f10 2023-11-28T21:34:29.278 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.273+0000 7f5b3b7fe640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b280663e0 0x7f5b280688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.278 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.273+0000 7f5b3b7fe640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b280663e0 0x7f5b280688a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c144170 tx=0x7f5b30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:29.280 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.277+0000 7f5b38ff9640 1 -- 172.21.15.12:0/1841799035 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5b2c0eb010 con 0x7f5b3c066f10 2023-11-28T21:34:29.439 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.437+0000 7f5b42632640 1 -- 172.21.15.12:0/1841799035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b3c0c99e0 con 0x7f5b3c066f10 2023-11-28T21:34:29.440 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.437+0000 7f5b38ff9640 1 -- 172.21.15.12:0/1841799035 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10920 (secure 0 0 0) 0x7f5b2c0b0070 con 0x7f5b3c066f10 2023-11-28T21:34:29.440 INFO:teuthology.misc.health.smithi012.stdout: 2023-11-28T21:34:29.440 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":13,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:29.023059+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:34:29.442 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 -- 172.21.15.12:0/1841799035 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b280663e0 msgr2=0x7f5b280688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.442 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b280663e0 0x7f5b280688a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c144170 tx=0x7f5b30008040 comp rx=0 tx=0).stop 2023-11-28T21:34:29.442 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 -- 172.21.15.12:0/1841799035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c066f10 msgr2=0x7f5b3c15cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.442 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c066f10 0x7f5b3c15cb00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009710 tx=0x7f5b2c0190b0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.442 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 -- 172.21.15.12:0/1841799035 shutdown_connections 2023-11-28T21:34:29.442 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b280663e0 0x7f5b280688a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.442 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b3c154060 0x7f5b3c0c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.442 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b3c0ca2e0 0x7f5b3c0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.442 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 --2- 172.21.15.12:0/1841799035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b3c066f10 0x7f5b3c15cb00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.443 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 -- 172.21.15.12:0/1841799035 >> 172.21.15.12:0/1841799035 conn(0x7f5b3c0cf5b0 msgr2=0x7f5b3c1443e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:29.443 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 -- 172.21.15.12:0/1841799035 shutdown_connections 2023-11-28T21:34:29.443 INFO:teuthology.misc.health.smithi012.stderr:2023-11-28T21:34:29.441+0000 7f5b42632640 1 -- 172.21.15.12:0/1841799035 wait complete. 2023-11-28T21:34:29.460 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T21:34:29.460 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T21:34:29.460 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T21:34:29.460 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T21:34:29.464 INFO:tasks.ceph.mds.a:Started 2023-11-28T21:34:29.465 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T21:34:29.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T21:34:29.469 INFO:tasks.ceph.mds.c:Started 2023-11-28T21:34:29.469 INFO:tasks.ceph.mds.e:Restarting daemon 2023-11-28T21:34:29.469 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-11-28T21:34:29.473 INFO:tasks.ceph.mds.e:Started 2023-11-28T21:34:29.473 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T21:34:29.473 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T21:34:29.476 INFO:tasks.ceph.mds.b:Started 2023-11-28T21:34:29.476 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T21:34:29.477 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T21:34:29.480 INFO:tasks.ceph.mds.d:Started 2023-11-28T21:34:29.480 INFO:tasks.ceph.mds.f:Restarting daemon 2023-11-28T21:34:29.480 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-11-28T21:34:29.483 INFO:tasks.ceph.mds.f:Started 2023-11-28T21:34:29.483 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T21:34:29.484 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:34:29.554 INFO:tasks.ceph.mds.f.smithi144.stdout:starting mds.f at 2023-11-28T21:34:29.563 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-11-28T21:34:29.564 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-11-28T21:34:29.566 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-11-28T21:34:29.567 INFO:tasks.ceph.mds.c.smithi012.stdout:starting mds.c at 2023-11-28T21:34:29.568 INFO:tasks.ceph.mds.e.smithi012.stdout:starting mds.e at 2023-11-28T21:34:29.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.633+0000 7f0aceddf640 1 Processor -- start 2023-11-28T21:34:29.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.633+0000 7f0aceddf640 1 -- start start 2023-11-28T21:34:29.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0aceddf640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0ac815d320 0x7f0ac806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0aceddf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac806aaf0 0x7f0ac80c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0aceddf640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac815e270 con 0x7f0ac8154100 2023-11-28T21:34:29.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0aceddf640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac80c4e60 con 0x7f0ac815d320 2023-11-28T21:34:29.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0aceddf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac815d740 con 0x7f0ac806aaf0 2023-11-28T21:34:29.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0ac7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0ac815d320 0x7f0ac806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0accb7b640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0ac8154100 0x7f0ac80c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:33460/0 (socket says 172.21.15.12:33460) 2023-11-28T21:34:29.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0ac7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0ac815d320 0x7f0ac806a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59130/0 (socket says 172.21.15.12:59130) 2023-11-28T21:34:29.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0ac7fff640 1 -- 172.21.15.12:0/3424750747 learned_addr learned my addr 172.21.15.12:0/3424750747 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:29.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0acd37c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac806aaf0 0x7f0ac80c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0ac7fff640 1 -- 172.21.15.12:0/3424750747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac806aaf0 msgr2=0x7f0ac80c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:29.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0ac7fff640 1 --2- 172.21.15.12:0/3424750747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac806aaf0 0x7f0ac80c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0ac7fff640 1 -- 172.21.15.12:0/3424750747 >> v1:172.21.15.144:6789/0 conn(0x7f0ac8154100 legacy=0x7f0ac80c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.637+0000 7f0ac7fff640 1 -- 172.21.15.12:0/3424750747 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac81544e0 con 0x7f0ac815d320 2023-11-28T21:34:29.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0ac7fff640 1 --2- 172.21.15.12:0/3424750747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac815d320 0x7f0ac806a5b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009870 tx=0x7f0ab401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b29d6fe432dab7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0ac77fe640 1 -- 172.21.15.12:0/3424750747 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40ab620 con 0x7f0ac815d320 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0ac77fe640 1 -- 172.21.15.12:0/3424750747 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab40ab780 con 0x7f0ac815d320 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0ac77fe640 1 -- 172.21.15.12:0/3424750747 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40abb10 con 0x7f0ac815d320 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0aceddf640 1 -- 172.21.15.12:0/3424750747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac815d320 msgr2=0x7f0ac806a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0aceddf640 1 --2- 172.21.15.12:0/3424750747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac815d320 0x7f0ac806a5b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009870 tx=0x7f0ab401c510 comp rx=0 tx=0).stop 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0aceddf640 1 -- 172.21.15.12:0/3424750747 shutdown_connections 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0aceddf640 1 --2- 172.21.15.12:0/3424750747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac806aaf0 0x7f0ac80c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0aceddf640 1 --2- 172.21.15.12:0/3424750747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac815d320 0x7f0ac806a5b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0aceddf640 1 -- 172.21.15.12:0/3424750747 >> 172.21.15.12:0/3424750747 conn(0x7f0ac806c690 msgr2=0x7f0ac8066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0aceddf640 1 -- 172.21.15.12:0/3424750747 shutdown_connections 2023-11-28T21:34:29.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0aceddf640 1 -- 172.21.15.12:0/3424750747 wait complete. 2023-11-28T21:34:29.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0aceddf640 1 Processor -- start 2023-11-28T21:34:29.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.641+0000 7f0aceddf640 1 -- start start 2023-11-28T21:34:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0aceddf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac806aaf0 0x7f0ac81523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0aceddf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac8154100 0x7f0ac815c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0aceddf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac815d320 0x7f0ac8140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0aceddf640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac8147230 con 0x7f0ac8154100 2023-11-28T21:34:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0aceddf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac8151f60 con 0x7f0ac815d320 2023-11-28T21:34:29.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0aceddf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac8151c80 con 0x7f0ac806aaf0 2023-11-28T21:34:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac8154100 0x7f0ac815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac8154100 0x7f0ac815c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59136/0 (socket says 172.21.15.12:59136) 2023-11-28T21:34:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac7fff640 1 -- 172.21.15.12:0/3296238307 learned_addr learned my addr 172.21.15.12:0/3296238307 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0accb7b640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac806aaf0 0x7f0ac81523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0acd37c640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac815d320 0x7f0ac8140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac7fff640 1 -- 172.21.15.12:0/3296238307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac806aaf0 msgr2=0x7f0ac81523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac7fff640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac806aaf0 0x7f0ac81523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac7fff640 1 -- 172.21.15.12:0/3296238307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac815d320 msgr2=0x7f0ac8140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac7fff640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac815d320 0x7f0ac8140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac7fff640 1 -- 172.21.15.12:0/3296238307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40094d0 con 0x7f0ac8154100 2023-11-28T21:34:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac7fff640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac8154100 0x7f0ac815c950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ab401a140 tx=0x7f0ab401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac57fa640 1 -- 172.21.15.12:0/3296238307 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40abe80 con 0x7f0ac8154100 2023-11-28T21:34:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac57fa640 1 -- 172.21.15.12:0/3296238307 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab40be3f0 con 0x7f0ac8154100 2023-11-28T21:34:29.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0aceddf640 1 -- 172.21.15.12:0/3296238307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac81f6d70 con 0x7f0ac8154100 2023-11-28T21:34:29.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac57fa640 1 -- 172.21.15.12:0/3296238307 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40be780 con 0x7f0ac8154100 2023-11-28T21:34:29.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0aceddf640 1 -- 172.21.15.12:0/3296238307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac8147590 con 0x7f0ac8154100 2023-11-28T21:34:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0aceddf640 1 -- 172.21.15.12:0/3296238307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac8067c40 con 0x7f0ac8154100 2023-11-28T21:34:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.645+0000 7f0ac57fa640 1 -- 172.21.15.12:0/3296238307 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 85796+0+0 (secure 0 0 0) 0x7f0ab40b43f0 con 0x7f0ac8154100 2023-11-28T21:34:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.649+0000 7f0ac57fa640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0aac0663e0 0x7f0aac0688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.649+0000 7f0ac57fa640 1 -- 172.21.15.12:0/3296238307 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f0ab411d4b0 con 0x7f0ac8154100 2023-11-28T21:34:29.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.649+0000 7f0accb7b640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0aac0663e0 0x7f0aac0688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.649+0000 7f0accb7b640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0aac0663e0 0x7f0aac0688a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0abc000f70 tx=0x7f0abc007e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:29.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.653+0000 7f0ac57fa640 1 -- 172.21.15.12:0/3296238307 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0ab40eaf80 con 0x7f0ac8154100 2023-11-28T21:34:29.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.813+0000 7f0aceddf640 1 -- 172.21.15.12:0/3296238307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ac8074f60 con 0x7f0ac8154100 2023-11-28T21:34:29.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.813+0000 7f0ac57fa640 1 -- 172.21.15.12:0/3296238307 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10920 (secure 0 0 0) 0x7f0ab40ee5a0 con 0x7f0ac8154100 2023-11-28T21:34:29.815 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:29.815 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:29.023059+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:34:29.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 -- 172.21.15.12:0/3296238307 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0aac0663e0 msgr2=0x7f0aac0688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0aac0663e0 0x7f0aac0688a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0abc000f70 tx=0x7f0abc007e50 comp rx=0 tx=0).stop 2023-11-28T21:34:29.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 -- 172.21.15.12:0/3296238307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac8154100 msgr2=0x7f0ac815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac8154100 0x7f0ac815c950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ab401a140 tx=0x7f0ab401ad50 comp rx=0 tx=0).stop 2023-11-28T21:34:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 -- 172.21.15.12:0/3296238307 shutdown_connections 2023-11-28T21:34:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0aac0663e0 0x7f0aac0688a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ac815d320 0x7f0ac8140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ac8154100 0x7f0ac815c950 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 --2- 172.21.15.12:0/3296238307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ac806aaf0 0x7f0ac81523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 -- 172.21.15.12:0/3296238307 >> 172.21.15.12:0/3296238307 conn(0x7f0ac806c690 msgr2=0x7f0ac80cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 -- 172.21.15.12:0/3296238307 shutdown_connections 2023-11-28T21:34:29.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.817+0000 7f0aceddf640 1 -- 172.21.15.12:0/3296238307 wait complete. 2023-11-28T21:34:29.837 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-11-28T21:34:29.837 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:34:29.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.981+0000 7f367defe640 1 Processor -- start 2023-11-28T21:34:29.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.981+0000 7f367defe640 1 -- start start 2023-11-28T21:34:29.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f367defe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3678143070 0x7f367815a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f367defe640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f367806f050 con 0x7f367815b340 2023-11-28T21:34:29.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f367defe640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f367815ae50 con 0x7f367815b760 2023-11-28T21:34:29.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f367defe640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f367815bb70 con 0x7f3678143070 2023-11-28T21:34:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f3676ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f367815b340 0x7f367806a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54318/0 (socket says 172.21.15.12:54318) 2023-11-28T21:34:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f3676ffd640 1 -- 172.21.15.12:0/3244279579 learned_addr learned my addr 172.21.15.12:0/3244279579 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36777fe640 1 --2- 172.21.15.12:0/3244279579 >> v2:172.21.15.144:3300/0 conn(0x7f3678143070 0x7f367815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675397973 0 0) 0x7f367806f050 con 0x7f367815b340 2023-11-28T21:34:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3660003530 con 0x7f367815b340 2023-11-28T21:34:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 833106374 0 0) 0x7f3660003530 con 0x7f367815b340 2023-11-28T21:34:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 >> v2:172.21.15.144:3300/0 conn(0x7f3678143070 msgr2=0x7f367815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 --2- 172.21.15.12:0/3244279579 >> v2:172.21.15.144:3300/0 conn(0x7f3678143070 0x7f367815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 >> v1:172.21.15.144:6789/0 conn(0x7f367815b760 legacy=0x7f367806acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367806b3d0 con 0x7f367815b340 2023-11-28T21:34:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f366c001680 con 0x7f367815b340 2023-11-28T21:34:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 >> v1:172.21.15.12:6789/0 conn(0x7f367815b340 legacy=0x7f367806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36600039f0 0x7f3660003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36600047d0 0x7f3660004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f366000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367815bb70 con 0x7f3660005190 2023-11-28T21:34:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366000c0c0 con 0x7f36600047d0 2023-11-28T21:34:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366000c280 con 0x7f36600039f0 2023-11-28T21:34:29.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f3677fff640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f366000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f36777fe640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36600039f0 0x7f3660003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f3676ffd640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36600047d0 0x7f3660004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f3677fff640 1 -- 172.21.15.12:0/3244279579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36600039f0 msgr2=0x7f3660003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f3677fff640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36600039f0 0x7f3660003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f3677fff640 1 -- 172.21.15.12:0/3244279579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36600047d0 msgr2=0x7f3660004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f3677fff640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36600047d0 0x7f3660004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.985+0000 7f3677fff640 1 -- 172.21.15.12:0/3244279579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3660003530 con 0x7f3660005190 2023-11-28T21:34:29.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f3677fff640 1 -- 172.21.15.12:0/3244279579 reap_dead start 2023-11-28T21:34:29.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f3677fff640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f366000b990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3670001e70 tx=0x7f367001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c39308dec67394 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:29.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670003b80 con 0x7f3660005190 2023-11-28T21:34:29.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3670003d00 con 0x7f3660005190 2023-11-28T21:34:29.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f36767fc640 1 -- 172.21.15.12:0/3244279579 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3670005260 con 0x7f3660005190 2023-11-28T21:34:29.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f367defe640 1 -- 172.21.15.12:0/3244279579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 msgr2=0x7f366000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f367defe640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f366000b990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3670001e70 tx=0x7f367001c760 comp rx=0 tx=0).stop 2023-11-28T21:34:29.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f367defe640 1 -- 172.21.15.12:0/3244279579 shutdown_connections 2023-11-28T21:34:29.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f367defe640 1 --2- 172.21.15.12:0/3244279579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f366000b990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f367defe640 1 -- 172.21.15.12:0/3244279579 >> 172.21.15.12:0/3244279579 conn(0x7f367806b6c0 msgr2=0x7f367806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:29.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f367defe640 1 -- 172.21.15.12:0/3244279579 shutdown_connections 2023-11-28T21:34:29.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.989+0000 7f367defe640 1 -- 172.21.15.12:0/3244279579 wait complete. 2023-11-28T21:34:29.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 Processor -- start 2023-11-28T21:34:29.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 -- start start 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f3678155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f367813d390 0x7f367813bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3678156b90 0x7f367815a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367813b9a0 con 0x7f3660005190 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36781539e0 con 0x7f3678156b90 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3678153c10 con 0x7f367813d390 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f36777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f3678155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f36777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f3678155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59160/0 (socket says 172.21.15.12:59160) 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f36777fe640 1 -- 172.21.15.12:0/3202706872 learned_addr learned my addr 172.21.15.12:0/3202706872 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f3676ffd640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f367813d390 0x7f367813bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f3677fff640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3678156b90 0x7f367815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f36777fe640 1 -- 172.21.15.12:0/3202706872 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f367813d390 msgr2=0x7f367813bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f36777fe640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f367813d390 0x7f367813bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f36777fe640 1 -- 172.21.15.12:0/3202706872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3678156b90 msgr2=0x7f367815a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f36777fe640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3678156b90 0x7f367815a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f36777fe640 1 -- 172.21.15.12:0/3202706872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367813e020 con 0x7f3660005190 2023-11-28T21:34:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f36777fe640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f3678155110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3668011960 tx=0x7f3668011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:29.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f365ffff640 1 -- 172.21.15.12:0/3202706872 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366800faf0 con 0x7f3660005190 2023-11-28T21:34:29.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f365ffff640 1 -- 172.21.15.12:0/3202706872 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366800fc50 con 0x7f3660005190 2023-11-28T21:34:29.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f365ffff640 1 -- 172.21.15.12:0/3202706872 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3668014440 con 0x7f3660005190 2023-11-28T21:34:29.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 -- 172.21.15.12:0/3202706872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367813e2e0 con 0x7f3660005190 2023-11-28T21:34:29.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 -- 172.21.15.12:0/3202706872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367813b5c0 con 0x7f3660005190 2023-11-28T21:34:29.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.993+0000 7f367defe640 1 -- 172.21.15.12:0/3202706872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367806a100 con 0x7f3660005190 2023-11-28T21:34:29.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.997+0000 7f365ffff640 1 -- 172.21.15.12:0/3202706872 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 85796+0+0 (secure 0 0 0) 0x7f36680145a0 con 0x7f3660005190 2023-11-28T21:34:29.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.997+0000 7f365ffff640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36640663e0 0x7f36640688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:29.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.997+0000 7f365ffff640 1 -- 172.21.15.12:0/3202706872 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f36680841d0 con 0x7f3660005190 2023-11-28T21:34:29.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.997+0000 7f3676ffd640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36640663e0 0x7f36640688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:29.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:29.997+0000 7f3676ffd640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36640663e0 0x7f36640688a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f366c001060 tx=0x7f366c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:30.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.001+0000 7f365ffff640 1 -- 172.21.15.12:0/3202706872 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3668051ad0 con 0x7f3660005190 2023-11-28T21:34:30.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 -- 172.21.15.12:0/3202706872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3678074f60 con 0x7f3660005190 2023-11-28T21:34:30.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f365ffff640 1 -- 172.21.15.12:0/3202706872 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f3668022090 con 0x7f3660005190 2023-11-28T21:34:30.199 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:30.200 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-28T21:34:30.201 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 2 2023-11-28T21:34:30.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 -- 172.21.15.12:0/3202706872 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36640663e0 msgr2=0x7f36640688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:30.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36640663e0 0x7f36640688a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f366c001060 tx=0x7f366c009040 comp rx=0 tx=0).stop 2023-11-28T21:34:30.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 -- 172.21.15.12:0/3202706872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 msgr2=0x7f3678155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:30.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f3678155110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3668011960 tx=0x7f3668011e20 comp rx=0 tx=0).stop 2023-11-28T21:34:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 -- 172.21.15.12:0/3202706872 shutdown_connections 2023-11-28T21:34:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36640663e0 0x7f36640688a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3678156b90 0x7f367815a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:30.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f367813d390 0x7f367813bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:30.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 --2- 172.21.15.12:0/3202706872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3660005190 0x7f3678155110 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:30.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 -- 172.21.15.12:0/3202706872 >> 172.21.15.12:0/3202706872 conn(0x7f367806b6c0 msgr2=0x7f36780ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:30.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 -- 172.21.15.12:0/3202706872 shutdown_connections 2023-11-28T21:34:30.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:30.197+0000 7f367defe640 1 -- 172.21.15.12:0/3202706872 wait complete. 2023-11-28T21:34:32.220 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:34:32.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.373+0000 7f765fe9c640 1 Processor -- start 2023-11-28T21:34:32.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.373+0000 7f765fe9c640 1 -- start start 2023-11-28T21:34:32.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.373+0000 7f765fe9c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7658158420 0x7f765806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.373+0000 7f765fe9c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76580c3460 con 0x7f76580c38f0 2023-11-28T21:34:32.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.373+0000 7f765fe9c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76580c2b70 con 0x7f7658161600 2023-11-28T21:34:32.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.373+0000 7f765fe9c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7658158800 con 0x7f7658158420 2023-11-28T21:34:32.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.373+0000 7f765d437640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f76580c38f0 0x7f76580c3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54322/0 (socket says 172.21.15.12:54322) 2023-11-28T21:34:32.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.373+0000 7f765d437640 1 -- 172.21.15.12:0/1227807398 learned_addr learned my addr 172.21.15.12:0/1227807398 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.373+0000 7f765dc38640 1 --2- 172.21.15.12:0/1227807398 >> v2:172.21.15.144:3300/0 conn(0x7f7658158420 0x7f765806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494837681 0 0) 0x7f76580c3460 con 0x7f76580c38f0 2023-11-28T21:34:32.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7644003530 con 0x7f76580c38f0 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3043269239 0 0) 0x7f7644003530 con 0x7f76580c38f0 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 >> v2:172.21.15.144:3300/0 conn(0x7f7658158420 msgr2=0x7f765806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 --2- 172.21.15.12:0/1227807398 >> v2:172.21.15.144:3300/0 conn(0x7f7658158420 0x7f765806d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 >> v1:172.21.15.144:6789/0 conn(0x7f7658161600 legacy=0x7f7658161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765806e090 con 0x7f76580c38f0 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f7648001680 con 0x7f76580c38f0 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 >> v1:172.21.15.12:6789/0 conn(0x7f76580c38f0 legacy=0x7f76580c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76440039f0 0x7f7644003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76440047d0 0x7f7644004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7644005190 0x7f764400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7658158800 con 0x7f7644005190 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764400c0c0 con 0x7f76440039f0 2023-11-28T21:34:32.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764400c280 con 0x7f76440047d0 2023-11-28T21:34:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765e439640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7644005190 0x7f764400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765dc38640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76440039f0 0x7f7644003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765d437640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76440047d0 0x7f7644004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765e439640 1 -- 172.21.15.12:0/1227807398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76440047d0 msgr2=0x7f7644004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765e439640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76440047d0 0x7f7644004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765e439640 1 -- 172.21.15.12:0/1227807398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76440039f0 msgr2=0x7f7644003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765e439640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76440039f0 0x7f7644003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.377+0000 7f765e439640 1 -- 172.21.15.12:0/1227807398 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7644003530 con 0x7f7644005190 2023-11-28T21:34:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765e439640 1 -- 172.21.15.12:0/1227807398 reap_dead start 2023-11-28T21:34:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765e439640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7644005190 0x7f764400b990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7650001e70 tx=0x7f765001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4537c3588b160eae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650003b80 con 0x7f7644005190 2023-11-28T21:34:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7650003d00 con 0x7f7644005190 2023-11-28T21:34:32.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765cc36640 1 -- 172.21.15.12:0/1227807398 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650005260 con 0x7f7644005190 2023-11-28T21:34:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 -- 172.21.15.12:0/1227807398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7644005190 msgr2=0x7f764400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7644005190 0x7f764400b990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7650001e70 tx=0x7f765001c760 comp rx=0 tx=0).stop 2023-11-28T21:34:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 -- 172.21.15.12:0/1227807398 shutdown_connections 2023-11-28T21:34:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 --2- 172.21.15.12:0/1227807398 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7644005190 0x7f764400b990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 -- 172.21.15.12:0/1227807398 >> 172.21.15.12:0/1227807398 conn(0x7f7658066360 msgr2=0x7f7658066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 -- 172.21.15.12:0/1227807398 shutdown_connections 2023-11-28T21:34:32.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 -- 172.21.15.12:0/1227807398 wait complete. 2023-11-28T21:34:32.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 Processor -- start 2023-11-28T21:34:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 -- start start 2023-11-28T21:34:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7644005190 0x7f76581f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76580c9cc0 0x7f76581540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76581e7b50 0x7f765814e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7658152c00 con 0x7f76581e7b50 2023-11-28T21:34:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76581f7fe0 con 0x7f76580c9cc0 2023-11-28T21:34:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.381+0000 7f765fe9c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76581549d0 con 0x7f7644005190 2023-11-28T21:34:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765e439640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76581e7b50 0x7f765814e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765e439640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76581e7b50 0x7f765814e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59184/0 (socket says 172.21.15.12:59184) 2023-11-28T21:34:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765e439640 1 -- 172.21.15.12:0/3873266091 learned_addr learned my addr 172.21.15.12:0/3873266091 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765dc38640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7644005190 0x7f76581f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765d437640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76580c9cc0 0x7f76581540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765e439640 1 -- 172.21.15.12:0/3873266091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7644005190 msgr2=0x7f76581f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765e439640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7644005190 0x7f76581f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765e439640 1 -- 172.21.15.12:0/3873266091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76580c9cc0 msgr2=0x7f76581540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765e439640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76580c9cc0 0x7f76581540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765e439640 1 -- 172.21.15.12:0/3873266091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658152e50 con 0x7f76581e7b50 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765e439640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76581e7b50 0x7f765814e790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f765001cc40 tx=0x7f7644005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f764e7fc640 1 -- 172.21.15.12:0/3873266091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500ac070 con 0x7f76581e7b50 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f764e7fc640 1 -- 172.21.15.12:0/3873266091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7650001260 con 0x7f76581e7b50 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f764e7fc640 1 -- 172.21.15.12:0/3873266091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650003ba0 con 0x7f76581e7b50 2023-11-28T21:34:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765fe9c640 1 -- 172.21.15.12:0/3873266091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7658153130 con 0x7f76581e7b50 2023-11-28T21:34:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765fe9c640 1 -- 172.21.15.12:0/3873266091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76581536f0 con 0x7f76581e7b50 2023-11-28T21:34:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f765fe9c640 1 -- 172.21.15.12:0/3873266091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7658153930 con 0x7f76581e7b50 2023-11-28T21:34:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.385+0000 7f764e7fc640 1 -- 172.21.15.12:0/3873266091 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 85796+0+0 (secure 0 0 0) 0x7f7650006b60 con 0x7f76581e7b50 2023-11-28T21:34:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.389+0000 7f764e7fc640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f763c066410 0x7f763c0688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.389+0000 7f764e7fc640 1 -- 172.21.15.12:0/3873266091 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f765011c520 con 0x7f76581e7b50 2023-11-28T21:34:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.389+0000 7f765dc38640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f763c066410 0x7f763c0688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.389+0000 7f765dc38640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f763c066410 0x7f763c0688d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7654009b60 tx=0x7f7654009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.393+0000 7f764e7fc640 1 -- 172.21.15.12:0/3873266091 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7658153930 con 0x7f76581e7b50 2023-11-28T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 -- 172.21.15.12:0/3873266091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7658153930 con 0x7f76581e7b50 2023-11-28T21:34:32.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f764e7fc640 1 -- 172.21.15.12:0/3873266091 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5280 (secure 0 0 0) 0x7f7658153930 con 0x7f76581e7b50 2023-11-28T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4261,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-28T21:34:32.583 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2023-11-28T21:34:32.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 -- 172.21.15.12:0/3873266091 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f763c066410 msgr2=0x7f763c0688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f763c066410 0x7f763c0688d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7654009b60 tx=0x7f7654009190 comp rx=0 tx=0).stop 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 -- 172.21.15.12:0/3873266091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76581e7b50 msgr2=0x7f765814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76581e7b50 0x7f765814e790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f765001cc40 tx=0x7f7644005800 comp rx=0 tx=0).stop 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 -- 172.21.15.12:0/3873266091 shutdown_connections 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f763c066410 0x7f763c0688d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f76581e7b50 0x7f765814e790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76580c9cc0 0x7f76581540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 --2- 172.21.15.12:0/3873266091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7644005190 0x7f76581f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 -- 172.21.15.12:0/3873266091 >> 172.21.15.12:0/3873266091 conn(0x7f7658066360 msgr2=0x7f76580716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 -- 172.21.15.12:0/3873266091 shutdown_connections 2023-11-28T21:34:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.581+0000 7f765fe9c640 1 -- 172.21.15.12:0/3873266091 wait complete. 2023-11-28T21:34:32.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:34:32.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02d009640 1 Processor -- start 2023-11-28T21:34:32.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02d009640 1 -- start start 2023-11-28T21:34:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02d009640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb028154100 0x7fb02815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02d009640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb0280ca3f0 0x7fb0280c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02d009640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02815e270 con 0x7fb0280c9fd0 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02d009640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0280c4e60 con 0x7fb0280ca3f0 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02d009640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0281544e0 con 0x7fb028154100 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02659c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb0280c9fd0 0x7fb02806a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54332/0 (socket says 172.21.15.12:54332) 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02659c640 1 -- 172.21.15.12:0/2574985983 learned_addr learned my addr 172.21.15.12:0/2574985983 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb026d9d640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb028154100 0x7fb02815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02759e640 1 --2- 172.21.15.12:0/2574985983 >> v2:172.21.15.144:3300/0 conn(0x7fb0280ca3f0 0x7fb0280c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631377034 0 0) 0x7fb02815e270 con 0x7fb0280c9fd0 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb010003530 con 0x7fb0280c9fd0 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2334533447 0 0) 0x7fb010003530 con 0x7fb0280c9fd0 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb028154100 msgr2=0x7fb02815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb028154100 0x7fb02815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 >> v2:172.21.15.144:3300/0 conn(0x7fb0280ca3f0 msgr2=0x7fb0280c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 --2- 172.21.15.12:0/2574985983 >> v2:172.21.15.144:3300/0 conn(0x7fb0280ca3f0 0x7fb0280c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02815d9d0 con 0x7fb0280c9fd0 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fb01c001680 con 0x7fb0280c9fd0 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 >> v1:172.21.15.12:6789/0 conn(0x7fb0280c9fd0 legacy=0x7fb02806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0100039f0 0x7fb010003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb028154100 0x7fb0100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb010004d10 0x7fb010005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280c4e60 con 0x7fb0100039f0 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0281544e0 con 0x7fb010004d10 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01000b990 con 0x7fb028154100 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb026d9d640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0100039f0 0x7fb010003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02659c640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb028154100 0x7fb0100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02759e640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb010004d10 0x7fb010005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb026d9d640 1 -- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb028154100 msgr2=0x7fb0100047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:32.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb026d9d640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb028154100 0x7fb0100047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb026d9d640 1 -- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb010004d10 msgr2=0x7fb010005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:32.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb026d9d640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb010004d10 0x7fb010005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb026d9d640 1 -- 172.21.15.12:0/2574985983 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010003530 con 0x7fb0100039f0 2023-11-28T21:34:32.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.753+0000 7fb02759e640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb010004d10 0x7fb010005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:32.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb026d9d640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0100039f0 0x7fb010003e10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb018009ae0 tx=0x7fb01801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c565c831a4e08dc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:32.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0180a6510 con 0x7fb0100039f0 2023-11-28T21:34:32.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0180a6670 con 0x7fb0100039f0 2023-11-28T21:34:32.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb025d9b640 1 -- 172.21.15.12:0/2574985983 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0180a6a20 con 0x7fb0100039f0 2023-11-28T21:34:32.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb02d009640 1 -- 172.21.15.12:0/2574985983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0100039f0 msgr2=0x7fb010003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb02d009640 1 --2- 172.21.15.12:0/2574985983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0100039f0 0x7fb010003e10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb018009ae0 tx=0x7fb01801ca20 comp rx=0 tx=0).stop 2023-11-28T21:34:32.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb02759e640 1 -- 172.21.15.12:0/2574985983 reap_dead start 2023-11-28T21:34:32.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb02d009640 1 -- 172.21.15.12:0/2574985983 shutdown_connections 2023-11-28T21:34:32.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb02d009640 1 -- 172.21.15.12:0/2574985983 >> 172.21.15.12:0/2574985983 conn(0x7fb02806c690 msgr2=0x7fb028066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:32.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb02d009640 1 -- 172.21.15.12:0/2574985983 shutdown_connections 2023-11-28T21:34:32.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.757+0000 7fb02d009640 1 -- 172.21.15.12:0/2574985983 wait complete. 2023-11-28T21:34:32.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02d009640 1 Processor -- start 2023-11-28T21:34:32.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02d009640 1 -- start start 2023-11-28T21:34:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02d009640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0100039f0 0x7fb028162010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02d009640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0281632b0 0x7fb028149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02d009640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0281512f0 0x7fb0281f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02d009640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb028162550 con 0x7fb0281632b0 2023-11-28T21:34:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02d009640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb028161060 con 0x7fb0100039f0 2023-11-28T21:34:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02d009640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb028146910 con 0x7fb0281512f0 2023-11-28T21:34:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0281632b0 0x7fb028149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0281632b0 0x7fb028149da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59200/0 (socket says 172.21.15.12:59200) 2023-11-28T21:34:32.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02659c640 1 -- 172.21.15.12:0/4291379967 learned_addr learned my addr 172.21.15.12:0/4291379967 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:32.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb026d9d640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0100039f0 0x7fb028162010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02759e640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0281512f0 0x7fb0281f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02659c640 1 -- 172.21.15.12:0/4291379967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0281512f0 msgr2=0x7fb0281f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02659c640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0281512f0 0x7fb0281f3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02659c640 1 -- 172.21.15.12:0/4291379967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0100039f0 msgr2=0x7fb028162010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02659c640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0100039f0 0x7fb028162010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02659c640 1 -- 172.21.15.12:0/4291379967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb028146bd0 con 0x7fb0281632b0 2023-11-28T21:34:32.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02659c640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0281632b0 0x7fb028149da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb01c007290 tx=0x7fb01c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:32.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb00f7fe640 1 -- 172.21.15.12:0/4291379967 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c00f9c0 con 0x7fb0281632b0 2023-11-28T21:34:32.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb00f7fe640 1 -- 172.21.15.12:0/4291379967 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c00fb20 con 0x7fb0281632b0 2023-11-28T21:34:32.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb00f7fe640 1 -- 172.21.15.12:0/4291379967 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c00feb0 con 0x7fb0281632b0 2023-11-28T21:34:32.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.761+0000 7fb02d009640 1 -- 172.21.15.12:0/4291379967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb028146eb0 con 0x7fb0281632b0 2023-11-28T21:34:32.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.765+0000 7fb02d009640 1 -- 172.21.15.12:0/4291379967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb028160a10 con 0x7fb0281632b0 2023-11-28T21:34:32.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.765+0000 7fb02d009640 1 -- 172.21.15.12:0/4291379967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb028161210 con 0x7fb0281632b0 2023-11-28T21:34:32.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.765+0000 7fb00f7fe640 1 -- 172.21.15.12:0/4291379967 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 85796+0+0 (secure 0 0 0) 0x7fb01c01d460 con 0x7fb0281632b0 2023-11-28T21:34:32.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.765+0000 7fb00f7fe640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb0140663e0 0x7fb0140688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:32.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.765+0000 7fb00f7fe640 1 -- 172.21.15.12:0/4291379967 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb01c083c30 con 0x7fb0281632b0 2023-11-28T21:34:32.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.765+0000 7fb026d9d640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb0140663e0 0x7fb0140688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:32.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.765+0000 7fb026d9d640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb0140663e0 0x7fb0140688a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb0180af000 tx=0x7fb0180046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:32.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.769+0000 7fb00f7fe640 1 -- 172.21.15.12:0/4291379967 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb028161210 con 0x7fb0281632b0 2023-11-28T21:34:32.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 -- 172.21.15.12:0/4291379967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb028067c40 con 0x7fb0281632b0 2023-11-28T21:34:32.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb00f7fe640 1 -- 172.21.15.12:0/4291379967 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12862 (secure 0 0 0) 0x7fb01c0514e0 con 0x7fb0281632b0 2023-11-28T21:34:32.930 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:32.930 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":16,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:32.031158+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:34:32.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 -- 172.21.15.12:0/4291379967 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb0140663e0 msgr2=0x7fb0140688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb0140663e0 0x7fb0140688a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb0180af000 tx=0x7fb0180046d0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 -- 172.21.15.12:0/4291379967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0281632b0 msgr2=0x7fb028149da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:32.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0281632b0 0x7fb028149da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb01c007290 tx=0x7fb01c007750 comp rx=0 tx=0).stop 2023-11-28T21:34:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 -- 172.21.15.12:0/4291379967 shutdown_connections 2023-11-28T21:34:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb0140663e0 0x7fb0140688a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0281512f0 0x7fb0281f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0281632b0 0x7fb028149da0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 --2- 172.21.15.12:0/4291379967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0100039f0 0x7fb028162010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 -- 172.21.15.12:0/4291379967 >> 172.21.15.12:0/4291379967 conn(0x7fb02806c690 msgr2=0x7fb028163e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 -- 172.21.15.12:0/4291379967 shutdown_connections 2023-11-28T21:34:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:32.929+0000 7fb02d009640 1 -- 172.21.15.12:0/4291379967 wait complete. 2023-11-28T21:34:32.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T21:34:29.878190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T21:34:32.952 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T21:34:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad7817640 1 Processor -- start 2023-11-28T21:34:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad7817640 1 -- start start 2023-11-28T21:34:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad7817640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1ad0151af0 0x7f1ad00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad7817640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1ad0151ed0 0x7f1ad00c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad7817640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad006e300 0x7f1ad00c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad7817640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad00707f0 con 0x7f1ad0151ed0 2023-11-28T21:34:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad7817640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad00c7b90 con 0x7f1ad0151af0 2023-11-28T21:34:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad7817640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad0071c20 con 0x7f1ad006e300 2023-11-28T21:34:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad4db2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1ad0151ed0 0x7f1ad00c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad4db2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1ad0151ed0 0x7f1ad00c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59236/0 (socket says 172.21.15.12:59236) 2023-11-28T21:34:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad4db2640 1 -- 172.21.15.12:0/451996069 learned_addr learned my addr 172.21.15.12:0/451996069 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad5db4640 1 --2- 172.21.15.12:0/451996069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad006e300 0x7f1ad00c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.101+0000 7f1ad55b3640 1 --2- 172.21.15.12:0/451996069 >> v2:172.21.15.144:3300/0 conn(0x7f1ad0151af0 0x7f1ad00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.105+0000 7f1ad4db2640 1 -- 172.21.15.12:0/451996069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad006e300 msgr2=0x7f1ad00c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.105+0000 7f1ad4db2640 1 --2- 172.21.15.12:0/451996069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad006e300 0x7f1ad00c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.105+0000 7f1ad4db2640 1 -- 172.21.15.12:0/451996069 >> v2:172.21.15.144:3300/0 conn(0x7f1ad0151af0 msgr2=0x7f1ad00c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.105+0000 7f1ad4db2640 1 --2- 172.21.15.12:0/451996069 >> v2:172.21.15.144:3300/0 conn(0x7f1ad0151af0 0x7f1ad00c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.105+0000 7f1ad4db2640 1 -- 172.21.15.12:0/451996069 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad0157b70 con 0x7f1ad0151ed0 2023-11-28T21:34:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.105+0000 7f1ad4db2640 1 --2- 172.21.15.12:0/451996069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad0151ed0 0x7f1ad00c49d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00c7d20 tx=0x7f1ac001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5042b5f8d6685de7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.105+0000 7f1ac7fff640 1 -- 172.21.15.12:0/451996069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00ac410 con 0x7f1ad0151ed0 2023-11-28T21:34:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.105+0000 7f1ac7fff640 1 -- 172.21.15.12:0/451996069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac00ac570 con 0x7f1ad0151ed0 2023-11-28T21:34:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.105+0000 7f1ac7fff640 1 -- 172.21.15.12:0/451996069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00ac900 con 0x7f1ad0151ed0 2023-11-28T21:34:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 -- 172.21.15.12:0/451996069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad0151ed0 msgr2=0x7f1ad00c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 --2- 172.21.15.12:0/451996069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad0151ed0 0x7f1ad00c49d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00c7d20 tx=0x7f1ac001c500 comp rx=0 tx=0).stop 2023-11-28T21:34:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 -- 172.21.15.12:0/451996069 shutdown_connections 2023-11-28T21:34:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 --2- 172.21.15.12:0/451996069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad006e300 0x7f1ad00c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 --2- 172.21.15.12:0/451996069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad0151ed0 0x7f1ad00c49d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 --2- 172.21.15.12:0/451996069 >> v2:172.21.15.144:3300/0 conn(0x7f1ad0151af0 0x7f1ad00c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 -- 172.21.15.12:0/451996069 >> 172.21.15.12:0/451996069 conn(0x7f1ad0066360 msgr2=0x7f1ad0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 -- 172.21.15.12:0/451996069 shutdown_connections 2023-11-28T21:34:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 -- 172.21.15.12:0/451996069 wait complete. 2023-11-28T21:34:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 Processor -- start 2023-11-28T21:34:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 -- start start 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad006e300 0x7f1ad01f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad0151af0 0x7f1ad0160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad00ca690 0x7f1ad014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad01f2e90 con 0x7f1ad00ca690 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad014d020 con 0x7f1ad006e300 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad7817640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad014c690 con 0x7f1ad0151af0 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad5db4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad00ca690 0x7f1ad014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad5db4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad00ca690 0x7f1ad014bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59238/0 (socket says 172.21.15.12:59238) 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad5db4640 1 -- 172.21.15.12:0/2176589871 learned_addr learned my addr 172.21.15.12:0/2176589871 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad55b3640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad006e300 0x7f1ad01f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad4db2640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad0151af0 0x7f1ad0160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad5db4640 1 -- 172.21.15.12:0/2176589871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad0151af0 msgr2=0x7f1ad0160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad5db4640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad0151af0 0x7f1ad0160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad5db4640 1 -- 172.21.15.12:0/2176589871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad006e300 msgr2=0x7f1ad01f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad5db4640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad006e300 0x7f1ad01f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.109+0000 7f1ad5db4640 1 -- 172.21.15.12:0/2176589871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00094d0 con 0x7f1ad00ca690 2023-11-28T21:34:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.113+0000 7f1ad5db4640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad00ca690 0x7f1ad014bda0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800eed0 tx=0x7f1ac800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:33.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.113+0000 7f1ac5ffb640 1 -- 172.21.15.12:0/2176589871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8010040 con 0x7f1ad00ca690 2023-11-28T21:34:33.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.113+0000 7f1ac5ffb640 1 -- 172.21.15.12:0/2176589871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac8014400 con 0x7f1ad00ca690 2023-11-28T21:34:33.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.113+0000 7f1ac5ffb640 1 -- 172.21.15.12:0/2176589871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac80147b0 con 0x7f1ad00ca690 2023-11-28T21:34:33.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.113+0000 7f1ad7817640 1 -- 172.21.15.12:0/2176589871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad014c950 con 0x7f1ad00ca690 2023-11-28T21:34:33.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.113+0000 7f1ad7817640 1 -- 172.21.15.12:0/2176589871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad014d270 con 0x7f1ad00ca690 2023-11-28T21:34:33.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.113+0000 7f1ad7817640 1 -- 172.21.15.12:0/2176589871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad0074f60 con 0x7f1ad00ca690 2023-11-28T21:34:33.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.113+0000 7f1ac5ffb640 1 -- 172.21.15.12:0/2176589871 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 85796+0+0 (secure 0 0 0) 0x7f1ac8014910 con 0x7f1ad00ca690 2023-11-28T21:34:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.117+0000 7f1ac5ffb640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ab40663e0 0x7f1ab40688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.117+0000 7f1ac5ffb640 1 -- 172.21.15.12:0/2176589871 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1ac8084390 con 0x7f1ad00ca690 2023-11-28T21:34:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.117+0000 7f1ad55b3640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ab40663e0 0x7f1ab40688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.117+0000 7f1ad55b3640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ab40663e0 0x7f1ab40688a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009b60 tx=0x7f1acc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.121+0000 7f1ac5ffb640 1 -- 172.21.15.12:0/2176589871 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1ac8051ce0 con 0x7f1ad00ca690 2023-11-28T21:34:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.281+0000 7f1ad7817640 1 -- 172.21.15.12:0/2176589871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ad0072c10 con 0x7f1ad00ca690 2023-11-28T21:34:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.281+0000 7f1ac5ffb640 1 -- 172.21.15.12:0/2176589871 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ac80180a0 con 0x7f1ad00ca690 2023-11-28T21:34:33.283 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-11-28T21:34:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 -- 172.21.15.12:0/2176589871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ab40663e0 msgr2=0x7f1ab40688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ab40663e0 0x7f1ab40688a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009b60 tx=0x7f1acc009190 comp rx=0 tx=0).stop 2023-11-28T21:34:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 -- 172.21.15.12:0/2176589871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad00ca690 msgr2=0x7f1ad014bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad00ca690 0x7f1ad014bda0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800eed0 tx=0x7f1ac800b480 comp rx=0 tx=0).stop 2023-11-28T21:34:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 -- 172.21.15.12:0/2176589871 shutdown_connections 2023-11-28T21:34:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ab40663e0 0x7f1ab40688a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ad00ca690 0x7f1ad014bda0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad0151af0 0x7f1ad0160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 --2- 172.21.15.12:0/2176589871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad006e300 0x7f1ad01f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 -- 172.21.15.12:0/2176589871 >> 172.21.15.12:0/2176589871 conn(0x7f1ad0066360 msgr2=0x7f1ad014f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 -- 172.21.15.12:0/2176589871 shutdown_connections 2023-11-28T21:34:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.285+0000 7f1ad7817640 1 -- 172.21.15.12:0/2176589871 wait complete. 2023-11-28T21:34:33.306 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T21:34:33.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.449+0000 7f81b73aa640 1 Processor -- start 2023-11-28T21:34:33.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.449+0000 7f81b73aa640 1 -- start start 2023-11-28T21:34:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.449+0000 7f81b73aa640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f81b015d320 0x7f81b006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b73aa640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b015e270 con 0x7f81b0154100 2023-11-28T21:34:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b73aa640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b00c4e60 con 0x7f81b006aaf0 2023-11-28T21:34:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b73aa640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b015d740 con 0x7f81b015d320 2023-11-28T21:34:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b5146640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f81b0154100 0x7f81b00c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54346/0 (socket says 172.21.15.12:54346) 2023-11-28T21:34:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b5146640 1 -- 172.21.15.12:0/3793093260 learned_addr learned my addr 172.21.15.12:0/3793093260 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:33.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b4945640 1 --2- 172.21.15.12:0/3793093260 >> v2:172.21.15.144:3300/0 conn(0x7f81b015d320 0x7f81b006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605188021 0 0) 0x7f81b015e270 con 0x7f81b0154100 2023-11-28T21:34:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f819c003530 con 0x7f81b0154100 2023-11-28T21:34:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2055014940 0 0) 0x7f819c003530 con 0x7f81b0154100 2023-11-28T21:34:33.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 >> v2:172.21.15.144:3300/0 conn(0x7f81b015d320 msgr2=0x7f81b006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 --2- 172.21.15.12:0/3793093260 >> v2:172.21.15.144:3300/0 conn(0x7f81b015d320 0x7f81b006a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 >> v1:172.21.15.144:6789/0 conn(0x7f81b006aaf0 legacy=0x7f81b00c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b01544e0 con 0x7f81b0154100 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f81ac001680 con 0x7f81b0154100 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 >> v1:172.21.15.12:6789/0 conn(0x7f81b0154100 legacy=0x7f81b00c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c0039f0 0x7f819c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f819c0047d0 0x7f819c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f819c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b015d740 con 0x7f819c005190 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c00c0c0 con 0x7f819c0047d0 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c00c280 con 0x7f819c0039f0 2023-11-28T21:34:33.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b5947640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f819c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b5146640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c0039f0 0x7f819c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b4945640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f819c0047d0 0x7f819c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b5947640 1 -- 172.21.15.12:0/3793093260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c0039f0 msgr2=0x7f819c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b5947640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c0039f0 0x7f819c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b5947640 1 -- 172.21.15.12:0/3793093260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f819c0047d0 msgr2=0x7f819c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b5947640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f819c0047d0 0x7f819c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b5947640 1 -- 172.21.15.12:0/3793093260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c003530 con 0x7f819c005190 2023-11-28T21:34:33.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.453+0000 7f81b4945640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f819c0047d0 0x7f819c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:33.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81b5947640 1 -- 172.21.15.12:0/3793093260 reap_dead start 2023-11-28T21:34:33.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81b5947640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f819c00b990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81a8001e70 tx=0x7f81a801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee6e0d353cd0f3d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:33.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8003b80 con 0x7f819c005190 2023-11-28T21:34:33.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a8003d20 con 0x7f819c005190 2023-11-28T21:34:33.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81a7fff640 1 -- 172.21.15.12:0/3793093260 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8005260 con 0x7f819c005190 2023-11-28T21:34:33.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81b73aa640 1 -- 172.21.15.12:0/3793093260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 msgr2=0x7f819c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81b73aa640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f819c00b990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81a8001e70 tx=0x7f81a801c760 comp rx=0 tx=0).stop 2023-11-28T21:34:33.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81b73aa640 1 -- 172.21.15.12:0/3793093260 shutdown_connections 2023-11-28T21:34:33.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81b73aa640 1 --2- 172.21.15.12:0/3793093260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f819c00b990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81b73aa640 1 -- 172.21.15.12:0/3793093260 >> 172.21.15.12:0/3793093260 conn(0x7f81b006c690 msgr2=0x7f81b0066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:33.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81b73aa640 1 -- 172.21.15.12:0/3793093260 shutdown_connections 2023-11-28T21:34:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.457+0000 7f81b73aa640 1 -- 172.21.15.12:0/3793093260 wait complete. 2023-11-28T21:34:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b73aa640 1 Processor -- start 2023-11-28T21:34:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b73aa640 1 -- start start 2023-11-28T21:34:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b73aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f81b01481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b73aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81b01417b0 0x7f81b0141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b73aa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0142350 0x7f81b01427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b73aa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0145df0 con 0x7f819c005190 2023-11-28T21:34:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b73aa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0140c70 con 0x7f81b0142350 2023-11-28T21:34:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b73aa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b00b4c40 con 0x7f81b01417b0 2023-11-28T21:34:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b5146640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f81b01481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b5146640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f81b01481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59266/0 (socket says 172.21.15.12:59266) 2023-11-28T21:34:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b5146640 1 -- 172.21.15.12:0/1751675905 learned_addr learned my addr 172.21.15.12:0/1751675905 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b4945640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81b01417b0 0x7f81b0141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b5947640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0142350 0x7f81b01427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b5146640 1 -- 172.21.15.12:0/1751675905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81b01417b0 msgr2=0x7f81b0141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b5146640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81b01417b0 0x7f81b0141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b5146640 1 -- 172.21.15.12:0/1751675905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0142350 msgr2=0x7f81b01427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b5146640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0142350 0x7f81b01427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b5146640 1 -- 172.21.15.12:0/1751675905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b00b4f10 con 0x7f819c005190 2023-11-28T21:34:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b5146640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f81b01481a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81ac001060 tx=0x7f81ac00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81a5ffb640 1 -- 172.21.15.12:0/1751675905 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00ee60 con 0x7f819c005190 2023-11-28T21:34:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81a5ffb640 1 -- 172.21.15.12:0/1751675905 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac00c8d0 con 0x7f819c005190 2023-11-28T21:34:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81a5ffb640 1 -- 172.21.15.12:0/1751675905 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00cc80 con 0x7f819c005190 2023-11-28T21:34:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.461+0000 7f81b73aa640 1 -- 172.21.15.12:0/1751675905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b01452b0 con 0x7f819c005190 2023-11-28T21:34:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.465+0000 7f81b73aa640 1 -- 172.21.15.12:0/1751675905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b015aeb0 con 0x7f819c005190 2023-11-28T21:34:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.465+0000 7f81b73aa640 1 -- 172.21.15.12:0/1751675905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b0067c40 con 0x7f819c005190 2023-11-28T21:34:33.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.465+0000 7f81a5ffb640 1 -- 172.21.15.12:0/1751675905 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 85796+0+0 (secure 0 0 0) 0x7f81ac013420 con 0x7f819c005190 2023-11-28T21:34:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.465+0000 7f81a5ffb640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8194066410 0x7f81940688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.465+0000 7f81a5ffb640 1 -- 172.21.15.12:0/1751675905 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f81ac083630 con 0x7f819c005190 2023-11-28T21:34:33.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.469+0000 7f81b4945640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8194066410 0x7f81940688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.469+0000 7f81b4945640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8194066410 0x7f81940688d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f81b0148c80 tx=0x7f81a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:33.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.473+0000 7f81a5ffb640 1 -- 172.21.15.12:0/1751675905 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f81ac050ed0 con 0x7f819c005190 2023-11-28T21:34:33.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 -- 172.21.15.12:0/1751675905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f81b0074f60 con 0x7f819c005190 2023-11-28T21:34:33.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81a5ffb640 1 -- 172.21.15.12:0/1751675905 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f81ac0544f0 con 0x7f819c005190 2023-11-28T21:34:33.638 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:33.638 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2023-11-28T21:34:33.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2023-11-28T21:34:33.638 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2023-11-28T21:34:33.638 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2023-11-28T21:34:33.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-11-28T21:34:33.639 INFO:teuthology.orchestra.run.smithi012.stdout:] 2023-11-28T21:34:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 -- 172.21.15.12:0/1751675905 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8194066410 msgr2=0x7f81940688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8194066410 0x7f81940688d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f81b0148c80 tx=0x7f81a0008040 comp rx=0 tx=0).stop 2023-11-28T21:34:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 -- 172.21.15.12:0/1751675905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 msgr2=0x7f81b01481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f81b01481a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81ac001060 tx=0x7f81ac00ec10 comp rx=0 tx=0).stop 2023-11-28T21:34:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 -- 172.21.15.12:0/1751675905 shutdown_connections 2023-11-28T21:34:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8194066410 0x7f81940688d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81b0142350 0x7f81b01427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81b01417b0 0x7f81b0141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 --2- 172.21.15.12:0/1751675905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f819c005190 0x7f81b01481a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 -- 172.21.15.12:0/1751675905 >> 172.21.15.12:0/1751675905 conn(0x7f81b006c690 msgr2=0x7f81b00c5140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 -- 172.21.15.12:0/1751675905 shutdown_connections 2023-11-28T21:34:33.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.637+0000 7f81b73aa640 1 -- 172.21.15.12:0/1751675905 wait complete. 2023-11-28T21:34:33.659 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T21:34:33.659 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T21:34:33.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.809+0000 7f7754d96640 1 Processor -- start 2023-11-28T21:34:33.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.809+0000 7f7754d96640 1 -- start start 2023-11-28T21:34:33.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f7754d96640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f77501540a0 0x7f77500c9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f7754d96640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775015a420 0x7f77500ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f7754d96640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77500727b0 con 0x7f77500ca0a0 2023-11-28T21:34:33.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f7754d96640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7750154480 con 0x7f77501540a0 2023-11-28T21:34:33.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f7754d96640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775006d680 con 0x7f775015a420 2023-11-28T21:34:33.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f7747fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f77500ca0a0 0x7f7750159d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54352/0 (socket says 172.21.15.12:54352) 2023-11-28T21:34:33.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f7747fff640 1 -- 172.21.15.12:0/574121610 learned_addr learned my addr 172.21.15.12:0/574121610 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:33.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774e59c640 1 --2- 172.21.15.12:0/574121610 >> v2:172.21.15.144:3300/0 conn(0x7f77501540a0 0x7f77500c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774ed9d640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775015a420 0x7f77500ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463558709 0 0) 0x7f77500727b0 con 0x7f77500ca0a0 2023-11-28T21:34:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f773c003530 con 0x7f77500ca0a0 2023-11-28T21:34:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1057796387 0 0) 0x7f773c003530 con 0x7f77500ca0a0 2023-11-28T21:34:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775015a420 msgr2=0x7f77500ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775015a420 0x7f77500ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 >> v2:172.21.15.144:3300/0 conn(0x7f77501540a0 msgr2=0x7f77500c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:33.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 --2- 172.21.15.12:0/574121610 >> v2:172.21.15.144:3300/0 conn(0x7f77501540a0 0x7f77500c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7750072a30 con 0x7f77500ca0a0 2023-11-28T21:34:33.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f7738001680 con 0x7f77500ca0a0 2023-11-28T21:34:33.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 >> v1:172.21.15.12:6789/0 conn(0x7f77500ca0a0 legacy=0x7f7750159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773c003980 0x7f773c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775015a420 0x7f773c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f773c004d10 0x7f773c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750154480 con 0x7f773c004d10 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775006d680 con 0x7f773c003980 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c00b990 con 0x7f775015a420 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774ed9d640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f773c004d10 0x7f773c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774e59c640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773c003980 0x7f773c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f7747fff640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775015a420 0x7f773c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774ed9d640 1 -- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775015a420 msgr2=0x7f773c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774ed9d640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775015a420 0x7f773c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774ed9d640 1 -- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773c003980 msgr2=0x7f773c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774ed9d640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773c003980 0x7f773c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.813+0000 7f774ed9d640 1 -- 172.21.15.12:0/574121610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c003530 con 0x7f773c004d10 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f774ed9d640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f773c004d10 0x7f773c005190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7750068bf0 tx=0x7f774801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af769667e741733c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77480a5c10 con 0x7f773c004d10 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774800f040 con 0x7f773c004d10 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f774dd9b640 1 -- 172.21.15.12:0/574121610 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77480ad440 con 0x7f773c004d10 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f7754d96640 1 -- 172.21.15.12:0/574121610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f773c004d10 msgr2=0x7f773c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f7754d96640 1 --2- 172.21.15.12:0/574121610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f773c004d10 0x7f773c005190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7750068bf0 tx=0x7f774801cc00 comp rx=0 tx=0).stop 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f774ed9d640 1 -- 172.21.15.12:0/574121610 reap_dead start 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f7754d96640 1 -- 172.21.15.12:0/574121610 shutdown_connections 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f7754d96640 1 -- 172.21.15.12:0/574121610 >> 172.21.15.12:0/574121610 conn(0x7f7750066360 msgr2=0x7f7750066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f7754d96640 1 -- 172.21.15.12:0/574121610 shutdown_connections 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f7754d96640 1 -- 172.21.15.12:0/574121610 wait complete. 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f7754d96640 1 Processor -- start 2023-11-28T21:34:33.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.817+0000 7f7754d96640 1 -- start start 2023-11-28T21:34:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7754d96640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773c004d10 0x7f775015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7754d96640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7750154680 0x7f775014e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7754d96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7750154b00 0x7f775014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7754d96640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77501596a0 con 0x7f7750154b00 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7754d96640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775015cf00 con 0x7f7750154680 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7754d96640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7750159b30 con 0x7f773c004d10 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f774ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7750154b00 0x7f775014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f774ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7750154b00 0x7f775014eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59288/0 (socket says 172.21.15.12:59288) 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f774ed9d640 1 -- 172.21.15.12:0/2797598489 learned_addr learned my addr 172.21.15.12:0/2797598489 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7747fff640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7750154680 0x7f775014e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f774e59c640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773c004d10 0x7f775015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f774ed9d640 1 -- 172.21.15.12:0/2797598489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773c004d10 msgr2=0x7f775015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f774ed9d640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773c004d10 0x7f775015c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f774ed9d640 1 -- 172.21.15.12:0/2797598489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7750154680 msgr2=0x7f775014e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f774ed9d640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7750154680 0x7f775014e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f774ed9d640 1 -- 172.21.15.12:0/2797598489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77501f38a0 con 0x7f7750154b00 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7747fff640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7750154680 0x7f775014e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:33.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f774ed9d640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7750154b00 0x7f775014eb10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7748002240 tx=0x7f7748009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:33.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7746ffd640 1 -- 172.21.15.12:0/2797598489 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77480039a0 con 0x7f7750154b00 2023-11-28T21:34:33.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7746ffd640 1 -- 172.21.15.12:0/2797598489 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7748003b00 con 0x7f7750154b00 2023-11-28T21:34:33.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7754d96640 1 -- 172.21.15.12:0/2797598489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7750158a90 con 0x7f7750154b00 2023-11-28T21:34:33.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7746ffd640 1 -- 172.21.15.12:0/2797598489 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748003e90 con 0x7f7750154b00 2023-11-28T21:34:33.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7754d96640 1 -- 172.21.15.12:0/2797598489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7750159050 con 0x7f7750154b00 2023-11-28T21:34:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.821+0000 7f7754d96640 1 -- 172.21.15.12:0/2797598489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7750159810 con 0x7f7750154b00 2023-11-28T21:34:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.825+0000 7f7746ffd640 1 -- 172.21.15.12:0/2797598489 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 85796+0+0 (secure 0 0 0) 0x7f77480b4040 con 0x7f7750154b00 2023-11-28T21:34:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.825+0000 7f7746ffd640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7730066410 0x7f77300688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.825+0000 7f7746ffd640 1 -- 172.21.15.12:0/2797598489 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f774811d820 con 0x7f7750154b00 2023-11-28T21:34:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.825+0000 7f774e59c640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7730066410 0x7f77300688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.825+0000 7f774e59c640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7730066410 0x7f77300688d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7740009b20 tx=0x7f7740008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:33.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.829+0000 7f7746ffd640 1 -- 172.21.15.12:0/2797598489 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f77480eb170 con 0x7f7750154b00 2023-11-28T21:34:33.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:33.993+0000 7f7754d96640 1 -- 172.21.15.12:0/2797598489 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f775004e850 con 0x7f7750154b00 2023-11-28T21:34:34.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.428+0000 7f7746ffd640 1 -- 172.21.15.12:0/2797598489 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f77480eb540 con 0x7f7750154b00 2023-11-28T21:34:34.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.960+0000 7f7746ffd640 1 -- 172.21.15.12:0/2797598489 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7748117590 con 0x7f7750154b00 2023-11-28T21:34:34.964 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 -- 172.21.15.12:0/2797598489 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7730066410 msgr2=0x7f77300688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7730066410 0x7f77300688d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7740009b20 tx=0x7f7740008040 comp rx=0 tx=0).stop 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 -- 172.21.15.12:0/2797598489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7750154b00 msgr2=0x7f775014eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7750154b00 0x7f775014eb10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7748002240 tx=0x7f7748009750 comp rx=0 tx=0).stop 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 -- 172.21.15.12:0/2797598489 shutdown_connections 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7730066410 0x7f77300688d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7750154b00 0x7f775014eb10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7750154680 0x7f775014e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 --2- 172.21.15.12:0/2797598489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773c004d10 0x7f775015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 -- 172.21.15.12:0/2797598489 >> 172.21.15.12:0/2797598489 conn(0x7f7750066360 msgr2=0x7f77500cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 -- 172.21.15.12:0/2797598489 shutdown_connections 2023-11-28T21:34:34.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:34.964+0000 7f7754d96640 1 -- 172.21.15.12:0/2797598489 wait complete. 2023-11-28T21:34:34.994 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-28T21:34:35.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb3d2a640 1 Processor -- start 2023-11-28T21:34:35.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb3d2a640 1 -- start start 2023-11-28T21:34:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb3d2a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6cac15d560 0x7f6cac06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb3d2a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6cac0c66a0 0x7f6cac0c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb3d2a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cac06a5b0 con 0x7f6cac0c7650 2023-11-28T21:34:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb3d2a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac0c4b50 con 0x7f6cac15d560 2023-11-28T21:34:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb3d2a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac06b130 con 0x7f6cac0c66a0 2023-11-28T21:34:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb1ac6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6cac15d560 0x7f6cac06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb1ac6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6cac15d560 0x7f6cac06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56882/0 (socket says 172.21.15.12:56882) 2023-11-28T21:34:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb1ac6640 1 -- 172.21.15.12:0/2147377740 learned_addr learned my addr 172.21.15.12:0/2147377740 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb22c7640 1 --1- 172.21.15.12:0/2147377740 >> v1:172.21.15.12:6789/0 conn(0x7f6cac0c7650 0x7f6cac15d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54788/0 (socket says 172.21.15.12:54788) 2023-11-28T21:34:35.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb12c5640 1 --2- 172.21.15.12:0/2147377740 >> v2:172.21.15.144:3300/0 conn(0x7f6cac0c66a0 0x7f6cac0c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:35.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb1ac6640 1 -- 172.21.15.12:0/2147377740 >> v2:172.21.15.144:3300/0 conn(0x7f6cac0c66a0 msgr2=0x7f6cac0c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:35.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb1ac6640 1 --2- 172.21.15.12:0/2147377740 >> v2:172.21.15.144:3300/0 conn(0x7f6cac0c66a0 0x7f6cac0c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb1ac6640 1 -- 172.21.15.12:0/2147377740 >> v1:172.21.15.12:6789/0 conn(0x7f6cac0c7650 legacy=0x7f6cac15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:35.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb1ac6640 1 -- 172.21.15.12:0/2147377740 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac15e050 con 0x7f6cac15d560 2023-11-28T21:34:35.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.144+0000 7f6cb12c5640 1 --2- 172.21.15.12:0/2147377740 >> v2:172.21.15.144:3300/0 conn(0x7f6cac0c66a0 0x7f6cac0c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:34:35.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb1ac6640 1 --2- 172.21.15.12:0/2147377740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15d560 0x7f6cac06b5c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0c4dc0 tx=0x7f6ca801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ba4b8886826093 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:35.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb0ac4640 1 -- 172.21.15.12:0/2147377740 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80aa640 con 0x7f6cac15d560 2023-11-28T21:34:35.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb0ac4640 1 -- 172.21.15.12:0/2147377740 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca80aa7a0 con 0x7f6cac15d560 2023-11-28T21:34:35.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb0ac4640 1 -- 172.21.15.12:0/2147377740 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80aab30 con 0x7f6cac15d560 2023-11-28T21:34:35.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/2147377740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15d560 msgr2=0x7f6cac06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:35.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb3d2a640 1 --2- 172.21.15.12:0/2147377740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15d560 0x7f6cac06b5c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0c4dc0 tx=0x7f6ca801c4e0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/2147377740 shutdown_connections 2023-11-28T21:34:35.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb3d2a640 1 --2- 172.21.15.12:0/2147377740 >> v2:172.21.15.144:3300/0 conn(0x7f6cac0c66a0 0x7f6cac0c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb3d2a640 1 --2- 172.21.15.12:0/2147377740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac15d560 0x7f6cac06b5c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/2147377740 >> 172.21.15.12:0/2147377740 conn(0x7f6cac06d5e0 msgr2=0x7f6cac06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:35.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/2147377740 shutdown_connections 2023-11-28T21:34:35.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.148+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/2147377740 wait complete. 2023-11-28T21:34:35.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb3d2a640 1 Processor -- start 2023-11-28T21:34:35.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb3d2a640 1 -- start start 2023-11-28T21:34:35.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb3d2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0c66a0 0x7f6cac14f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:35.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb3d2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac0c7650 0x7f6cac15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:35.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb3d2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac15d560 0x7f6cac15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:35.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb3d2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15cd90 con 0x7f6cac0c7650 2023-11-28T21:34:35.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb3d2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac145d50 con 0x7f6cac15d560 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb3d2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac1461e0 con 0x7f6cac0c66a0 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb12c5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac0c7650 0x7f6cac15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb12c5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac0c7650 0x7f6cac15aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56886/0 (socket says 172.21.15.12:56886) 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb12c5640 1 -- 172.21.15.12:0/1135305427 learned_addr learned my addr 172.21.15.12:0/1135305427 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb22c7640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac15d560 0x7f6cac15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb12c5640 1 -- 172.21.15.12:0/1135305427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0c66a0 msgr2=0x7f6cac14f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb1ac6640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0c66a0 0x7f6cac14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb12c5640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0c66a0 0x7f6cac14f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb12c5640 1 -- 172.21.15.12:0/1135305427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac15d560 msgr2=0x7f6cac15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb12c5640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac15d560 0x7f6cac15b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb12c5640 1 -- 172.21.15.12:0/1135305427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca80094d0 con 0x7f6cac0c7650 2023-11-28T21:34:35.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb22c7640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac15d560 0x7f6cac15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:34:35.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb1ac6640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0c66a0 0x7f6cac14f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:34:35.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb12c5640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac0c7650 0x7f6cac15aed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00c7f0 tx=0x7f6c9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:35.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6ca27fc640 1 -- 172.21.15.12:0/1135305427 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c007820 con 0x7f6cac0c7650 2023-11-28T21:34:35.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6ca27fc640 1 -- 172.21.15.12:0/1135305427 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c007980 con 0x7f6cac0c7650 2023-11-28T21:34:35.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/1135305427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac145180 con 0x7f6cac0c7650 2023-11-28T21:34:35.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6ca27fc640 1 -- 172.21.15.12:0/1135305427 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c007d30 con 0x7f6cac0c7650 2023-11-28T21:34:35.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.152+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/1135305427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac145700 con 0x7f6cac0c7650 2023-11-28T21:34:35.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.156+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/1135305427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac04e850 con 0x7f6cac0c7650 2023-11-28T21:34:35.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.156+0000 7f6ca27fc640 1 -- 172.21.15.12:0/1135305427 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6c9c0113f0 con 0x7f6cac0c7650 2023-11-28T21:34:35.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.156+0000 7f6ca27fc640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c900664a0 0x7f6c90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:35.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.156+0000 7f6ca27fc640 1 -- 172.21.15.12:0/1135305427 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6c9c083ff0 con 0x7f6cac0c7650 2023-11-28T21:34:35.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.156+0000 7f6cb1ac6640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c900664a0 0x7f6c90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:35.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.156+0000 7f6cb1ac6640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c900664a0 0x7f6c90068960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80ae000 tx=0x7f6ca8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:35.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.160+0000 7f6ca27fc640 1 -- 172.21.15.12:0/1135305427 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c9c0b5280 con 0x7f6cac0c7650 2023-11-28T21:34:35.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.320+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/1135305427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f6cac145e90 con 0x7f6cac0c7650 2023-11-28T21:34:35.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.964+0000 7f6ca27fc640 1 -- 172.21.15.12:0/1135305427 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f6c9c01f0d0 con 0x7f6cac0c7650 2023-11-28T21:34:35.969 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2023-11-28T21:34:35.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/1135305427 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c900664a0 msgr2=0x7f6c90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c900664a0 0x7f6c90068960 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80ae000 tx=0x7f6ca8008040 comp rx=0 tx=0).stop 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/1135305427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac0c7650 msgr2=0x7f6cac15aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac0c7650 0x7f6cac15aed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00c7f0 tx=0x7f6c9c00ccb0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/1135305427 shutdown_connections 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c900664a0 0x7f6c90068960 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cac15d560 0x7f6cac15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6cac0c7650 0x7f6cac15aed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 --2- 172.21.15.12:0/1135305427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cac0c66a0 0x7f6cac14f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/1135305427 >> 172.21.15.12:0/1135305427 conn(0x7f6cac06d5e0 msgr2=0x7f6cac151780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/1135305427 shutdown_connections 2023-11-28T21:34:35.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:35.968+0000 7f6cb3d2a640 1 -- 172.21.15.12:0/1135305427 wait complete. 2023-11-28T21:34:36.018 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T21:34:36.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.164+0000 7f4e9e846640 1 Processor -- start 2023-11-28T21:34:36.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.164+0000 7f4e9e846640 1 -- start start 2023-11-28T21:34:36.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9e846640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e98151ed0 0x7f4e980c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:36.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9e846640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e98157b70 con 0x7f4e9806e300 2023-11-28T21:34:36.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9e846640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e98157e50 con 0x7f4e98151af0 2023-11-28T21:34:36.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9e846640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e980707f0 con 0x7f4e98151ed0 2023-11-28T21:34:36.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9cde3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4e9806e300 0x7f4e98072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54792/0 (socket says 172.21.15.12:54792) 2023-11-28T21:34:36.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9cde3640 1 -- 172.21.15.12:0/286131586 learned_addr learned my addr 172.21.15.12:0/286131586 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:36.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e977fe640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e98151ed0 0x7f4e980c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:36.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626390797 0 0) 0x7f4e98157b70 con 0x7f4e9806e300 2023-11-28T21:34:36.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e80003530 con 0x7f4e9806e300 2023-11-28T21:34:36.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573090188 0 0) 0x7f4e98157e50 con 0x7f4e98151af0 2023-11-28T21:34:36.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e98157b70 con 0x7f4e98151af0 2023-11-28T21:34:36.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 327109416 0 0) 0x7f4e80003530 con 0x7f4e9806e300 2023-11-28T21:34:36.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e98151ed0 msgr2=0x7f4e980c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:36.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e98151ed0 0x7f4e980c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:36.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 >> v1:172.21.15.144:6789/0 conn(0x7f4e98151af0 legacy=0x7f4e980c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e98158130 con 0x7f4e9806e300 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f4e90001a00 con 0x7f4e9806e300 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 >> v1:172.21.15.12:6789/0 conn(0x7f4e9806e300 legacy=0x7f4e98072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e80003bb0 0x7f4e80003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e800047d0 0x7f4e80004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80005190 0x7f4e8000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e980707f0 con 0x7f4e80005190 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8000bed0 con 0x7f4e800047d0 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8000c090 con 0x7f4e80003bb0 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9cde3640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80005190 0x7f4e8000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e97fff640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e80003bb0 0x7f4e80003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e977fe640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e800047d0 0x7f4e80004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9cde3640 1 -- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e80003bb0 msgr2=0x7f4e80003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:36.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9cde3640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e80003bb0 0x7f4e80003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:36.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9cde3640 1 -- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e800047d0 msgr2=0x7f4e80004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:36.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9cde3640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e800047d0 0x7f4e80004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:36.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e9cde3640 1 -- 172.21.15.12:0/286131586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e80003530 con 0x7f4e80005190 2023-11-28T21:34:36.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e97fff640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e80003bb0 0x7f4e80003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:36.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.168+0000 7f4e977fe640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e800047d0 0x7f4e80004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:36.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e9cde3640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80005190 0x7f4e8000b990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e98071280 tx=0x7f4e9001f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc9507714618d06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:36.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e900aa610 con 0x7f4e80005190 2023-11-28T21:34:36.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e900aa770 con 0x7f4e80005190 2023-11-28T21:34:36.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e96ffd640 1 -- 172.21.15.12:0/286131586 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e900aab20 con 0x7f4e80005190 2023-11-28T21:34:36.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e9e846640 1 -- 172.21.15.12:0/286131586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80005190 msgr2=0x7f4e8000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:36.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e9e846640 1 --2- 172.21.15.12:0/286131586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e80005190 0x7f4e8000b990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4e98071280 tx=0x7f4e9001f9e0 comp rx=0 tx=0).stop 2023-11-28T21:34:36.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e9cde3640 1 -- 172.21.15.12:0/286131586 reap_dead start 2023-11-28T21:34:36.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e9e846640 1 -- 172.21.15.12:0/286131586 shutdown_connections 2023-11-28T21:34:36.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e9e846640 1 -- 172.21.15.12:0/286131586 >> 172.21.15.12:0/286131586 conn(0x7f4e98066360 msgr2=0x7f4e98066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:36.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e9e846640 1 -- 172.21.15.12:0/286131586 shutdown_connections 2023-11-28T21:34:36.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e9e846640 1 -- 172.21.15.12:0/286131586 wait complete. 2023-11-28T21:34:36.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.172+0000 7f4e9e846640 1 Processor -- start 2023-11-28T21:34:36.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9e846640 1 -- start start 2023-11-28T21:34:36.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9e846640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e80005190 0x7f4e981f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9e846640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e9814b720 0x7f4e98156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9e846640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e98156580 0x7f4e981ed010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9e846640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e98153920 con 0x7f4e9814b720 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9e846640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981ecb10 con 0x7f4e80005190 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9e846640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e98156b60 con 0x7f4e98156580 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e977fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e9814b720 0x7f4e98156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e977fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e9814b720 0x7f4e98156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56910/0 (socket says 172.21.15.12:56910) 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e977fe640 1 -- 172.21.15.12:0/2471412648 learned_addr learned my addr 172.21.15.12:0/2471412648 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9cde3640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e98156580 0x7f4e981ed010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e97fff640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e80005190 0x7f4e981f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e977fe640 1 -- 172.21.15.12:0/2471412648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e98156580 msgr2=0x7f4e981ed010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:36.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e977fe640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e98156580 0x7f4e981ed010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e977fe640 1 -- 172.21.15.12:0/2471412648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e80005190 msgr2=0x7f4e981f0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e977fe640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e80005190 0x7f4e981f0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e977fe640 1 -- 172.21.15.12:0/2471412648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e98156e40 con 0x7f4e9814b720 2023-11-28T21:34:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e977fe640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e9814b720 0x7f4e98156040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c011600 tx=0x7f4e8c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e94ff9640 1 -- 172.21.15.12:0/2471412648 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c004230 con 0x7f4e9814b720 2023-11-28T21:34:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e94ff9640 1 -- 172.21.15.12:0/2471412648 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c004390 con 0x7f4e9814b720 2023-11-28T21:34:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e94ff9640 1 -- 172.21.15.12:0/2471412648 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00f960 con 0x7f4e9814b720 2023-11-28T21:34:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9e846640 1 -- 172.21.15.12:0/2471412648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e98157120 con 0x7f4e9814b720 2023-11-28T21:34:36.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9e846640 1 -- 172.21.15.12:0/2471412648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e981576e0 con 0x7f4e9814b720 2023-11-28T21:34:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.176+0000 7f4e9e846640 1 -- 172.21.15.12:0/2471412648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e981578a0 con 0x7f4e9814b720 2023-11-28T21:34:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.180+0000 7f4e94ff9640 1 -- 172.21.15.12:0/2471412648 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4e8c00fac0 con 0x7f4e9814b720 2023-11-28T21:34:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.180+0000 7f4e94ff9640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e84066470 0x7f4e84068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:36.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.180+0000 7f4e94ff9640 1 -- 172.21.15.12:0/2471412648 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4e8c084110 con 0x7f4e9814b720 2023-11-28T21:34:36.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.180+0000 7f4e97fff640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e84066470 0x7f4e84068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:36.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.180+0000 7f4e97fff640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e84066470 0x7f4e84068930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4e88000f70 tx=0x7f4e88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:36.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.184+0000 7f4e94ff9640 1 -- 172.21.15.12:0/2471412648 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e8c051550 con 0x7f4e9814b720 2023-11-28T21:34:36.330 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:34:36.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:34:36.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:36.376+0000 7f4e9e846640 1 -- 172.21.15.12:0/2471412648 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4e981ecc50 con 0x7f4e9814b720 2023-11-28T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.316+0000 7f4e94ff9640 1 -- 172.21.15.12:0/2471412648 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4e8c054b70 con 0x7f4e9814b720 2023-11-28T21:34:37.319 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 2 and data pool 3 2023-11-28T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.320+0000 7f4e9e846640 1 -- 172.21.15.12:0/2471412648 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e84066470 msgr2=0x7f4e84068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:37.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.320+0000 7f4e9e846640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e84066470 0x7f4e84068930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4e88000f70 tx=0x7f4e88009040 comp rx=0 tx=0).stop 2023-11-28T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.320+0000 7f4e9e846640 1 -- 172.21.15.12:0/2471412648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e9814b720 msgr2=0x7f4e98156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.320+0000 7f4e9e846640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e9814b720 0x7f4e98156040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c011600 tx=0x7f4e8c011ac0 comp rx=0 tx=0).stop 2023-11-28T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.324+0000 7f4e9e846640 1 -- 172.21.15.12:0/2471412648 shutdown_connections 2023-11-28T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.324+0000 7f4e9e846640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e84066470 0x7f4e84068930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.324+0000 7f4e9e846640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e98156580 0x7f4e981ed010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.324+0000 7f4e9e846640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e9814b720 0x7f4e98156040 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.324+0000 7f4e9e846640 1 --2- 172.21.15.12:0/2471412648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e80005190 0x7f4e981f0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:37.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.324+0000 7f4e9e846640 1 -- 172.21.15.12:0/2471412648 >> 172.21.15.12:0/2471412648 conn(0x7f4e98066360 msgr2=0x7f4e981e28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.324+0000 7f4e9e846640 1 -- 172.21.15.12:0/2471412648 shutdown_connections 2023-11-28T21:34:37.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.324+0000 7f4e9e846640 1 -- 172.21.15.12:0/2471412648 wait complete. 2023-11-28T21:34:37.358 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-28T21:34:37.358 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-28T21:34:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d2227b640 1 Processor -- start 2023-11-28T21:34:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d2227b640 1 -- start start 2023-11-28T21:34:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d2227b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2d1c15a530 0x7f2d1c150460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d2227b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a9a0 0x7f2d1c1509a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d2227b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d1c158b40 con 0x7f2d1c15a150 2023-11-28T21:34:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d2227b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d1c06d2e0 con 0x7f2d1c15a530 2023-11-28T21:34:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d2227b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c158e20 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d20a78640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2d1c15a530 0x7f2d1c150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d20a78640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2d1c15a530 0x7f2d1c150460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56924/0 (socket says 172.21.15.12:56924) 2023-11-28T21:34:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d20a78640 1 -- 172.21.15.12:0/3761074936 learned_addr learned my addr 172.21.15.12:0/3761074936 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d21a7a640 1 --2- 172.21.15.12:0/3761074936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a9a0 0x7f2d1c1509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:37.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d20a78640 1 -- 172.21.15.12:0/3761074936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a9a0 msgr2=0x7f2d1c1509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:37.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d20a78640 1 --2- 172.21.15.12:0/3761074936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a9a0 0x7f2d1c1509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:37.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d20a78640 1 -- 172.21.15.12:0/3761074936 >> v1:172.21.15.144:6789/0 conn(0x7f2d1c15a150 legacy=0x7f2d1c06cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:37.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.508+0000 7f2d20a78640 1 -- 172.21.15.12:0/3761074936 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c159100 con 0x7f2d1c15a530 2023-11-28T21:34:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d20a78640 1 --2- 172.21.15.12:0/3761074936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a530 0x7f2d1c150460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009870 tx=0x7f2d0c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f7f003118051bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d13fff640 1 -- 172.21.15.12:0/3761074936 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0aa670 con 0x7f2d1c15a530 2023-11-28T21:34:37.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d13fff640 1 -- 172.21.15.12:0/3761074936 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c0aa7d0 con 0x7f2d1c15a530 2023-11-28T21:34:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d13fff640 1 -- 172.21.15.12:0/3761074936 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0aab60 con 0x7f2d1c15a530 2023-11-28T21:34:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d2227b640 1 -- 172.21.15.12:0/3761074936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a530 msgr2=0x7f2d1c150460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d2227b640 1 --2- 172.21.15.12:0/3761074936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a530 0x7f2d1c150460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009870 tx=0x7f2d0c01c510 comp rx=0 tx=0).stop 2023-11-28T21:34:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d2227b640 1 -- 172.21.15.12:0/3761074936 shutdown_connections 2023-11-28T21:34:37.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d2227b640 1 --2- 172.21.15.12:0/3761074936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a9a0 0x7f2d1c1509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:37.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d2227b640 1 --2- 172.21.15.12:0/3761074936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a530 0x7f2d1c150460 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:37.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d2227b640 1 -- 172.21.15.12:0/3761074936 >> 172.21.15.12:0/3761074936 conn(0x7f2d1c0c7cb0 msgr2=0x7f2d1c0c80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:37.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d2227b640 1 -- 172.21.15.12:0/3761074936 shutdown_connections 2023-11-28T21:34:37.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.512+0000 7f2d2227b640 1 -- 172.21.15.12:0/3761074936 wait complete. 2023-11-28T21:34:37.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 Processor -- start 2023-11-28T21:34:37.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 -- start start 2023-11-28T21:34:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c15a150 0x7f2d1c06d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a530 0x7f2d1c0c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a9a0 0x7f2d1c1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c0c3d50 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c0c9630 con 0x7f2d1c15a150 2023-11-28T21:34:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c145400 con 0x7f2d1c15a530 2023-11-28T21:34:37.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d21a7a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a9a0 0x7f2d1c1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d21a7a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a9a0 0x7f2d1c1f37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56928/0 (socket says 172.21.15.12:56928) 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d21a7a640 1 -- 172.21.15.12:0/3182279014 learned_addr learned my addr 172.21.15.12:0/3182279014 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d21279640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c15a150 0x7f2d1c06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d20a78640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a530 0x7f2d1c0c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d21a7a640 1 -- 172.21.15.12:0/3182279014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a530 msgr2=0x7f2d1c0c2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d21a7a640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a530 0x7f2d1c0c2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d21a7a640 1 -- 172.21.15.12:0/3182279014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c15a150 msgr2=0x7f2d1c06d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d21a7a640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c15a150 0x7f2d1c06d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d21a7a640 1 -- 172.21.15.12:0/3182279014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0094d0 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d20a78640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a530 0x7f2d1c0c2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d21a7a640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a9a0 0x7f2d1c1f37b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d14007e60 tx=0x7f2d1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:37.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d11ffb640 1 -- 172.21.15.12:0/3182279014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14014420 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d11ffb640 1 -- 172.21.15.12:0/3182279014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d140145a0 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d11ffb640 1 -- 172.21.15.12:0/3182279014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14014950 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 -- 172.21.15.12:0/3182279014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1c14feb0 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 -- 172.21.15.12:0/3182279014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c147140 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.516+0000 7f2d2227b640 1 -- 172.21.15.12:0/3182279014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1c146f40 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.520+0000 7f2d11ffb640 1 -- 172.21.15.12:0/3182279014 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2d14014ab0 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.520+0000 7f2d11ffb640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d000664c0 0x7f2d00068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.520+0000 7f2d11ffb640 1 -- 172.21.15.12:0/3182279014 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d14084770 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.520+0000 7f2d21279640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d000664c0 0x7f2d00068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.520+0000 7f2d21279640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d000664c0 0x7f2d00068980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2d18000f70 tx=0x7f2d18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:37.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.524+0000 7f2d11ffb640 1 -- 172.21.15.12:0/3182279014 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d14051bd0 con 0x7f2d1c15a9a0 2023-11-28T21:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:37.688+0000 7f2d2227b640 1 -- 172.21.15.12:0/3182279014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f2d1c074f60 con 0x7f2d1c15a9a0 2023-11-28T21:34:38.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d11ffb640 1 -- 172.21.15.12:0/3182279014 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f2d140551f0 con 0x7f2d1c15a9a0 2023-11-28T21:34:38.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 -- 172.21.15.12:0/3182279014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d000664c0 msgr2=0x7f2d00068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:38.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d000664c0 0x7f2d00068980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2d18000f70 tx=0x7f2d18009040 comp rx=0 tx=0).stop 2023-11-28T21:34:38.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 -- 172.21.15.12:0/3182279014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a9a0 msgr2=0x7f2d1c1f37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:38.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a9a0 0x7f2d1c1f37b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d14007e60 tx=0x7f2d1400e4b0 comp rx=0 tx=0).stop 2023-11-28T21:34:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 -- 172.21.15.12:0/3182279014 shutdown_connections 2023-11-28T21:34:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d000664c0 0x7f2d00068980 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d1c15a9a0 0x7f2d1c1f37b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:38.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d1c15a530 0x7f2d1c0c2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:38.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 --2- 172.21.15.12:0/3182279014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d1c15a150 0x7f2d1c06d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:38.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 -- 172.21.15.12:0/3182279014 >> 172.21.15.12:0/3182279014 conn(0x7f2d1c0c7cb0 msgr2=0x7f2d1c0c8090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:38.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 -- 172.21.15.12:0/3182279014 shutdown_connections 2023-11-28T21:34:38.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.308+0000 7f2d2227b640 1 -- 172.21.15.12:0/3182279014 wait complete. 2023-11-28T21:34:38.336 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T21:34:38.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.488+0000 7f49adff8640 1 Processor -- start 2023-11-28T21:34:38.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.488+0000 7f49adff8640 1 -- start start 2023-11-28T21:34:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49adff8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f49a80c27c0 0x7f49a80c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49adff8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a80c6520 con 0x7f49a80c7610 2023-11-28T21:34:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49adff8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a8156e90 con 0x7f49a814ffd0 2023-11-28T21:34:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49adff8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a80c6ef0 con 0x7f49a80c27c0 2023-11-28T21:34:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49a7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f49a80c27c0 0x7f49a80c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:38.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49a7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f49a80c27c0 0x7f49a80c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56936/0 (socket says 172.21.15.12:56936) 2023-11-28T21:34:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49a7fff640 1 -- 172.21.15.12:0/294668294 learned_addr learned my addr 172.21.15.12:0/294668294 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49a6ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f49a80c7610 0x7f49a815eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54804/0 (socket says 172.21.15.12:54804) 2023-11-28T21:34:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49a7fff640 1 -- 172.21.15.12:0/294668294 >> v1:172.21.15.144:6789/0 conn(0x7f49a814ffd0 legacy=0x7f49a806ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49a7fff640 1 -- 172.21.15.12:0/294668294 >> v1:172.21.15.12:6789/0 conn(0x7f49a80c7610 legacy=0x7f49a815eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49a67fc640 1 -- 172.21.15.12:0/294668294 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496386663 0 0) 0x7f49a80c6520 con 0x7f49a80c7610 2023-11-28T21:34:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.492+0000 7f49a7fff640 1 -- 172.21.15.12:0/294668294 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80c5ed0 con 0x7f49a80c27c0 2023-11-28T21:34:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49a7fff640 1 --2- 172.21.15.12:0/294668294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 0x7f49a80c69b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009e50 tx=0x7f49a001f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d425ca7029873e79 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49a67fc640 1 -- 172.21.15.12:0/294668294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a00af400 con 0x7f49a80c27c0 2023-11-28T21:34:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49a67fc640 1 -- 172.21.15.12:0/294668294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a00ab050 con 0x7f49a80c27c0 2023-11-28T21:34:38.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49a67fc640 1 -- 172.21.15.12:0/294668294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a00af780 con 0x7f49a80c27c0 2023-11-28T21:34:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 -- 172.21.15.12:0/294668294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 msgr2=0x7f49a80c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 --2- 172.21.15.12:0/294668294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 0x7f49a80c69b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009e50 tx=0x7f49a001f1f0 comp rx=0 tx=0).stop 2023-11-28T21:34:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 -- 172.21.15.12:0/294668294 shutdown_connections 2023-11-28T21:34:38.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 --2- 172.21.15.12:0/294668294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 0x7f49a80c69b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:38.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 -- 172.21.15.12:0/294668294 >> 172.21.15.12:0/294668294 conn(0x7f49a806d5e0 msgr2=0x7f49a806d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:38.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 -- 172.21.15.12:0/294668294 shutdown_connections 2023-11-28T21:34:38.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 -- 172.21.15.12:0/294668294 wait complete. 2023-11-28T21:34:38.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 Processor -- start 2023-11-28T21:34:38.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 -- start start 2023-11-28T21:34:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 0x7f49a8150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a80c7610 0x7f49a80c3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a814ffd0 0x7f49a81ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a80c3e10 con 0x7f49a80c27c0 2023-11-28T21:34:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a81e2480 con 0x7f49a80c7610 2023-11-28T21:34:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.496+0000 7f49adff8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a81efd10 con 0x7f49a814ffd0 2023-11-28T21:34:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 0x7f49a8150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 0x7f49a8150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56946/0 (socket says 172.21.15.12:56946) 2023-11-28T21:34:38.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a77fe640 1 -- 172.21.15.12:0/1444134553 learned_addr learned my addr 172.21.15.12:0/1444134553 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a6ffd640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a80c7610 0x7f49a80c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a7fff640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a814ffd0 0x7f49a81ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a77fe640 1 -- 172.21.15.12:0/1444134553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a814ffd0 msgr2=0x7f49a81ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a77fe640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a814ffd0 0x7f49a81ef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a77fe640 1 -- 172.21.15.12:0/1444134553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a80c7610 msgr2=0x7f49a80c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a77fe640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a80c7610 0x7f49a80c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a77fe640 1 -- 172.21.15.12:0/1444134553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a0009b30 con 0x7f49a80c27c0 2023-11-28T21:34:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a7fff640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a814ffd0 0x7f49a81ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:34:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49a77fe640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 0x7f49a8150d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f499800c3c0 tx=0x7f499800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:38.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f498ffff640 1 -- 172.21.15.12:0/1444134553 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49980052d0 con 0x7f49a80c27c0 2023-11-28T21:34:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f498ffff640 1 -- 172.21.15.12:0/1444134553 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4998005430 con 0x7f49a80c27c0 2023-11-28T21:34:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f498ffff640 1 -- 172.21.15.12:0/1444134553 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998003840 con 0x7f49a80c27c0 2023-11-28T21:34:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49adff8640 1 -- 172.21.15.12:0/1444134553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a81efff0 con 0x7f49a80c27c0 2023-11-28T21:34:38.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49adff8640 1 -- 172.21.15.12:0/1444134553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a81e29b0 con 0x7f49a80c27c0 2023-11-28T21:34:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f49adff8640 1 -- 172.21.15.12:0/1444134553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a804e850 con 0x7f49a80c27c0 2023-11-28T21:34:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f498ffff640 1 -- 172.21.15.12:0/1444134553 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f49980039a0 con 0x7f49a80c27c0 2023-11-28T21:34:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.500+0000 7f498ffff640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4994066470 0x7f4994068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.504+0000 7f498ffff640 1 -- 172.21.15.12:0/1444134553 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f49980834b0 con 0x7f49a80c27c0 2023-11-28T21:34:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.504+0000 7f49a6ffd640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4994066470 0x7f4994068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:38.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.504+0000 7f49a6ffd640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4994066470 0x7f4994068930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49a8074250 tx=0x7f499c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:38.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.508+0000 7f498ffff640 1 -- 172.21.15.12:0/1444134553 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4998050840 con 0x7f49a80c27c0 2023-11-28T21:34:38.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:38.664+0000 7f49adff8640 1 -- 172.21.15.12:0/1444134553 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f49a80c73f0 con 0x7f49a80c27c0 2023-11-28T21:34:40.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.328+0000 7f498ffff640 1 -- 172.21.15.12:0/1444134553 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f4998053e60 con 0x7f49a80c27c0 2023-11-28T21:34:40.331 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' created 2023-11-28T21:34:40.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 -- 172.21.15.12:0/1444134553 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4994066470 msgr2=0x7f4994068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:40.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4994066470 0x7f4994068930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49a8074250 tx=0x7f499c009040 comp rx=0 tx=0).stop 2023-11-28T21:34:40.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 -- 172.21.15.12:0/1444134553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 msgr2=0x7f49a8150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:40.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 0x7f49a8150d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f499800c3c0 tx=0x7f499800c880 comp rx=0 tx=0).stop 2023-11-28T21:34:40.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 -- 172.21.15.12:0/1444134553 shutdown_connections 2023-11-28T21:34:40.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4994066470 0x7f4994068930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:40.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a814ffd0 0x7f49a81ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:40.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a80c7610 0x7f49a80c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:40.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 --2- 172.21.15.12:0/1444134553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49a80c27c0 0x7f49a8150d70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:40.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 -- 172.21.15.12:0/1444134553 >> 172.21.15.12:0/1444134553 conn(0x7f49a806d5e0 msgr2=0x7f49a80c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:40.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 -- 172.21.15.12:0/1444134553 shutdown_connections 2023-11-28T21:34:40.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.332+0000 7f49adff8640 1 -- 172.21.15.12:0/1444134553 wait complete. 2023-11-28T21:34:40.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-28T21:34:40.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.516+0000 7ff5d9916640 1 Processor -- start 2023-11-28T21:34:40.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.516+0000 7ff5d9916640 1 -- start start 2023-11-28T21:34:40.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.516+0000 7ff5d9916640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d4140390 0x7ff5d40725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:40.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.516+0000 7ff5d9916640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff5d415b0c0 0x7ff5d40c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:40.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.516+0000 7ff5d9916640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d40c6920 con 0x7ff5d4072af0 2023-11-28T21:34:40.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.516+0000 7ff5d9916640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d4071e20 con 0x7ff5d415b0c0 2023-11-28T21:34:40.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.516+0000 7ff5d9916640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d415b5b0 con 0x7ff5d4140390 2023-11-28T21:34:40.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.516+0000 7ff5d37fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff5d4072af0 0x7ff5d4157af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54812/0 (socket says 172.21.15.12:54812) 2023-11-28T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.516+0000 7ff5d37fe640 1 -- 172.21.15.12:0/4261011948 learned_addr learned my addr 172.21.15.12:0/4261011948 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/4261011948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d4140390 0x7ff5d40725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5cbfff640 1 --2- 172.21.15.12:0/4261011948 >> v2:172.21.15.144:3300/0 conn(0x7ff5d415b0c0 0x7ff5d40c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5d27fc640 1 -- 172.21.15.12:0/4261011948 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931191551 0 0) 0x7ff5d40c6920 con 0x7ff5d4072af0 2023-11-28T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5d27fc640 1 -- 172.21.15.12:0/4261011948 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5c0003530 con 0x7ff5d4072af0 2023-11-28T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5d2ffd640 1 -- 172.21.15.12:0/4261011948 >> v2:172.21.15.144:3300/0 conn(0x7ff5d415b0c0 msgr2=0x7ff5d40c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/4261011948 >> v2:172.21.15.144:3300/0 conn(0x7ff5d415b0c0 0x7ff5d40c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5d2ffd640 1 -- 172.21.15.12:0/4261011948 >> v1:172.21.15.12:6789/0 conn(0x7ff5d4072af0 legacy=0x7ff5d4157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5d27fc640 1 -- 172.21.15.12:0/4261011948 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2391742659 0 0) 0x7ff5c0003530 con 0x7ff5d4072af0 2023-11-28T21:34:40.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5d2ffd640 1 -- 172.21.15.12:0/4261011948 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d415b890 con 0x7ff5d4140390 2023-11-28T21:34:40.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/4261011948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d4140390 0x7ff5d40725b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097c0 tx=0x7ff5c401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f2ba3a3375b0dec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:40.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.520+0000 7ff5d27fc640 1 -- 172.21.15.12:0/4261011948 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40ab650 con 0x7ff5d4140390 2023-11-28T21:34:40.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d27fc640 1 -- 172.21.15.12:0/4261011948 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c40ab7b0 con 0x7ff5d4140390 2023-11-28T21:34:40.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d27fc640 1 -- 172.21.15.12:0/4261011948 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40abb20 con 0x7ff5d4140390 2023-11-28T21:34:40.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 -- 172.21.15.12:0/4261011948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d4140390 msgr2=0x7ff5d40725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:40.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 --2- 172.21.15.12:0/4261011948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d4140390 0x7ff5d40725b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097c0 tx=0x7ff5c401c450 comp rx=0 tx=0).stop 2023-11-28T21:34:40.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 -- 172.21.15.12:0/4261011948 shutdown_connections 2023-11-28T21:34:40.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 --2- 172.21.15.12:0/4261011948 >> v2:172.21.15.144:3300/0 conn(0x7ff5d415b0c0 0x7ff5d40c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:40.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 --2- 172.21.15.12:0/4261011948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d4140390 0x7ff5d40725b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:40.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 -- 172.21.15.12:0/4261011948 >> 172.21.15.12:0/4261011948 conn(0x7ff5d406d5e0 msgr2=0x7ff5d406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:40.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 -- 172.21.15.12:0/4261011948 shutdown_connections 2023-11-28T21:34:40.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 -- 172.21.15.12:0/4261011948 wait complete. 2023-11-28T21:34:40.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 Processor -- start 2023-11-28T21:34:40.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 -- start start 2023-11-28T21:34:40.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d4072af0 0x7ff5d41f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d4140390 0x7ff5d4161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d415b0c0 0x7ff5d4155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4161c50 con 0x7ff5d4140390 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4157670 con 0x7ff5d4072af0 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d9916640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d4157060 con 0x7ff5d415b0c0 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d4140390 0x7ff5d4161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d4140390 0x7ff5d4161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56958/0 (socket says 172.21.15.12:56958) 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5cbfff640 1 -- 172.21.15.12:0/830721824 learned_addr learned my addr 172.21.15.12:0/830721824 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d4072af0 0x7ff5d41f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.524+0000 7ff5d37fe640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d415b0c0 0x7ff5d4155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:40.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5cbfff640 1 -- 172.21.15.12:0/830721824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d415b0c0 msgr2=0x7ff5d4155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5cbfff640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d415b0c0 0x7ff5d4155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5cbfff640 1 -- 172.21.15.12:0/830721824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d4072af0 msgr2=0x7ff5d41f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5cbfff640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d4072af0 0x7ff5d41f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5cbfff640 1 -- 172.21.15.12:0/830721824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c4009420 con 0x7ff5d4140390 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5d37fe640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d415b0c0 0x7ff5d4155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5cbfff640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d4140390 0x7ff5d4161120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00e900 tx=0x7ff5bc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/830721824 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc009690 con 0x7ff5d4140390 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/830721824 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc00ca00 con 0x7ff5d4140390 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/830721824 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc00cdb0 con 0x7ff5d4140390 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5d9916640 1 -- 172.21.15.12:0/830721824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d41579e0 con 0x7ff5d4140390 2023-11-28T21:34:40.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5d9916640 1 -- 172.21.15.12:0/830721824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d4157300 con 0x7ff5d4140390 2023-11-28T21:34:40.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5d9916640 1 -- 172.21.15.12:0/830721824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d4155c10 con 0x7ff5d4140390 2023-11-28T21:34:40.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.528+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/830721824 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff5bc013470 con 0x7ff5d4140390 2023-11-28T21:34:40.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.532+0000 7ff5d0ff9640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5b40664a0 0x7ff5b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:40.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.532+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/830721824 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6428+0+0 (secure 0 0 0) 0x7ff5bc084360 con 0x7ff5d4140390 2023-11-28T21:34:40.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.532+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5b40664a0 0x7ff5b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:40.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.532+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5b40664a0 0x7ff5b4068960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40af000 tx=0x7ff5c40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:40.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.536+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/830721824 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff5bc051520 con 0x7ff5d4140390 2023-11-28T21:34:40.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:40.692+0000 7ff5d9916640 1 -- 172.21.15.12:0/830721824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7ff5d40716f0 con 0x7ff5d4140390 2023-11-28T21:34:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.332+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/830721824 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff5bc054b40 con 0x7ff5d4140390 2023-11-28T21:34:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:set pool 4 allow_ec_overwrites to true 2023-11-28T21:34:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.336+0000 7ff5d9916640 1 -- 172.21.15.12:0/830721824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5b40664a0 msgr2=0x7ff5b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.336+0000 7ff5d9916640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5b40664a0 0x7ff5b4068960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40af000 tx=0x7ff5c40a7000 comp rx=0 tx=0).stop 2023-11-28T21:34:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.336+0000 7ff5d9916640 1 -- 172.21.15.12:0/830721824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d4140390 msgr2=0x7ff5d4161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:41.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.336+0000 7ff5d9916640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d4140390 0x7ff5d4161120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00e900 tx=0x7ff5bc00edc0 comp rx=0 tx=0).stop 2023-11-28T21:34:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.340+0000 7ff5d9916640 1 -- 172.21.15.12:0/830721824 shutdown_connections 2023-11-28T21:34:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.340+0000 7ff5d9916640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5b40664a0 0x7ff5b4068960 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.340+0000 7ff5d9916640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5d415b0c0 0x7ff5d4155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.340+0000 7ff5d9916640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d4140390 0x7ff5d4161120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.340+0000 7ff5d9916640 1 --2- 172.21.15.12:0/830721824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5d4072af0 0x7ff5d41f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:41.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.340+0000 7ff5d9916640 1 -- 172.21.15.12:0/830721824 >> 172.21.15.12:0/830721824 conn(0x7ff5d406d5e0 msgr2=0x7ff5d40c49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:41.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.344+0000 7ff5d9916640 1 -- 172.21.15.12:0/830721824 shutdown_connections 2023-11-28T21:34:41.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.344+0000 7ff5d9916640 1 -- 172.21.15.12:0/830721824 wait complete. 2023-11-28T21:34:41.391 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-28T21:34:41.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.540+0000 7f027c5df640 1 Processor -- start 2023-11-28T21:34:41.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.540+0000 7f027c5df640 1 -- start start 2023-11-28T21:34:41.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f027c5df640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0274151af0 0x7f02740c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:41.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f027c5df640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0274157b70 con 0x7f0274151ed0 2023-11-28T21:34:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f027c5df640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0274157e50 con 0x7f027406e300 2023-11-28T21:34:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f027c5df640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02740707f0 con 0x7f0274151af0 2023-11-28T21:34:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f027a37b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0274151af0 0x7f02740c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f027a37b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0274151af0 0x7f02740c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56960/0 (socket says 172.21.15.12:56960) 2023-11-28T21:34:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f027a37b640 1 -- 172.21.15.12:0/189559555 learned_addr learned my addr 172.21.15.12:0/189559555 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:41.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f027a37b640 1 -- 172.21.15.12:0/189559555 >> v1:172.21.15.144:6789/0 conn(0x7f027406e300 legacy=0x7f0274072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:41.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f027a37b640 1 -- 172.21.15.12:0/189559555 >> v1:172.21.15.12:6789/0 conn(0x7f0274151ed0 legacy=0x7f02740c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:41.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f027a37b640 1 -- 172.21.15.12:0/189559555 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0274158130 con 0x7f0274151af0 2023-11-28T21:34:41.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.544+0000 7f0279379640 1 -- 172.21.15.12:0/189559555 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159732130 0 0) 0x7f0274157b70 con 0x7f0274151ed0 2023-11-28T21:34:41.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027a37b640 1 --2- 172.21.15.12:0/189559555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 0x7f02740c7650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0270009870 tx=0x7f027001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b375c78329b97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:41.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f0279379640 1 -- 172.21.15.12:0/189559555 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02700ab600 con 0x7f0274151af0 2023-11-28T21:34:41.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f0279379640 1 -- 172.21.15.12:0/189559555 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02700a7050 con 0x7f0274151af0 2023-11-28T21:34:41.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f0279379640 1 -- 172.21.15.12:0/189559555 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02700ab980 con 0x7f0274151af0 2023-11-28T21:34:41.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 -- 172.21.15.12:0/189559555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 msgr2=0x7f02740c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:41.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 --2- 172.21.15.12:0/189559555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 0x7f02740c7650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0270009870 tx=0x7f027001c540 comp rx=0 tx=0).stop 2023-11-28T21:34:41.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 -- 172.21.15.12:0/189559555 shutdown_connections 2023-11-28T21:34:41.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 --2- 172.21.15.12:0/189559555 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 0x7f02740c7650 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:41.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 -- 172.21.15.12:0/189559555 >> 172.21.15.12:0/189559555 conn(0x7f0274066360 msgr2=0x7f0274066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:41.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 -- 172.21.15.12:0/189559555 shutdown_connections 2023-11-28T21:34:41.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 -- 172.21.15.12:0/189559555 wait complete. 2023-11-28T21:34:41.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 Processor -- start 2023-11-28T21:34:41.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 -- start start 2023-11-28T21:34:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f027406e300 0x7f0274144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 0x7f0274143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0274146ad0 0x7f0274140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0274148fb0 con 0x7f0274151af0 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02741f69d0 con 0x7f0274146ad0 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.548+0000 7f027c5df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02741f63c0 con 0x7f027406e300 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f0279b7a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 0x7f0274143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f0279b7a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 0x7f0274143ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56970/0 (socket says 172.21.15.12:56970) 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f0279b7a640 1 -- 172.21.15.12:0/4027212171 learned_addr learned my addr 172.21.15.12:0/4027212171 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f027ab7c640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0274146ad0 0x7f0274140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f027a37b640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f027406e300 0x7f0274144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f0279b7a640 1 -- 172.21.15.12:0/4027212171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f027406e300 msgr2=0x7f0274144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f0279b7a640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f027406e300 0x7f0274144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f0279b7a640 1 -- 172.21.15.12:0/4027212171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0274146ad0 msgr2=0x7f0274140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f0279b7a640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0274146ad0 0x7f0274140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f0279b7a640 1 -- 172.21.15.12:0/4027212171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02700094d0 con 0x7f0274151af0 2023-11-28T21:34:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f0279b7a640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 0x7f0274143ce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f026400c410 tx=0x7f026400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f026affd640 1 -- 172.21.15.12:0/4027212171 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640054e0 con 0x7f0274151af0 2023-11-28T21:34:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f026affd640 1 -- 172.21.15.12:0/4027212171 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0264003600 con 0x7f0274151af0 2023-11-28T21:34:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f027c5df640 1 -- 172.21.15.12:0/4027212171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0274148550 con 0x7f0274151af0 2023-11-28T21:34:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f026affd640 1 -- 172.21.15.12:0/4027212171 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640039b0 con 0x7f0274151af0 2023-11-28T21:34:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f027c5df640 1 -- 172.21.15.12:0/4027212171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02741f6660 con 0x7f0274151af0 2023-11-28T21:34:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f027c5df640 1 -- 172.21.15.12:0/4027212171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02741f6820 con 0x7f0274151af0 2023-11-28T21:34:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.552+0000 7f026affd640 1 -- 172.21.15.12:0/4027212171 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0264003b10 con 0x7f0274151af0 2023-11-28T21:34:41.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.556+0000 7f026affd640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02580664a0 0x7f0258068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:41.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.556+0000 7f026affd640 1 -- 172.21.15.12:0/4027212171 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f02640835e0 con 0x7f0274151af0 2023-11-28T21:34:41.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.556+0000 7f027a37b640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02580664a0 0x7f0258068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:41.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.556+0000 7f027a37b640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02580664a0 0x7f0258068960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02740c8e60 tx=0x7f02700a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:41.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.556+0000 7f026affd640 1 -- 172.21.15.12:0/4027212171 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0264050820 con 0x7f0274151af0 2023-11-28T21:34:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:41.752+0000 7f027c5df640 1 -- 172.21.15.12:0/4027212171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f02741f6b40 con 0x7f0274151af0 2023-11-28T21:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f026affd640 1 -- 172.21.15.12:0/4027212171 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v5) v1 ==== 137+0+0 (secure 0 0 0) 0x7f0264053e40 con 0x7f0274151af0 2023-11-28T21:34:42.365 INFO:teuthology.orchestra.run.smithi012.stderr:added data pool 4 to fsmap 2023-11-28T21:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 -- 172.21.15.12:0/4027212171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02580664a0 msgr2=0x7f0258068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02580664a0 0x7f0258068960 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02740c8e60 tx=0x7f02700a6000 comp rx=0 tx=0).stop 2023-11-28T21:34:42.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 -- 172.21.15.12:0/4027212171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 msgr2=0x7f0274143ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 0x7f0274143ce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f026400c410 tx=0x7f026400c8d0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 -- 172.21.15.12:0/4027212171 shutdown_connections 2023-11-28T21:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02580664a0 0x7f0258068960 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0274146ad0 0x7f0274140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0274151af0 0x7f0274143ce0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 --2- 172.21.15.12:0/4027212171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f027406e300 0x7f0274144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 -- 172.21.15.12:0/4027212171 >> 172.21.15.12:0/4027212171 conn(0x7f0274066360 msgr2=0x7f027406cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 -- 172.21.15.12:0/4027212171 shutdown_connections 2023-11-28T21:34:42.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.364+0000 7f027c5df640 1 -- 172.21.15.12:0/4027212171 wait complete. 2023-11-28T21:34:42.388 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:34:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.528+0000 7f2f2c90f640 1 Processor -- start 2023-11-28T21:34:42.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.528+0000 7f2f2c90f640 1 -- start start 2023-11-28T21:34:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.528+0000 7f2f2c90f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f28151af0 0x7f2f280c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.528+0000 7f2f2c90f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2f28151ed0 0x7f2f280c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.528+0000 7f2f2c90f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f28157b70 con 0x7f2f2806e300 2023-11-28T21:34:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.528+0000 7f2f2c90f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f280707f0 con 0x7f2f28151af0 2023-11-28T21:34:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.528+0000 7f2f2c90f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f280c7b90 con 0x7f2f28151ed0 2023-11-28T21:34:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.528+0000 7f2f2659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f28151af0 0x7f2f280c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f2659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f28151af0 0x7f2f280c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56978/0 (socket says 172.21.15.12:56978) 2023-11-28T21:34:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f2659c640 1 -- 172.21.15.12:0/2443574819 learned_addr learned my addr 172.21.15.12:0/2443574819 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f26d9d640 1 --1- 172.21.15.12:0/2443574819 >> v1:172.21.15.12:6789/0 conn(0x7f2f2806e300 0x7f2f28072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54834/0 (socket says 172.21.15.12:54834) 2023-11-28T21:34:42.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f1ffff640 1 --2- 172.21.15.12:0/2443574819 >> v2:172.21.15.144:3300/0 conn(0x7f2f28151ed0 0x7f2f280c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f2659c640 1 -- 172.21.15.12:0/2443574819 >> v2:172.21.15.144:3300/0 conn(0x7f2f28151ed0 msgr2=0x7f2f280c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f2659c640 1 --2- 172.21.15.12:0/2443574819 >> v2:172.21.15.144:3300/0 conn(0x7f2f28151ed0 0x7f2f280c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f2659c640 1 -- 172.21.15.12:0/2443574819 >> v1:172.21.15.12:6789/0 conn(0x7f2f2806e300 legacy=0x7f2f28072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f2659c640 1 -- 172.21.15.12:0/2443574819 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f28157e50 con 0x7f2f28151af0 2023-11-28T21:34:42.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f2659c640 1 --2- 172.21.15.12:0/2443574819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 0x7f2f280c7650 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2f280c8e60 tx=0x7f2f1801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ca9d66591e54ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:42.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f25d9b640 1 -- 172.21.15.12:0/2443574819 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180ab600 con 0x7f2f28151af0 2023-11-28T21:34:42.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f25d9b640 1 -- 172.21.15.12:0/2443574819 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f180a7050 con 0x7f2f28151af0 2023-11-28T21:34:42.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.532+0000 7f2f25d9b640 1 -- 172.21.15.12:0/2443574819 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180ab980 con 0x7f2f28151af0 2023-11-28T21:34:42.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2443574819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 msgr2=0x7f2f280c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 --2- 172.21.15.12:0/2443574819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 0x7f2f280c7650 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2f280c8e60 tx=0x7f2f1801c500 comp rx=0 tx=0).stop 2023-11-28T21:34:42.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2443574819 shutdown_connections 2023-11-28T21:34:42.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 --2- 172.21.15.12:0/2443574819 >> v2:172.21.15.144:3300/0 conn(0x7f2f28151ed0 0x7f2f280c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 --2- 172.21.15.12:0/2443574819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 0x7f2f280c7650 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2443574819 >> 172.21.15.12:0/2443574819 conn(0x7f2f28066360 msgr2=0x7f2f28066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:42.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2443574819 shutdown_connections 2023-11-28T21:34:42.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2443574819 wait complete. 2023-11-28T21:34:42.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 Processor -- start 2023-11-28T21:34:42.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 -- start start 2023-11-28T21:34:42.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 0x7f2f280c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 0x7f2f28140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f28143a70 0x7f2f28140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281406c0 con 0x7f2f28151af0 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281f6d60 con 0x7f2f28143a70 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2c90f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f28143f60 con 0x7f2f2806e300 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f1ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 0x7f2f28140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f1ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 0x7f2f28140aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56986/0 (socket says 172.21.15.12:56986) 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f1ffff640 1 -- 172.21.15.12:0/2328055177 learned_addr learned my addr 172.21.15.12:0/2328055177 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2659c640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 0x7f2f280c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f26d9d640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f28143a70 0x7f2f28140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f1ffff640 1 -- 172.21.15.12:0/2328055177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 msgr2=0x7f2f280c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f1ffff640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 0x7f2f280c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f1ffff640 1 -- 172.21.15.12:0/2328055177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f28143a70 msgr2=0x7f2f28140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f1ffff640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f28143a70 0x7f2f28140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f1ffff640 1 -- 172.21.15.12:0/2328055177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f180094d0 con 0x7f2f28151af0 2023-11-28T21:34:42.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f2659c640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 0x7f2f280c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:42.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.536+0000 7f2f1ffff640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 0x7f2f28140aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000d5c0 tx=0x7f2f1000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.540+0000 7f2f1effd640 1 -- 172.21.15.12:0/2328055177 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10004240 con 0x7f2f28151af0 2023-11-28T21:34:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.540+0000 7f2f1effd640 1 -- 172.21.15.12:0/2328055177 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f100043a0 con 0x7f2f28151af0 2023-11-28T21:34:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.540+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2328055177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f281f6150 con 0x7f2f28151af0 2023-11-28T21:34:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.540+0000 7f2f1effd640 1 -- 172.21.15.12:0/2328055177 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10003840 con 0x7f2f28151af0 2023-11-28T21:34:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.540+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2328055177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f281f6710 con 0x7f2f28151af0 2023-11-28T21:34:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.540+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2328055177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f28074f60 con 0x7f2f28151af0 2023-11-28T21:34:42.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.540+0000 7f2f1effd640 1 -- 172.21.15.12:0/2328055177 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2f100039a0 con 0x7f2f28151af0 2023-11-28T21:34:42.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.540+0000 7f2f1effd640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f08066470 0x7f2f08068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.540+0000 7f2f1effd640 1 -- 172.21.15.12:0/2328055177 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2f100842e0 con 0x7f2f28151af0 2023-11-28T21:34:42.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.540+0000 7f2f2659c640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f08066470 0x7f2f08068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.544+0000 7f2f2659c640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f08066470 0x7f2f08068930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f180ad000 tx=0x7f2f18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:42.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.544+0000 7f2f1effd640 1 -- 172.21.15.12:0/2328055177 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2f10051460 con 0x7f2f28151af0 2023-11-28T21:34:42.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.740+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2328055177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f281f6ea0 con 0x7f2f28151af0 2023-11-28T21:34:42.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.740+0000 7f2f1effd640 1 -- 172.21.15.12:0/2328055177 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+6495 (secure 0 0 0) 0x7f2f10054a80 con 0x7f2f28151af0 2023-11-28T21:34:42.745 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:42.745 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:34:42.347765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T21:34:42.745 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 6 2023-11-28T21:34:42.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2328055177 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f08066470 msgr2=0x7f2f08068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f08066470 0x7f2f08068930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f180ad000 tx=0x7f2f18008040 comp rx=0 tx=0).stop 2023-11-28T21:34:42.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2328055177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 msgr2=0x7f2f28140aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 0x7f2f28140aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000d5c0 tx=0x7f2f1000da80 comp rx=0 tx=0).stop 2023-11-28T21:34:42.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2328055177 shutdown_connections 2023-11-28T21:34:42.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f08066470 0x7f2f08068930 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f28143a70 0x7f2f28140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151af0 0x7f2f28140aa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 --2- 172.21.15.12:0/2328055177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 0x7f2f280c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2328055177 >> 172.21.15.12:0/2328055177 conn(0x7f2f28066360 msgr2=0x7f2f28150460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:42.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2328055177 shutdown_connections 2023-11-28T21:34:42.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.744+0000 7f2f2c90f640 1 -- 172.21.15.12:0/2328055177 wait complete. 2023-11-28T21:34:42.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:34:42.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb97dfb8640 1 Processor -- start 2023-11-28T21:34:42.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb97dfb8640 1 -- start start 2023-11-28T21:34:42.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb97dfb8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb978151af0 0x7fb9780c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb97dfb8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb978151ed0 0x7fb9780c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb97dfb8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb978157b70 con 0x7fb97806e300 2023-11-28T21:34:42.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb97dfb8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9780707f0 con 0x7fb978151ed0 2023-11-28T21:34:42.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb97dfb8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9780c7b90 con 0x7fb978151af0 2023-11-28T21:34:42.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb977fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb97806e300 0x7fb978072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54846/0 (socket says 172.21.15.12:54846) 2023-11-28T21:34:42.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb977fff640 1 -- 172.21.15.12:0/3380793484 learned_addr learned my addr 172.21.15.12:0/3380793484 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb976ffd640 1 --2- 172.21.15.12:0/3380793484 >> v2:172.21.15.144:3300/0 conn(0x7fb978151ed0 0x7fb9780c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9777fe640 1 --2- 172.21.15.12:0/3380793484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb978151af0 0x7fb9780c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604572864 0 0) 0x7fb978157b70 con 0x7fb97806e300 2023-11-28T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb960003530 con 0x7fb97806e300 2023-11-28T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4119169871 0 0) 0x7fb960003530 con 0x7fb97806e300 2023-11-28T21:34:42.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb978151af0 msgr2=0x7fb9780c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 --2- 172.21.15.12:0/3380793484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb978151af0 0x7fb9780c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 >> v2:172.21.15.144:3300/0 conn(0x7fb978151ed0 msgr2=0x7fb9780c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 --2- 172.21.15.12:0/3380793484 >> v2:172.21.15.144:3300/0 conn(0x7fb978151ed0 0x7fb9780c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb978157e50 con 0x7fb97806e300 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fb970001a00 con 0x7fb97806e300 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 >> v1:172.21.15.12:6789/0 conn(0x7fb97806e300 legacy=0x7fb978072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 --2- 172.21.15.12:0/3380793484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600039f0 0x7fb960003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 --2- 172.21.15.12:0/3380793484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9600047d0 0x7fb960004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 --2- 172.21.15.12:0/3380793484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb978151af0 0x7fb960005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9780707f0 con 0x7fb9600039f0 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9780c7b90 con 0x7fb9600047d0 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96000b990 con 0x7fb978151af0 2023-11-28T21:34:42.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9777fe640 1 --2- 172.21.15.12:0/3380793484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600039f0 0x7fb960003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9777fe640 1 -- 172.21.15.12:0/3380793484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb978151af0 msgr2=0x7fb960005190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T21:34:42.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9777fe640 1 --2- 172.21.15.12:0/3380793484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb978151af0 0x7fb960005190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9777fe640 1 -- 172.21.15.12:0/3380793484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9600047d0 msgr2=0x7fb960004c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T21:34:42.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9777fe640 1 --2- 172.21.15.12:0/3380793484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9600047d0 0x7fb960004c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.908+0000 7fb9777fe640 1 -- 172.21.15.12:0/3380793484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb960003530 con 0x7fb9600039f0 2023-11-28T21:34:42.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb9777fe640 1 --2- 172.21.15.12:0/3380793484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600039f0 0x7fb960003e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb968009af0 tx=0x7fb96801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5bc4828816ea25 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:42.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9680a9430 con 0x7fb9600039f0 2023-11-28T21:34:42.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9680a9590 con 0x7fb9600039f0 2023-11-28T21:34:42.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb9767fc640 1 -- 172.21.15.12:0/3380793484 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9680a9920 con 0x7fb9600039f0 2023-11-28T21:34:42.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb97dfb8640 1 -- 172.21.15.12:0/3380793484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600039f0 msgr2=0x7fb960003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb97dfb8640 1 --2- 172.21.15.12:0/3380793484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600039f0 0x7fb960003e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb968009af0 tx=0x7fb96801ca30 comp rx=0 tx=0).stop 2023-11-28T21:34:42.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb977fff640 1 -- 172.21.15.12:0/3380793484 reap_dead start 2023-11-28T21:34:42.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb97dfb8640 1 -- 172.21.15.12:0/3380793484 shutdown_connections 2023-11-28T21:34:42.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb97dfb8640 1 -- 172.21.15.12:0/3380793484 >> 172.21.15.12:0/3380793484 conn(0x7fb978066360 msgr2=0x7fb978066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:42.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb97dfb8640 1 -- 172.21.15.12:0/3380793484 shutdown_connections 2023-11-28T21:34:42.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.912+0000 7fb97dfb8640 1 -- 172.21.15.12:0/3380793484 wait complete. 2023-11-28T21:34:42.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb97dfb8640 1 Processor -- start 2023-11-28T21:34:42.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb97dfb8640 1 -- start start 2023-11-28T21:34:42.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb97dfb8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600039f0 0x7fb9781607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb97dfb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9780c2310 0x7fb978148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb97dfb8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9781f4440 0x7fb9781479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb97dfb8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97814a2b0 con 0x7fb9780c2310 2023-11-28T21:34:42.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb97dfb8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97814a640 con 0x7fb9781f4440 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb97dfb8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97815f490 con 0x7fb9600039f0 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb976ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9780c2310 0x7fb978148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb976ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9780c2310 0x7fb978148ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57016/0 (socket says 172.21.15.12:57016) 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb976ffd640 1 -- 172.21.15.12:0/898956160 learned_addr learned my addr 172.21.15.12:0/898956160 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb977fff640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9781f4440 0x7fb9781479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb9777fe640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600039f0 0x7fb9781607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb976ffd640 1 -- 172.21.15.12:0/898956160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600039f0 msgr2=0x7fb9781607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb976ffd640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600039f0 0x7fb9781607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb976ffd640 1 -- 172.21.15.12:0/898956160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9781f4440 msgr2=0x7fb9781479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb976ffd640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9781f4440 0x7fb9781479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:42.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb976ffd640 1 -- 172.21.15.12:0/898956160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97815f710 con 0x7fb9780c2310 2023-11-28T21:34:42.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb977fff640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9781f4440 0x7fb9781479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:34:42.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb9777fe640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600039f0 0x7fb9781607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:34:42.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb976ffd640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9780c2310 0x7fb978148ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00e880 tx=0x7fb96c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:42.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb95ffff640 1 -- 172.21.15.12:0/898956160 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c009650 con 0x7fb9780c2310 2023-11-28T21:34:42.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb95ffff640 1 -- 172.21.15.12:0/898956160 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c00c8e0 con 0x7fb9780c2310 2023-11-28T21:34:42.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb95ffff640 1 -- 172.21.15.12:0/898956160 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00cc70 con 0x7fb9780c2310 2023-11-28T21:34:42.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb97dfb8640 1 -- 172.21.15.12:0/898956160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97815f9f0 con 0x7fb9780c2310 2023-11-28T21:34:42.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.916+0000 7fb97dfb8640 1 -- 172.21.15.12:0/898956160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97815ffb0 con 0x7fb9780c2310 2023-11-28T21:34:42.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.920+0000 7fb97dfb8640 1 -- 172.21.15.12:0/898956160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb978074f60 con 0x7fb9780c2310 2023-11-28T21:34:42.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.920+0000 7fb95ffff640 1 -- 172.21.15.12:0/898956160 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb96c013470 con 0x7fb9780c2310 2023-11-28T21:34:42.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.920+0000 7fb95ffff640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb9640664a0 0x7fb964068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:42.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.920+0000 7fb95ffff640 1 -- 172.21.15.12:0/898956160 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb96c0842e0 con 0x7fb9780c2310 2023-11-28T21:34:42.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.920+0000 7fb9777fe640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb9640664a0 0x7fb964068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:42.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.920+0000 7fb9777fe640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb9640664a0 0x7fb964068960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb9680ae000 tx=0x7fb9680b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:42.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:42.924+0000 7fb95ffff640 1 -- 172.21.15.12:0/898956160 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb96c0513d0 con 0x7fb9780c2310 2023-11-28T21:34:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.084+0000 7fb97dfb8640 1 -- 172.21.15.12:0/898956160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb978150240 con 0x7fb9780c2310 2023-11-28T21:34:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.084+0000 7fb95ffff640 1 -- 172.21.15.12:0/898956160 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7fb96c0549f0 con 0x7fb9780c2310 2023-11-28T21:34:43.088 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:43.088 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:34:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 -- 172.21.15.12:0/898956160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb9640664a0 msgr2=0x7fb964068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb9640664a0 0x7fb964068960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb9680ae000 tx=0x7fb9680b3040 comp rx=0 tx=0).stop 2023-11-28T21:34:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 -- 172.21.15.12:0/898956160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9780c2310 msgr2=0x7fb978148ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9780c2310 0x7fb978148ad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00e880 tx=0x7fb96c00ed40 comp rx=0 tx=0).stop 2023-11-28T21:34:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 -- 172.21.15.12:0/898956160 shutdown_connections 2023-11-28T21:34:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb9640664a0 0x7fb964068960 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9781f4440 0x7fb9781479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9780c2310 0x7fb978148ad0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 --2- 172.21.15.12:0/898956160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600039f0 0x7fb9781607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 -- 172.21.15.12:0/898956160 >> 172.21.15.12:0/898956160 conn(0x7fb978066360 msgr2=0x7fb97806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 -- 172.21.15.12:0/898956160 shutdown_connections 2023-11-28T21:34:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.088+0000 7fb97dfb8640 1 -- 172.21.15.12:0/898956160 wait complete. 2023-11-28T21:34:43.109 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:34:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.248+0000 7f1318180640 1 Processor -- start 2023-11-28T21:34:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.248+0000 7f1318180640 1 -- start start 2023-11-28T21:34:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f1318180640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1310158420 0x7f131006d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f1318180640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f13100c38f0 0x7f13100c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f1318180640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13100c3460 con 0x7f131015e010 2023-11-28T21:34:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f1318180640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1310158800 con 0x7f13100c38f0 2023-11-28T21:34:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f1318180640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1310162610 con 0x7f1310158420 2023-11-28T21:34:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f131571b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f13100c38f0 0x7f13100c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f131571b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f13100c38f0 0x7f13100c3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57026/0 (socket says 172.21.15.12:57026) 2023-11-28T21:34:43.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f131571b640 1 -- 172.21.15.12:0/3102757968 learned_addr learned my addr 172.21.15.12:0/3102757968 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f131571b640 1 -- 172.21.15.12:0/3102757968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1310158420 msgr2=0x7f131006d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f131571b640 1 --2- 172.21.15.12:0/3102757968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1310158420 0x7f131006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f131571b640 1 -- 172.21.15.12:0/3102757968 >> v1:172.21.15.144:6789/0 conn(0x7f131015e010 legacy=0x7f131015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.252+0000 7f131571b640 1 -- 172.21.15.12:0/3102757968 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13100c2b70 con 0x7f13100c38f0 2023-11-28T21:34:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f131571b640 1 --2- 172.21.15.12:0/3102757968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 0x7f13100c3d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1300009870 tx=0x7f130001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d973e273286f065 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1314f1a640 1 -- 172.21.15.12:0/3102757968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13000ab600 con 0x7f13100c38f0 2023-11-28T21:34:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1314f1a640 1 -- 172.21.15.12:0/3102757968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13000a7050 con 0x7f13100c38f0 2023-11-28T21:34:43.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1314f1a640 1 -- 172.21.15.12:0/3102757968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13000ab980 con 0x7f13100c38f0 2023-11-28T21:34:43.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 -- 172.21.15.12:0/3102757968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 msgr2=0x7f13100c3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 --2- 172.21.15.12:0/3102757968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 0x7f13100c3d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1300009870 tx=0x7f130001c540 comp rx=0 tx=0).stop 2023-11-28T21:34:43.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 -- 172.21.15.12:0/3102757968 shutdown_connections 2023-11-28T21:34:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 --2- 172.21.15.12:0/3102757968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 0x7f13100c3d60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 --2- 172.21.15.12:0/3102757968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1310158420 0x7f131006d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 -- 172.21.15.12:0/3102757968 >> 172.21.15.12:0/3102757968 conn(0x7f1310066360 msgr2=0x7f1310066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 -- 172.21.15.12:0/3102757968 shutdown_connections 2023-11-28T21:34:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 -- 172.21.15.12:0/3102757968 wait complete. 2023-11-28T21:34:43.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 Processor -- start 2023-11-28T21:34:43.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 -- start start 2023-11-28T21:34:43.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 0x7f131014e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1310158420 0x7f131014e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.256+0000 7f1318180640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131015e010 0x7f13100c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1318180640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1310155bc0 con 0x7f13100c38f0 2023-11-28T21:34:43.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1318180640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13101f7030 con 0x7f1310158420 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1318180640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13101f7220 con 0x7f131015e010 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1315f1c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 0x7f131014e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1315f1c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 0x7f131014e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57040/0 (socket says 172.21.15.12:57040) 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1315f1c640 1 -- 172.21.15.12:0/174859525 learned_addr learned my addr 172.21.15.12:0/174859525 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f131571b640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1310158420 0x7f131014e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f131671d640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131015e010 0x7f13100c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1315f1c640 1 -- 172.21.15.12:0/174859525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131015e010 msgr2=0x7f13100c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1315f1c640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131015e010 0x7f13100c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1315f1c640 1 -- 172.21.15.12:0/174859525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1310158420 msgr2=0x7f131014e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1315f1c640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1310158420 0x7f131014e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1315f1c640 1 -- 172.21.15.12:0/174859525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13000094d0 con 0x7f13100c38f0 2023-11-28T21:34:43.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1315f1c640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 0x7f131014e180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f130c00b550 tx=0x7f130c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:43.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f13067fc640 1 -- 172.21.15.12:0/174859525 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c010870 con 0x7f13100c38f0 2023-11-28T21:34:43.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f13067fc640 1 -- 172.21.15.12:0/174859525 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130c0109f0 con 0x7f13100c38f0 2023-11-28T21:34:43.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f13067fc640 1 -- 172.21.15.12:0/174859525 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c010da0 con 0x7f13100c38f0 2023-11-28T21:34:43.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1318180640 1 -- 172.21.15.12:0/174859525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13101f7500 con 0x7f13100c38f0 2023-11-28T21:34:43.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1318180640 1 -- 172.21.15.12:0/174859525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13101f7ac0 con 0x7f13100c38f0 2023-11-28T21:34:43.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f1318180640 1 -- 172.21.15.12:0/174859525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1310074f60 con 0x7f13100c38f0 2023-11-28T21:34:43.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.260+0000 7f13067fc640 1 -- 172.21.15.12:0/174859525 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f130c00f3c0 con 0x7f13100c38f0 2023-11-28T21:34:43.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.264+0000 7f13067fc640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f12f40664a0 0x7f12f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.264+0000 7f13067fc640 1 -- 172.21.15.12:0/174859525 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f130c084290 con 0x7f13100c38f0 2023-11-28T21:34:43.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.264+0000 7f131571b640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f12f40664a0 0x7f12f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.264+0000 7f131571b640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f12f40664a0 0x7f12f4068960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f131006fb30 tx=0x7f1300008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:43.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.268+0000 7f13067fc640 1 -- 172.21.15.12:0/174859525 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f130c0512f0 con 0x7f13100c38f0 2023-11-28T21:34:43.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.428+0000 7f1318180640 1 -- 172.21.15.12:0/174859525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1310066840 con 0x7f13100c38f0 2023-11-28T21:34:43.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.428+0000 7f13067fc640 1 -- 172.21.15.12:0/174859525 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7f130c054910 con 0x7f13100c38f0 2023-11-28T21:34:43.431 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:43.431 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:34:43.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 -- 172.21.15.12:0/174859525 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f12f40664a0 msgr2=0x7f12f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f12f40664a0 0x7f12f4068960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f131006fb30 tx=0x7f1300008040 comp rx=0 tx=0).stop 2023-11-28T21:34:43.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 -- 172.21.15.12:0/174859525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 msgr2=0x7f131014e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 0x7f131014e180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f130c00b550 tx=0x7f130c00ba10 comp rx=0 tx=0).stop 2023-11-28T21:34:43.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 -- 172.21.15.12:0/174859525 shutdown_connections 2023-11-28T21:34:43.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f12f40664a0 0x7f12f4068960 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f131015e010 0x7f13100c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1310158420 0x7f131014e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 --2- 172.21.15.12:0/174859525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13100c38f0 0x7f131014e180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 -- 172.21.15.12:0/174859525 >> 172.21.15.12:0/174859525 conn(0x7f1310066360 msgr2=0x7f1310071620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:43.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 -- 172.21.15.12:0/174859525 shutdown_connections 2023-11-28T21:34:43.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.432+0000 7f1318180640 1 -- 172.21.15.12:0/174859525 wait complete. 2023-11-28T21:34:43.465 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:34:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f150c93b640 1 Processor -- start 2023-11-28T21:34:43.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f150c93b640 1 -- start start 2023-11-28T21:34:43.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f150c93b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1508153fb0 0x7f15080ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f150c93b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15080b41e0 0x7f15080caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f150c93b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150815d390 con 0x7f150815a980 2023-11-28T21:34:43.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f150c93b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15080b2b60 con 0x7f1508153fb0 2023-11-28T21:34:43.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f150c93b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150815e1b0 con 0x7f15080b41e0 2023-11-28T21:34:43.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f1506ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f150815a980 0x7f15080b3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54852/0 (socket says 172.21.15.12:54852) 2023-11-28T21:34:43.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f1506ffd640 1 -- 172.21.15.12:0/4102908479 learned_addr learned my addr 172.21.15.12:0/4102908479 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:43.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15077fe640 1 --2- 172.21.15.12:0/4102908479 >> v2:172.21.15.144:3300/0 conn(0x7f1508153fb0 0x7f15080ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f1507fff640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15080b41e0 0x7f15080caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134253738 0 0) 0x7f150815d390 con 0x7f150815a980 2023-11-28T21:34:43.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f0003530 con 0x7f150815a980 2023-11-28T21:34:43.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 214050351 0 0) 0x7f14f0003530 con 0x7f150815a980 2023-11-28T21:34:43.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15080b41e0 msgr2=0x7f15080caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:43.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15080b41e0 0x7f15080caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 >> v2:172.21.15.144:3300/0 conn(0x7f1508153fb0 msgr2=0x7f15080ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:43.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 --2- 172.21.15.12:0/4102908479 >> v2:172.21.15.144:3300/0 conn(0x7f1508153fb0 0x7f15080ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150815d630 con 0x7f150815a980 2023-11-28T21:34:43.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f14fc001680 con 0x7f150815a980 2023-11-28T21:34:43.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 >> v1:172.21.15.12:6789/0 conn(0x7f150815a980 legacy=0x7f15080b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14f0003980 0x7f14f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f00047d0 0x7f14f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15080b41e0 0x7f14f0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15080b2b60 con 0x7f14f0003980 2023-11-28T21:34:43.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150815e1b0 con 0x7f14f00047d0 2023-11-28T21:34:43.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f000b990 con 0x7f15080b41e0 2023-11-28T21:34:43.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15077fe640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14f0003980 0x7f14f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f1506ffd640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f00047d0 0x7f14f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f1507fff640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15080b41e0 0x7f14f0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15077fe640 1 -- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15080b41e0 msgr2=0x7f14f0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:43.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15077fe640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15080b41e0 0x7f14f0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15077fe640 1 -- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f00047d0 msgr2=0x7f14f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:43.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15077fe640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14f00047d0 0x7f14f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.608+0000 7f15077fe640 1 -- 172.21.15.12:0/4102908479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0003530 con 0x7f14f0003980 2023-11-28T21:34:43.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f15077fe640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14f0003980 0x7f14f0003da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009ae0 tx=0x7f14f801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30791e38fd7a9c7f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:43.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f80a6470 con 0x7f14f0003980 2023-11-28T21:34:43.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f80a65d0 con 0x7f14f0003980 2023-11-28T21:34:43.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f15067fc640 1 -- 172.21.15.12:0/4102908479 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f80a6980 con 0x7f14f0003980 2023-11-28T21:34:43.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f150c93b640 1 -- 172.21.15.12:0/4102908479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14f0003980 msgr2=0x7f14f0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f150c93b640 1 --2- 172.21.15.12:0/4102908479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14f0003980 0x7f14f0003da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009ae0 tx=0x7f14f801ca00 comp rx=0 tx=0).stop 2023-11-28T21:34:43.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f1507fff640 1 -- 172.21.15.12:0/4102908479 reap_dead start 2023-11-28T21:34:43.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f150c93b640 1 -- 172.21.15.12:0/4102908479 shutdown_connections 2023-11-28T21:34:43.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f150c93b640 1 -- 172.21.15.12:0/4102908479 >> 172.21.15.12:0/4102908479 conn(0x7f1508071ce0 msgr2=0x7f15080720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:43.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f150c93b640 1 -- 172.21.15.12:0/4102908479 shutdown_connections 2023-11-28T21:34:43.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.612+0000 7f150c93b640 1 -- 172.21.15.12:0/4102908479 wait complete. 2023-11-28T21:34:43.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f150c93b640 1 Processor -- start 2023-11-28T21:34:43.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f150c93b640 1 -- start start 2023-11-28T21:34:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f150c93b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14f0003980 0x7f15081fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f150c93b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15080cd860 0x7f1508159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f150c93b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1508147ff0 0x7f15080c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f150c93b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081fb290 con 0x7f15080cd860 2023-11-28T21:34:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f150c93b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15080c78e0 con 0x7f1508147ff0 2023-11-28T21:34:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f150c93b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1508159540 con 0x7f14f0003980 2023-11-28T21:34:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1506ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15080cd860 0x7f1508159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1506ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15080cd860 0x7f1508159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57062/0 (socket says 172.21.15.12:57062) 2023-11-28T21:34:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1506ffd640 1 -- 172.21.15.12:0/357955770 learned_addr learned my addr 172.21.15.12:0/357955770 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f15077fe640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14f0003980 0x7f15081fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1507fff640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1508147ff0 0x7f15080c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1506ffd640 1 -- 172.21.15.12:0/357955770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14f0003980 msgr2=0x7f15081fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1506ffd640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14f0003980 0x7f15081fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1506ffd640 1 -- 172.21.15.12:0/357955770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1508147ff0 msgr2=0x7f15080c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1506ffd640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1508147ff0 0x7f15080c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1506ffd640 1 -- 172.21.15.12:0/357955770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15081faf20 con 0x7f15080cd860 2023-11-28T21:34:43.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1507fff640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1508147ff0 0x7f15080c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:43.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f1506ffd640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15080cd860 0x7f1508159820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00e780 tx=0x7f14fc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:43.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f14effff640 1 -- 172.21.15.12:0/357955770 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc00c6c0 con 0x7f15080cd860 2023-11-28T21:34:43.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f14effff640 1 -- 172.21.15.12:0/357955770 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc00c820 con 0x7f15080cd860 2023-11-28T21:34:43.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f150c93b640 1 -- 172.21.15.12:0/357955770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15081fbc60 con 0x7f15080cd860 2023-11-28T21:34:43.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f14effff640 1 -- 172.21.15.12:0/357955770 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc00cbd0 con 0x7f15080cd860 2023-11-28T21:34:43.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.616+0000 7f150c93b640 1 -- 172.21.15.12:0/357955770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1508159d60 con 0x7f15080cd860 2023-11-28T21:34:43.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.620+0000 7f150c93b640 1 -- 172.21.15.12:0/357955770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15081fb0b0 con 0x7f15080cd860 2023-11-28T21:34:43.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.620+0000 7f14effff640 1 -- 172.21.15.12:0/357955770 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f14fc00cd30 con 0x7f15080cd860 2023-11-28T21:34:43.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.620+0000 7f14effff640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f14f4066470 0x7f14f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.620+0000 7f14effff640 1 -- 172.21.15.12:0/357955770 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f14fc084070 con 0x7f15080cd860 2023-11-28T21:34:43.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.620+0000 7f15077fe640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f14f4066470 0x7f14f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.620+0000 7f15077fe640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f14f4066470 0x7f14f4068930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14f80ad000 tx=0x7f14f80b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:43.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.624+0000 7f14effff640 1 -- 172.21.15.12:0/357955770 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f14fc0b5280 con 0x7f15080cd860 2023-11-28T21:34:43.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.812+0000 7f150c93b640 1 -- 172.21.15.12:0/357955770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1508159680 con 0x7f15080cd860 2023-11-28T21:34:43.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f14effff640 1 -- 172.21.15.12:0/357955770 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+6495 (secure 0 0 0) 0x7f1508159680 con 0x7f15080cd860 2023-11-28T21:34:43.817 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:43.817 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:34:42.347765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T21:34:43.817 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 6 2023-11-28T21:34:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 -- 172.21.15.12:0/357955770 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f14f4066470 msgr2=0x7f14f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f14f4066470 0x7f14f4068930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14f80ad000 tx=0x7f14f80b3040 comp rx=0 tx=0).stop 2023-11-28T21:34:43.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 -- 172.21.15.12:0/357955770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15080cd860 msgr2=0x7f1508159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15080cd860 0x7f1508159820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00e780 tx=0x7f14fc00ec40 comp rx=0 tx=0).stop 2023-11-28T21:34:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 -- 172.21.15.12:0/357955770 shutdown_connections 2023-11-28T21:34:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f14f4066470 0x7f14f4068930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1508147ff0 0x7f15080c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15080cd860 0x7f1508159820 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 --2- 172.21.15.12:0/357955770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14f0003980 0x7f15081fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 -- 172.21.15.12:0/357955770 >> 172.21.15.12:0/357955770 conn(0x7f1508071ce0 msgr2=0x7f150814e260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 -- 172.21.15.12:0/357955770 shutdown_connections 2023-11-28T21:34:43.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.816+0000 7f150c93b640 1 -- 172.21.15.12:0/357955770 wait complete. 2023-11-28T21:34:43.838 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:34:43.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.976+0000 7f2ef68fe640 1 Processor -- start 2023-11-28T21:34:43.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2ef68fe640 1 -- start start 2023-11-28T21:34:43.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2ef68fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef0151af0 0x7f2ef00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2ef68fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2ef0151ed0 0x7f2ef00c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2ef68fe640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef0157b70 con 0x7f2ef006e300 2023-11-28T21:34:43.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2ef68fe640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef00707f0 con 0x7f2ef0151ed0 2023-11-28T21:34:43.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2ef68fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00c7b90 con 0x7f2ef0151af0 2023-11-28T21:34:43.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2ef4e9b640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2ef006e300 0x7f2ef0072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54868/0 (socket says 172.21.15.12:54868) 2023-11-28T21:34:43.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2ef4e9b640 1 -- 172.21.15.12:0/822335218 learned_addr learned my addr 172.21.15.12:0/822335218 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:43.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeffff640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef0151af0 0x7f2ef00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eef7fe640 1 --2- 172.21.15.12:0/822335218 >> v2:172.21.15.144:3300/0 conn(0x7f2ef0151ed0 0x7f2ef00c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969969940 0 0) 0x7f2ef0157b70 con 0x7f2ef006e300 2023-11-28T21:34:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed8003530 con 0x7f2ef006e300 2023-11-28T21:34:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1855394772 0 0) 0x7f2ed8003530 con 0x7f2ef006e300 2023-11-28T21:34:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef0151af0 msgr2=0x7f2ef00c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef0151af0 0x7f2ef00c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 >> v2:172.21.15.144:3300/0 conn(0x7f2ef0151ed0 msgr2=0x7f2ef00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 --2- 172.21.15.12:0/822335218 >> v2:172.21.15.144:3300/0 conn(0x7f2ef0151ed0 0x7f2ef00c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef0157e50 con 0x7f2ef006e300 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f2ee8001a00 con 0x7f2ef006e300 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 >> v1:172.21.15.12:6789/0 conn(0x7f2ef006e300 legacy=0x7f2ef0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ed80039f0 0x7f2ed8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 0x7f2ed8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef0151af0 0x7f2ed8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00707f0 con 0x7f2ed80047d0 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00c7b90 con 0x7f2ed80039f0 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed800b9c0 con 0x7f2ef0151af0 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eef7fe640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 0x7f2ed8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeffff640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ed80039f0 0x7f2ed8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2ef4e9b640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef0151af0 0x7f2ed8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eef7fe640 1 -- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef0151af0 msgr2=0x7f2ed8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eef7fe640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef0151af0 0x7f2ed8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eef7fe640 1 -- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ed80039f0 msgr2=0x7f2ed8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eef7fe640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ed80039f0 0x7f2ed8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eef7fe640 1 -- 172.21.15.12:0/822335218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed8003530 con 0x7f2ed80047d0 2023-11-28T21:34:43.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.980+0000 7f2eeffff640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ed80039f0 0x7f2ed8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2eef7fe640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 0x7f2ed8004c50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009ab0 tx=0x7f2ee401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d0c32cca3ef5c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40a5550 con 0x7f2ed80047d0 2023-11-28T21:34:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee40a56b0 con 0x7f2ed80047d0 2023-11-28T21:34:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2eeeffd640 1 -- 172.21.15.12:0/822335218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40a5a20 con 0x7f2ed80047d0 2023-11-28T21:34:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2ef68fe640 1 -- 172.21.15.12:0/822335218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 msgr2=0x7f2ed8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2ef68fe640 1 --2- 172.21.15.12:0/822335218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 0x7f2ed8004c50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009ab0 tx=0x7f2ee401ca40 comp rx=0 tx=0).stop 2023-11-28T21:34:43.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2ef4e9b640 1 -- 172.21.15.12:0/822335218 reap_dead start 2023-11-28T21:34:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2ef68fe640 1 -- 172.21.15.12:0/822335218 shutdown_connections 2023-11-28T21:34:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2ef68fe640 1 -- 172.21.15.12:0/822335218 >> 172.21.15.12:0/822335218 conn(0x7f2ef0066360 msgr2=0x7f2ef0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2ef68fe640 1 -- 172.21.15.12:0/822335218 shutdown_connections 2023-11-28T21:34:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.984+0000 7f2ef68fe640 1 -- 172.21.15.12:0/822335218 wait complete. 2023-11-28T21:34:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef68fe640 1 Processor -- start 2023-11-28T21:34:43.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef68fe640 1 -- start start 2023-11-28T21:34:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef68fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 0x7f2ef00c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef68fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef00c9cb0 0x7f2ef014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef68fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ef00ca110 0x7f2ef014ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef68fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef01560b0 con 0x7f2ed80047d0 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef68fe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef0156bf0 con 0x7f2ef00ca110 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef68fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef014b070 con 0x7f2ef00c9cb0 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eeffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 0x7f2ef00c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eeffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 0x7f2ef00c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57078/0 (socket says 172.21.15.12:57078) 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eeffff640 1 -- 172.21.15.12:0/1655148066 learned_addr learned my addr 172.21.15.12:0/1655148066 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eef7fe640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef00c9cb0 0x7f2ef014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef4e9b640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ef00ca110 0x7f2ef014ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eeffff640 1 -- 172.21.15.12:0/1655148066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef00c9cb0 msgr2=0x7f2ef014a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eeffff640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef00c9cb0 0x7f2ef014a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eeffff640 1 -- 172.21.15.12:0/1655148066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ef00ca110 msgr2=0x7f2ef014ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eeffff640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ef00ca110 0x7f2ef014ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eeffff640 1 -- 172.21.15.12:0/1655148066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef014b850 con 0x7f2ed80047d0 2023-11-28T21:34:43.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eeffff640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 0x7f2ef00c95e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009fd0 tx=0x7f2ee0011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eecff9640 1 -- 172.21.15.12:0/1655148066 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0004220 con 0x7f2ed80047d0 2023-11-28T21:34:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eecff9640 1 -- 172.21.15.12:0/1655148066 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee0004380 con 0x7f2ed80047d0 2023-11-28T21:34:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef68fe640 1 -- 172.21.15.12:0/1655148066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef01562c0 con 0x7f2ed80047d0 2023-11-28T21:34:43.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2eecff9640 1 -- 172.21.15.12:0/1655148066 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000f9e0 con 0x7f2ed80047d0 2023-11-28T21:34:43.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.988+0000 7f2ef68fe640 1 -- 172.21.15.12:0/1655148066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef0156880 con 0x7f2ed80047d0 2023-11-28T21:34:43.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.992+0000 7f2ef68fe640 1 -- 172.21.15.12:0/1655148066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef0074f60 con 0x7f2ed80047d0 2023-11-28T21:34:43.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.992+0000 7f2eecff9640 1 -- 172.21.15.12:0/1655148066 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2ee000fb40 con 0x7f2ed80047d0 2023-11-28T21:34:43.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.992+0000 7f2eecff9640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2edc066470 0x7f2edc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:43.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.992+0000 7f2eecff9640 1 -- 172.21.15.12:0/1655148066 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2ee00847c0 con 0x7f2ed80047d0 2023-11-28T21:34:43.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.992+0000 7f2eef7fe640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2edc066470 0x7f2edc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:43.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.992+0000 7f2eef7fe640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2edc066470 0x7f2edc068930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40af000 tx=0x7f2ee40046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:43.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:43.996+0000 7f2eecff9640 1 -- 172.21.15.12:0/1655148066 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2ee00518b0 con 0x7f2ed80047d0 2023-11-28T21:34:44.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.156+0000 7f2ef68fe640 1 -- 172.21.15.12:0/1655148066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ef0072c10 con 0x7f2ed80047d0 2023-11-28T21:34:44.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.156+0000 7f2eecff9640 1 -- 172.21.15.12:0/1655148066 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7f2ee0054ed0 con 0x7f2ed80047d0 2023-11-28T21:34:44.161 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:44.161 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:34:44.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 -- 172.21.15.12:0/1655148066 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2edc066470 msgr2=0x7f2edc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:44.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2edc066470 0x7f2edc068930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40af000 tx=0x7f2ee40046d0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 -- 172.21.15.12:0/1655148066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 msgr2=0x7f2ef00c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:44.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 0x7f2ef00c95e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009fd0 tx=0x7f2ee0011b30 comp rx=0 tx=0).stop 2023-11-28T21:34:44.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 -- 172.21.15.12:0/1655148066 shutdown_connections 2023-11-28T21:34:44.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2edc066470 0x7f2edc068930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ef00ca110 0x7f2ef014ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ef00c9cb0 0x7f2ef014a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 --2- 172.21.15.12:0/1655148066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ed80047d0 0x7f2ef00c95e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 -- 172.21.15.12:0/1655148066 >> 172.21.15.12:0/1655148066 conn(0x7f2ef0066360 msgr2=0x7f2ef006d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:44.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 -- 172.21.15.12:0/1655148066 shutdown_connections 2023-11-28T21:34:44.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.160+0000 7f2ef68fe640 1 -- 172.21.15.12:0/1655148066 wait complete. 2023-11-28T21:34:44.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T21:34:36.378416+0000', 'modified': '2023-11-28T21:34:42.347765+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4261}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4261': {'gid': 4261, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.144:6833/3053317078', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3053317078}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3053317078}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-28T21:34:44.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T21:34:44.787 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-11-28T21:34:44.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23c085a640 1 Processor -- start 2023-11-28T21:34:44.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23c085a640 1 -- start start 2023-11-28T21:34:44.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23c085a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f23bc06e300 0x7f23bc156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23c085a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23bc067410 con 0x7f23bc06cf40 2023-11-28T21:34:44.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23c085a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23bc06e090 con 0x7f23bc150c20 2023-11-28T21:34:44.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23c085a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23bc157f30 con 0x7f23bc06e300 2023-11-28T21:34:44.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23b9d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f23bc06e300 0x7f23bc156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23b9d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f23bc06e300 0x7f23bc156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53476/0 (socket says 172.21.15.12:53476) 2023-11-28T21:34:44.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23b9d9b640 1 -- 172.21.15.12:0/843106140 learned_addr learned my addr 172.21.15.12:0/843106140 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:44.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23bad9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f23bc06cf40 0x7f23bc158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43380/0 (socket says 172.21.15.12:43380) 2023-11-28T21:34:44.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23b9d9b640 1 -- 172.21.15.12:0/843106140 >> v1:172.21.15.144:6789/0 conn(0x7f23bc150c20 legacy=0x7f23bc0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:44.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23b9d9b640 1 -- 172.21.15.12:0/843106140 >> v1:172.21.15.12:6789/0 conn(0x7f23bc06cf40 legacy=0x7f23bc158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:44.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23b9d9b640 1 -- 172.21.15.12:0/843106140 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc151000 con 0x7f23bc06e300 2023-11-28T21:34:44.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.940+0000 7f23b959a640 1 -- 172.21.15.12:0/843106140 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165977650 0 0) 0x7f23bc067410 con 0x7f23bc06cf40 2023-11-28T21:34:44.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23b9d9b640 1 --2- 172.21.15.12:0/843106140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 0x7f23bc156910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009870 tx=0x7f23b001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad9c31b71cd4be5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:44.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23b959a640 1 -- 172.21.15.12:0/843106140 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00ab650 con 0x7f23bc06e300 2023-11-28T21:34:44.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23b959a640 1 -- 172.21.15.12:0/843106140 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b00ab7b0 con 0x7f23bc06e300 2023-11-28T21:34:44.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23b959a640 1 -- 172.21.15.12:0/843106140 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00abb40 con 0x7f23bc06e300 2023-11-28T21:34:44.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23c085a640 1 -- 172.21.15.12:0/843106140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 msgr2=0x7f23bc156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:44.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23c085a640 1 --2- 172.21.15.12:0/843106140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 0x7f23bc156910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009870 tx=0x7f23b001c540 comp rx=0 tx=0).stop 2023-11-28T21:34:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23c085a640 1 -- 172.21.15.12:0/843106140 shutdown_connections 2023-11-28T21:34:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23c085a640 1 --2- 172.21.15.12:0/843106140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 0x7f23bc156910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23c085a640 1 -- 172.21.15.12:0/843106140 >> 172.21.15.12:0/843106140 conn(0x7f23bc066360 msgr2=0x7f23bc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23c085a640 1 -- 172.21.15.12:0/843106140 shutdown_connections 2023-11-28T21:34:44.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.944+0000 7f23c085a640 1 -- 172.21.15.12:0/843106140 wait complete. 2023-11-28T21:34:44.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23c085a640 1 Processor -- start 2023-11-28T21:34:44.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23c085a640 1 -- start start 2023-11-28T21:34:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23c085a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23bc06cf40 0x7f23bc1f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23c085a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 0x7f23bc1f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23c085a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23bc150c20 0x7f23bc0c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23c085a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0c3190 con 0x7f23bc06e300 2023-11-28T21:34:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23c085a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1f3200 con 0x7f23bc06cf40 2023-11-28T21:34:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23c085a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0c5430 con 0x7f23bc150c20 2023-11-28T21:34:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23b9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 0x7f23bc1f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23b9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 0x7f23bc1f2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53488/0 (socket says 172.21.15.12:53488) 2023-11-28T21:34:44.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23b9d9b640 1 -- 172.21.15.12:0/55144941 learned_addr learned my addr 172.21.15.12:0/55144941 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23ba59c640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23bc06cf40 0x7f23bc1f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23bad9d640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23bc150c20 0x7f23bc0c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23b9d9b640 1 -- 172.21.15.12:0/55144941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23bc150c20 msgr2=0x7f23bc0c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23b9d9b640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23bc150c20 0x7f23bc0c4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23b9d9b640 1 -- 172.21.15.12:0/55144941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23bc06cf40 msgr2=0x7f23bc1f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23b9d9b640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23bc06cf40 0x7f23bc1f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23b9d9b640 1 -- 172.21.15.12:0/55144941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b00094d0 con 0x7f23bc06e300 2023-11-28T21:34:44.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23b9d9b640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 0x7f23bc1f2cc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f23b001a170 tx=0x7f23b001ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:44.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23aaffd640 1 -- 172.21.15.12:0/55144941 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00b4800 con 0x7f23bc06e300 2023-11-28T21:34:44.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23aaffd640 1 -- 172.21.15.12:0/55144941 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b00b4980 con 0x7f23bc06e300 2023-11-28T21:34:44.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23aaffd640 1 -- 172.21.15.12:0/55144941 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00b4d30 con 0x7f23bc06e300 2023-11-28T21:34:44.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23c085a640 1 -- 172.21.15.12:0/55144941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc0c4a90 con 0x7f23bc06e300 2023-11-28T21:34:44.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.948+0000 7f23c085a640 1 -- 172.21.15.12:0/55144941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc0c47b0 con 0x7f23bc06e300 2023-11-28T21:34:44.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.952+0000 7f23aaffd640 1 -- 172.21.15.12:0/55144941 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f23b00b3380 con 0x7f23bc06e300 2023-11-28T21:34:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.952+0000 7f23aaffd640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f239c066470 0x7f239c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.952+0000 7f23aaffd640 1 -- 172.21.15.12:0/55144941 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f23b011e9e0 con 0x7f23bc06e300 2023-11-28T21:34:44.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.952+0000 7f23ba59c640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f239c066470 0x7f239c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.952+0000 7f23c3bcf000 1 Processor -- start 2023-11-28T21:34:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.952+0000 7f23ba59c640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f239c066470 0x7f239c068930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f23ac007fd0 tx=0x7f23ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:44.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.952+0000 7f23c3bcf000 1 -- start start 2023-11-28T21:34:44.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 0x55d7384bbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7382ea130 con 0x55d7384c1310 2023-11-28T21:34:44.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7382d79a0 con 0x55d7384c09f0 2023-11-28T21:34:44.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7382d4ed0 con 0x55d7384bb930 2023-11-28T21:34:44.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23bad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23bad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384c16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53496/0 (socket says 172.21.15.12:53496) 2023-11-28T21:34:44.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23bad9d640 1 -- 172.21.15.12:0/1645893205 learned_addr learned my addr 172.21.15.12:0/1645893205 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:44.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23ba59c640 1 --2- 172.21.15.12:0/1645893205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 0x55d7384bbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23b9d9b640 1 --2- 172.21.15.12:0/1645893205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23bad9d640 1 -- 172.21.15.12:0/1645893205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 msgr2=0x55d7384bbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:44.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23bad9d640 1 --2- 172.21.15.12:0/1645893205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 0x55d7384bbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23bad9d640 1 -- 172.21.15.12:0/1645893205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 msgr2=0x55d7384c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:44.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23bad9d640 1 --2- 172.21.15.12:0/1645893205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23bad9d640 1 -- 172.21.15.12:0/1645893205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d737fc8890 con 0x55d7384c1310 2023-11-28T21:34:44.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23bad9d640 1 --2- 172.21.15.12:0/1645893205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384c16f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f23b4007330 tx=0x7f23b40077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b236f76fa4a8813 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:44.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f239bfff640 1 -- 172.21.15.12:0/1645893205 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4004d80 con 0x55d7384c1310 2023-11-28T21:34:44.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f239bfff640 1 -- 172.21.15.12:0/1645893205 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b4001c20 con 0x55d7384c1310 2023-11-28T21:34:44.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f239bfff640 1 -- 172.21.15.12:0/1645893205 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4001fd0 con 0x55d7384c1310 2023-11-28T21:34:44.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 -- 172.21.15.12:0/1645893205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 msgr2=0x55d7384c16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:44.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/1645893205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384c16f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f23b4007330 tx=0x7f23b40077f0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 -- 172.21.15.12:0/1645893205 shutdown_connections 2023-11-28T21:34:44.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/1645893205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384c16f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/1645893205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/1645893205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 0x55d7384bbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 -- 172.21.15.12:0/1645893205 >> 172.21.15.12:0/1645893205 conn(0x55d7384bc7a0 msgr2=0x55d7384beb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:44.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 -- 172.21.15.12:0/1645893205 shutdown_connections 2023-11-28T21:34:44.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 -- 172.21.15.12:0/1645893205 wait complete. 2023-11-28T21:34:44.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 -- start start 2023-11-28T21:34:44.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.956+0000 7f23c3bcf000 1 Processor -- start 2023-11-28T21:34:44.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 0x55d7384bad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384bb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7382d07c0 con 0x55d7384c1310 2023-11-28T21:34:44.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d738288bf0 con 0x55d7384c09f0 2023-11-28T21:34:44.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d73828ea90 con 0x55d7384bb930 2023-11-28T21:34:44.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 10 client.0 _ll_register_callbacks cb 0x55d73846b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T21:34:44.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23bad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23bad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53510/0 (socket says 172.21.15.12:53510) 2023-11-28T21:34:44.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23bad9d640 1 -- 172.21.15.12:0/2471715745 learned_addr learned my addr 172.21.15.12:0/2471715745 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:44.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23ba59c640 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 0x55d7384bad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23b9d9b640 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384bb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23bad9d640 1 -- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 msgr2=0x55d7384bad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:44.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23bad9d640 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 0x55d7384bad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23bad9d640 1 -- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 msgr2=0x55d7384bb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:44.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23bad9d640 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384bb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:44.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23bad9d640 1 -- 172.21.15.12:0/2471715745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b400f040 con 0x55d7384c1310 2023-11-28T21:34:44.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23ba59c640 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 0x55d7384bad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:44.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23bad9d640 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384ca990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55d738349330 tx=0x7f23b4001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457477e8ad07f08c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:44.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23997fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-11-28T21:34:44.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23997fa640 1 -- 172.21.15.12:0/2471715745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4002070 con 0x55d7384c1310 2023-11-28T21:34:44.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23997fa640 1 -- 172.21.15.12:0/2471715745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f23b4014460 con 0x55d7384c1310 2023-11-28T21:34:44.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 1 -- 172.21.15.12:0/2471715745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d738292470 con 0x55d7384c1310 2023-11-28T21:34:44.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23997fa640 1 -- 172.21.15.12:0/2471715745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b4014620 con 0x55d7384c1310 2023-11-28T21:34:44.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 10 client.4389 fetch_fsmap learned FSMap version 6 2023-11-28T21:34:44.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 1 -- 172.21.15.12:0/2471715745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d738291190 con 0x55d7384c1310 2023-11-28T21:34:44.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23997fa640 1 -- 172.21.15.12:0/2471715745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d738292470 con 0x55d7384c1310 2023-11-28T21:34:44.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23997fa640 1 -- 172.21.15.12:0/2471715745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 6) v1 ==== 4452+0+0 (secure 0 0 0) 0x7f23b4014780 con 0x55d7384c1310 2023-11-28T21:34:44.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 10 client.4389 fetch_fsmap finished waiting for FSMap version 6 2023-11-28T21:34:44.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 10 client.4389 resolve_mds: resolved name 'f' to daemon mds.f 2023-11-28T21:34:44.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 20 client.4389 populate_metadata read hostname 'smithi012' 2023-11-28T21:34:44.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384cb640 0x55d7384cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:44.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 4 client.4389 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T21:34:44.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23c3bcf000 1 -- 172.21.15.12:0/2471715745 --> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d738295b50 con 0x55d7384cb640 2023-11-28T21:34:44.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23ba59c640 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384cb640 0x55d7384cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:44.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23ba59c640 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384cb640 0x55d7384cba20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6063c81d6c405a7f server_cookie=5e40fe5100ee5db1 in_seq=0 out_seq=0 2023-11-28T21:34:44.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:44.960+0000 7f23997fa640 10 client.4389 ms_handle_connect on v2:172.21.15.144:6832/3053317078 2023-11-28T21:34:47.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23997fa640 1 -- 172.21.15.12:0/2471715745 <== mds.0 v2:172.21.15.144:6832/3053317078 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d738295b50 con 0x55d7384cb640 2023-11-28T21:34:47.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23997fa640 10 client.4389 handle_command_reply: tid=0 2023-11-28T21:34:47.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 client.4389 shutdown 2023-11-28T21:34:47.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 -- 172.21.15.12:0/2471715745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 msgr2=0x55d7384ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:47.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384ca990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55d738349330 tx=0x7f23b4001260 comp rx=0 tx=0).stop 2023-11-28T21:34:47.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 -- 172.21.15.12:0/2471715745 shutdown_connections 2023-11-28T21:34:47.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384cb640 0x55d7384cba20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c1310 0x55d7384ca990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/2471715745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384bb930 0x55d7384bad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23997fa640 0 client.4389 ms_handle_reset on v2:172.21.15.144:6832/3053317078 2023-11-28T21:34:47.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 -- 172.21.15.12:0/2471715745 >> 172.21.15.12:0/2471715745 conn(0x55d7384bc7a0 msgr2=0x55d738485ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:47.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 -- 172.21.15.12:0/2471715745 shutdown_connections 2023-11-28T21:34:47.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 1 -- 172.21.15.12:0/2471715745 wait complete. 2023-11-28T21:34:47.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.368+0000 7f23c3bcf000 20 client.4389 trim_cache size 0 max 16384 2023-11-28T21:34:47.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.384+0000 7f23c3bcf000 1 Processor -- start 2023-11-28T21:34:47.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.384+0000 7f23c3bcf000 1 -- start start 2023-11-28T21:34:47.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384bb930 0x55d7384c9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384c1310 0x55d738483ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d737f89940 con 0x55d7384bb930 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7382cc130 con 0x55d7384c09f0 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7382aae30 con 0x55d7384c1310 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23ba59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384bb930 0x55d7384c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23ba59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384bb930 0x55d7384c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53514/0 (socket says 172.21.15.12:53514) 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23ba59c640 1 -- 172.21.15.12:0/1162178972 learned_addr learned my addr 172.21.15.12:0/1162178972 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23b9d9b640 1 --2- 172.21.15.12:0/1162178972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23bad9d640 1 --2- 172.21.15.12:0/1162178972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384c1310 0x55d738483ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23ba59c640 1 -- 172.21.15.12:0/1162178972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384c1310 msgr2=0x55d738483ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23ba59c640 1 --2- 172.21.15.12:0/1162178972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384c1310 0x55d738483ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23ba59c640 1 -- 172.21.15.12:0/1162178972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 msgr2=0x55d7384c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:47.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23ba59c640 1 --2- 172.21.15.12:0/1162178972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23ba59c640 1 -- 172.21.15.12:0/1162178972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d738277490 con 0x55d7384bb930 2023-11-28T21:34:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23ba59c640 1 --2- 172.21.15.12:0/1162178972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384bb930 0x55d7384c9730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23ac007a00 tx=0x7f23ac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3728cb35d4bda1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23a8ff9640 1 -- 172.21.15.12:0/1162178972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac027ea0 con 0x55d7384bb930 2023-11-28T21:34:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23a8ff9640 1 -- 172.21.15.12:0/1162178972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac0148d0 con 0x55d7384bb930 2023-11-28T21:34:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23a8ff9640 1 -- 172.21.15.12:0/1162178972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac014a40 con 0x55d7384bb930 2023-11-28T21:34:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 -- 172.21.15.12:0/1162178972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384bb930 msgr2=0x55d7384c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/1162178972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384bb930 0x55d7384c9730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23ac007a00 tx=0x7f23ac012000 comp rx=0 tx=0).stop 2023-11-28T21:34:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 -- 172.21.15.12:0/1162178972 shutdown_connections 2023-11-28T21:34:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/1162178972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384c1310 0x55d738483ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/1162178972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384c09f0 0x55d7384c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/1162178972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384bb930 0x55d7384c9730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 -- 172.21.15.12:0/1162178972 >> 172.21.15.12:0/1162178972 conn(0x55d7384bc7a0 msgr2=0x55d7384ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 -- 172.21.15.12:0/1162178972 shutdown_connections 2023-11-28T21:34:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.388+0000 7f23c3bcf000 1 -- 172.21.15.12:0/1162178972 wait complete. 2023-11-28T21:34:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 1 -- start start 2023-11-28T21:34:47.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 1 Processor -- start 2023-11-28T21:34:47.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384bb930 0x55d738483c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c09f0 0x55d7384e7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384c1310 0x55d7384e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7382cf650 con 0x55d7384c09f0 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7382d47c0 con 0x55d7384bb930 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d738357fa0 con 0x55d7384c1310 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 10 client.0 _ll_register_callbacks cb 0x55d73844bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23b9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c09f0 0x55d7384e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23b9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c09f0 0x55d7384e7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53522/0 (socket says 172.21.15.12:53522) 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23b9d9b640 1 -- 172.21.15.12:0/161365906 learned_addr learned my addr 172.21.15.12:0/161365906 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23ba59c640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384bb930 0x55d738483c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23bad9d640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384c1310 0x55d7384e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23b9d9b640 1 -- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384c1310 msgr2=0x55d7384e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23b9d9b640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384c1310 0x55d7384e7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23b9d9b640 1 -- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384bb930 msgr2=0x55d738483c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:47.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23b9d9b640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384bb930 0x55d738483c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23b9d9b640 1 -- 172.21.15.12:0/161365906 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ac019040 con 0x55d7384c09f0 2023-11-28T21:34:47.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23b9d9b640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c09f0 0x55d7384e7690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23b00ebdd0 tx=0x7f23b00192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5147bdc7386967 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:47.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23997fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-11-28T21:34:47.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23997fa640 1 -- 172.21.15.12:0/161365906 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00f7920 con 0x55d7384c09f0 2023-11-28T21:34:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23997fa640 1 -- 172.21.15.12:0/161365906 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f23b00f0b50 con 0x55d7384c09f0 2023-11-28T21:34:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 1 -- 172.21.15.12:0/161365906 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7382cf110 con 0x55d7384c09f0 2023-11-28T21:34:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23997fa640 1 -- 172.21.15.12:0/161365906 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b01071d0 con 0x55d7384c09f0 2023-11-28T21:34:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 10 client.4395 fetch_fsmap learned FSMap version 6 2023-11-28T21:34:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 1 -- 172.21.15.12:0/161365906 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d737fc8cc0 con 0x55d7384c09f0 2023-11-28T21:34:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23997fa640 1 -- 172.21.15.12:0/161365906 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d7382cf110 con 0x55d7384c09f0 2023-11-28T21:34:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23997fa640 1 -- 172.21.15.12:0/161365906 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 6) v1 ==== 4452+0+0 (secure 0 0 0) 0x7f23b00f0d10 con 0x55d7384c09f0 2023-11-28T21:34:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 10 client.4395 fetch_fsmap finished waiting for FSMap version 6 2023-11-28T21:34:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 10 client.4395 resolve_mds: resolved name 'f' to daemon mds.f 2023-11-28T21:34:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.392+0000 7f23c3bcf000 20 client.4395 populate_metadata read hostname 'smithi012' 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384e8110 0x55d738526a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 4 client.4395 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 -- 172.21.15.12:0/161365906 --> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d7382fe9a0 con 0x55d7384e8110 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23ba59c640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384e8110 0x55d738526a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23ba59c640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384e8110 0x55d738526a80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7389676cb81c7e server_cookie=484574c06ed6641 in_seq=0 out_seq=0 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23997fa640 10 client.4395 ms_handle_connect on v2:172.21.15.144:6832/3053317078 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23ba59c640 1 -- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384e8110 msgr2=0x55d738526a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23ba59c640 1 -- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384e8110 msgr2=0x55d738526a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23ba59c640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384e8110 0x55d738526a80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23ba59c640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384e8110 0x55d738526a80 unknown :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2023-11-28T21:34:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23ba59c640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384e8110 0x55d738526a80 unknown :-1 s=BANNER_CONNECTING pgs=14 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23ba59c640 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384e8110 0x55d738526a80 crc :-1 s=READY pgs=14 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7389676cb81c7e server_cookie=484574c06ed6641 in_seq=0 out_seq=1 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23997fa640 10 client.4395 ms_handle_connect on v2:172.21.15.144:6832/3053317078 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23997fa640 1 -- 172.21.15.12:0/161365906 <== mds.0 v2:172.21.15.144:6832/3053317078 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55d7382fe9a0 con 0x55d7384e8110 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23997fa640 10 client.4395 handle_command_reply: tid=0 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 client.4395 shutdown 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 -- 172.21.15.12:0/161365906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c09f0 msgr2=0x55d7384e7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c09f0 0x55d7384e7690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23b00ebdd0 tx=0x7f23b00192e0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 -- 172.21.15.12:0/161365906 shutdown_connections 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x55d7384e8110 0x55d738526a80 crc :-1 s=READY pgs=14 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d7384c1310 0x55d7384e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d7384c09f0 0x55d7384e7690 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 --2- 172.21.15.12:0/161365906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d7384bb930 0x55d738483c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23997fa640 0 client.4395 ms_handle_reset on v2:172.21.15.144:6832/3053317078 2023-11-28T21:34:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 -- 172.21.15.12:0/161365906 >> 172.21.15.12:0/161365906 conn(0x55d7384bc7a0 msgr2=0x55d7384cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 -- 172.21.15.12:0/161365906 shutdown_connections 2023-11-28T21:34:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 1 -- 172.21.15.12:0/161365906 wait complete. 2023-11-28T21:34:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.396+0000 7f23c3bcf000 20 client.4395 trim_cache size 0 max 16384 2023-11-28T21:34:47.403 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:34:47.403 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "2975d7d0-8668-45d5-addd-c7b2ef978be7", 2023-11-28T21:34:47.403 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2023-11-28T21:34:47.403 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4261, 2023-11-28T21:34:47.404 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2023-11-28T21:34:47.404 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2023-11-28T21:34:47.404 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2023-11-28T21:34:47.404 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 10.072689239000001, 2023-11-28T21:34:47.404 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 5, 2023-11-28T21:34:47.404 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 24, 2023-11-28T21:34:47.404 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 24, 2023-11-28T21:34:47.404 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 17.843037729999999 2023-11-28T21:34:47.404 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:34:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 -- 172.21.15.12:0/55144941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f239c066470 msgr2=0x7f239c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f239c066470 0x7f239c068930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f23ac007fd0 tx=0x7f23ac009040 comp rx=0 tx=0).stop 2023-11-28T21:34:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 -- 172.21.15.12:0/55144941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 msgr2=0x7f23bc1f2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 0x7f23bc1f2cc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f23b001a170 tx=0x7f23b001ad80 comp rx=0 tx=0).stop 2023-11-28T21:34:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 -- 172.21.15.12:0/55144941 shutdown_connections 2023-11-28T21:34:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f239c066470 0x7f239c068930 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23bc150c20 0x7f23bc0c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f23bc06e300 0x7f23bc1f2cc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 --2- 172.21.15.12:0/55144941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23bc06cf40 0x7f23bc1f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 -- 172.21.15.12:0/55144941 >> 172.21.15.12:0/55144941 conn(0x7f23bc066360 msgr2=0x7f23bc06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 -- 172.21.15.12:0/55144941 shutdown_connections 2023-11-28T21:34:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.400+0000 7f23c085a640 1 -- 172.21.15.12:0/55144941 wait complete. 2023-11-28T21:34:47.424 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T21:34:47.424 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T21:34:47.424 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T21:34:47.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2cb6e79640 1 Processor -- start 2023-11-28T21:34:47.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2cb6e79640 1 -- start start 2023-11-28T21:34:47.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2cb6e79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cb0150c20 0x7f2cb0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2cb6e79640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2cb006e300 0x7f2cb00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2cb6e79640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2cb006cf40 0x7f2cb00c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2cb6e79640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb0067410 con 0x7f2cb006e300 2023-11-28T21:34:47.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2cb6e79640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb006e090 con 0x7f2cb006cf40 2023-11-28T21:34:47.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2cb6e79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb0157f30 con 0x7f2cb0150c20 2023-11-28T21:34:47.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2caffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2cb006e300 0x7f2cb00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2caffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2cb006e300 0x7f2cb00c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53532/0 (socket says 172.21.15.12:53532) 2023-11-28T21:34:47.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2caffff640 1 -- 172.21.15.12:0/47094074 learned_addr learned my addr 172.21.15.12:0/47094074 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:47.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.568+0000 7f2cb4c15640 1 --2- 172.21.15.12:0/47094074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cb0150c20 0x7f2cb0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2cb5416640 1 --2- 172.21.15.12:0/47094074 >> v2:172.21.15.144:3300/0 conn(0x7f2cb006cf40 0x7f2cb00c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2caffff640 1 -- 172.21.15.12:0/47094074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cb0150c20 msgr2=0x7f2cb0156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:47.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2caffff640 1 --2- 172.21.15.12:0/47094074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cb0150c20 0x7f2cb0156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2caffff640 1 -- 172.21.15.12:0/47094074 >> v2:172.21.15.144:3300/0 conn(0x7f2cb006cf40 msgr2=0x7f2cb00c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:47.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2caffff640 1 --2- 172.21.15.12:0/47094074 >> v2:172.21.15.144:3300/0 conn(0x7f2cb006cf40 0x7f2cb00c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2caffff640 1 -- 172.21.15.12:0/47094074 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb0151000 con 0x7f2cb006e300 2023-11-28T21:34:47.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2cb5416640 1 --2- 172.21.15.12:0/47094074 >> v2:172.21.15.144:3300/0 conn(0x7f2cb006cf40 0x7f2cb00c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:47.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2caffff640 1 --2- 172.21.15.12:0/47094074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 0x7f2cb00c6dd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00c7d70 tx=0x7f2c9c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22adef4007b58f5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:47.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2caf7fe640 1 -- 172.21.15.12:0/47094074 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0ac630 con 0x7f2cb006e300 2023-11-28T21:34:47.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2caf7fe640 1 -- 172.21.15.12:0/47094074 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c0ac790 con 0x7f2cb006e300 2023-11-28T21:34:47.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2caf7fe640 1 -- 172.21.15.12:0/47094074 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0acb20 con 0x7f2cb006e300 2023-11-28T21:34:47.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.572+0000 7f2cb6e79640 1 -- 172.21.15.12:0/47094074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 msgr2=0x7f2cb00c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:47.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 --2- 172.21.15.12:0/47094074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 0x7f2cb00c6dd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00c7d70 tx=0x7f2c9c01c500 comp rx=0 tx=0).stop 2023-11-28T21:34:47.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 -- 172.21.15.12:0/47094074 shutdown_connections 2023-11-28T21:34:47.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 --2- 172.21.15.12:0/47094074 >> v2:172.21.15.144:3300/0 conn(0x7f2cb006cf40 0x7f2cb00c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 --2- 172.21.15.12:0/47094074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 0x7f2cb00c6dd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 --2- 172.21.15.12:0/47094074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cb0150c20 0x7f2cb0156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 -- 172.21.15.12:0/47094074 >> 172.21.15.12:0/47094074 conn(0x7f2cb0066360 msgr2=0x7f2cb0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:47.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 -- 172.21.15.12:0/47094074 shutdown_connections 2023-11-28T21:34:47.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 -- 172.21.15.12:0/47094074 wait complete. 2023-11-28T21:34:47.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 Processor -- start 2023-11-28T21:34:47.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 -- start start 2023-11-28T21:34:47.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cb006cf40 0x7f2cb015a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 0x7f2cb01f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb0150c20 0x7f2cb01f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb01f3030 con 0x7f2cb006e300 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb015f9e0 con 0x7f2cb0150c20 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb6e79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb01f62b0 con 0x7f2cb006cf40 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2caffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 0x7f2cb01f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2caffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 0x7f2cb01f34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53536/0 (socket says 172.21.15.12:53536) 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2caffff640 1 -- 172.21.15.12:0/2486354085 learned_addr learned my addr 172.21.15.12:0/2486354085 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb4c15640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cb006cf40 0x7f2cb015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2cb5416640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb0150c20 0x7f2cb01f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2caffff640 1 -- 172.21.15.12:0/2486354085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cb006cf40 msgr2=0x7f2cb015a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2caffff640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cb006cf40 0x7f2cb015a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2caffff640 1 -- 172.21.15.12:0/2486354085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb0150c20 msgr2=0x7f2cb01f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:47.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2caffff640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb0150c20 0x7f2cb01f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2caffff640 1 -- 172.21.15.12:0/2486354085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0094d0 con 0x7f2cb006e300 2023-11-28T21:34:47.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.576+0000 7f2caffff640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 0x7f2cb01f34c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0af040 tx=0x7f2c9c0192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:47.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cad7fa640 1 -- 172.21.15.12:0/2486354085 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0b5640 con 0x7f2cb006e300 2023-11-28T21:34:47.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cad7fa640 1 -- 172.21.15.12:0/2486354085 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c0b57a0 con 0x7f2cb006e300 2023-11-28T21:34:47.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cb6e79640 1 -- 172.21.15.12:0/2486354085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb015f440 con 0x7f2cb006e300 2023-11-28T21:34:47.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cad7fa640 1 -- 172.21.15.12:0/2486354085 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0b5b10 con 0x7f2cb006e300 2023-11-28T21:34:47.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cb6e79640 1 -- 172.21.15.12:0/2486354085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb015f700 con 0x7f2cb006e300 2023-11-28T21:34:47.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cb6e79640 1 -- 172.21.15.12:0/2486354085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb01f31a0 con 0x7f2cb006e300 2023-11-28T21:34:47.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cad7fa640 1 -- 172.21.15.12:0/2486354085 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2c9c0b3070 con 0x7f2cb006e300 2023-11-28T21:34:47.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cad7fa640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2c940664a0 0x7f2c94068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:47.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cad7fa640 1 -- 172.21.15.12:0/2486354085 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2c9c11e210 con 0x7f2cb006e300 2023-11-28T21:34:47.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cb4c15640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2c940664a0 0x7f2c94068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:47.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.580+0000 7f2cb4c15640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2c940664a0 0x7f2c94068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400b930 tx=0x7f2ca4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:47.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.584+0000 7f2cad7fa640 1 -- 172.21.15.12:0/2486354085 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c9c0eb420 con 0x7f2cb006e300 2023-11-28T21:34:47.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 -- 172.21.15.12:0/2486354085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2cb0074f60 con 0x7f2cb006e300 2023-11-28T21:34:47.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cad7fa640 1 -- 172.21.15.12:0/2486354085 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2c9c0eea40 con 0x7f2cb006e300 2023-11-28T21:34:47.786 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T21:34:47.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 -- 172.21.15.12:0/2486354085 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2c940664a0 msgr2=0x7f2c94068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:47.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2c940664a0 0x7f2c94068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400b930 tx=0x7f2ca4008040 comp rx=0 tx=0).stop 2023-11-28T21:34:47.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 -- 172.21.15.12:0/2486354085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 msgr2=0x7f2cb01f34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:47.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 0x7f2cb01f34c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0af040 tx=0x7f2c9c0192a0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 -- 172.21.15.12:0/2486354085 shutdown_connections 2023-11-28T21:34:47.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2c940664a0 0x7f2c94068960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb0150c20 0x7f2cb01f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cb006e300 0x7f2cb01f34c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 --2- 172.21.15.12:0/2486354085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cb006cf40 0x7f2cb015a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:47.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 -- 172.21.15.12:0/2486354085 >> 172.21.15.12:0/2486354085 conn(0x7f2cb0066360 msgr2=0x7f2cb00c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:47.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 -- 172.21.15.12:0/2486354085 shutdown_connections 2023-11-28T21:34:47.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:47.784+0000 7f2cb6e79640 1 -- 172.21.15.12:0/2486354085 wait complete. 2023-11-28T21:34:47.807 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T21:34:47.807 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T21:34:47.807 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2023-11-28T21:34:47.808 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-28T21:34:47.808 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T21:34:47.808 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T21:34:47.808 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T21:34:47.808 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T21:34:47.809 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T21:34:47.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:47.809 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2023-11-28T21:34:47.819 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T21:34:47.819 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T21:34:47.819 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T21:34:47.819 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-11-28T21:34:47.819 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2023-11-28T21:34:47.819 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-28T21:34:47.820 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42388sec preferred_lft 42388sec 2023-11-28T21:34:47.821 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2023-11-28T21:34:47.821 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-11-28T21:34:47.821 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T21:34:47.821 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T21:34:47.821 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2023-11-28T21:34:47.821 INFO:teuthology.orchestra.run.smithi012.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T21:34:47.821 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 02:42:f3:cf:11:01 brd ff:ff:ff:ff:ff:ff 2023-11-28T21:34:47.821 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T21:34:47.821 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-11-28T21:34:47.822 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T21:34:47.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:47.822 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-11-28T21:34:47.823 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-11-28T21:34:47.823 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2023-11-28T21:34:47.823 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2023-11-28T21:34:47.823 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2023-11-28T21:34:47.823 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T21:34:47.823 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-11-28T21:34:47.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:47.934 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T21:34:47.954 INFO:teuthology.orchestra.run.smithi012.stdout:1 2023-11-28T21:34:47.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:47.956 DEBUG:teuthology.orchestra.run.smithi012:> route 2023-11-28T21:34:48.023 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2023-11-28T21:34:48.023 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T21:34:48.024 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-28T21:34:48.024 INFO:teuthology.orchestra.run.smithi012.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T21:34:48.024 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-28T21:34:48.024 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T21:34:48.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:48.025 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-11-28T21:34:48.025 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-11-28T21:34:48.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T21:34:48.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T21:34:48.026 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T21:34:48.026 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-11-28T21:34:48.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:48.120 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-11-28T21:34:48.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:48.130 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2023-11-28T21:34:48.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:48.182 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-11-28T21:34:48.182 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-11-28T21:34:48.182 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T21:34:48.182 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-28T21:34:48.182 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-11-28T21:34:48.265 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-28T21:34:48.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:48.266 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-11-28T21:34:48.266 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-11-28T21:34:48.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-28T21:34:48.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T21:34:48.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-28T21:34:48.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-28T21:34:48.266 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-28T21:34:48.266 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-11-28T21:34:48.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:48.444 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-11-28T21:34:48.444 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-11-28T21:34:48.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2023-11-28T21:34:48.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T21:34:48.444 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-11-28T21:34:48.485 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-28T21:34:48.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T21:34:48.486 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-28T21:34:48.541 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T21:34:48.541 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T21:34:48.542 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-28T21:34:48.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2023-11-28T21:34:48.659 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T21:34:48.714 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2023-11-28T21:34:48.715 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2023-11-28T21:34:48.715 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2023-11-28T21:34:48.715 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2023-11-28T21:34:48.715 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2023-11-28T21:34:48.715 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2023-11-28T21:34:48.715 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2023-11-28T21:34:48.715 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2023-11-28T21:34:48.715 INFO:teuthology.orchestra.run.smithi012.stdout:/run/lock 2023-11-28T21:34:48.715 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/fs/nfsd 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/run/rpc_pipefs 2023-11-28T21:34:48.716 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2023-11-28T21:34:48.717 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2023-11-28T21:34:48.717 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T21:34:48.717 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T21:34:48.717 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T21:34:48.717 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T21:34:48.717 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2023-11-28T21:34:48.717 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T21:34:48.717 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T21:34:48.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:48.718 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2023-11-28T21:34:48.773 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T21:34:48.773 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-11-28T21:34:48.820 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2023-11-28T21:34:48.843 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T21:34:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2023-11-28T21:34:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2023-11-28T21:34:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2023-11-28T21:34:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2023-11-28T21:34:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2023-11-28T21:34:48.895 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2023-11-28T21:34:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2023-11-28T21:34:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2023-11-28T21:34:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/run/lock 2023-11-28T21:34:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2023-11-28T21:34:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2023-11-28T21:34:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2023-11-28T21:34:48.896 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T21:34:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2023-11-28T21:34:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2023-11-28T21:34:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2023-11-28T21:34:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2023-11-28T21:34:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/fs/nfsd 2023-11-28T21:34:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2023-11-28T21:34:48.897 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2023-11-28T21:34:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T21:34:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/run/rpc_pipefs 2023-11-28T21:34:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2023-11-28T21:34:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2023-11-28T21:34:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T21:34:48.898 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2023-11-28T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T21:34:48.899 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T21:34:48.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:48.900 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2023-11-28T21:34:48.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:2023-11-28T21:34:48.924+0000 7f40b422a580 -1 init, newargv = 0x55f779f3ce90 newargc=15 2023-11-28T21:34:48.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[42748]: starting ceph client 2023-11-28T21:34:48.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[42748]: starting fuse 2023-11-28T21:34:50.038 DEBUG:teuthology.orchestra.run.smithi012:> sudo modprobe fuse 2023-11-28T21:34:50.064 DEBUG:teuthology.orchestra.run.smithi012:> cat /proc/self/mounts | awk '{print $2}' 2023-11-28T21:34:50.115 INFO:teuthology.orchestra.run.smithi012.stdout:/sys 2023-11-28T21:34:50.115 INFO:teuthology.orchestra.run.smithi012.stdout:/proc 2023-11-28T21:34:50.115 INFO:teuthology.orchestra.run.smithi012.stdout:/dev 2023-11-28T21:34:50.115 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/pts 2023-11-28T21:34:50.115 INFO:teuthology.orchestra.run.smithi012.stdout:/run 2023-11-28T21:34:50.115 INFO:teuthology.orchestra.run.smithi012.stdout:/ 2023-11-28T21:34:50.115 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/security 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/shm 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/run/lock 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/cgroup 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/pstore 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/bpf 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/sys/fs/binfmt_misc 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/hugepages 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/dev/mqueue 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/debug 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/tracing 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/proc/fs/nfsd 2023-11-28T21:34:50.116 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/fs/fuse/connections 2023-11-28T21:34:50.117 INFO:teuthology.orchestra.run.smithi012.stdout:/sys/kernel/config 2023-11-28T21:34:50.117 INFO:teuthology.orchestra.run.smithi012.stdout:/run/credentials/systemd-sysusers.service 2023-11-28T21:34:50.117 INFO:teuthology.orchestra.run.smithi012.stdout:/run/rpc_pipefs 2023-11-28T21:34:50.117 INFO:teuthology.orchestra.run.smithi012.stdout:/run/user/1000 2023-11-28T21:34:50.117 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph 2023-11-28T21:34:50.117 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-0 2023-11-28T21:34:50.117 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-1 2023-11-28T21:34:50.117 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-2 2023-11-28T21:34:50.118 INFO:teuthology.orchestra.run.smithi012.stdout:/var/lib/ceph/osd/ceph-3 2023-11-28T21:34:50.118 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns 2023-11-28T21:34:50.118 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T21:34:50.118 INFO:teuthology.orchestra.run.smithi012.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T21:34:50.118 INFO:teuthology.orchestra.run.smithi012.stdout:/home/ubuntu/cephtest/mnt.admin 2023-11-28T21:34:50.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:50.119 DEBUG:teuthology.orchestra.run.smithi012:> ls /sys/fs/fuse/connections 2023-11-28T21:34:50.169 INFO:teuthology.orchestra.run.smithi012.stdout:43 2023-11-28T21:34:50.170 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-11-28T21:34:50.170 DEBUG:teuthology.orchestra.run.smithi012:> sudo stdin-killer -- python3 -c ' 2023-11-28T21:34:50.170 DEBUG:teuthology.orchestra.run.smithi012:> import glob 2023-11-28T21:34:50.170 DEBUG:teuthology.orchestra.run.smithi012:> import re 2023-11-28T21:34:50.170 DEBUG:teuthology.orchestra.run.smithi012:> import os 2023-11-28T21:34:50.170 DEBUG:teuthology.orchestra.run.smithi012:> import subprocess 2023-11-28T21:34:50.171 DEBUG:teuthology.orchestra.run.smithi012:> 2023-11-28T21:34:50.171 DEBUG:teuthology.orchestra.run.smithi012:> def _find_admin_socket(client_name): 2023-11-28T21:34:50.171 DEBUG:teuthology.orchestra.run.smithi012:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-28T21:34:50.171 DEBUG:teuthology.orchestra.run.smithi012:> files = glob.glob(asok_path) 2023-11-28T21:34:50.171 DEBUG:teuthology.orchestra.run.smithi012:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-28T21:34:50.172 DEBUG:teuthology.orchestra.run.smithi012:> 2023-11-28T21:34:50.172 DEBUG:teuthology.orchestra.run.smithi012:> # Given a non-glob path, it better be there 2023-11-28T21:34:50.172 DEBUG:teuthology.orchestra.run.smithi012:> if "*" not in asok_path: 2023-11-28T21:34:50.172 DEBUG:teuthology.orchestra.run.smithi012:> assert(len(files) == 1) 2023-11-28T21:34:50.172 DEBUG:teuthology.orchestra.run.smithi012:> return files[0] 2023-11-28T21:34:50.172 DEBUG:teuthology.orchestra.run.smithi012:> 2023-11-28T21:34:50.172 DEBUG:teuthology.orchestra.run.smithi012:> for f in files: 2023-11-28T21:34:50.173 DEBUG:teuthology.orchestra.run.smithi012:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T21:34:50.173 DEBUG:teuthology.orchestra.run.smithi012:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T21:34:50.173 DEBUG:teuthology.orchestra.run.smithi012:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T21:34:50.173 DEBUG:teuthology.orchestra.run.smithi012:> contents = proc_f.read() 2023-11-28T21:34:50.173 DEBUG:teuthology.orchestra.run.smithi012:> if mountpoint in contents: 2023-11-28T21:34:50.173 DEBUG:teuthology.orchestra.run.smithi012:> return f 2023-11-28T21:34:50.173 DEBUG:teuthology.orchestra.run.smithi012:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T21:34:50.173 DEBUG:teuthology.orchestra.run.smithi012:> 2023-11-28T21:34:50.174 DEBUG:teuthology.orchestra.run.smithi012:> print(_find_admin_socket("client.admin")) 2023-11-28T21:34:50.174 DEBUG:teuthology.orchestra.run.smithi012:> ' 2023-11-28T21:34:50.297 INFO:teuthology.orchestra.run.smithi012.stdout:/var/run/ceph/ceph-client.admin.42748.asok 2023-11-28T21:34:50.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-28T21:34:50.308 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42748.asok 2023-11-28T21:34:50.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:50.309 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42748.asok status 2023-11-28T21:34:50.482 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:34:50.482 INFO:teuthology.orchestra.run.smithi012.stdout: "metadata": { 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_sha1": "43dd94c190073cf16cff1e711dd7f5a7b1cbc883", 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 18.0.0-7520-g43dd94c1 (43dd94c190073cf16cff1e711dd7f5a7b1cbc883) reef (dev)", 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "entity_id": "admin", 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": "smithi012", 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "pid": "42748", 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "root": "/" 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_count": 0, 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "dentry_pinned_count": 0, 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4407, 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "inst": { 2023-11-28T21:34:50.483 INFO:teuthology.orchestra.run.smithi012.stdout: "name": { 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "client", 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "num": 4407 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2609245321 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: } 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": { 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2023-11-28T21:34:50.484 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "192.168.0.1:0", 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2609245321 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "inst_str": "client.4407 192.168.0.1:0/2609245321", 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "addr_str": "192.168.0.1:0/2609245321", 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "inode_count": 1, 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_epoch": 5, 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch": 24, 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_epoch_barrier": 0, 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklisted": false, 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs" 2023-11-28T21:34:50.485 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:34:50.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:50.498 DEBUG:teuthology.orchestra.run.smithi012:> stat --file-system '--printf=%T 2023-11-28T21:34:50.498 DEBUG:teuthology.orchestra.run.smithi012:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-28T21:34:50.508 INFO:teuthology.orchestra.run.smithi012.stdout:fuseblk 2023-11-28T21:34:50.508 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-28T21:34:50.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:50.508 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-28T21:34:50.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:50.571 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-28T21:34:50.596 INFO:tasks.ceph.mds.f.smithi144.stderr:2023-11-28T21:34:50.595+0000 7fcaf2464640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T21:34:50.662 INFO:teuthology.orchestra.run.smithi012.stdout:# file: . 2023-11-28T21:34:50.663 INFO:teuthology.orchestra.run.smithi012.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-28T21:34:50.663 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:50.664 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:50.664 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T21:34:50.674 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7d014aa640 1 Processor -- start 2023-11-28T21:34:50.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7d014aa640 1 -- start start 2023-11-28T21:34:50.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7d014aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc1500d0 0x7f7cfc0c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7d014aa640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7cfc156af0 0x7f7cfc0c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7d014aa640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0c6ce0 con 0x7f7cfc0c3500 2023-11-28T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7d014aa640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc0c7170 con 0x7f7cfc156af0 2023-11-28T21:34:50.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7d014aa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0c5df0 con 0x7f7cfc1500d0 2023-11-28T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7cfb7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7cfc0c3500 0x7f7cfc06d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43384/0 (socket says 172.21.15.12:43384) 2023-11-28T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7cfb7fe640 1 -- 172.21.15.12:0/3598096238 learned_addr learned my addr 172.21.15.12:0/3598096238 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:50.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7cfaffd640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc1500d0 0x7f7cfc0c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7cf3fff640 1 --2- 172.21.15.12:0/3598096238 >> v2:172.21.15.144:3300/0 conn(0x7f7cfc156af0 0x7f7cfc0c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720255447 0 0) 0x7f7cfc0c6ce0 con 0x7f7cfc0c3500 2023-11-28T21:34:50.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.856+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce8003530 con 0x7f7cfc0c3500 2023-11-28T21:34:50.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 416368336 0 0) 0x7f7ce8003530 con 0x7f7cfc0c3500 2023-11-28T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc1500d0 msgr2=0x7f7cfc0c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc1500d0 0x7f7cfc0c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 >> v2:172.21.15.144:3300/0 conn(0x7f7cfc156af0 msgr2=0x7f7cfc0c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 --2- 172.21.15.12:0/3598096238 >> v2:172.21.15.144:3300/0 conn(0x7f7cfc156af0 0x7f7cfc0c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0c60d0 con 0x7f7cfc0c3500 2023-11-28T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f7cf4001a00 con 0x7f7cfc0c3500 2023-11-28T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 >> v1:172.21.15.12:6789/0 conn(0x7f7cfc0c3500 legacy=0x7f7cfc06d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ce8003a20 0x7f7ce8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:50.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ce80047d0 0x7f7ce8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc1500d0 0x7f7ce8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0c7170 con 0x7f7ce80047d0 2023-11-28T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0c5df0 con 0x7f7ce8003a20 2023-11-28T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce800ba70 con 0x7f7cfc1500d0 2023-11-28T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cf3fff640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ce80047d0 0x7f7ce8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfaffd640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ce8003a20 0x7f7ce8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfb7fe640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc1500d0 0x7f7ce8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:50.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cf3fff640 1 -- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc1500d0 msgr2=0x7f7ce8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cf3fff640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc1500d0 0x7f7ce8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cf3fff640 1 -- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ce8003a20 msgr2=0x7f7ce8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cf3fff640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ce8003a20 0x7f7ce8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cf3fff640 1 -- 172.21.15.12:0/3598096238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce8003530 con 0x7f7ce80047d0 2023-11-28T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cfaffd640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ce8003a20 0x7f7ce8003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.860+0000 7f7cf3fff640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ce80047d0 0x7f7ce8004c50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009ab0 tx=0x7f7ce401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35af4a81b7ccc9f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40a6490 con 0x7f7ce80047d0 2023-11-28T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce40a65f0 con 0x7f7ce80047d0 2023-11-28T21:34:50.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7cfa7fc640 1 -- 172.21.15.12:0/3598096238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40a69a0 con 0x7f7ce80047d0 2023-11-28T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 -- 172.21.15.12:0/3598096238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ce80047d0 msgr2=0x7f7ce8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 --2- 172.21.15.12:0/3598096238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ce80047d0 0x7f7ce8004c50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009ab0 tx=0x7f7ce401ca20 comp rx=0 tx=0).stop 2023-11-28T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7cfb7fe640 1 -- 172.21.15.12:0/3598096238 reap_dead start 2023-11-28T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 -- 172.21.15.12:0/3598096238 shutdown_connections 2023-11-28T21:34:50.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 -- 172.21.15.12:0/3598096238 >> 172.21.15.12:0/3598096238 conn(0x7f7cfc066360 msgr2=0x7f7cfc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 -- 172.21.15.12:0/3598096238 shutdown_connections 2023-11-28T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 -- 172.21.15.12:0/3598096238 wait complete. 2023-11-28T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 Processor -- start 2023-11-28T21:34:50.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 -- start start 2023-11-28T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce80047d0 0x7f7cfc071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072440 0x7f7cfc153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cfc160de0 0x7f7cfc159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc154ef0 con 0x7f7cfc160de0 2023-11-28T21:34:50.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc162fc0 con 0x7f7cfc072440 2023-11-28T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7d014aa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1f31c0 con 0x7f7ce80047d0 2023-11-28T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7cfb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cfc160de0 0x7f7cfc159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:50.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7cfb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cfc160de0 0x7f7cfc159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53552/0 (socket says 172.21.15.12:53552) 2023-11-28T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7cfb7fe640 1 -- 172.21.15.12:0/2390239079 learned_addr learned my addr 172.21.15.12:0/2390239079 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7cf3fff640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072440 0x7f7cfc153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.864+0000 7f7cfaffd640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce80047d0 0x7f7cfc071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cfb7fe640 1 -- 172.21.15.12:0/2390239079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce80047d0 msgr2=0x7f7cfc071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:50.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cfb7fe640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce80047d0 0x7f7cfc071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cfb7fe640 1 -- 172.21.15.12:0/2390239079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072440 msgr2=0x7f7cfc153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cfb7fe640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072440 0x7f7cfc153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cfb7fe640 1 -- 172.21.15.12:0/2390239079 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc1f3480 con 0x7f7cfc160de0 2023-11-28T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cfaffd640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce80047d0 0x7f7cfc071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cfb7fe640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cfc160de0 0x7f7cfc159970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc15a720 tx=0x7f7cf4007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cf37fe640 1 -- 172.21.15.12:0/2390239079 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400eb40 con 0x7f7cfc160de0 2023-11-28T21:34:50.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cf37fe640 1 -- 172.21.15.12:0/2390239079 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf400ecc0 con 0x7f7cfc160de0 2023-11-28T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cf37fe640 1 -- 172.21.15.12:0/2390239079 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4013420 con 0x7f7cfc160de0 2023-11-28T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7d014aa640 1 -- 172.21.15.12:0/2390239079 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc155120 con 0x7f7cfc160de0 2023-11-28T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7d014aa640 1 -- 172.21.15.12:0/2390239079 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc1556e0 con 0x7f7cfc160de0 2023-11-28T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7d014aa640 1 -- 172.21.15.12:0/2390239079 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc074f60 con 0x7f7cfc160de0 2023-11-28T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.868+0000 7f7cf37fe640 1 -- 172.21.15.12:0/2390239079 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7cf4013580 con 0x7f7cfc160de0 2023-11-28T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.872+0000 7f7cf37fe640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7cdc066470 0x7f7cdc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:50.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.872+0000 7f7cf37fe640 1 -- 172.21.15.12:0/2390239079 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7cf40845c0 con 0x7f7cfc160de0 2023-11-28T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.872+0000 7f7cfaffd640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7cdc066470 0x7f7cdc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.872+0000 7f7cfaffd640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7cdc066470 0x7f7cdc068930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009a90 tx=0x7f7cec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:50.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:50.876+0000 7f7cf37fe640 1 -- 172.21.15.12:0/2390239079 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7cf40516b0 con 0x7f7cfc160de0 2023-11-28T21:34:51.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.032+0000 7f7d014aa640 1 -- 172.21.15.12:0/2390239079 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cfc0c7320 con 0x7f7cfc160de0 2023-11-28T21:34:51.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7cf37fe640 1 -- 172.21.15.12:0/2390239079 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7f7cf4054cd0 con 0x7f7cfc160de0 2023-11-28T21:34:51.038 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:51.039 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:34:51.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 -- 172.21.15.12:0/2390239079 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7cdc066470 msgr2=0x7f7cdc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7cdc066470 0x7f7cdc068930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009a90 tx=0x7f7cec008040 comp rx=0 tx=0).stop 2023-11-28T21:34:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 -- 172.21.15.12:0/2390239079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cfc160de0 msgr2=0x7f7cfc159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cfc160de0 0x7f7cfc159970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc15a720 tx=0x7f7cf4007bc0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 -- 172.21.15.12:0/2390239079 shutdown_connections 2023-11-28T21:34:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7cdc066470 0x7f7cdc068930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7cfc160de0 0x7f7cfc159970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072440 0x7f7cfc153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 --2- 172.21.15.12:0/2390239079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ce80047d0 0x7f7cfc071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 -- 172.21.15.12:0/2390239079 >> 172.21.15.12:0/2390239079 conn(0x7f7cfc066360 msgr2=0x7f7cfc0c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:51.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 -- 172.21.15.12:0/2390239079 shutdown_connections 2023-11-28T21:34:51.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.036+0000 7f7d014aa640 1 -- 172.21.15.12:0/2390239079 wait complete. 2023-11-28T21:34:51.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T21:34:29.878190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T21:34:34.039032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T21:34:35.369379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T21:34:39.373079+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T21:34:51.059 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T21:34:51.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f75a1a09640 1 Processor -- start 2023-11-28T21:34:51.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f75a1a09640 1 -- start start 2023-11-28T21:34:51.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f75a1a09640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f759c0c9790 0x7f759c0c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f75a1a09640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f759c0c9be0 0x7f759c0c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f75a1a09640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759c15c0d0 con 0x7f759c151f50 2023-11-28T21:34:51.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f75a1a09640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759c152330 con 0x7f759c0c9790 2023-11-28T21:34:51.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f75a1a09640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c15b2b0 con 0x7f759c0c9be0 2023-11-28T21:34:51.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f7593fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f759c0c9790 0x7f759c0c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f7593fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f759c0c9790 0x7f759c0c49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53554/0 (socket says 172.21.15.12:53554) 2023-11-28T21:34:51.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f7593fff640 1 -- 172.21.15.12:0/4178056087 learned_addr learned my addr 172.21.15.12:0/4178056087 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:51.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f759b7fe640 1 --2- 172.21.15.12:0/4178056087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f759c0c9be0 0x7f759c0c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f7593fff640 1 -- 172.21.15.12:0/4178056087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f759c0c9be0 msgr2=0x7f759c0c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:51.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f7593fff640 1 --2- 172.21.15.12:0/4178056087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f759c0c9be0 0x7f759c0c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f7593fff640 1 -- 172.21.15.12:0/4178056087 >> v1:172.21.15.144:6789/0 conn(0x7f759c151f50 legacy=0x7f759c0724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:51.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.204+0000 7f7593fff640 1 -- 172.21.15.12:0/4178056087 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c15b550 con 0x7f759c0c9790 2023-11-28T21:34:51.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f7593fff640 1 --2- 172.21.15.12:0/4178056087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9790 0x7f759c0c49c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7584009870 tx=0x7f758401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773d7a816d77707d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:51.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f759a7fc640 1 -- 172.21.15.12:0/4178056087 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840a8a30 con 0x7f759c0c9790 2023-11-28T21:34:51.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f759a7fc640 1 -- 172.21.15.12:0/4178056087 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75840a8b90 con 0x7f759c0c9790 2023-11-28T21:34:51.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f759a7fc640 1 -- 172.21.15.12:0/4178056087 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840b0440 con 0x7f759c0c9790 2023-11-28T21:34:51.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f75a1a09640 1 -- 172.21.15.12:0/4178056087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9790 msgr2=0x7f759c0c49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f75a1a09640 1 --2- 172.21.15.12:0/4178056087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9790 0x7f759c0c49c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7584009870 tx=0x7f758401ed30 comp rx=0 tx=0).stop 2023-11-28T21:34:51.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f75a1a09640 1 -- 172.21.15.12:0/4178056087 shutdown_connections 2023-11-28T21:34:51.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f75a1a09640 1 --2- 172.21.15.12:0/4178056087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f759c0c9be0 0x7f759c0c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f75a1a09640 1 --2- 172.21.15.12:0/4178056087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9790 0x7f759c0c49c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f75a1a09640 1 -- 172.21.15.12:0/4178056087 >> 172.21.15.12:0/4178056087 conn(0x7f759c066360 msgr2=0x7f759c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:51.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f75a1a09640 1 -- 172.21.15.12:0/4178056087 shutdown_connections 2023-11-28T21:34:51.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.208+0000 7f75a1a09640 1 -- 172.21.15.12:0/4178056087 wait complete. 2023-11-28T21:34:51.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 Processor -- start 2023-11-28T21:34:51.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 -- start start 2023-11-28T21:34:51.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f759c0c9790 0x7f759c0c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9be0 0x7f759c14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f759c151f50 0x7f759c1f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c1f40c0 con 0x7f759c0c9be0 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0cd5c0 con 0x7f759c151f50 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c06d6f0 con 0x7f759c0c9790 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7593fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9be0 0x7f759c14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7593fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9be0 0x7f759c14d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53564/0 (socket says 172.21.15.12:53564) 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7593fff640 1 -- 172.21.15.12:0/3076216526 learned_addr learned my addr 172.21.15.12:0/3076216526 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f759b7fe640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f759c151f50 0x7f759c1f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f759affd640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f759c0c9790 0x7f759c0c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7593fff640 1 -- 172.21.15.12:0/3076216526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f759c0c9790 msgr2=0x7f759c0c16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7593fff640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f759c0c9790 0x7f759c0c16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7593fff640 1 -- 172.21.15.12:0/3076216526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f759c151f50 msgr2=0x7f759c1f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7593fff640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f759c151f50 0x7f759c1f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7593fff640 1 -- 172.21.15.12:0/3076216526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75840094d0 con 0x7f759c0c9be0 2023-11-28T21:34:51.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7593fff640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9be0 0x7f759c14d560 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f75840b3040 tx=0x7f758401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:51.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7598ff9640 1 -- 172.21.15.12:0/3076216526 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840be070 con 0x7f759c0c9be0 2023-11-28T21:34:51.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7598ff9640 1 -- 172.21.15.12:0/3076216526 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75840ba400 con 0x7f759c0c9be0 2023-11-28T21:34:51.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f7598ff9640 1 -- 172.21.15.12:0/3076216526 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840ba7d0 con 0x7f759c0c9be0 2023-11-28T21:34:51.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 -- 172.21.15.12:0/3076216526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c0c1c30 con 0x7f759c0c9be0 2023-11-28T21:34:51.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 -- 172.21.15.12:0/3076216526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c06ce50 con 0x7f759c0c9be0 2023-11-28T21:34:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.212+0000 7f75a1a09640 1 -- 172.21.15.12:0/3076216526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c1f3e20 con 0x7f759c0c9be0 2023-11-28T21:34:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.216+0000 7f7598ff9640 1 -- 172.21.15.12:0/3076216526 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f75840c3410 con 0x7f759c0c9be0 2023-11-28T21:34:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.216+0000 7f7598ff9640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f757c066470 0x7f757c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.216+0000 7f7598ff9640 1 -- 172.21.15.12:0/3076216526 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7584121e00 con 0x7f759c0c9be0 2023-11-28T21:34:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.216+0000 7f759affd640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f757c066470 0x7f757c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.216+0000 7f759affd640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f757c066470 0x7f757c068930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f758c000f70 tx=0x7f758c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.220+0000 7f7598ff9640 1 -- 172.21.15.12:0/3076216526 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f759c1f3e20 con 0x7f759c0c9be0 2023-11-28T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 -- 172.21.15.12:0/3076216526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f759c1f3e20 con 0x7f759c0c9be0 2023-11-28T21:34:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f7598ff9640 1 -- 172.21.15.12:0/3076216526 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f759c1f3e20 con 0x7f759c0c9be0 2023-11-28T21:34:51.382 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-11-28T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 -- 172.21.15.12:0/3076216526 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f757c066470 msgr2=0x7f757c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f757c066470 0x7f757c068930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f758c000f70 tx=0x7f758c009040 comp rx=0 tx=0).stop 2023-11-28T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 -- 172.21.15.12:0/3076216526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9be0 msgr2=0x7f759c14d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9be0 0x7f759c14d560 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f75840b3040 tx=0x7f758401bad0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 -- 172.21.15.12:0/3076216526 shutdown_connections 2023-11-28T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f757c066470 0x7f757c068930 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f759c151f50 0x7f759c1f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f759c0c9be0 0x7f759c14d560 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 --2- 172.21.15.12:0/3076216526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f759c0c9790 0x7f759c0c16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 -- 172.21.15.12:0/3076216526 >> 172.21.15.12:0/3076216526 conn(0x7f759c066360 msgr2=0x7f759c072200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 -- 172.21.15.12:0/3076216526 shutdown_connections 2023-11-28T21:34:51.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.380+0000 7f75a1a09640 1 -- 172.21.15.12:0/3076216526 wait complete. 2023-11-28T21:34:51.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T21:34:51.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.544+0000 7f8fdc079640 1 Processor -- start 2023-11-28T21:34:51.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.544+0000 7f8fdc079640 1 -- start start 2023-11-28T21:34:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fdc079640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd406b130 0x7f8fd40c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fdc079640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fd40cb020 0x7f8fd4159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fdc079640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd40cb4b0 con 0x7f8fd40cac40 2023-11-28T21:34:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fdc079640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd406b550 con 0x7f8fd40cb020 2023-11-28T21:34:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fdc079640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd415a5e0 con 0x7f8fd406b130 2023-11-28T21:34:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fd9e15640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8fd40cac40 0x7f8fd40ca030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43388/0 (socket says 172.21.15.12:43388) 2023-11-28T21:34:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fd9e15640 1 -- 172.21.15.12:0/1706527589 learned_addr learned my addr 172.21.15.12:0/1706527589 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fd9614640 1 --2- 172.21.15.12:0/1706527589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd406b130 0x7f8fd40c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fda616640 1 --2- 172.21.15.12:0/1706527589 >> v2:172.21.15.144:3300/0 conn(0x7f8fd40cb020 0x7f8fd4159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fd8e13640 1 -- 172.21.15.12:0/1706527589 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421667024 0 0) 0x7f8fd40cb4b0 con 0x7f8fd40cac40 2023-11-28T21:34:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fd8e13640 1 -- 172.21.15.12:0/1706527589 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc0003530 con 0x7f8fd40cac40 2023-11-28T21:34:51.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fd9614640 1 -- 172.21.15.12:0/1706527589 >> v2:172.21.15.144:3300/0 conn(0x7f8fd40cb020 msgr2=0x7f8fd4159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:51.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fd8e13640 1 -- 172.21.15.12:0/1706527589 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 658497813 0 0) 0x7f8fc0003530 con 0x7f8fd40cac40 2023-11-28T21:34:51.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fd9614640 1 --2- 172.21.15.12:0/1706527589 >> v2:172.21.15.144:3300/0 conn(0x7f8fd40cb020 0x7f8fd4159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fd9614640 1 -- 172.21.15.12:0/1706527589 >> v1:172.21.15.12:6789/0 conn(0x7f8fd40cac40 legacy=0x7f8fd40ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.548+0000 7f8fd9614640 1 -- 172.21.15.12:0/1706527589 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd415a8c0 con 0x7f8fd406b130 2023-11-28T21:34:51.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fd9614640 1 --2- 172.21.15.12:0/1706527589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd406b130 0x7f8fd40c4980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40097c0 tx=0x7f8fc401eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef313ac55551d514 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:51.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fd8e13640 1 -- 172.21.15.12:0/1706527589 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0001680 con 0x7f8fd406b130 2023-11-28T21:34:51.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fd8e13640 1 -- 172.21.15.12:0/1706527589 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc40a85d0 con 0x7f8fd406b130 2023-11-28T21:34:51.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fd8e13640 1 -- 172.21.15.12:0/1706527589 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40a8940 con 0x7f8fd406b130 2023-11-28T21:34:51.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fdc079640 1 -- 172.21.15.12:0/1706527589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd406b130 msgr2=0x7f8fd40c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fdc079640 1 --2- 172.21.15.12:0/1706527589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd406b130 0x7f8fd40c4980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40097c0 tx=0x7f8fc401eb90 comp rx=0 tx=0).stop 2023-11-28T21:34:51.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fdc079640 1 -- 172.21.15.12:0/1706527589 shutdown_connections 2023-11-28T21:34:51.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fdc079640 1 --2- 172.21.15.12:0/1706527589 >> v2:172.21.15.144:3300/0 conn(0x7f8fd40cb020 0x7f8fd4159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fdc079640 1 --2- 172.21.15.12:0/1706527589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd406b130 0x7f8fd40c4980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fdc079640 1 -- 172.21.15.12:0/1706527589 >> 172.21.15.12:0/1706527589 conn(0x7f8fd40720e0 msgr2=0x7f8fd40724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:51.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fdc079640 1 -- 172.21.15.12:0/1706527589 shutdown_connections 2023-11-28T21:34:51.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.552+0000 7f8fdc079640 1 -- 172.21.15.12:0/1706527589 wait complete. 2023-11-28T21:34:51.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 Processor -- start 2023-11-28T21:34:51.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 -- start start 2023-11-28T21:34:51.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd406b130 0x7f8fd4150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd40cac40 0x7f8fd41ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd40cb020 0x7f8fd41efee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4159d50 con 0x7f8fd40cb020 2023-11-28T21:34:51.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4159490 con 0x7f8fd406b130 2023-11-28T21:34:51.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd41f3cf0 con 0x7f8fd40cac40 2023-11-28T21:34:51.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fda616640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd40cb020 0x7f8fd41efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fda616640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd40cb020 0x7f8fd41efee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53578/0 (socket says 172.21.15.12:53578) 2023-11-28T21:34:51.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fda616640 1 -- 172.21.15.12:0/2978513461 learned_addr learned my addr 172.21.15.12:0/2978513461 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:51.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fd9e15640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd406b130 0x7f8fd4150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fd9614640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd40cac40 0x7f8fd41ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fda616640 1 -- 172.21.15.12:0/2978513461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd40cac40 msgr2=0x7f8fd41ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fda616640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd40cac40 0x7f8fd41ef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fda616640 1 -- 172.21.15.12:0/2978513461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd406b130 msgr2=0x7f8fd4150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fda616640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd406b130 0x7f8fd4150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fda616640 1 -- 172.21.15.12:0/2978513461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc4009420 con 0x7f8fd40cb020 2023-11-28T21:34:51.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fda616640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd40cb020 0x7f8fd41efee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00efc0 tx=0x7f8fcc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:51.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fca7fc640 1 -- 172.21.15.12:0/2978513461 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc014480 con 0x7f8fd40cb020 2023-11-28T21:34:51.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fca7fc640 1 -- 172.21.15.12:0/2978513461 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc00f040 con 0x7f8fd40cb020 2023-11-28T21:34:51.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 -- 172.21.15.12:0/2978513461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd4158ed0 con 0x7f8fd40cb020 2023-11-28T21:34:51.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fca7fc640 1 -- 172.21.15.12:0/2978513461 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc014820 con 0x7f8fd40cb020 2023-11-28T21:34:51.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 -- 172.21.15.12:0/2978513461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd41599e0 con 0x7f8fd40cb020 2023-11-28T21:34:51.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.556+0000 7f8fdc079640 1 -- 172.21.15.12:0/2978513461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd4067c70 con 0x7f8fd40cb020 2023-11-28T21:34:51.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.560+0000 7f8fca7fc640 1 -- 172.21.15.12:0/2978513461 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8fcc014980 con 0x7f8fd40cb020 2023-11-28T21:34:51.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.560+0000 7f8fca7fc640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fb8066470 0x7f8fb8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.560+0000 7f8fca7fc640 1 -- 172.21.15.12:0/2978513461 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8fcc084ce0 con 0x7f8fd40cb020 2023-11-28T21:34:51.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.560+0000 7f8fd9e15640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fb8066470 0x7f8fb8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.560+0000 7f8fd9e15640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fb8066470 0x7f8fb8068930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00011f0 tx=0x7f8fd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:51.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.564+0000 7f8fca7fc640 1 -- 172.21.15.12:0/2978513461 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8fcc051e70 con 0x7f8fd40cb020 2023-11-28T21:34:51.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 -- 172.21.15.12:0/2978513461 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fd4074f60 con 0x7f8fd40cb020 2023-11-28T21:34:51.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fca7fc640 1 -- 172.21.15.12:0/2978513461 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8fcc0180a0 con 0x7f8fd40cb020 2023-11-28T21:34:51.722 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:34:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 -- 172.21.15.12:0/2978513461 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fb8066470 msgr2=0x7f8fb8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fb8066470 0x7f8fb8068930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00011f0 tx=0x7f8fd0009040 comp rx=0 tx=0).stop 2023-11-28T21:34:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 -- 172.21.15.12:0/2978513461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd40cb020 msgr2=0x7f8fd41efee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd40cb020 0x7f8fd41efee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00efc0 tx=0x7f8fcc00c490 comp rx=0 tx=0).stop 2023-11-28T21:34:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 -- 172.21.15.12:0/2978513461 shutdown_connections 2023-11-28T21:34:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fb8066470 0x7f8fb8068930 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd40cb020 0x7f8fd41efee0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd40cac40 0x7f8fd41ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 --2- 172.21.15.12:0/2978513461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd406b130 0x7f8fd4150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 -- 172.21.15.12:0/2978513461 >> 172.21.15.12:0/2978513461 conn(0x7f8fd40720e0 msgr2=0x7f8fd40cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 -- 172.21.15.12:0/2978513461 shutdown_connections 2023-11-28T21:34:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.720+0000 7f8fdc079640 1 -- 172.21.15.12:0/2978513461 wait complete. 2023-11-28T21:34:51.743 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T21:34:51.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.884+0000 7fee4405a640 1 Processor -- start 2023-11-28T21:34:51.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.884+0000 7fee4405a640 1 -- start start 2023-11-28T21:34:51.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.888+0000 7fee4405a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee3c151af0 0x7fee3c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.888+0000 7fee4405a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3c157b70 con 0x7fee3c06e300 2023-11-28T21:34:51.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.888+0000 7fee4405a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3c157e50 con 0x7fee3c151ed0 2023-11-28T21:34:51.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.888+0000 7fee4405a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3c0707f0 con 0x7fee3c151af0 2023-11-28T21:34:51.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.888+0000 7fee425f7640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fee3c06e300 0x7fee3c072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43396/0 (socket says 172.21.15.12:43396) 2023-11-28T21:34:51.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.888+0000 7fee425f7640 1 -- 172.21.15.12:0/2351101195 learned_addr learned my addr 172.21.15.12:0/2351101195 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:51.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.888+0000 7fee41df6640 1 --2- 172.21.15.12:0/2351101195 >> v2:172.21.15.144:3300/0 conn(0x7fee3c151af0 0x7fee3c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.888+0000 7fee41df6640 1 -- 172.21.15.12:0/2351101195 >> v1:172.21.15.144:6789/0 conn(0x7fee3c151ed0 legacy=0x7fee3c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.888+0000 7fee41df6640 1 -- 172.21.15.12:0/2351101195 >> v1:172.21.15.12:6789/0 conn(0x7fee3c06e300 legacy=0x7fee3c072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.888+0000 7fee41df6640 1 -- 172.21.15.12:0/2351101195 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c158130 con 0x7fee3c151af0 2023-11-28T21:34:51.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee41df6640 1 --2- 172.21.15.12:0/2351101195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee3c151af0 0x7fee3c0c7650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fee380097c0 tx=0x7fee3801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73d764ef34eb7acd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:51.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee40df4640 1 -- 172.21.15.12:0/2351101195 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380ab600 con 0x7fee3c151af0 2023-11-28T21:34:51.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee40df4640 1 -- 172.21.15.12:0/2351101195 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee380a7050 con 0x7fee3c151af0 2023-11-28T21:34:51.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee40df4640 1 -- 172.21.15.12:0/2351101195 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380ab980 con 0x7fee3c151af0 2023-11-28T21:34:51.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee4405a640 1 -- 172.21.15.12:0/2351101195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee3c151af0 msgr2=0x7fee3c0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee4405a640 1 --2- 172.21.15.12:0/2351101195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee3c151af0 0x7fee3c0c7650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fee380097c0 tx=0x7fee3801c480 comp rx=0 tx=0).stop 2023-11-28T21:34:51.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee4405a640 1 -- 172.21.15.12:0/2351101195 shutdown_connections 2023-11-28T21:34:51.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee4405a640 1 --2- 172.21.15.12:0/2351101195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee3c151af0 0x7fee3c0c7650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee4405a640 1 -- 172.21.15.12:0/2351101195 >> 172.21.15.12:0/2351101195 conn(0x7fee3c066360 msgr2=0x7fee3c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:51.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee4405a640 1 -- 172.21.15.12:0/2351101195 shutdown_connections 2023-11-28T21:34:51.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.892+0000 7fee4405a640 1 -- 172.21.15.12:0/2351101195 wait complete. 2023-11-28T21:34:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 Processor -- start 2023-11-28T21:34:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 -- start start 2023-11-28T21:34:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee3c06e300 0x7fee3c141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee3c151af0 0x7fee3c06fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee3c1421d0 0x7fee3c1fb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0c9fd0 con 0x7fee3c06e300 2023-11-28T21:34:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c1f7690 con 0x7fee3c151af0 2023-11-28T21:34:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c162690 con 0x7fee3c1421d0 2023-11-28T21:34:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee41df6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee3c06e300 0x7fee3c141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee41df6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee3c06e300 0x7fee3c141ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53590/0 (socket says 172.21.15.12:53590) 2023-11-28T21:34:51.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee41df6640 1 -- 172.21.15.12:0/821687660 learned_addr learned my addr 172.21.15.12:0/821687660 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:51.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee425f7640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee3c1421d0 0x7fee3c1fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee415f5640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee3c151af0 0x7fee3c06fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee41df6640 1 -- 172.21.15.12:0/821687660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee3c1421d0 msgr2=0x7fee3c1fb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee41df6640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee3c1421d0 0x7fee3c1fb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee41df6640 1 -- 172.21.15.12:0/821687660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee3c151af0 msgr2=0x7fee3c06fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:51.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee41df6640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee3c151af0 0x7fee3c06fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:51.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee41df6640 1 -- 172.21.15.12:0/821687660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee38009420 con 0x7fee3c06e300 2023-11-28T21:34:51.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee41df6640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee3c06e300 0x7fee3c141ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee380097c0 tx=0x7fee38019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:51.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee327fc640 1 -- 172.21.15.12:0/821687660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380b4840 con 0x7fee3c06e300 2023-11-28T21:34:51.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee327fc640 1 -- 172.21.15.12:0/821687660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee380b49c0 con 0x7fee3c06e300 2023-11-28T21:34:51.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 -- 172.21.15.12:0/821687660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c163180 con 0x7fee3c06e300 2023-11-28T21:34:51.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee327fc640 1 -- 172.21.15.12:0/821687660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380b4d70 con 0x7fee3c06e300 2023-11-28T21:34:51.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 -- 172.21.15.12:0/821687660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c1fadc0 con 0x7fee3c06e300 2023-11-28T21:34:51.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.896+0000 7fee4405a640 1 -- 172.21.15.12:0/821687660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3c074f60 con 0x7fee3c06e300 2023-11-28T21:34:51.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.900+0000 7fee327fc640 1 -- 172.21.15.12:0/821687660 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fee380b2040 con 0x7fee3c06e300 2023-11-28T21:34:51.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.900+0000 7fee327fc640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee20066470 0x7fee20068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:51.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.900+0000 7fee327fc640 1 -- 172.21.15.12:0/821687660 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fee3811e590 con 0x7fee3c06e300 2023-11-28T21:34:51.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.900+0000 7fee415f5640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee20066470 0x7fee20068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:51.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.900+0000 7fee415f5640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee20066470 0x7fee20068930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee3c162620 tx=0x7fee2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:51.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:51.904+0000 7fee327fc640 1 -- 172.21.15.12:0/821687660 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee380eb710 con 0x7fee3c06e300 2023-11-28T21:34:52.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 -- 172.21.15.12:0/821687660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee3c1578b0 con 0x7fee3c06e300 2023-11-28T21:34:52.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee327fc640 1 -- 172.21.15.12:0/821687660 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee380eed30 con 0x7fee3c06e300 2023-11-28T21:34:52.070 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-11-28T21:34:52.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 -- 172.21.15.12:0/821687660 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee20066470 msgr2=0x7fee20068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee20066470 0x7fee20068930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee3c162620 tx=0x7fee2c009040 comp rx=0 tx=0).stop 2023-11-28T21:34:52.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 -- 172.21.15.12:0/821687660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee3c06e300 msgr2=0x7fee3c141ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee3c06e300 0x7fee3c141ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee380097c0 tx=0x7fee38019220 comp rx=0 tx=0).stop 2023-11-28T21:34:52.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 -- 172.21.15.12:0/821687660 shutdown_connections 2023-11-28T21:34:52.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee20066470 0x7fee20068930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee3c1421d0 0x7fee3c1fb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee3c151af0 0x7fee3c06fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 --2- 172.21.15.12:0/821687660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee3c06e300 0x7fee3c141ad0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 -- 172.21.15.12:0/821687660 >> 172.21.15.12:0/821687660 conn(0x7fee3c066360 msgr2=0x7fee3c1500e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:52.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 -- 172.21.15.12:0/821687660 shutdown_connections 2023-11-28T21:34:52.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.068+0000 7fee4405a640 1 -- 172.21.15.12:0/821687660 wait complete. 2023-11-28T21:34:52.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T21:34:52.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.232+0000 7f7d84bdf640 1 Processor -- start 2023-11-28T21:34:52.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.232+0000 7f7d84bdf640 1 -- start start 2023-11-28T21:34:52.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d84bdf640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7d80151af0 0x7f7d800c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d84bdf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d8006e300 0x7f7d80072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d84bdf640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d80157b70 con 0x7f7d80151ed0 2023-11-28T21:34:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d84bdf640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d800707f0 con 0x7f7d80151af0 2023-11-28T21:34:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d84bdf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d800c7b90 con 0x7f7d8006e300 2023-11-28T21:34:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d7e59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7d80151af0 0x7f7d800c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d7e59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7d80151af0 0x7f7d800c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53596/0 (socket says 172.21.15.12:53596) 2023-11-28T21:34:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d7e59c640 1 -- 172.21.15.12:0/4053979411 learned_addr learned my addr 172.21.15.12:0/4053979411 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d7ed9d640 1 --2- 172.21.15.12:0/4053979411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d8006e300 0x7f7d80072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d7e59c640 1 -- 172.21.15.12:0/4053979411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d8006e300 msgr2=0x7f7d80072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:52.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d7e59c640 1 --2- 172.21.15.12:0/4053979411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d8006e300 0x7f7d80072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d7e59c640 1 -- 172.21.15.12:0/4053979411 >> v1:172.21.15.144:6789/0 conn(0x7f7d80151ed0 legacy=0x7f7d800c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:52.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.236+0000 7f7d7e59c640 1 -- 172.21.15.12:0/4053979411 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d80157e50 con 0x7f7d80151af0 2023-11-28T21:34:52.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d7e59c640 1 --2- 172.21.15.12:0/4053979411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d80151af0 0x7f7d800c7650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009870 tx=0x7f7d7001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5440acb37350f0b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:52.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d7dd9b640 1 -- 172.21.15.12:0/4053979411 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d700ab600 con 0x7f7d80151af0 2023-11-28T21:34:52.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d7dd9b640 1 -- 172.21.15.12:0/4053979411 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d700a7050 con 0x7f7d80151af0 2023-11-28T21:34:52.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d7dd9b640 1 -- 172.21.15.12:0/4053979411 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d700ab980 con 0x7f7d80151af0 2023-11-28T21:34:52.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 -- 172.21.15.12:0/4053979411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d80151af0 msgr2=0x7f7d800c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 --2- 172.21.15.12:0/4053979411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d80151af0 0x7f7d800c7650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009870 tx=0x7f7d7001c540 comp rx=0 tx=0).stop 2023-11-28T21:34:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 -- 172.21.15.12:0/4053979411 shutdown_connections 2023-11-28T21:34:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 --2- 172.21.15.12:0/4053979411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d8006e300 0x7f7d80072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 --2- 172.21.15.12:0/4053979411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d80151af0 0x7f7d800c7650 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 -- 172.21.15.12:0/4053979411 >> 172.21.15.12:0/4053979411 conn(0x7f7d80066360 msgr2=0x7f7d80066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 -- 172.21.15.12:0/4053979411 shutdown_connections 2023-11-28T21:34:52.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 -- 172.21.15.12:0/4053979411 wait complete. 2023-11-28T21:34:52.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 Processor -- start 2023-11-28T21:34:52.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 -- start start 2023-11-28T21:34:52.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d8006e300 0x7f7d80155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d80151af0 0x7f7d8014cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d8014c080 0x7f7d8013c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80156d30 con 0x7f7d8014c080 2023-11-28T21:34:52.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8014d610 con 0x7f7d80151af0 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.240+0000 7f7d84bdf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d801f6690 con 0x7f7d8006e300 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d7ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d8014c080 0x7f7d8013c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d7ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d8014c080 0x7f7d8013c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53606/0 (socket says 172.21.15.12:53606) 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d7ed9d640 1 -- 172.21.15.12:0/2095607869 learned_addr learned my addr 172.21.15.12:0/2095607869 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d7e59c640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d8006e300 0x7f7d80155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d77fff640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d80151af0 0x7f7d8014cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d7ed9d640 1 -- 172.21.15.12:0/2095607869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d8006e300 msgr2=0x7f7d80155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d7ed9d640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d8006e300 0x7f7d80155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d7ed9d640 1 -- 172.21.15.12:0/2095607869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d80151af0 msgr2=0x7f7d8014cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d7ed9d640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d80151af0 0x7f7d8014cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d7ed9d640 1 -- 172.21.15.12:0/2095607869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d700094d0 con 0x7f7d8014c080 2023-11-28T21:34:52.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d7ed9d640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d8014c080 0x7f7d8013c7f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7d78007e30 tx=0x7f7d7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:52.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d76ffd640 1 -- 172.21.15.12:0/2095607869 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78014620 con 0x7f7d8014c080 2023-11-28T21:34:52.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d76ffd640 1 -- 172.21.15.12:0/2095607869 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7800ee90 con 0x7f7d8014c080 2023-11-28T21:34:52.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d76ffd640 1 -- 172.21.15.12:0/2095607869 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780149c0 con 0x7f7d8014c080 2023-11-28T21:34:52.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d84bdf640 1 -- 172.21.15.12:0/2095607869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d801f6970 con 0x7f7d8014c080 2023-11-28T21:34:52.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d84bdf640 1 -- 172.21.15.12:0/2095607869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d801f6f30 con 0x7f7d8014c080 2023-11-28T21:34:52.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d84bdf640 1 -- 172.21.15.12:0/2095607869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d80155240 con 0x7f7d8014c080 2023-11-28T21:34:52.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.244+0000 7f7d76ffd640 1 -- 172.21.15.12:0/2095607869 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7d78014b20 con 0x7f7d8014c080 2023-11-28T21:34:52.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.248+0000 7f7d76ffd640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d60066470 0x7f7d60068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.248+0000 7f7d76ffd640 1 -- 172.21.15.12:0/2095607869 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7d78084a80 con 0x7f7d8014c080 2023-11-28T21:34:52.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.248+0000 7f7d7e59c640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d60066470 0x7f7d60068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.248+0000 7f7d7e59c640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d60066470 0x7f7d60068930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d800c8e60 tx=0x7f7d70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:52.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.252+0000 7f7d76ffd640 1 -- 172.21.15.12:0/2095607869 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7d78051c10 con 0x7f7d8014c080 2023-11-28T21:34:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.408+0000 7f7d84bdf640 1 -- 172.21.15.12:0/2095607869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7d80074f60 con 0x7f7d8014c080 2023-11-28T21:34:52.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.408+0000 7f7d76ffd640 1 -- 172.21.15.12:0/2095607869 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7d78055230 con 0x7f7d8014c080 2023-11-28T21:34:52.413 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:34:52.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 -- 172.21.15.12:0/2095607869 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d60066470 msgr2=0x7f7d60068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d60066470 0x7f7d60068930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d800c8e60 tx=0x7f7d70008040 comp rx=0 tx=0).stop 2023-11-28T21:34:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 -- 172.21.15.12:0/2095607869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d8014c080 msgr2=0x7f7d8013c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d8014c080 0x7f7d8013c7f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7d78007e30 tx=0x7f7d7800e4b0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 -- 172.21.15.12:0/2095607869 shutdown_connections 2023-11-28T21:34:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d60066470 0x7f7d60068930 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7d8014c080 0x7f7d8013c7f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d80151af0 0x7f7d8014cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 --2- 172.21.15.12:0/2095607869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d8006e300 0x7f7d80155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 -- 172.21.15.12:0/2095607869 >> 172.21.15.12:0/2095607869 conn(0x7f7d80066360 msgr2=0x7f7d8006cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 -- 172.21.15.12:0/2095607869 shutdown_connections 2023-11-28T21:34:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.412+0000 7f7d84bdf640 1 -- 172.21.15.12:0/2095607869 wait complete. 2023-11-28T21:34:52.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T21:34:52.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.576+0000 7fd676f96640 1 Processor -- start 2023-11-28T21:34:52.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.576+0000 7fd676f96640 1 -- start start 2023-11-28T21:34:52.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd676f96640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006a5b0 0x7fd6700c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd676f96640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd670070bd0 0x7fd670071060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd676f96640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6700c6350 con 0x7fd670153ee0 2023-11-28T21:34:52.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd676f96640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6700c6b90 con 0x7fd670070bd0 2023-11-28T21:34:52.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd676f96640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd670072410 con 0x7fd67006a5b0 2023-11-28T21:34:52.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd674d32640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd670153ee0 0x7fd67015a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43404/0 (socket says 172.21.15.12:43404) 2023-11-28T21:34:52.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd674d32640 1 -- 172.21.15.12:0/3587760632 learned_addr learned my addr 172.21.15.12:0/3587760632 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:52.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd675533640 1 --2- 172.21.15.12:0/3587760632 >> v2:172.21.15.144:3300/0 conn(0x7fd670070bd0 0x7fd670071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66ffff640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006a5b0 0x7fd6700c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789660418 0 0) 0x7fd6700c6350 con 0x7fd670153ee0 2023-11-28T21:34:52.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd660003530 con 0x7fd670153ee0 2023-11-28T21:34:52.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1845134973 0 0) 0x7fd660003530 con 0x7fd670153ee0 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006a5b0 msgr2=0x7fd6700c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006a5b0 0x7fd6700c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 >> v2:172.21.15.144:3300/0 conn(0x7fd670070bd0 msgr2=0x7fd670071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 --2- 172.21.15.12:0/3587760632 >> v2:172.21.15.144:3300/0 conn(0x7fd670070bd0 0x7fd670071060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6700c65d0 con 0x7fd670153ee0 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fd664001680 con 0x7fd670153ee0 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 >> v1:172.21.15.12:6789/0 conn(0x7fd670153ee0 legacy=0x7fd67015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6600039f0 0x7fd660003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006a5b0 0x7fd6600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd660004d10 0x7fd660005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700c6b90 con 0x7fd660004d10 2023-11-28T21:34:52.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66000b990 con 0x7fd6600039f0 2023-11-28T21:34:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66000bb50 con 0x7fd67006a5b0 2023-11-28T21:34:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd675533640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd660004d10 0x7fd660005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd66ffff640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006a5b0 0x7fd6600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd674d32640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6600039f0 0x7fd660003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd675533640 1 -- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006a5b0 msgr2=0x7fd6600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd675533640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd67006a5b0 0x7fd6600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd675533640 1 -- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6600039f0 msgr2=0x7fd660003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd675533640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6600039f0 0x7fd660003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd675533640 1 -- 172.21.15.12:0/3587760632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660003530 con 0x7fd660004d10 2023-11-28T21:34:52.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.580+0000 7fd674d32640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6600039f0 0x7fd660003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:52.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd675533640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd660004d10 0x7fd660005190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd668009fc0 tx=0x7fd66801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b5c98a38b18b30 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6680a5cb0 con 0x7fd660004d10 2023-11-28T21:34:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66800f040 con 0x7fd660004d10 2023-11-28T21:34:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd66f7fe640 1 -- 172.21.15.12:0/3587760632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6680ad680 con 0x7fd660004d10 2023-11-28T21:34:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd676f96640 1 -- 172.21.15.12:0/3587760632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd660004d10 msgr2=0x7fd660005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd676f96640 1 --2- 172.21.15.12:0/3587760632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd660004d10 0x7fd660005190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd668009fc0 tx=0x7fd66801cca0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd675533640 1 -- 172.21.15.12:0/3587760632 reap_dead start 2023-11-28T21:34:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd676f96640 1 -- 172.21.15.12:0/3587760632 shutdown_connections 2023-11-28T21:34:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd676f96640 1 -- 172.21.15.12:0/3587760632 >> 172.21.15.12:0/3587760632 conn(0x7fd6700c8450 msgr2=0x7fd6700c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd676f96640 1 -- 172.21.15.12:0/3587760632 shutdown_connections 2023-11-28T21:34:52.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd676f96640 1 -- 172.21.15.12:0/3587760632 wait complete. 2023-11-28T21:34:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd676f96640 1 Processor -- start 2023-11-28T21:34:52.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.584+0000 7fd676f96640 1 -- start start 2023-11-28T21:34:52.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd676f96640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd660004d10 0x7fd6701ef940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd676f96640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6700c4e50 0x7fd6700c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd676f96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67014f7f0 0x7fd67014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd676f96640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6701efe80 con 0x7fd67014f7f0 2023-11-28T21:34:52.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd676f96640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6701f0900 con 0x7fd6700c4e50 2023-11-28T21:34:52.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd676f96640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6701501b0 con 0x7fd660004d10 2023-11-28T21:34:52.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd675533640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67014f7f0 0x7fd67014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd675533640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67014f7f0 0x7fd67014fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53624/0 (socket says 172.21.15.12:53624) 2023-11-28T21:34:52.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd675533640 1 -- 172.21.15.12:0/73444100 learned_addr learned my addr 172.21.15.12:0/73444100 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:52.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd674d32640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd660004d10 0x7fd6701ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd66ffff640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6700c4e50 0x7fd6700c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd675533640 1 -- 172.21.15.12:0/73444100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd660004d10 msgr2=0x7fd6701ef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd675533640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd660004d10 0x7fd6701ef940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd675533640 1 -- 172.21.15.12:0/73444100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6700c4e50 msgr2=0x7fd6700c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd675533640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6700c4e50 0x7fd6700c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd675533640 1 -- 172.21.15.12:0/73444100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6701f0060 con 0x7fd67014f7f0 2023-11-28T21:34:52.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd66ffff640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6700c4e50 0x7fd6700c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:52.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd675533640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67014f7f0 0x7fd67014fc70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6680987d0 tx=0x7fd668009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:52.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd66d7fa640 1 -- 172.21.15.12:0/73444100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6680039a0 con 0x7fd67014f7f0 2023-11-28T21:34:52.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd66d7fa640 1 -- 172.21.15.12:0/73444100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd668003b20 con 0x7fd67014f7f0 2023-11-28T21:34:52.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd66d7fa640 1 -- 172.21.15.12:0/73444100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6680ae890 con 0x7fd67014f7f0 2023-11-28T21:34:52.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd676f96640 1 -- 172.21.15.12:0/73444100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6701f0340 con 0x7fd67014f7f0 2023-11-28T21:34:52.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd676f96640 1 -- 172.21.15.12:0/73444100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67014eff0 con 0x7fd67014f7f0 2023-11-28T21:34:52.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.588+0000 7fd676f96640 1 -- 172.21.15.12:0/73444100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67014ee50 con 0x7fd67014f7f0 2023-11-28T21:34:52.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.592+0000 7fd66d7fa640 1 -- 172.21.15.12:0/73444100 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd6680b4040 con 0x7fd67014f7f0 2023-11-28T21:34:52.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.592+0000 7fd66d7fa640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd654066470 0x7fd654068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.592+0000 7fd66d7fa640 1 -- 172.21.15.12:0/73444100 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd66811e180 con 0x7fd67014f7f0 2023-11-28T21:34:52.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.592+0000 7fd674d32640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd654066470 0x7fd654068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.592+0000 7fd674d32640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd654066470 0x7fd654068930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd664000c10 tx=0x7fd664007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:52.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.596+0000 7fd66d7fa640 1 -- 172.21.15.12:0/73444100 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd67014ee50 con 0x7fd67014f7f0 2023-11-28T21:34:52.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.764+0000 7fd676f96640 1 -- 172.21.15.12:0/73444100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd67014ee50 con 0x7fd67014f7f0 2023-11-28T21:34:52.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.764+0000 7fd66d7fa640 1 -- 172.21.15.12:0/73444100 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd67014ee50 con 0x7fd67014f7f0 2023-11-28T21:34:52.769 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2023-11-28T21:34:52.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 -- 172.21.15.12:0/73444100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd654066470 msgr2=0x7fd654068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd654066470 0x7fd654068930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd664000c10 tx=0x7fd664007040 comp rx=0 tx=0).stop 2023-11-28T21:34:52.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 -- 172.21.15.12:0/73444100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67014f7f0 msgr2=0x7fd67014fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67014f7f0 0x7fd67014fc70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6680987d0 tx=0x7fd668009750 comp rx=0 tx=0).stop 2023-11-28T21:34:52.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 -- 172.21.15.12:0/73444100 shutdown_connections 2023-11-28T21:34:52.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd654066470 0x7fd654068930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd67014f7f0 0x7fd67014fc70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6700c4e50 0x7fd6700c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 --2- 172.21.15.12:0/73444100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd660004d10 0x7fd6701ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 -- 172.21.15.12:0/73444100 >> 172.21.15.12:0/73444100 conn(0x7fd6700c8450 msgr2=0x7fd67006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:52.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 -- 172.21.15.12:0/73444100 shutdown_connections 2023-11-28T21:34:52.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.768+0000 7fd676f96640 1 -- 172.21.15.12:0/73444100 wait complete. 2023-11-28T21:34:52.790 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:34:52.790 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T21:34:52.800 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:34:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.980+0000 7ff62246b640 1 Processor -- start 2023-11-28T21:34:52.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.980+0000 7ff62246b640 1 -- start start 2023-11-28T21:34:52.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.980+0000 7ff62246b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff61c153ee0 0x7ff61c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.980+0000 7ff62246b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff61c06a5b0 0x7ff61c0c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.980+0000 7ff62246b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c0c6350 con 0x7ff61c070bd0 2023-11-28T21:34:52.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.980+0000 7ff62246b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c0c6b90 con 0x7ff61c153ee0 2023-11-28T21:34:52.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.980+0000 7ff62246b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c072410 con 0x7ff61c06a5b0 2023-11-28T21:34:52.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.980+0000 7ff61bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff61c153ee0 0x7ff61c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff61c153ee0 0x7ff61c15a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53626/0 (socket says 172.21.15.12:53626) 2023-11-28T21:34:52.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61bfff640 1 -- 172.21.15.12:0/290495355 learned_addr learned my addr 172.21.15.12:0/290495355 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:52.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61b7fe640 1 --2- 172.21.15.12:0/290495355 >> v2:172.21.15.144:3300/0 conn(0x7ff61c06a5b0 0x7ff61c0c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61bfff640 1 -- 172.21.15.12:0/290495355 >> v2:172.21.15.144:3300/0 conn(0x7ff61c06a5b0 msgr2=0x7ff61c0c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:52.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61bfff640 1 --2- 172.21.15.12:0/290495355 >> v2:172.21.15.144:3300/0 conn(0x7ff61c06a5b0 0x7ff61c0c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61bfff640 1 -- 172.21.15.12:0/290495355 >> v1:172.21.15.144:6789/0 conn(0x7ff61c070bd0 legacy=0x7ff61c071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:52.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61bfff640 1 -- 172.21.15.12:0/290495355 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c0c65d0 con 0x7ff61c153ee0 2023-11-28T21:34:52.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61bfff640 1 --2- 172.21.15.12:0/290495355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c153ee0 0x7ff61c15a680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff61c06c020 tx=0x7ff60c01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6065f7350afb3f72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:52.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61affd640 1 -- 172.21.15.12:0/290495355 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0a89b0 con 0x7ff61c153ee0 2023-11-28T21:34:52.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61affd640 1 -- 172.21.15.12:0/290495355 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c0a8b10 con 0x7ff61c153ee0 2023-11-28T21:34:52.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.984+0000 7ff61affd640 1 -- 172.21.15.12:0/290495355 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0a8ea0 con 0x7ff61c153ee0 2023-11-28T21:34:52.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 -- 172.21.15.12:0/290495355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c153ee0 msgr2=0x7ff61c15a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 --2- 172.21.15.12:0/290495355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c153ee0 0x7ff61c15a680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff61c06c020 tx=0x7ff60c01ecb0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 -- 172.21.15.12:0/290495355 shutdown_connections 2023-11-28T21:34:52.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 --2- 172.21.15.12:0/290495355 >> v2:172.21.15.144:3300/0 conn(0x7ff61c06a5b0 0x7ff61c0c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 --2- 172.21.15.12:0/290495355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c153ee0 0x7ff61c15a680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 -- 172.21.15.12:0/290495355 >> 172.21.15.12:0/290495355 conn(0x7ff61c0c8450 msgr2=0x7ff61c0c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:52.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 -- 172.21.15.12:0/290495355 shutdown_connections 2023-11-28T21:34:52.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 -- 172.21.15.12:0/290495355 wait complete. 2023-11-28T21:34:52.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 Processor -- start 2023-11-28T21:34:52.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 -- start start 2023-11-28T21:34:52.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff61c06a5b0 0x7ff61c159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c070bd0 0x7ff61c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff61c153ee0 0x7ff61c1f0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c155ff0 con 0x7ff61c070bd0 2023-11-28T21:34:52.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c158280 con 0x7ff61c06a5b0 2023-11-28T21:34:52.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff62246b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c1f0870 con 0x7ff61c153ee0 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff61b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c070bd0 0x7ff61c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff61b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c070bd0 0x7ff61c154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53634/0 (socket says 172.21.15.12:53634) 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff61b7fe640 1 -- 172.21.15.12:0/3816730474 learned_addr learned my addr 172.21.15.12:0/3816730474 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff61bfff640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff61c06a5b0 0x7ff61c159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff620a08640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff61c153ee0 0x7ff61c1f0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff61b7fe640 1 -- 172.21.15.12:0/3816730474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff61c153ee0 msgr2=0x7ff61c1f0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff61b7fe640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff61c153ee0 0x7ff61c1f0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff61b7fe640 1 -- 172.21.15.12:0/3816730474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff61c06a5b0 msgr2=0x7ff61c159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff61b7fe640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff61c06a5b0 0x7ff61c159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff61b7fe640 1 -- 172.21.15.12:0/3816730474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0094d0 con 0x7ff61c070bd0 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff620a08640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff61c153ee0 0x7ff61c1f0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:34:52.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.988+0000 7ff61b7fe640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c070bd0 0x7ff61c154580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff61000d800 tx=0x7ff61000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:52.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff618ff9640 1 -- 172.21.15.12:0/3816730474 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610018070 con 0x7ff61c070bd0 2023-11-28T21:34:52.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff618ff9640 1 -- 172.21.15.12:0/3816730474 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6100059a0 con 0x7ff61c070bd0 2023-11-28T21:34:52.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff618ff9640 1 -- 172.21.15.12:0/3816730474 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610005b50 con 0x7ff61c070bd0 2023-11-28T21:34:52.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff62246b640 1 -- 172.21.15.12:0/3816730474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c157a70 con 0x7ff61c070bd0 2023-11-28T21:34:52.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff62246b640 1 -- 172.21.15.12:0/3816730474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c157fa0 con 0x7ff61c070bd0 2023-11-28T21:34:52.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff62246b640 1 -- 172.21.15.12:0/3816730474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c074f60 con 0x7ff61c070bd0 2023-11-28T21:34:52.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff618ff9640 1 -- 172.21.15.12:0/3816730474 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff610005cb0 con 0x7ff61c070bd0 2023-11-28T21:34:52.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff618ff9640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6080664a0 0x7ff608068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:52.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff618ff9640 1 -- 172.21.15.12:0/3816730474 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff610083f90 con 0x7ff61c070bd0 2023-11-28T21:34:52.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff61bfff640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6080664a0 0x7ff608068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:52.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.992+0000 7ff61bfff640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6080664a0 0x7ff608068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0b2000 tx=0x7ff60c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:53.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:52.996+0000 7ff618ff9640 1 -- 172.21.15.12:0/3816730474 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6100b5280 con 0x7ff61c070bd0 2023-11-28T21:34:53.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.156+0000 7ff62246b640 1 -- 172.21.15.12:0/3816730474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff61c152d70 con 0x7ff61c070bd0 2023-11-28T21:34:53.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.156+0000 7ff618ff9640 1 -- 172.21.15.12:0/3816730474 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7ff6100511b0 con 0x7ff61c070bd0 2023-11-28T21:34:53.160 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:34:53.161 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:34:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 -- 172.21.15.12:0/3816730474 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6080664a0 msgr2=0x7ff608068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6080664a0 0x7ff608068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff60c0b2000 tx=0x7ff60c008040 comp rx=0 tx=0).stop 2023-11-28T21:34:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 -- 172.21.15.12:0/3816730474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c070bd0 msgr2=0x7ff61c154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c070bd0 0x7ff61c154580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff61000d800 tx=0x7ff61000dcc0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 -- 172.21.15.12:0/3816730474 shutdown_connections 2023-11-28T21:34:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6080664a0 0x7ff608068960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff61c153ee0 0x7ff61c1f0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff61c070bd0 0x7ff61c154580 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 --2- 172.21.15.12:0/3816730474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff61c06a5b0 0x7ff61c159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 -- 172.21.15.12:0/3816730474 >> 172.21.15.12:0/3816730474 conn(0x7ff61c0c8450 msgr2=0x7ff61c151710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 -- 172.21.15.12:0/3816730474 shutdown_connections 2023-11-28T21:34:53.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.160+0000 7ff62246b640 1 -- 172.21.15.12:0/3816730474 wait complete. 2023-11-28T21:34:53.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T21:34:29.878190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T21:34:34.039032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T21:34:35.369379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T21:34:39.373079+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T21:34:53.182 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T21:34:53.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43c5c91640 1 Processor -- start 2023-11-28T21:34:53.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43c5c91640 1 -- start start 2023-11-28T21:34:53.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43c5c91640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f43c0153ee0 0x7f43c015a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43c5c91640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f43c006a5b0 0x7f43c00c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43c5c91640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c00c6350 con 0x7f43c0070bd0 2023-11-28T21:34:53.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43c5c91640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c00c6b90 con 0x7f43c006a5b0 2023-11-28T21:34:53.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43c5c91640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c0072410 con 0x7f43c0153ee0 2023-11-28T21:34:53.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43beffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f43c006a5b0 0x7f43c00c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43beffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f43c006a5b0 0x7f43c00c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53648/0 (socket says 172.21.15.12:53648) 2023-11-28T21:34:53.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43beffd640 1 -- 172.21.15.12:0/399385969 learned_addr learned my addr 172.21.15.12:0/399385969 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:53.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43bf7fe640 1 --2- 172.21.15.12:0/399385969 >> v2:172.21.15.144:3300/0 conn(0x7f43c0153ee0 0x7f43c015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43beffd640 1 -- 172.21.15.12:0/399385969 >> v2:172.21.15.144:3300/0 conn(0x7f43c0153ee0 msgr2=0x7f43c015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:53.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43beffd640 1 --2- 172.21.15.12:0/399385969 >> v2:172.21.15.144:3300/0 conn(0x7f43c0153ee0 0x7f43c015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43beffd640 1 -- 172.21.15.12:0/399385969 >> v1:172.21.15.144:6789/0 conn(0x7f43c0070bd0 legacy=0x7f43c0071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.328+0000 7f43beffd640 1 -- 172.21.15.12:0/399385969 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c00c65d0 con 0x7f43c006a5b0 2023-11-28T21:34:53.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43beffd640 1 --2- 172.21.15.12:0/399385969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 0x7f43c00c7a80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f43c006c250 tx=0x7f43b401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5645ed76ba7686 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:53.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43be7fc640 1 -- 172.21.15.12:0/399385969 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b40a89f0 con 0x7f43c006a5b0 2023-11-28T21:34:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43be7fc640 1 -- 172.21.15.12:0/399385969 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b40a8b50 con 0x7f43c006a5b0 2023-11-28T21:34:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43be7fc640 1 -- 172.21.15.12:0/399385969 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b40b0440 con 0x7f43c006a5b0 2023-11-28T21:34:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43c5c91640 1 -- 172.21.15.12:0/399385969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 msgr2=0x7f43c00c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43c5c91640 1 --2- 172.21.15.12:0/399385969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 0x7f43c00c7a80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f43c006c250 tx=0x7f43b401ecf0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43c5c91640 1 -- 172.21.15.12:0/399385969 shutdown_connections 2023-11-28T21:34:53.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43c5c91640 1 --2- 172.21.15.12:0/399385969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 0x7f43c00c7a80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43c5c91640 1 --2- 172.21.15.12:0/399385969 >> v2:172.21.15.144:3300/0 conn(0x7f43c0153ee0 0x7f43c015a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43c5c91640 1 -- 172.21.15.12:0/399385969 >> 172.21.15.12:0/399385969 conn(0x7f43c00c8450 msgr2=0x7f43c00c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:53.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43c5c91640 1 -- 172.21.15.12:0/399385969 shutdown_connections 2023-11-28T21:34:53.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.332+0000 7f43c5c91640 1 -- 172.21.15.12:0/399385969 wait complete. 2023-11-28T21:34:53.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43c5c91640 1 Processor -- start 2023-11-28T21:34:53.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43c5c91640 1 -- start start 2023-11-28T21:34:53.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43c5c91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 0x7f43c01582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43c5c91640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43c0070bd0 0x7f43c00c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43c5c91640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43c0153ee0 0x7f43c00c3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43c5c91640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c01f0060 con 0x7f43c006a5b0 2023-11-28T21:34:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43c5c91640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c01f1030 con 0x7f43c0153ee0 2023-11-28T21:34:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43c5c91640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c00c39b0 con 0x7f43c0070bd0 2023-11-28T21:34:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43bf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 0x7f43c01582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43bf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 0x7f43c01582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53660/0 (socket says 172.21.15.12:53660) 2023-11-28T21:34:53.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3372423612 learned_addr learned my addr 172.21.15.12:0/3372423612 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:53.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43bffff640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43c0153ee0 0x7f43c00c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43beffd640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43c0070bd0 0x7f43c00c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3372423612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43c0070bd0 msgr2=0x7f43c00c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43bf7fe640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43c0070bd0 0x7f43c00c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3372423612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43c0153ee0 msgr2=0x7f43c00c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43bf7fe640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43c0153ee0 0x7f43c00c3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.336+0000 7f43bf7fe640 1 -- 172.21.15.12:0/3372423612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b4009480 con 0x7f43c006a5b0 2023-11-28T21:34:53.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43bf7fe640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 0x7f43c01582a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f43b000e8a0 tx=0x7f43b000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:53.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43a7fff640 1 -- 172.21.15.12:0/3372423612 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0009510 con 0x7f43c006a5b0 2023-11-28T21:34:53.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43a7fff640 1 -- 172.21.15.12:0/3372423612 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b000c940 con 0x7f43c006a5b0 2023-11-28T21:34:53.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43a7fff640 1 -- 172.21.15.12:0/3372423612 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000ccb0 con 0x7f43c006a5b0 2023-11-28T21:34:53.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43c5c91640 1 -- 172.21.15.12:0/3372423612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c0157c40 con 0x7f43c006a5b0 2023-11-28T21:34:53.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43c5c91640 1 -- 172.21.15.12:0/3372423612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c01f0550 con 0x7f43c006a5b0 2023-11-28T21:34:53.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43c5c91640 1 -- 172.21.15.12:0/3372423612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c0074f60 con 0x7f43c006a5b0 2023-11-28T21:34:53.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43a7fff640 1 -- 172.21.15.12:0/3372423612 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f43b0013470 con 0x7f43c006a5b0 2023-11-28T21:34:53.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43a7fff640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f43ac066470 0x7f43ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43a7fff640 1 -- 172.21.15.12:0/3372423612 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f43b0084220 con 0x7f43c006a5b0 2023-11-28T21:34:53.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.340+0000 7f43beffd640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f43ac066470 0x7f43ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.344+0000 7f43beffd640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f43ac066470 0x7f43ac068930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43b40b2000 tx=0x7f43b40aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:53.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.348+0000 7f43a7fff640 1 -- 172.21.15.12:0/3372423612 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f43b00513d0 con 0x7f43c006a5b0 2023-11-28T21:34:53.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.508+0000 7f43c5c91640 1 -- 172.21.15.12:0/3372423612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43c00667a0 con 0x7f43c006a5b0 2023-11-28T21:34:53.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.508+0000 7f43a7fff640 1 -- 172.21.15.12:0/3372423612 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43c00667a0 con 0x7f43c006a5b0 2023-11-28T21:34:53.511 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-11-28T21:34:53.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 -- 172.21.15.12:0/3372423612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f43ac066470 msgr2=0x7f43ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f43ac066470 0x7f43ac068930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43b40b2000 tx=0x7f43b40aa070 comp rx=0 tx=0).stop 2023-11-28T21:34:53.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 -- 172.21.15.12:0/3372423612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 msgr2=0x7f43c01582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 0x7f43c01582a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f43b000e8a0 tx=0x7f43b000eca0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 -- 172.21.15.12:0/3372423612 shutdown_connections 2023-11-28T21:34:53.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f43ac066470 0x7f43ac068930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43c0153ee0 0x7f43c00c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43c0070bd0 0x7f43c00c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 --2- 172.21.15.12:0/3372423612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f43c006a5b0 0x7f43c01582a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 -- 172.21.15.12:0/3372423612 >> 172.21.15.12:0/3372423612 conn(0x7f43c00c8450 msgr2=0x7f43c0071030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:53.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 -- 172.21.15.12:0/3372423612 shutdown_connections 2023-11-28T21:34:53.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.512+0000 7f43c5c91640 1 -- 172.21.15.12:0/3372423612 wait complete. 2023-11-28T21:34:53.533 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T21:34:53.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.676+0000 7f99cb879640 1 Processor -- start 2023-11-28T21:34:53.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.676+0000 7f99cb879640 1 -- start start 2023-11-28T21:34:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.676+0000 7f99cb879640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99c4072750 0x7f99c40c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.676+0000 7f99cb879640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f99c40c8010 0x7f99c4151910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.676+0000 7f99cb879640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c40c7b80 con 0x7f99c4072b30 2023-11-28T21:34:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.676+0000 7f99cb879640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c40c2b10 con 0x7f99c40c8010 2023-11-28T21:34:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.676+0000 7f99cb879640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40c50d0 con 0x7f99c4072750 2023-11-28T21:34:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99c8e14640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f99c40c8010 0x7f99c4151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99c9e16640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f99c4072b30 0x7f99c4151e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43418/0 (socket says 172.21.15.12:43418) 2023-11-28T21:34:53.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99c9e16640 1 -- 172.21.15.12:0/3673928683 learned_addr learned my addr 172.21.15.12:0/3673928683 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:53.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99c9615640 1 --2- 172.21.15.12:0/3673928683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99c4072750 0x7f99c40c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99c8e14640 1 -- 172.21.15.12:0/3673928683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99c4072750 msgr2=0x7f99c40c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:53.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99c8e14640 1 --2- 172.21.15.12:0/3673928683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99c4072750 0x7f99c40c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99c8e14640 1 -- 172.21.15.12:0/3673928683 >> v1:172.21.15.12:6789/0 conn(0x7f99c4072b30 legacy=0x7f99c4151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99bbfff640 1 -- 172.21.15.12:0/3673928683 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110547784 0 0) 0x7f99c40c7b80 con 0x7f99c4072b30 2023-11-28T21:34:53.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99c8e14640 1 -- 172.21.15.12:0/3673928683 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4066990 con 0x7f99c40c8010 2023-11-28T21:34:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99c8e14640 1 --2- 172.21.15.12:0/3673928683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 0x7f99c4151910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99b40097a0 tx=0x7f99b401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f43b84890e58f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99bbfff640 1 -- 172.21.15.12:0/3673928683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b40a7070 con 0x7f99c40c8010 2023-11-28T21:34:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99bbfff640 1 -- 172.21.15.12:0/3673928683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b40ab430 con 0x7f99c40c8010 2023-11-28T21:34:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.680+0000 7f99bbfff640 1 -- 172.21.15.12:0/3673928683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b40ab7c0 con 0x7f99c40c8010 2023-11-28T21:34:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 -- 172.21.15.12:0/3673928683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 msgr2=0x7f99c4151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 --2- 172.21.15.12:0/3673928683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 0x7f99c4151910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99b40097a0 tx=0x7f99b401c430 comp rx=0 tx=0).stop 2023-11-28T21:34:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 -- 172.21.15.12:0/3673928683 shutdown_connections 2023-11-28T21:34:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 --2- 172.21.15.12:0/3673928683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 0x7f99c4151910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 --2- 172.21.15.12:0/3673928683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99c4072750 0x7f99c40c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 -- 172.21.15.12:0/3673928683 >> 172.21.15.12:0/3673928683 conn(0x7f99c40c8450 msgr2=0x7f99c40c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 -- 172.21.15.12:0/3673928683 shutdown_connections 2023-11-28T21:34:53.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 -- 172.21.15.12:0/3673928683 wait complete. 2023-11-28T21:34:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 Processor -- start 2023-11-28T21:34:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 -- start start 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99c4072750 0x7f99c414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c4072b30 0x7f99c4071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 0x7f99c41f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c41f5b60 con 0x7f99c40c8010 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c4150f90 con 0x7f99c4072b30 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99cb879640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c4151250 con 0x7f99c4072750 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c9e16640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 0x7f99c41f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c9e16640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 0x7f99c41f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53672/0 (socket says 172.21.15.12:53672) 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c9e16640 1 -- 172.21.15.12:0/4188778190 learned_addr learned my addr 172.21.15.12:0/4188778190 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c8e14640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c4072b30 0x7f99c4071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c9615640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99c4072750 0x7f99c414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c9e16640 1 -- 172.21.15.12:0/4188778190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99c4072750 msgr2=0x7f99c414f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c9e16640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99c4072750 0x7f99c414f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c9e16640 1 -- 172.21.15.12:0/4188778190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c4072b30 msgr2=0x7f99c4071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c9e16640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c4072b30 0x7f99c4071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c9e16640 1 -- 172.21.15.12:0/4188778190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b40093b0 con 0x7f99c40c8010 2023-11-28T21:34:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.684+0000 7f99c8e14640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c4072b30 0x7f99c4071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:34:53.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99c9e16640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 0x7f99c41f25f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99bc001570 tx=0x7f99bc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99b9ffb640 1 -- 172.21.15.12:0/4188778190 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00a640 con 0x7f99c40c8010 2023-11-28T21:34:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99b9ffb640 1 -- 172.21.15.12:0/4188778190 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc00a7a0 con 0x7f99c40c8010 2023-11-28T21:34:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99b9ffb640 1 -- 172.21.15.12:0/4188778190 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00ab90 con 0x7f99c40c8010 2023-11-28T21:34:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99cb879640 1 -- 172.21.15.12:0/4188778190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c41f5db0 con 0x7f99c40c8010 2023-11-28T21:34:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99cb879640 1 -- 172.21.15.12:0/4188778190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c41f6370 con 0x7f99c40c8010 2023-11-28T21:34:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99cb879640 1 -- 172.21.15.12:0/4188778190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c41f6560 con 0x7f99c40c8010 2023-11-28T21:34:53.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99b9ffb640 1 -- 172.21.15.12:0/4188778190 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f99bc00acf0 con 0x7f99c40c8010 2023-11-28T21:34:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99b9ffb640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99a80664c0 0x7f99a8068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99b9ffb640 1 -- 172.21.15.12:0/4188778190 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f99bc0839e0 con 0x7f99c40c8010 2023-11-28T21:34:53.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.688+0000 7f99c9615640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99a80664c0 0x7f99a8068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:53.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.692+0000 7f99c9615640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99a80664c0 0x7f99a8068980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99c000b9d0 tx=0x7f99c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:53.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.692+0000 7f99b9ffb640 1 -- 172.21.15.12:0/4188778190 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f99bc050c00 con 0x7f99c40c8010 2023-11-28T21:34:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.848+0000 7f99cb879640 1 -- 172.21.15.12:0/4188778190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99c4074f60 con 0x7f99c40c8010 2023-11-28T21:34:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99b9ffb640 1 -- 172.21.15.12:0/4188778190 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f99bc018020 con 0x7f99c40c8010 2023-11-28T21:34:53.855 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:34:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 -- 172.21.15.12:0/4188778190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99a80664c0 msgr2=0x7f99a8068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99a80664c0 0x7f99a8068980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99c000b9d0 tx=0x7f99c0008040 comp rx=0 tx=0).stop 2023-11-28T21:34:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 -- 172.21.15.12:0/4188778190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 msgr2=0x7f99c41f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 0x7f99c41f25f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99bc001570 tx=0x7f99bc00cc70 comp rx=0 tx=0).stop 2023-11-28T21:34:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 -- 172.21.15.12:0/4188778190 shutdown_connections 2023-11-28T21:34:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99a80664c0 0x7f99a8068980 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99c40c8010 0x7f99c41f25f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99c4072b30 0x7f99c4071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 --2- 172.21.15.12:0/4188778190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99c4072750 0x7f99c414f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 -- 172.21.15.12:0/4188778190 >> 172.21.15.12:0/4188778190 conn(0x7f99c40c8450 msgr2=0x7f99c4160660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 -- 172.21.15.12:0/4188778190 shutdown_connections 2023-11-28T21:34:53.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:53.852+0000 7f99cb879640 1 -- 172.21.15.12:0/4188778190 wait complete. 2023-11-28T21:34:53.875 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T21:34:54.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.016+0000 7f5a45fb8640 1 Processor -- start 2023-11-28T21:34:54.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.016+0000 7f5a45fb8640 1 -- start start 2023-11-28T21:34:54.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.020+0000 7f5a45fb8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5a4015a910 0x7f5a40070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.020+0000 7f5a45fb8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4006f050 con 0x7f5a400710e0 2023-11-28T21:34:54.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.020+0000 7f5a45fb8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4006f580 con 0x7f5a40143070 2023-11-28T21:34:54.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.020+0000 7f5a45fb8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4015e330 con 0x7f5a4015a910 2023-11-28T21:34:54.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.020+0000 7f5a3effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5a4015a910 0x7f5a40070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.020+0000 7f5a3ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5a400710e0 0x7f5a400714f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43426/0 (socket says 172.21.15.12:43426) 2023-11-28T21:34:54.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.020+0000 7f5a3ffff640 1 -- 172.21.15.12:0/4179131332 learned_addr learned my addr 172.21.15.12:0/4179131332 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:34:54.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.020+0000 7f5a3effd640 1 -- 172.21.15.12:0/4179131332 >> v1:172.21.15.144:6789/0 conn(0x7f5a40143070 legacy=0x7f5a4015b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.020+0000 7f5a3effd640 1 -- 172.21.15.12:0/4179131332 >> v1:172.21.15.12:6789/0 conn(0x7f5a400710e0 legacy=0x7f5a400714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.020+0000 7f5a3effd640 1 -- 172.21.15.12:0/4179131332 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4006f2a0 con 0x7f5a4015a910 2023-11-28T21:34:54.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a3effd640 1 --2- 172.21.15.12:0/4179131332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 0x7f5a40070ba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5a340097c0 tx=0x7f5a3401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b9f590a66a442e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a3e7fc640 1 -- 172.21.15.12:0/4179131332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a340ab650 con 0x7f5a4015a910 2023-11-28T21:34:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a3e7fc640 1 -- 172.21.15.12:0/4179131332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a340ab7b0 con 0x7f5a4015a910 2023-11-28T21:34:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a3e7fc640 1 -- 172.21.15.12:0/4179131332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a340abb40 con 0x7f5a4015a910 2023-11-28T21:34:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a45fb8640 1 -- 172.21.15.12:0/4179131332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 msgr2=0x7f5a40070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a45fb8640 1 --2- 172.21.15.12:0/4179131332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 0x7f5a40070ba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5a340097c0 tx=0x7f5a3401c450 comp rx=0 tx=0).stop 2023-11-28T21:34:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a45fb8640 1 -- 172.21.15.12:0/4179131332 shutdown_connections 2023-11-28T21:34:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a45fb8640 1 --2- 172.21.15.12:0/4179131332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 0x7f5a40070ba0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a45fb8640 1 -- 172.21.15.12:0/4179131332 >> 172.21.15.12:0/4179131332 conn(0x7f5a4006b6c0 msgr2=0x7f5a4006bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a45fb8640 1 -- 172.21.15.12:0/4179131332 shutdown_connections 2023-11-28T21:34:54.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a45fb8640 1 -- 172.21.15.12:0/4179131332 wait complete. 2023-11-28T21:34:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a45fb8640 1 Processor -- start 2023-11-28T21:34:54.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.024+0000 7f5a45fb8640 1 -- start start 2023-11-28T21:34:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a45fb8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a400710e0 0x7f5a4013c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a45fb8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a40143070 0x7f5a4013cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a45fb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 0x7f5a401fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a45fb8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a401378c0 con 0x7f5a4015a910 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a45fb8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4013c3f0 con 0x7f5a40143070 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a45fb8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4013de70 con 0x7f5a400710e0 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 0x7f5a401fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 0x7f5a401fa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53686/0 (socket says 172.21.15.12:53686) 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3ffff640 1 -- 172.21.15.12:0/3267467723 learned_addr learned my addr 172.21.15.12:0/3267467723 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3effd640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a40143070 0x7f5a4013cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3f7fe640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a400710e0 0x7f5a4013c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3ffff640 1 -- 172.21.15.12:0/3267467723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a400710e0 msgr2=0x7f5a4013c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3ffff640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a400710e0 0x7f5a4013c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3ffff640 1 -- 172.21.15.12:0/3267467723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a40143070 msgr2=0x7f5a4013cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3ffff640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a40143070 0x7f5a4013cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3ffff640 1 -- 172.21.15.12:0/3267467723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a34009420 con 0x7f5a4015a910 2023-11-28T21:34:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a3ffff640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 0x7f5a401fa4c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800ca80 tx=0x7f5a3800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a27fff640 1 -- 172.21.15.12:0/3267467723 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800ac30 con 0x7f5a4015a910 2023-11-28T21:34:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a27fff640 1 -- 172.21.15.12:0/3267467723 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800ad90 con 0x7f5a4015a910 2023-11-28T21:34:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a27fff640 1 -- 172.21.15.12:0/3267467723 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38012600 con 0x7f5a4015a910 2023-11-28T21:34:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a45fb8640 1 -- 172.21.15.12:0/3267467723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a40135570 con 0x7f5a4015a910 2023-11-28T21:34:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a45fb8640 1 -- 172.21.15.12:0/3267467723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4013e0f0 con 0x7f5a4015a910 2023-11-28T21:34:54.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a45fb8640 1 -- 172.21.15.12:0/3267467723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4013c5d0 con 0x7f5a4015a910 2023-11-28T21:34:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.028+0000 7f5a27fff640 1 -- 172.21.15.12:0/3267467723 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5a38007070 con 0x7f5a4015a910 2023-11-28T21:34:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.032+0000 7f5a27fff640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a2c066470 0x7f5a2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.032+0000 7f5a27fff640 1 -- 172.21.15.12:0/3267467723 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5a380840b0 con 0x7f5a4015a910 2023-11-28T21:34:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.032+0000 7f5a3f7fe640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a2c066470 0x7f5a2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.032+0000 7f5a3f7fe640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a2c066470 0x7f5a2c068930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5a30000f70 tx=0x7f5a30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:54.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.036+0000 7f5a27fff640 1 -- 172.21.15.12:0/3267467723 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5a4013c5d0 con 0x7f5a4015a910 2023-11-28T21:34:54.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 -- 172.21.15.12:0/3267467723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a4013c5d0 con 0x7f5a4015a910 2023-11-28T21:34:54.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a27fff640 1 -- 172.21.15.12:0/3267467723 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a4013c5d0 con 0x7f5a4015a910 2023-11-28T21:34:54.195 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 -- 172.21.15.12:0/3267467723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a2c066470 msgr2=0x7f5a2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a2c066470 0x7f5a2c068930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5a30000f70 tx=0x7f5a30009040 comp rx=0 tx=0).stop 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 -- 172.21.15.12:0/3267467723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 msgr2=0x7f5a401fa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 0x7f5a401fa4c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800ca80 tx=0x7f5a3800cf40 comp rx=0 tx=0).stop 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 -- 172.21.15.12:0/3267467723 shutdown_connections 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a2c066470 0x7f5a2c068930 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a4015a910 0x7f5a401fa4c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a40143070 0x7f5a4013cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 --2- 172.21.15.12:0/3267467723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a400710e0 0x7f5a4013c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 -- 172.21.15.12:0/3267467723 >> 172.21.15.12:0/3267467723 conn(0x7f5a4006b6c0 msgr2=0x7f5a4006de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 -- 172.21.15.12:0/3267467723 shutdown_connections 2023-11-28T21:34:54.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.192+0000 7f5a45fb8640 1 -- 172.21.15.12:0/3267467723 wait complete. 2023-11-28T21:34:54.216 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T21:34:54.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3597b8640 1 Processor -- start 2023-11-28T21:34:54.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3597b8640 1 -- start start 2023-11-28T21:34:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3597b8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd354154100 0x7fd35415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3597b8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd3540ca3f0 0x7fd3540c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3597b8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35415e270 con 0x7fd3540c9fd0 2023-11-28T21:34:54.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3597b8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3540c4e60 con 0x7fd3540ca3f0 2023-11-28T21:34:54.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3597b8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3541544e0 con 0x7fd354154100 2023-11-28T21:34:54.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3537fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd3540ca3f0 0x7fd3540c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3537fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd3540ca3f0 0x7fd3540c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53694/0 (socket says 172.21.15.12:53694) 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3537fe640 1 -- 172.21.15.12:0/2240016743 learned_addr learned my addr 172.21.15.12:0/2240016743 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd352ffd640 1 --2- 172.21.15.12:0/2240016743 >> v2:172.21.15.144:3300/0 conn(0x7fd354154100 0x7fd35415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3537fe640 1 -- 172.21.15.12:0/2240016743 >> v2:172.21.15.144:3300/0 conn(0x7fd354154100 msgr2=0x7fd35415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3537fe640 1 --2- 172.21.15.12:0/2240016743 >> v2:172.21.15.144:3300/0 conn(0x7fd354154100 0x7fd35415d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3537fe640 1 -- 172.21.15.12:0/2240016743 >> v1:172.21.15.144:6789/0 conn(0x7fd3540c9fd0 legacy=0x7fd35406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.360+0000 7fd3537fe640 1 -- 172.21.15.12:0/2240016743 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35415d9d0 con 0x7fd3540ca3f0 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3537fe640 1 --2- 172.21.15.12:0/2240016743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540ca3f0 0x7fd3540c52f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd354067120 tx=0x7fd34c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07587f726907b2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3527fc640 1 -- 172.21.15.12:0/2240016743 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0a5860 con 0x7fd3540ca3f0 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3527fc640 1 -- 172.21.15.12:0/2240016743 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c0a59c0 con 0x7fd3540ca3f0 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3527fc640 1 -- 172.21.15.12:0/2240016743 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0a5d50 con 0x7fd3540ca3f0 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3597b8640 1 -- 172.21.15.12:0/2240016743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540ca3f0 msgr2=0x7fd3540c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3597b8640 1 --2- 172.21.15.12:0/2240016743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540ca3f0 0x7fd3540c52f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd354067120 tx=0x7fd34c01c9a0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3597b8640 1 -- 172.21.15.12:0/2240016743 shutdown_connections 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3597b8640 1 --2- 172.21.15.12:0/2240016743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540ca3f0 0x7fd3540c52f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3597b8640 1 --2- 172.21.15.12:0/2240016743 >> v2:172.21.15.144:3300/0 conn(0x7fd354154100 0x7fd35415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3597b8640 1 -- 172.21.15.12:0/2240016743 >> 172.21.15.12:0/2240016743 conn(0x7fd35406c690 msgr2=0x7fd354066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3597b8640 1 -- 172.21.15.12:0/2240016743 shutdown_connections 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.364+0000 7fd3597b8640 1 -- 172.21.15.12:0/2240016743 wait complete. 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3597b8640 1 Processor -- start 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3597b8640 1 -- start start 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3597b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540c9fd0 0x7fd354149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3597b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3540ca3f0 0x7fd354143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3597b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd354154100 0x7fd3541443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3597b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35415ab10 con 0x7fd3540c9fd0 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3597b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd354145fb0 con 0x7fd3540ca3f0 2023-11-28T21:34:54.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3597b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35415ad20 con 0x7fd354154100 2023-11-28T21:34:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd352ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540c9fd0 0x7fd354149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd352ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540c9fd0 0x7fd354149560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53702/0 (socket says 172.21.15.12:53702) 2023-11-28T21:34:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd352ffd640 1 -- 172.21.15.12:0/1132897893 learned_addr learned my addr 172.21.15.12:0/1132897893 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd34bfff640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3540ca3f0 0x7fd354143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3537fe640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd354154100 0x7fd3541443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd352ffd640 1 -- 172.21.15.12:0/1132897893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd354154100 msgr2=0x7fd3541443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd352ffd640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd354154100 0x7fd3541443e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd352ffd640 1 -- 172.21.15.12:0/1132897893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3540ca3f0 msgr2=0x7fd354143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd352ffd640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3540ca3f0 0x7fd354143ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd352ffd640 1 -- 172.21.15.12:0/1132897893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c009b30 con 0x7fd3540c9fd0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd352ffd640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540c9fd0 0x7fd354149560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd34400e870 tx=0x7fd34400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd34b7fe640 1 -- 172.21.15.12:0/1132897893 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344009510 con 0x7fd3540c9fd0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd34b7fe640 1 -- 172.21.15.12:0/1132897893 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34400eec0 con 0x7fd3540c9fd0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd34b7fe640 1 -- 172.21.15.12:0/1132897893 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34400cb40 con 0x7fd3540c9fd0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3597b8640 1 -- 172.21.15.12:0/1132897893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd354146270 con 0x7fd3540c9fd0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.368+0000 7fd3597b8640 1 -- 172.21.15.12:0/1132897893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd354146830 con 0x7fd3540c9fd0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.372+0000 7fd3597b8640 1 -- 172.21.15.12:0/1132897893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd354067c40 con 0x7fd3540c9fd0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.372+0000 7fd34b7fe640 1 -- 172.21.15.12:0/1132897893 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd34400cca0 con 0x7fd3540c9fd0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.372+0000 7fd34b7fe640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3340664a0 0x7fd334068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.372+0000 7fd34b7fe640 1 -- 172.21.15.12:0/1132897893 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd344084200 con 0x7fd3540c9fd0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.372+0000 7fd34bfff640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3340664a0 0x7fd334068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.372+0000 7fd34bfff640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3340664a0 0x7fd334068960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd3541518b0 tx=0x7fd33c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:54.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.376+0000 7fd34b7fe640 1 -- 172.21.15.12:0/1132897893 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd344051380 con 0x7fd3540c9fd0 2023-11-28T21:34:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 -- 172.21.15.12:0/1132897893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd354074f60 con 0x7fd3540c9fd0 2023-11-28T21:34:54.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd34b7fe640 1 -- 172.21.15.12:0/1132897893 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd3440549a0 con 0x7fd3540c9fd0 2023-11-28T21:34:54.539 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:34:54.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 -- 172.21.15.12:0/1132897893 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3340664a0 msgr2=0x7fd334068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3340664a0 0x7fd334068960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd3541518b0 tx=0x7fd33c009040 comp rx=0 tx=0).stop 2023-11-28T21:34:54.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 -- 172.21.15.12:0/1132897893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540c9fd0 msgr2=0x7fd354149560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540c9fd0 0x7fd354149560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd34400e870 tx=0x7fd34400ec70 comp rx=0 tx=0).stop 2023-11-28T21:34:54.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 -- 172.21.15.12:0/1132897893 shutdown_connections 2023-11-28T21:34:54.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3340664a0 0x7fd334068960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd354154100 0x7fd3541443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3540ca3f0 0x7fd354143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 --2- 172.21.15.12:0/1132897893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3540c9fd0 0x7fd354149560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 -- 172.21.15.12:0/1132897893 >> 172.21.15.12:0/1132897893 conn(0x7fd35406c690 msgr2=0x7fd354151ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 -- 172.21.15.12:0/1132897893 shutdown_connections 2023-11-28T21:34:54.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.536+0000 7fd3597b8640 1 -- 172.21.15.12:0/1132897893 wait complete. 2023-11-28T21:34:54.560 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T21:34:54.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f6446780640 1 Processor -- start 2023-11-28T21:34:54.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f6446780640 1 -- start start 2023-11-28T21:34:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f6446780640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6440150c20 0x7f6440156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f6446780640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f644006e300 0x7f64400c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f6446780640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6440067410 con 0x7f644006cf40 2023-11-28T21:34:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f6446780640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f644006e090 con 0x7f6440150c20 2023-11-28T21:34:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f6446780640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6440157f30 con 0x7f644006e300 2023-11-28T21:34:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f643ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6440150c20 0x7f6440156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f643ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6440150c20 0x7f6440156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54292/0 (socket says 172.21.15.12:54292) 2023-11-28T21:34:54.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f643ffff640 1 -- 172.21.15.12:0/3099044700 learned_addr learned my addr 172.21.15.12:0/3099044700 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f643f7fe640 1 --2- 172.21.15.12:0/3099044700 >> v2:172.21.15.144:3300/0 conn(0x7f644006e300 0x7f64400c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f643ffff640 1 -- 172.21.15.12:0/3099044700 >> v2:172.21.15.144:3300/0 conn(0x7f644006e300 msgr2=0x7f64400c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f643ffff640 1 --2- 172.21.15.12:0/3099044700 >> v2:172.21.15.144:3300/0 conn(0x7f644006e300 0x7f64400c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f643ffff640 1 -- 172.21.15.12:0/3099044700 >> v1:172.21.15.144:6789/0 conn(0x7f644006cf40 legacy=0x7f6440158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:34:54.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.704+0000 7f643ffff640 1 -- 172.21.15.12:0/3099044700 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6440151000 con 0x7f6440150c20 2023-11-28T21:34:54.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f643ffff640 1 --2- 172.21.15.12:0/3099044700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440150c20 0x7f6440156910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f64400c8e60 tx=0x7f643001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb18841580da7d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:54.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f643effd640 1 -- 172.21.15.12:0/3099044700 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64300ac410 con 0x7f6440150c20 2023-11-28T21:34:54.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f643effd640 1 -- 172.21.15.12:0/3099044700 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64300ac570 con 0x7f6440150c20 2023-11-28T21:34:54.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f643effd640 1 -- 172.21.15.12:0/3099044700 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64300ac900 con 0x7f6440150c20 2023-11-28T21:34:54.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f6446780640 1 -- 172.21.15.12:0/3099044700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440150c20 msgr2=0x7f6440156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f6446780640 1 --2- 172.21.15.12:0/3099044700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440150c20 0x7f6440156910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f64400c8e60 tx=0x7f643001c4c0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f6446780640 1 -- 172.21.15.12:0/3099044700 shutdown_connections 2023-11-28T21:34:54.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f6446780640 1 --2- 172.21.15.12:0/3099044700 >> v2:172.21.15.144:3300/0 conn(0x7f644006e300 0x7f64400c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f6446780640 1 --2- 172.21.15.12:0/3099044700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440150c20 0x7f6440156910 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f6446780640 1 -- 172.21.15.12:0/3099044700 >> 172.21.15.12:0/3099044700 conn(0x7f6440066360 msgr2=0x7f6440066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:54.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f6446780640 1 -- 172.21.15.12:0/3099044700 shutdown_connections 2023-11-28T21:34:54.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.708+0000 7f6446780640 1 -- 172.21.15.12:0/3099044700 wait complete. 2023-11-28T21:34:54.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6446780640 1 Processor -- start 2023-11-28T21:34:54.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6446780640 1 -- start start 2023-11-28T21:34:54.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6446780640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644006cf40 0x7f64401480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6446780640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644006e300 0x7f64401eec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6446780640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6440150c20 0x7f6440143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6446780640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644014fe80 con 0x7f644006e300 2023-11-28T21:34:54.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6446780640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6440150140 con 0x7f644006cf40 2023-11-28T21:34:54.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6446780640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6440150420 con 0x7f6440150c20 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644006e300 0x7f64401eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644006e300 0x7f64401eec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54298/0 (socket says 172.21.15.12:54298) 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643f7fe640 1 -- 172.21.15.12:0/3760400278 learned_addr learned my addr 172.21.15.12:0/3760400278 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643ffff640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644006cf40 0x7f64401480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6444d1d640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6440150c20 0x7f6440143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643f7fe640 1 -- 172.21.15.12:0/3760400278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6440150c20 msgr2=0x7f6440143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643f7fe640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6440150c20 0x7f6440143c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643f7fe640 1 -- 172.21.15.12:0/3760400278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644006cf40 msgr2=0x7f64401480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643f7fe640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644006cf40 0x7f64401480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643f7fe640 1 -- 172.21.15.12:0/3760400278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300094d0 con 0x7f644006e300 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643f7fe640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644006e300 0x7f64401eec80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f643400d7d0 tx=0x7f643400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:54.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643cff9640 1 -- 172.21.15.12:0/3760400278 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434018070 con 0x7f644006e300 2023-11-28T21:34:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643cff9640 1 -- 172.21.15.12:0/3760400278 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6434004020 con 0x7f644006e300 2023-11-28T21:34:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f643cff9640 1 -- 172.21.15.12:0/3760400278 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64340041d0 con 0x7f644006e300 2023-11-28T21:34:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6446780640 1 -- 172.21.15.12:0/3760400278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64401f21f0 con 0x7f644006e300 2023-11-28T21:34:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.712+0000 7f6446780640 1 -- 172.21.15.12:0/3760400278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64401f27b0 con 0x7f644006e300 2023-11-28T21:34:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.716+0000 7f6446780640 1 -- 172.21.15.12:0/3760400278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6440074f60 con 0x7f644006e300 2023-11-28T21:34:54.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.716+0000 7f643cff9640 1 -- 172.21.15.12:0/3760400278 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6434004330 con 0x7f644006e300 2023-11-28T21:34:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.716+0000 7f643cff9640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f642c066470 0x7f642c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:34:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.716+0000 7f643cff9640 1 -- 172.21.15.12:0/3760400278 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6434084180 con 0x7f644006e300 2023-11-28T21:34:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.716+0000 7f643ffff640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f642c066470 0x7f642c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:34:54.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.716+0000 7f643ffff640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f642c066470 0x7f642c068930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64300ae000 tx=0x7f6430008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:34:54.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.720+0000 7f643cff9640 1 -- 172.21.15.12:0/3760400278 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f64340511e0 con 0x7f644006e300 2023-11-28T21:34:54.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 -- 172.21.15.12:0/3760400278 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f64400c2800 con 0x7f644006e300 2023-11-28T21:34:54.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f643cff9640 1 -- 172.21.15.12:0/3760400278 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6434054800 con 0x7f644006e300 2023-11-28T21:34:54.895 INFO:teuthology.orchestra.run.smithi012.stderr:listed 0 entries 2023-11-28T21:34:54.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 -- 172.21.15.12:0/3760400278 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f642c066470 msgr2=0x7f642c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f642c066470 0x7f642c068930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64300ae000 tx=0x7f6430008040 comp rx=0 tx=0).stop 2023-11-28T21:34:54.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 -- 172.21.15.12:0/3760400278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644006e300 msgr2=0x7f64401eec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:34:54.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644006e300 0x7f64401eec80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f643400d7d0 tx=0x7f643400dc90 comp rx=0 tx=0).stop 2023-11-28T21:34:54.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 -- 172.21.15.12:0/3760400278 shutdown_connections 2023-11-28T21:34:54.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f642c066470 0x7f642c068930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6440150c20 0x7f6440143c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644006e300 0x7f64401eec80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 --2- 172.21.15.12:0/3760400278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f644006cf40 0x7f64401480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:34:54.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.892+0000 7f6446780640 1 -- 172.21.15.12:0/3760400278 >> 172.21.15.12:0/3760400278 conn(0x7f6440066360 msgr2=0x7f64400c9780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:34:54.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.896+0000 7f6446780640 1 -- 172.21.15.12:0/3760400278 shutdown_connections 2023-11-28T21:34:54.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:34:54.896+0000 7f6446780640 1 -- 172.21.15.12:0/3760400278 wait complete. 2023-11-28T21:34:54.916 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi012.front.sepia.ceph.com... 2023-11-28T21:34:54.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:34:54.919 DEBUG:teuthology.orchestra.run.smithi012:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-28T21:34:54.950 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi012.stderr:ceph-fuse[42748]: fuse finished with error 0 and tester_r 0 2023-11-28T21:34:54.952 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:35:01.053 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:35:01.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:01.054 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-28T21:35:01.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:01.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b77972640 1 Processor -- start 2023-11-28T21:35:01.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b77972640 1 -- start start 2023-11-28T21:35:01.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b77972640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b70150c20 0x7f7b70156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b77972640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b7006cf40 0x7f7b70158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b77972640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b70067410 con 0x7f7b7006e300 2023-11-28T21:35:01.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b77972640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b7006e090 con 0x7f7b70150c20 2023-11-28T21:35:01.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b77972640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b70157f30 con 0x7f7b7006cf40 2023-11-28T21:35:01.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b7570e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b70150c20 0x7f7b70156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b7570e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b70150c20 0x7f7b70156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54302/0 (socket says 172.21.15.12:54302) 2023-11-28T21:35:01.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b7570e640 1 -- 172.21.15.12:0/4124475132 learned_addr learned my addr 172.21.15.12:0/4124475132 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:01.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b75f0f640 1 --2- 172.21.15.12:0/4124475132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b7006cf40 0x7f7b70158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b7570e640 1 -- 172.21.15.12:0/4124475132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b7006cf40 msgr2=0x7f7b70158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:01.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b7570e640 1 --2- 172.21.15.12:0/4124475132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b7006cf40 0x7f7b70158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b7570e640 1 -- 172.21.15.12:0/4124475132 >> v1:172.21.15.144:6789/0 conn(0x7f7b7006e300 legacy=0x7f7b700c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.268+0000 7f7b7570e640 1 -- 172.21.15.12:0/4124475132 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70151000 con 0x7f7b70150c20 2023-11-28T21:35:01.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b7570e640 1 --2- 172.21.15.12:0/4124475132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b70150c20 0x7f7b70156910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009870 tx=0x7f7b6c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16e78bd51bba910 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b67fff640 1 -- 172.21.15.12:0/4124475132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0ab650 con 0x7f7b70150c20 2023-11-28T21:35:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b67fff640 1 -- 172.21.15.12:0/4124475132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c0ab7b0 con 0x7f7b70150c20 2023-11-28T21:35:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b67fff640 1 -- 172.21.15.12:0/4124475132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0abb40 con 0x7f7b70150c20 2023-11-28T21:35:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b77972640 1 -- 172.21.15.12:0/4124475132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b70150c20 msgr2=0x7f7b70156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b77972640 1 --2- 172.21.15.12:0/4124475132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b70150c20 0x7f7b70156910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009870 tx=0x7f7b6c01c540 comp rx=0 tx=0).stop 2023-11-28T21:35:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b77972640 1 -- 172.21.15.12:0/4124475132 shutdown_connections 2023-11-28T21:35:01.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b77972640 1 --2- 172.21.15.12:0/4124475132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b7006cf40 0x7f7b70158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b77972640 1 --2- 172.21.15.12:0/4124475132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b70150c20 0x7f7b70156910 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b77972640 1 -- 172.21.15.12:0/4124475132 >> 172.21.15.12:0/4124475132 conn(0x7f7b70066360 msgr2=0x7f7b70066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b77972640 1 -- 172.21.15.12:0/4124475132 shutdown_connections 2023-11-28T21:35:01.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.272+0000 7f7b77972640 1 -- 172.21.15.12:0/4124475132 wait complete. 2023-11-28T21:35:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b77972640 1 Processor -- start 2023-11-28T21:35:01.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b77972640 1 -- start start 2023-11-28T21:35:01.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b77972640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b7006cf40 0x7f7b701f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b77972640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7006e300 0x7f7b701f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b77972640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b70150c20 0x7f7b7014f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b77972640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b701419d0 con 0x7f7b7006cf40 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b77972640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b701f2fb0 con 0x7f7b7006e300 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b77972640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7014fb20 con 0x7f7b70150c20 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b7570e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b7006cf40 0x7f7b701f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b7570e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b7006cf40 0x7f7b701f6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54312/0 (socket says 172.21.15.12:54312) 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b7570e640 1 -- 172.21.15.12:0/2051748028 learned_addr learned my addr 172.21.15.12:0/2051748028 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b74f0d640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7006e300 0x7f7b701f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b75f0f640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b70150c20 0x7f7b7014f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b7570e640 1 -- 172.21.15.12:0/2051748028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b70150c20 msgr2=0x7f7b7014f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b7570e640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b70150c20 0x7f7b7014f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b7570e640 1 -- 172.21.15.12:0/2051748028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7006e300 msgr2=0x7f7b701f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b7570e640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7006e300 0x7f7b701f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b7570e640 1 -- 172.21.15.12:0/2051748028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c0094d0 con 0x7f7b7006cf40 2023-11-28T21:35:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b74f0d640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7006e300 0x7f7b701f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b7570e640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b7006cf40 0x7f7b701f6b60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0ad040 tx=0x7f7b6c01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:01.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b65ffb640 1 -- 172.21.15.12:0/2051748028 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0b4800 con 0x7f7b7006cf40 2023-11-28T21:35:01.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b65ffb640 1 -- 172.21.15.12:0/2051748028 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c0b4980 con 0x7f7b7006cf40 2023-11-28T21:35:01.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b65ffb640 1 -- 172.21.15.12:0/2051748028 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0b4d30 con 0x7f7b7006cf40 2023-11-28T21:35:01.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b77972640 1 -- 172.21.15.12:0/2051748028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b70141c00 con 0x7f7b7006cf40 2023-11-28T21:35:01.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.276+0000 7f7b77972640 1 -- 172.21.15.12:0/2051748028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b700c50e0 con 0x7f7b7006cf40 2023-11-28T21:35:01.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.280+0000 7f7b77972640 1 -- 172.21.15.12:0/2051748028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b700c5680 con 0x7f7b7006cf40 2023-11-28T21:35:01.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.280+0000 7f7b65ffb640 1 -- 172.21.15.12:0/2051748028 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7b6c0b3380 con 0x7f7b7006cf40 2023-11-28T21:35:01.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.280+0000 7f7b65ffb640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b54066470 0x7f7b54068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.280+0000 7f7b65ffb640 1 -- 172.21.15.12:0/2051748028 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7b6c11e870 con 0x7f7b7006cf40 2023-11-28T21:35:01.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.280+0000 7f7b74f0d640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b54066470 0x7f7b54068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.280+0000 7f7b74f0d640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b54066470 0x7f7b54068930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7b701489e0 tx=0x7f7b60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:01.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.284+0000 7f7b65ffb640 1 -- 172.21.15.12:0/2051748028 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7b6c0ebc60 con 0x7f7b7006cf40 2023-11-28T21:35:01.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.448+0000 7f7b77972640 1 -- 172.21.15.12:0/2051748028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b70074f60 con 0x7f7b7006cf40 2023-11-28T21:35:01.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.448+0000 7f7b65ffb640 1 -- 172.21.15.12:0/2051748028 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7f7b6c0ef280 con 0x7f7b7006cf40 2023-11-28T21:35:01.452 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:01.452 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 -- 172.21.15.12:0/2051748028 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b54066470 msgr2=0x7f7b54068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b54066470 0x7f7b54068930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7b701489e0 tx=0x7f7b60009040 comp rx=0 tx=0).stop 2023-11-28T21:35:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 -- 172.21.15.12:0/2051748028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b7006cf40 msgr2=0x7f7b701f6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b7006cf40 0x7f7b701f6b60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0ad040 tx=0x7f7b6c01ad80 comp rx=0 tx=0).stop 2023-11-28T21:35:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 -- 172.21.15.12:0/2051748028 shutdown_connections 2023-11-28T21:35:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b54066470 0x7f7b54068930 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b70150c20 0x7f7b7014f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7006e300 0x7f7b701f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 --2- 172.21.15.12:0/2051748028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b7006cf40 0x7f7b701f6b60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 -- 172.21.15.12:0/2051748028 >> 172.21.15.12:0/2051748028 conn(0x7f7b70066360 msgr2=0x7f7b7006b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 -- 172.21.15.12:0/2051748028 shutdown_connections 2023-11-28T21:35:01.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.452+0000 7f7b77972640 1 -- 172.21.15.12:0/2051748028 wait complete. 2023-11-28T21:35:01.474 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:01.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f5059d640 1 Processor -- start 2023-11-28T21:35:01.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f5059d640 1 -- start start 2023-11-28T21:35:01.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f5059d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8f48153e40 0x7f8f480ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f5059d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8f4815dc80 0x7f8f480cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f5059d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4815acc0 con 0x7f8f480cfba0 2023-11-28T21:35:01.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f5059d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f48159cb0 con 0x7f8f4815dc80 2023-11-28T21:35:01.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f5059d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480cffa0 con 0x7f8f48153e40 2023-11-28T21:35:01.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f4db38640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8f4815dc80 0x7f8f480cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f4db38640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8f4815dc80 0x7f8f480cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54328/0 (socket says 172.21.15.12:54328) 2023-11-28T21:35:01.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f4db38640 1 -- 172.21.15.12:0/61556272 learned_addr learned my addr 172.21.15.12:0/61556272 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:01.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f4e339640 1 --2- 172.21.15.12:0/61556272 >> v2:172.21.15.144:3300/0 conn(0x7f8f48153e40 0x7f8f480ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f4db38640 1 -- 172.21.15.12:0/61556272 >> v2:172.21.15.144:3300/0 conn(0x7f8f48153e40 msgr2=0x7f8f480ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:01.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f4db38640 1 --2- 172.21.15.12:0/61556272 >> v2:172.21.15.144:3300/0 conn(0x7f8f48153e40 0x7f8f480ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f4db38640 1 -- 172.21.15.12:0/61556272 >> v1:172.21.15.12:6789/0 conn(0x7f8f480cfba0 legacy=0x7f8f480cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f4d337640 1 -- 172.21.15.12:0/61556272 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229956934 0 0) 0x7f8f4815acc0 con 0x7f8f480cfba0 2023-11-28T21:35:01.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.616+0000 7f8f4db38640 1 -- 172.21.15.12:0/61556272 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f48159ea0 con 0x7f8f4815dc80 2023-11-28T21:35:01.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f4db38640 1 --2- 172.21.15.12:0/61556272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 0x7f8f480cf660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f480cd270 tx=0x7f8f3801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abca9416d259510e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:01.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f4d337640 1 -- 172.21.15.12:0/61556272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380aa430 con 0x7f8f4815dc80 2023-11-28T21:35:01.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f4d337640 1 -- 172.21.15.12:0/61556272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f380aa590 con 0x7f8f4815dc80 2023-11-28T21:35:01.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f4d337640 1 -- 172.21.15.12:0/61556272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380aa920 con 0x7f8f4815dc80 2023-11-28T21:35:01.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f5059d640 1 -- 172.21.15.12:0/61556272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 msgr2=0x7f8f480cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f5059d640 1 --2- 172.21.15.12:0/61556272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 0x7f8f480cf660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f480cd270 tx=0x7f8f3801c4d0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f5059d640 1 -- 172.21.15.12:0/61556272 shutdown_connections 2023-11-28T21:35:01.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f5059d640 1 --2- 172.21.15.12:0/61556272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 0x7f8f480cf660 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f5059d640 1 --2- 172.21.15.12:0/61556272 >> v2:172.21.15.144:3300/0 conn(0x7f8f48153e40 0x7f8f480ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f5059d640 1 -- 172.21.15.12:0/61556272 >> 172.21.15.12:0/61556272 conn(0x7f8f4806b7b0 msgr2=0x7f8f48066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:01.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f5059d640 1 -- 172.21.15.12:0/61556272 shutdown_connections 2023-11-28T21:35:01.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.620+0000 7f8f5059d640 1 -- 172.21.15.12:0/61556272 wait complete. 2023-11-28T21:35:01.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 Processor -- start 2023-11-28T21:35:01.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 -- start start 2023-11-28T21:35:01.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f480cfba0 0x7f8f480c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f48153e40 0x7f8f48149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 0x7f8f4814a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c7010 con 0x7f8f4815dc80 2023-11-28T21:35:01.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c82f0 con 0x7f8f480cfba0 2023-11-28T21:35:01.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c6ae0 con 0x7f8f48153e40 2023-11-28T21:35:01.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4eb3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 0x7f8f4814a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4eb3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 0x7f8f4814a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54330/0 (socket says 172.21.15.12:54330) 2023-11-28T21:35:01.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4eb3a640 1 -- 172.21.15.12:0/4160185225 learned_addr learned my addr 172.21.15.12:0/4160185225 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:01.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4e339640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f480cfba0 0x7f8f480c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4db38640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f48153e40 0x7f8f48149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4eb3a640 1 -- 172.21.15.12:0/4160185225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f48153e40 msgr2=0x7f8f48149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4eb3a640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f48153e40 0x7f8f48149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4eb3a640 1 -- 172.21.15.12:0/4160185225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f480cfba0 msgr2=0x7f8f480c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4eb3a640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f480cfba0 0x7f8f480c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4eb3a640 1 -- 172.21.15.12:0/4160185225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380094d0 con 0x7f8f4815dc80 2023-11-28T21:35:01.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f4eb3a640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 0x7f8f4814a530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f40002240 tx=0x7f8f4000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:01.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f3effd640 1 -- 172.21.15.12:0/4160185225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40003db0 con 0x7f8f4815dc80 2023-11-28T21:35:01.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f3effd640 1 -- 172.21.15.12:0/4160185225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4000ce30 con 0x7f8f4815dc80 2023-11-28T21:35:01.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f3effd640 1 -- 172.21.15.12:0/4160185225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40005270 con 0x7f8f4815dc80 2023-11-28T21:35:01.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 -- 172.21.15.12:0/4160185225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f48151650 con 0x7f8f4815dc80 2023-11-28T21:35:01.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 -- 172.21.15.12:0/4160185225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f481ec710 con 0x7f8f4815dc80 2023-11-28T21:35:01.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.624+0000 7f8f5059d640 1 -- 172.21.15.12:0/4160185225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f481517e0 con 0x7f8f4815dc80 2023-11-28T21:35:01.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.628+0000 7f8f3effd640 1 -- 172.21.15.12:0/4160185225 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8f400053d0 con 0x7f8f4815dc80 2023-11-28T21:35:01.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.628+0000 7f8f3effd640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8f2c0664c0 0x7f8f2c068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.628+0000 7f8f3effd640 1 -- 172.21.15.12:0/4160185225 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8f400838e0 con 0x7f8f4815dc80 2023-11-28T21:35:01.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.628+0000 7f8f4e339640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8f2c0664c0 0x7f8f2c068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.628+0000 7f8f4e339640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8f2c0664c0 0x7f8f2c068980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f44007a60 tx=0x7f8f44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:01.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.632+0000 7f8f3effd640 1 -- 172.21.15.12:0/4160185225 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8f40050b70 con 0x7f8f4815dc80 2023-11-28T21:35:01.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.788+0000 7f8f5059d640 1 -- 172.21.15.12:0/4160185225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f48074f60 con 0x7f8f4815dc80 2023-11-28T21:35:01.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f3effd640 1 -- 172.21.15.12:0/4160185225 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7f8f40017070 con 0x7f8f4815dc80 2023-11-28T21:35:01.794 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:01.794 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 -- 172.21.15.12:0/4160185225 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8f2c0664c0 msgr2=0x7f8f2c068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8f2c0664c0 0x7f8f2c068980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f44007a60 tx=0x7f8f44008040 comp rx=0 tx=0).stop 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 -- 172.21.15.12:0/4160185225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 msgr2=0x7f8f4814a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 0x7f8f4814a530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f40002240 tx=0x7f8f4000cbe0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 -- 172.21.15.12:0/4160185225 shutdown_connections 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8f2c0664c0 0x7f8f2c068980 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f4815dc80 0x7f8f4814a530 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f48153e40 0x7f8f48149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 --2- 172.21.15.12:0/4160185225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f480cfba0 0x7f8f480c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 -- 172.21.15.12:0/4160185225 >> 172.21.15.12:0/4160185225 conn(0x7f8f4806b7b0 msgr2=0x7f8f48157b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:01.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 -- 172.21.15.12:0/4160185225 shutdown_connections 2023-11-28T21:35:01.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.792+0000 7f8f5059d640 1 -- 172.21.15.12:0/4160185225 wait complete. 2023-11-28T21:35:01.815 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T21:35:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86da3b8640 1 Processor -- start 2023-11-28T21:35:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86da3b8640 1 -- start start 2023-11-28T21:35:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86da3b8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f86d406b6c0 0x7f86d406bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86da3b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d415d930 0x7f86d415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86da3b8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d406c040 con 0x7f86d4153f30 2023-11-28T21:35:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86da3b8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d415d4a0 con 0x7f86d406b6c0 2023-11-28T21:35:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86da3b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d415e2e0 con 0x7f86d415d930 2023-11-28T21:35:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86cbfff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f86d406b6c0 0x7f86d406bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86d8955640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d415d930 0x7f86d415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86d3fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f86d4153f30 0x7f86d40ca170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:56094/0 (socket says 172.21.15.12:56094) 2023-11-28T21:35:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86d3fff640 1 -- 172.21.15.12:0/2557081507 learned_addr learned my addr 172.21.15.12:0/2557081507 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86d8955640 1 -- 172.21.15.12:0/2557081507 >> v2:172.21.15.144:3300/0 conn(0x7f86d406b6c0 msgr2=0x7f86d406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86d8955640 1 --2- 172.21.15.12:0/2557081507 >> v2:172.21.15.144:3300/0 conn(0x7f86d406b6c0 0x7f86d406bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86d8955640 1 -- 172.21.15.12:0/2557081507 >> v1:172.21.15.144:6789/0 conn(0x7f86d4153f30 legacy=0x7f86d40ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86d8955640 1 -- 172.21.15.12:0/2557081507 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d415a130 con 0x7f86d415d930 2023-11-28T21:35:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.960+0000 7f86cbfff640 1 --2- 172.21.15.12:0/2557081507 >> v2:172.21.15.144:3300/0 conn(0x7f86d406b6c0 0x7f86d406bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.964+0000 7f86d8955640 1 --2- 172.21.15.12:0/2557081507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d415d930 0x7f86d415dda0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009d90 tx=0x7f86cc01f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2080f5c16334866 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.964+0000 7f86d37fe640 1 -- 172.21.15.12:0/2557081507 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0af600 con 0x7f86d415d930 2023-11-28T21:35:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.964+0000 7f86d37fe640 1 -- 172.21.15.12:0/2557081507 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc0ab050 con 0x7f86d415d930 2023-11-28T21:35:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.964+0000 7f86d37fe640 1 -- 172.21.15.12:0/2557081507 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0af980 con 0x7f86d415d930 2023-11-28T21:35:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.964+0000 7f86da3b8640 1 -- 172.21.15.12:0/2557081507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d415d930 msgr2=0x7f86d415dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.964+0000 7f86da3b8640 1 --2- 172.21.15.12:0/2557081507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d415d930 0x7f86d415dda0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009d90 tx=0x7f86cc01f130 comp rx=0 tx=0).stop 2023-11-28T21:35:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.964+0000 7f86da3b8640 1 -- 172.21.15.12:0/2557081507 shutdown_connections 2023-11-28T21:35:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.964+0000 7f86da3b8640 1 --2- 172.21.15.12:0/2557081507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d415d930 0x7f86d415dda0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.964+0000 7f86da3b8640 1 --2- 172.21.15.12:0/2557081507 >> v2:172.21.15.144:3300/0 conn(0x7f86d406b6c0 0x7f86d406bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.964+0000 7f86da3b8640 1 -- 172.21.15.12:0/2557081507 >> 172.21.15.12:0/2557081507 conn(0x7f86d4066530 msgr2=0x7f86d4066940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:01.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86da3b8640 1 -- 172.21.15.12:0/2557081507 shutdown_connections 2023-11-28T21:35:01.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86da3b8640 1 -- 172.21.15.12:0/2557081507 wait complete. 2023-11-28T21:35:01.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86da3b8640 1 Processor -- start 2023-11-28T21:35:01.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86da3b8640 1 -- start start 2023-11-28T21:35:01.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86da3b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d406b6c0 0x7f86d41494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86da3b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4153f30 0x7f86d4146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86da3b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d415d930 0x7f86d4145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86da3b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41ef9c0 con 0x7f86d4153f30 2023-11-28T21:35:01.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86da3b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41f09f0 con 0x7f86d415d930 2023-11-28T21:35:01.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86da3b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41ef700 con 0x7f86d406b6c0 2023-11-28T21:35:01.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4153f30 0x7f86d4146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4153f30 0x7f86d4146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54342/0 (socket says 172.21.15.12:54342) 2023-11-28T21:35:01.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86d3fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d406b6c0 0x7f86d41494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86cbfff640 1 -- 172.21.15.12:0/3621642821 learned_addr learned my addr 172.21.15.12:0/3621642821 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86d8955640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d415d930 0x7f86d4145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86cbfff640 1 -- 172.21.15.12:0/3621642821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d406b6c0 msgr2=0x7f86d41494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86cbfff640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d406b6c0 0x7f86d41494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86cbfff640 1 -- 172.21.15.12:0/3621642821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d415d930 msgr2=0x7f86d4145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:01.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86cbfff640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d415d930 0x7f86d4145bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86cbfff640 1 -- 172.21.15.12:0/3621642821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc009420 con 0x7f86d4153f30 2023-11-28T21:35:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.968+0000 7f86cbfff640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4153f30 0x7f86d4146b60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00ebe0 tx=0x7f86bc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.972+0000 7f86d17fa640 1 -- 172.21.15.12:0/3621642821 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc00eee0 con 0x7f86d4153f30 2023-11-28T21:35:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.972+0000 7f86d17fa640 1 -- 172.21.15.12:0/3621642821 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc00cc30 con 0x7f86d4153f30 2023-11-28T21:35:01.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.972+0000 7f86d17fa640 1 -- 172.21.15.12:0/3621642821 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc013470 con 0x7f86d4153f30 2023-11-28T21:35:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.972+0000 7f86da3b8640 1 -- 172.21.15.12:0/3621642821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc009a70 con 0x7f86d4153f30 2023-11-28T21:35:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.972+0000 7f86da3b8640 1 -- 172.21.15.12:0/3621642821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d4145110 con 0x7f86d4153f30 2023-11-28T21:35:01.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.972+0000 7f86da3b8640 1 -- 172.21.15.12:0/3621642821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d4074f60 con 0x7f86d4153f30 2023-11-28T21:35:01.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.972+0000 7f86d17fa640 1 -- 172.21.15.12:0/3621642821 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f86bc0135d0 con 0x7f86d4153f30 2023-11-28T21:35:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.972+0000 7f86d17fa640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86b4066470 0x7f86b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.972+0000 7f86d17fa640 1 -- 172.21.15.12:0/3621642821 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f86bc084460 con 0x7f86d4153f30 2023-11-28T21:35:01.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.972+0000 7f86d3fff640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86b4066470 0x7f86b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:01.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.976+0000 7f86d3fff640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86b4066470 0x7f86b4068930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f86c4005890 tx=0x7f86c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:01.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:01.976+0000 7f86d17fa640 1 -- 172.21.15.12:0/3621642821 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86bc051600 con 0x7f86d4153f30 2023-11-28T21:35:02.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.168+0000 7f86da3b8640 1 -- 172.21.15.12:0/3621642821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f86d40705e0 con 0x7f86d4153f30 2023-11-28T21:35:02.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.436+0000 7f86d17fa640 1 -- 172.21.15.12:0/3621642821 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f86bc054c20 con 0x7f86d4153f30 2023-11-28T21:35:02.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 -- 172.21.15.12:0/3621642821 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86b4066470 msgr2=0x7f86b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:02.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86b4066470 0x7f86b4068930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f86c4005890 tx=0x7f86c4009040 comp rx=0 tx=0).stop 2023-11-28T21:35:02.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 -- 172.21.15.12:0/3621642821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4153f30 msgr2=0x7f86d4146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:02.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4153f30 0x7f86d4146b60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00ebe0 tx=0x7f86bc00c520 comp rx=0 tx=0).stop 2023-11-28T21:35:02.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 -- 172.21.15.12:0/3621642821 shutdown_connections 2023-11-28T21:35:02.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86b4066470 0x7f86b4068930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:02.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d415d930 0x7f86d4145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86d4153f30 0x7f86d4146b60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 --2- 172.21.15.12:0/3621642821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86d406b6c0 0x7f86d41494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 -- 172.21.15.12:0/3621642821 >> 172.21.15.12:0/3621642821 conn(0x7f86d4066530 msgr2=0x7f86d414e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 -- 172.21.15.12:0/3621642821 shutdown_connections 2023-11-28T21:35:02.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.440+0000 7f86da3b8640 1 -- 172.21.15.12:0/3621642821 wait complete. 2023-11-28T21:35:02.462 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300} 2023-11-28T21:35:02.463 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-28T21:35:02.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.604+0000 7f7158380640 1 Processor -- start 2023-11-28T21:35:02.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.604+0000 7f7158380640 1 -- start start 2023-11-28T21:35:02.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f7158380640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f715006e300 0x7f7150156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:02.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f7158380640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f715006cf40 0x7f7150158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:02.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f7158380640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7150067410 con 0x7f7150150c20 2023-11-28T21:35:02.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f7158380640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715006e090 con 0x7f715006e300 2023-11-28T21:35:02.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f7158380640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7150157f30 con 0x7f715006cf40 2023-11-28T21:35:02.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f715591b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f715006e300 0x7f7150156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:02.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f715591b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f715006e300 0x7f7150156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54350/0 (socket says 172.21.15.12:54350) 2023-11-28T21:35:02.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f715591b640 1 -- 172.21.15.12:0/1778168860 learned_addr learned my addr 172.21.15.12:0/1778168860 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:02.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f715691d640 1 --2- 172.21.15.12:0/1778168860 >> v2:172.21.15.144:3300/0 conn(0x7f715006cf40 0x7f7150158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:02.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f715591b640 1 -- 172.21.15.12:0/1778168860 >> v2:172.21.15.144:3300/0 conn(0x7f715006cf40 msgr2=0x7f7150158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:02.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f715591b640 1 --2- 172.21.15.12:0/1778168860 >> v2:172.21.15.144:3300/0 conn(0x7f715006cf40 0x7f7150158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:02.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f715591b640 1 -- 172.21.15.12:0/1778168860 >> v1:172.21.15.12:6789/0 conn(0x7f7150150c20 legacy=0x7f71500c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:02.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f715591b640 1 -- 172.21.15.12:0/1778168860 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7150151000 con 0x7f715006e300 2023-11-28T21:35:02.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.608+0000 7f715511a640 1 -- 172.21.15.12:0/1778168860 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 222370203 0 0) 0x7f7150067410 con 0x7f7150150c20 2023-11-28T21:35:02.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f715591b640 1 --2- 172.21.15.12:0/1778168860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f715006e300 0x7f7150156910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f715006b9d0 tx=0x7f714001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d10632d09a5752 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:02.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f715511a640 1 -- 172.21.15.12:0/1778168860 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71400ab610 con 0x7f715006e300 2023-11-28T21:35:02.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f715511a640 1 -- 172.21.15.12:0/1778168860 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71400ab770 con 0x7f715006e300 2023-11-28T21:35:02.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f715511a640 1 -- 172.21.15.12:0/1778168860 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71400abb00 con 0x7f715006e300 2023-11-28T21:35:02.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 -- 172.21.15.12:0/1778168860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f715006e300 msgr2=0x7f7150156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:02.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 --2- 172.21.15.12:0/1778168860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f715006e300 0x7f7150156910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f715006b9d0 tx=0x7f714001c500 comp rx=0 tx=0).stop 2023-11-28T21:35:02.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 -- 172.21.15.12:0/1778168860 shutdown_connections 2023-11-28T21:35:02.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 --2- 172.21.15.12:0/1778168860 >> v2:172.21.15.144:3300/0 conn(0x7f715006cf40 0x7f7150158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:02.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 --2- 172.21.15.12:0/1778168860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f715006e300 0x7f7150156910 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:02.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 -- 172.21.15.12:0/1778168860 >> 172.21.15.12:0/1778168860 conn(0x7f7150066360 msgr2=0x7f7150066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:02.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 -- 172.21.15.12:0/1778168860 shutdown_connections 2023-11-28T21:35:02.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 -- 172.21.15.12:0/1778168860 wait complete. 2023-11-28T21:35:02.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 Processor -- start 2023-11-28T21:35:02.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 -- start start 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f715006cf40 0x7f715015f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f715006e300 0x7f7150155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7150150c20 0x7f7150154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71501f3aa0 con 0x7f7150150c20 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7150156110 con 0x7f715006cf40 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f7158380640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71501f3540 con 0x7f715006e300 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.612+0000 7f715691d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7150150c20 0x7f7150154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715691d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7150150c20 0x7f7150154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54354/0 (socket says 172.21.15.12:54354) 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715691d640 1 -- 172.21.15.12:0/93190277 learned_addr learned my addr 172.21.15.12:0/93190277 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715591b640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f715006e300 0x7f7150155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715611c640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f715006cf40 0x7f715015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:02.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715691d640 1 -- 172.21.15.12:0/93190277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f715006e300 msgr2=0x7f7150155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:02.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715691d640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f715006e300 0x7f7150155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:02.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715691d640 1 -- 172.21.15.12:0/93190277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f715006cf40 msgr2=0x7f715015f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:02.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715691d640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f715006cf40 0x7f715015f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:02.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715691d640 1 -- 172.21.15.12:0/93190277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71400094d0 con 0x7f7150150c20 2023-11-28T21:35:02.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715691d640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7150150c20 0x7f7150154a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7148007e70 tx=0x7f714800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:02.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f7146ffd640 1 -- 172.21.15.12:0/93190277 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148014620 con 0x7f7150150c20 2023-11-28T21:35:02.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f7146ffd640 1 -- 172.21.15.12:0/93190277 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71480147a0 con 0x7f7150150c20 2023-11-28T21:35:02.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f7158380640 1 -- 172.21.15.12:0/93190277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715014f470 con 0x7f7150150c20 2023-11-28T21:35:02.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f7146ffd640 1 -- 172.21.15.12:0/93190277 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148014b50 con 0x7f7150150c20 2023-11-28T21:35:02.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f7158380640 1 -- 172.21.15.12:0/93190277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71501f37c0 con 0x7f7150150c20 2023-11-28T21:35:02.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f7158380640 1 -- 172.21.15.12:0/93190277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715014f2d0 con 0x7f7150150c20 2023-11-28T21:35:02.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f7146ffd640 1 -- 172.21.15.12:0/93190277 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7148014cb0 con 0x7f7150150c20 2023-11-28T21:35:02.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f7146ffd640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71340664a0 0x7f7134068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:02.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f7146ffd640 1 -- 172.21.15.12:0/93190277 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7148084ad0 con 0x7f7150150c20 2023-11-28T21:35:02.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.616+0000 7f715611c640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71340664a0 0x7f7134068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:02.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.620+0000 7f715611c640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71340664a0 0x7f7134068960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f714c000f70 tx=0x7f714c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:02.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.620+0000 7f7146ffd640 1 -- 172.21.15.12:0/93190277 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f715014f2d0 con 0x7f7150150c20 2023-11-28T21:35:02.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:02.812+0000 7f7158380640 1 -- 172.21.15.12:0/93190277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f71501547b0 con 0x7f7150150c20 2023-11-28T21:35:03.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.284+0000 7f7146ffd640 1 -- 172.21.15.12:0/93190277 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f71501547b0 con 0x7f7150150c20 2023-11-28T21:35:03.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 -- 172.21.15.12:0/93190277 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71340664a0 msgr2=0x7f7134068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:03.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71340664a0 0x7f7134068960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f714c000f70 tx=0x7f714c009040 comp rx=0 tx=0).stop 2023-11-28T21:35:03.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 -- 172.21.15.12:0/93190277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7150150c20 msgr2=0x7f7150154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:03.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7150150c20 0x7f7150154a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7148007e70 tx=0x7f714800e4b0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 -- 172.21.15.12:0/93190277 shutdown_connections 2023-11-28T21:35:03.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71340664a0 0x7f7134068960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7150150c20 0x7f7150154a90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f715006e300 0x7f7150155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 --2- 172.21.15.12:0/93190277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f715006cf40 0x7f715015f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 -- 172.21.15.12:0/93190277 >> 172.21.15.12:0/93190277 conn(0x7f7150066360 msgr2=0x7f71500c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:03.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 -- 172.21.15.12:0/93190277 shutdown_connections 2023-11-28T21:35:03.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.288+0000 7f7158380640 1 -- 172.21.15.12:0/93190277 wait complete. 2023-11-28T21:35:03.313 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T21:35:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.460+0000 7f34b15df640 1 Processor -- start 2023-11-28T21:35:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.460+0000 7f34b15df640 1 -- start start 2023-11-28T21:35:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34b15df640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f34ac153e00 0x7f34ac0ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34b15df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac0caec0 0x7f34ac0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34b15df640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac15ab40 con 0x7f34ac0cba30 2023-11-28T21:35:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34b15df640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ac06a850 con 0x7f34ac153e00 2023-11-28T21:35:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34b15df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0b2670 con 0x7f34ac0caec0 2023-11-28T21:35:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34a27fc640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f34ac0cba30 0x7f34ac06ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50144/0 (socket says 172.21.15.12:50144) 2023-11-28T21:35:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34a27fc640 1 -- 172.21.15.12:0/669395649 learned_addr learned my addr 172.21.15.12:0/669395649 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34ab7fe640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac0caec0 0x7f34ac0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aaffd640 1 --2- 172.21.15.12:0/669395649 >> v2:172.21.15.144:3300/0 conn(0x7f34ac153e00 0x7f34ac0ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338598272 0 0) 0x7f34ac15ab40 con 0x7f34ac0cba30 2023-11-28T21:35:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3498003530 con 0x7f34ac0cba30 2023-11-28T21:35:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4225317843 0 0) 0x7f3498003530 con 0x7f34ac0cba30 2023-11-28T21:35:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac0caec0 msgr2=0x7f34ac0cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac0caec0 0x7f34ac0cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 >> v2:172.21.15.144:3300/0 conn(0x7f34ac153e00 msgr2=0x7f34ac0ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 --2- 172.21.15.12:0/669395649 >> v2:172.21.15.144:3300/0 conn(0x7f34ac153e00 0x7f34ac0ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac159fe0 con 0x7f34ac0cba30 2023-11-28T21:35:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f3494001680 con 0x7f34ac0cba30 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 >> v1:172.21.15.12:6789/0 conn(0x7f34ac0cba30 legacy=0x7f34ac06ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3498003980 0x7f3498003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac0caec0 0x7f34980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3498004d10 0x7f3498005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac06a850 con 0x7f3498003980 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0b2670 con 0x7f3498004d10 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349800ba10 con 0x7f34ac0caec0 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aaffd640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3498003980 0x7f3498003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34a27fc640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac0caec0 0x7f34980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34ab7fe640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3498004d10 0x7f3498005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aaffd640 1 -- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac0caec0 msgr2=0x7f34980047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aaffd640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac0caec0 0x7f34980047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aaffd640 1 -- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3498004d10 msgr2=0x7f3498005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:03.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aaffd640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3498004d10 0x7f3498005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.464+0000 7f34aaffd640 1 -- 172.21.15.12:0/669395649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3498003530 con 0x7f3498003980 2023-11-28T21:35:03.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34aaffd640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3498003980 0x7f3498003da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f349c009fc0 tx=0x7f349c01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc778bd04745c0d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:03.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c0a5c80 con 0x7f3498003980 2023-11-28T21:35:03.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c00f040 con 0x7f3498003980 2023-11-28T21:35:03.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34aa7fc640 1 -- 172.21.15.12:0/669395649 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c0ad440 con 0x7f3498003980 2023-11-28T21:35:03.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34b15df640 1 -- 172.21.15.12:0/669395649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3498003980 msgr2=0x7f3498003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:03.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34b15df640 1 --2- 172.21.15.12:0/669395649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3498003980 0x7f3498003da0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f349c009fc0 tx=0x7f349c01cc70 comp rx=0 tx=0).stop 2023-11-28T21:35:03.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34ab7fe640 1 -- 172.21.15.12:0/669395649 reap_dead start 2023-11-28T21:35:03.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34b15df640 1 -- 172.21.15.12:0/669395649 shutdown_connections 2023-11-28T21:35:03.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34b15df640 1 -- 172.21.15.12:0/669395649 >> 172.21.15.12:0/669395649 conn(0x7f34ac0cf4e0 msgr2=0x7f34ac0cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:03.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34b15df640 1 -- 172.21.15.12:0/669395649 shutdown_connections 2023-11-28T21:35:03.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34b15df640 1 -- 172.21.15.12:0/669395649 wait complete. 2023-11-28T21:35:03.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.468+0000 7f34b15df640 1 Processor -- start 2023-11-28T21:35:03.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34b15df640 1 -- start start 2023-11-28T21:35:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34b15df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3498003980 0x7f34ac1e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34b15df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ac15d180 0x7f34ac1f0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34b15df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac15caa0 0x7f34ac1e6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34b15df640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac15cf20 con 0x7f34ac15d180 2023-11-28T21:35:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34b15df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac15c610 con 0x7f3498003980 2023-11-28T21:35:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34b15df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac1512d0 con 0x7f34ac15caa0 2023-11-28T21:35:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ac15d180 0x7f34ac1f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:03.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ac15d180 0x7f34ac1f0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54374/0 (socket says 172.21.15.12:54374) 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a27fc640 1 -- 172.21.15.12:0/3183741472 learned_addr learned my addr 172.21.15.12:0/3183741472 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34aaffd640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3498003980 0x7f34ac1e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34ab7fe640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac15caa0 0x7f34ac1e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a27fc640 1 -- 172.21.15.12:0/3183741472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac15caa0 msgr2=0x7f34ac1e6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a27fc640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac15caa0 0x7f34ac1e6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a27fc640 1 -- 172.21.15.12:0/3183741472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3498003980 msgr2=0x7f34ac1e2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a27fc640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3498003980 0x7f34ac1e2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a27fc640 1 -- 172.21.15.12:0/3183741472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac15bfe0 con 0x7f34ac15d180 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34ab7fe640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac15caa0 0x7f34ac1e6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a27fc640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ac15d180 0x7f34ac1f0260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3494011970 tx=0x7f3494011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a3fff640 1 -- 172.21.15.12:0/3183741472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349400fce0 con 0x7f34ac15d180 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a3fff640 1 -- 172.21.15.12:0/3183741472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349400fe40 con 0x7f34ac15d180 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34a3fff640 1 -- 172.21.15.12:0/3183741472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3494015650 con 0x7f34ac15d180 2023-11-28T21:35:03.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34b15df640 1 -- 172.21.15.12:0/3183741472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ac15dae0 con 0x7f34ac15d180 2023-11-28T21:35:03.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34b15df640 1 -- 172.21.15.12:0/3183741472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac15c2a0 con 0x7f34ac15d180 2023-11-28T21:35:03.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.472+0000 7f34b15df640 1 -- 172.21.15.12:0/3183741472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ac15c430 con 0x7f34ac15d180 2023-11-28T21:35:03.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.476+0000 7f34a3fff640 1 -- 172.21.15.12:0/3183741472 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f34940157b0 con 0x7f34ac15d180 2023-11-28T21:35:03.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.476+0000 7f34a3fff640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f348c0664a0 0x7f348c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:03.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.476+0000 7f34a3fff640 1 -- 172.21.15.12:0/3183741472 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f34940876e0 con 0x7f34ac15d180 2023-11-28T21:35:03.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.476+0000 7f34aaffd640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f348c0664a0 0x7f348c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:03.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.476+0000 7f34aaffd640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f348c0664a0 0x7f348c068960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f349c0af000 tx=0x7f349c004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:03.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.480+0000 7f34a3fff640 1 -- 172.21.15.12:0/3183741472 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3494054740 con 0x7f34ac15d180 2023-11-28T21:35:03.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:03.676+0000 7f34b15df640 1 -- 172.21.15.12:0/3183741472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f34ac074f60 con 0x7f34ac15d180 2023-11-28T21:35:04.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.448+0000 7f34a3fff640 1 -- 172.21.15.12:0/3183741472 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3494057d60 con 0x7f34ac15d180 2023-11-28T21:35:04.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 -- 172.21.15.12:0/3183741472 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f348c0664a0 msgr2=0x7f348c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:04.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f348c0664a0 0x7f348c068960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f349c0af000 tx=0x7f349c004ac0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 -- 172.21.15.12:0/3183741472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ac15d180 msgr2=0x7f34ac1f0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:04.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ac15d180 0x7f34ac1f0260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3494011970 tx=0x7f3494011e30 comp rx=0 tx=0).stop 2023-11-28T21:35:04.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 -- 172.21.15.12:0/3183741472 shutdown_connections 2023-11-28T21:35:04.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f348c0664a0 0x7f348c068960 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34ac15caa0 0x7f34ac1e6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ac15d180 0x7f34ac1f0260 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 --2- 172.21.15.12:0/3183741472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3498003980 0x7f34ac1e2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 -- 172.21.15.12:0/3183741472 >> 172.21.15.12:0/3183741472 conn(0x7f34ac0cf4e0 msgr2=0x7f34ac1591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:04.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 -- 172.21.15.12:0/3183741472 shutdown_connections 2023-11-28T21:35:04.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.452+0000 7f34b15df640 1 -- 172.21.15.12:0/3183741472 wait complete. 2023-11-28T21:35:04.478 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-11-28T21:35:04.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21dc0fe640 1 Processor -- start 2023-11-28T21:35:04.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21dc0fe640 1 -- start start 2023-11-28T21:35:04.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21dc0fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21d4154100 0x7f21d415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:04.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21dc0fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f21d40c9fd0 0x7f21d406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:04.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21dc0fe640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d415e270 con 0x7f21d40ca3f0 2023-11-28T21:35:04.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21dc0fe640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d40c4e60 con 0x7f21d40c9fd0 2023-11-28T21:35:04.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21dc0fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d41544e0 con 0x7f21d4154100 2023-11-28T21:35:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21d9699640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f21d40c9fd0 0x7f21d406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21d9699640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f21d40c9fd0 0x7f21d406a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54376/0 (socket says 172.21.15.12:54376) 2023-11-28T21:35:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21d9699640 1 -- 172.21.15.12:0/3851302322 learned_addr learned my addr 172.21.15.12:0/3851302322 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21da69b640 1 --1- 172.21.15.12:0/3851302322 >> v1:172.21.15.12:6789/0 conn(0x7f21d40ca3f0 0x7f21d40c52f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50146/0 (socket says 172.21.15.12:50146) 2023-11-28T21:35:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21d9e9a640 1 --2- 172.21.15.12:0/3851302322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21d4154100 0x7f21d415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21d9699640 1 -- 172.21.15.12:0/3851302322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21d4154100 msgr2=0x7f21d415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21d9699640 1 --2- 172.21.15.12:0/3851302322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21d4154100 0x7f21d415d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21d9699640 1 -- 172.21.15.12:0/3851302322 >> v1:172.21.15.12:6789/0 conn(0x7f21d40ca3f0 legacy=0x7f21d40c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21d9699640 1 -- 172.21.15.12:0/3851302322 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d415d9d0 con 0x7f21d40c9fd0 2023-11-28T21:35:04.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.628+0000 7f21d8e98640 1 -- 172.21.15.12:0/3851302322 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716429948 0 0) 0x7f21d415e270 con 0x7f21d40ca3f0 2023-11-28T21:35:04.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21d9699640 1 --2- 172.21.15.12:0/3851302322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 0x7f21d406a5b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f21c40098a0 tx=0x7f21c401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab4825216e0d0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:04.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21d8e98640 1 -- 172.21.15.12:0/3851302322 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40aa650 con 0x7f21d40c9fd0 2023-11-28T21:35:04.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21d8e98640 1 -- 172.21.15.12:0/3851302322 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c40aa7b0 con 0x7f21d40c9fd0 2023-11-28T21:35:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21d8e98640 1 -- 172.21.15.12:0/3851302322 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40aab40 con 0x7f21d40c9fd0 2023-11-28T21:35:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21dc0fe640 1 -- 172.21.15.12:0/3851302322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 msgr2=0x7f21d406a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:04.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21dc0fe640 1 --2- 172.21.15.12:0/3851302322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 0x7f21d406a5b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f21c40098a0 tx=0x7f21c401c4f0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21dc0fe640 1 -- 172.21.15.12:0/3851302322 shutdown_connections 2023-11-28T21:35:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21dc0fe640 1 --2- 172.21.15.12:0/3851302322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 0x7f21d406a5b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21dc0fe640 1 --2- 172.21.15.12:0/3851302322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21d4154100 0x7f21d415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21dc0fe640 1 -- 172.21.15.12:0/3851302322 >> 172.21.15.12:0/3851302322 conn(0x7f21d406c690 msgr2=0x7f21d4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21dc0fe640 1 -- 172.21.15.12:0/3851302322 shutdown_connections 2023-11-28T21:35:04.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.632+0000 7f21dc0fe640 1 -- 172.21.15.12:0/3851302322 wait complete. 2023-11-28T21:35:04.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21dc0fe640 1 Processor -- start 2023-11-28T21:35:04.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21dc0fe640 1 -- start start 2023-11-28T21:35:04.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21dc0fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 0x7f21d41f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21dc0fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d40ca3f0 0x7f21d40cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21dc0fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21d4154100 0x7f21d40c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21dc0fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40c6d30 con 0x7f21d40c9fd0 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21dc0fe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40c8800 con 0x7f21d40ca3f0 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21dc0fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d41f6fb0 con 0x7f21d4154100 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9e9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 0x7f21d41f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9e9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 0x7f21d41f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54380/0 (socket says 172.21.15.12:54380) 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9e9a640 1 -- 172.21.15.12:0/1852995600 learned_addr learned my addr 172.21.15.12:0/1852995600 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9699640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d40ca3f0 0x7f21d40cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21da69b640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21d4154100 0x7f21d40c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9e9a640 1 -- 172.21.15.12:0/1852995600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21d4154100 msgr2=0x7f21d40c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:04.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9e9a640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21d4154100 0x7f21d40c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9e9a640 1 -- 172.21.15.12:0/1852995600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d40ca3f0 msgr2=0x7f21d40cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9e9a640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d40ca3f0 0x7f21d40cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9e9a640 1 -- 172.21.15.12:0/1852995600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40094d0 con 0x7f21d40c9fd0 2023-11-28T21:35:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9699640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d40ca3f0 0x7f21d40cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21d9e9a640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 0x7f21d41f6a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f21d000d800 tx=0x7f21d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21ca7fc640 1 -- 172.21.15.12:0/1852995600 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000b660 con 0x7f21d40c9fd0 2023-11-28T21:35:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21ca7fc640 1 -- 172.21.15.12:0/1852995600 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d000b7c0 con 0x7f21d40c9fd0 2023-11-28T21:35:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21ca7fc640 1 -- 172.21.15.12:0/1852995600 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000bb30 con 0x7f21d40c9fd0 2023-11-28T21:35:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21dc0fe640 1 -- 172.21.15.12:0/1852995600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d40c8fc0 con 0x7f21d40c9fd0 2023-11-28T21:35:04.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.636+0000 7f21dc0fe640 1 -- 172.21.15.12:0/1852995600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d40c9540 con 0x7f21d40c9fd0 2023-11-28T21:35:04.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.640+0000 7f21dc0fe640 1 -- 172.21.15.12:0/1852995600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d4067c40 con 0x7f21d40c9fd0 2023-11-28T21:35:04.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.640+0000 7f21ca7fc640 1 -- 172.21.15.12:0/1852995600 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f21d000bc90 con 0x7f21d40c9fd0 2023-11-28T21:35:04.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.640+0000 7f21ca7fc640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f21b80664a0 0x7f21b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:04.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.640+0000 7f21ca7fc640 1 -- 172.21.15.12:0/1852995600 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f21d0084700 con 0x7f21d40c9fd0 2023-11-28T21:35:04.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.640+0000 7f21d9699640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f21b80664a0 0x7f21b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:04.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.640+0000 7f21d9699640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f21b80664a0 0x7f21b8068960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21c40ae000 tx=0x7f21c40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.644+0000 7f21ca7fc640 1 -- 172.21.15.12:0/1852995600 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21d00517f0 con 0x7f21d40c9fd0 2023-11-28T21:35:04.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:04.840+0000 7f21dc0fe640 1 -- 172.21.15.12:0/1852995600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f21d4074f60 con 0x7f21d40c9fd0 2023-11-28T21:35:05.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21ca7fc640 1 -- 172.21.15.12:0/1852995600 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v10) v1 ==== 115+0+0 (secure 0 0 0) 0x7f21d0054e10 con 0x7f21d40c9fd0 2023-11-28T21:35:05.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 -- 172.21.15.12:0/1852995600 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f21b80664a0 msgr2=0x7f21b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:05.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f21b80664a0 0x7f21b8068960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21c40ae000 tx=0x7f21c40a7000 comp rx=0 tx=0).stop 2023-11-28T21:35:05.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 -- 172.21.15.12:0/1852995600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 msgr2=0x7f21d41f6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:05.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 0x7f21d41f6a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f21d000d800 tx=0x7f21d000dcc0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 -- 172.21.15.12:0/1852995600 shutdown_connections 2023-11-28T21:35:05.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f21b80664a0 0x7f21b8068960 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21d4154100 0x7f21d40c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21d40ca3f0 0x7f21d40cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 --2- 172.21.15.12:0/1852995600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21d40c9fd0 0x7f21d41f6a70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 -- 172.21.15.12:0/1852995600 >> 172.21.15.12:0/1852995600 conn(0x7f21d406c690 msgr2=0x7f21d4153230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:05.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 -- 172.21.15.12:0/1852995600 shutdown_connections 2023-11-28T21:35:05.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.452+0000 7f21dc0fe640 1 -- 172.21.15.12:0/1852995600 wait complete. 2023-11-28T21:35:05.481 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T21:35:05.481 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:35:05.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.624+0000 7fac2266b640 1 Processor -- start 2023-11-28T21:35:05.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.624+0000 7fac2266b640 1 -- start start 2023-11-28T21:35:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac2266b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c1540f0 0x7fac1c0c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac2266b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fac1c15b180 0x7fac1c15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac2266b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac1c15d210 0x7fac1c15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac2266b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1c06e200 con 0x7fac1c15d210 2023-11-28T21:35:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac2266b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1c06d2a0 con 0x7fac1c15b180 2023-11-28T21:35:05.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac2266b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c06d580 con 0x7fac1c1540f0 2023-11-28T21:35:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac20c08640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac1c15d210 0x7fac1c15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac20c08640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac1c15d210 0x7fac1c15d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37930/0 (socket says 172.21.15.12:37930) 2023-11-28T21:35:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac20c08640 1 -- 172.21.15.12:0/4126323600 learned_addr learned my addr 172.21.15.12:0/4126323600 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac1bfff640 1 --2- 172.21.15.12:0/4126323600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c1540f0 0x7fac1c0c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:05.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac1b7fe640 1 --2- 172.21.15.12:0/4126323600 >> v2:172.21.15.144:3300/0 conn(0x7fac1c15b180 0x7fac1c15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:05.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac20c08640 1 -- 172.21.15.12:0/4126323600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c1540f0 msgr2=0x7fac1c0c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:05.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac20c08640 1 --2- 172.21.15.12:0/4126323600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c1540f0 0x7fac1c0c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac20c08640 1 -- 172.21.15.12:0/4126323600 >> v2:172.21.15.144:3300/0 conn(0x7fac1c15b180 msgr2=0x7fac1c15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:05.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac20c08640 1 --2- 172.21.15.12:0/4126323600 >> v2:172.21.15.144:3300/0 conn(0x7fac1c15b180 0x7fac1c15b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.628+0000 7fac20c08640 1 -- 172.21.15.12:0/4126323600 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c06d860 con 0x7fac1c15d210 2023-11-28T21:35:05.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac20c08640 1 --2- 172.21.15.12:0/4126323600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c15d210 0x7fac1c15d6a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fac1c06c6d0 tx=0x7fac1401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a636d31819e4187 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:05.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac1affd640 1 -- 172.21.15.12:0/4126323600 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140a5730 con 0x7fac1c15d210 2023-11-28T21:35:05.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac1affd640 1 -- 172.21.15.12:0/4126323600 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac140a5890 con 0x7fac1c15d210 2023-11-28T21:35:05.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac1affd640 1 -- 172.21.15.12:0/4126323600 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140a5c20 con 0x7fac1c15d210 2023-11-28T21:35:05.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 -- 172.21.15.12:0/4126323600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c15d210 msgr2=0x7fac1c15d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:05.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 --2- 172.21.15.12:0/4126323600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c15d210 0x7fac1c15d6a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fac1c06c6d0 tx=0x7fac1401c9a0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 -- 172.21.15.12:0/4126323600 shutdown_connections 2023-11-28T21:35:05.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 --2- 172.21.15.12:0/4126323600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c15d210 0x7fac1c15d6a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 --2- 172.21.15.12:0/4126323600 >> v2:172.21.15.144:3300/0 conn(0x7fac1c15b180 0x7fac1c15b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 --2- 172.21.15.12:0/4126323600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c1540f0 0x7fac1c0c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 -- 172.21.15.12:0/4126323600 >> 172.21.15.12:0/4126323600 conn(0x7fac1c066360 msgr2=0x7fac1c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:05.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 -- 172.21.15.12:0/4126323600 shutdown_connections 2023-11-28T21:35:05.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 -- 172.21.15.12:0/4126323600 wait complete. 2023-11-28T21:35:05.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 Processor -- start 2023-11-28T21:35:05.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.632+0000 7fac2266b640 1 -- start start 2023-11-28T21:35:05.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac2266b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c1540f0 0x7fac1c1f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:05.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac2266b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c15b180 0x7fac1c152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:05.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac2266b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1c15d210 0x7fac1c1f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:05.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac2266b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1f31e0 con 0x7fac1c1540f0 2023-11-28T21:35:05.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac2266b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1f6ed0 con 0x7fac1c15d210 2023-11-28T21:35:05.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac2266b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c151980 con 0x7fac1c15b180 2023-11-28T21:35:05.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c1540f0 0x7fac1c1f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:05.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c1540f0 0x7fac1c1f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37946/0 (socket says 172.21.15.12:37946) 2023-11-28T21:35:05.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1bfff640 1 -- 172.21.15.12:0/507983219 learned_addr learned my addr 172.21.15.12:0/507983219 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:05.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1b7fe640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c15b180 0x7fac1c152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:05.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac20c08640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1c15d210 0x7fac1c1f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:05.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1bfff640 1 -- 172.21.15.12:0/507983219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c15b180 msgr2=0x7fac1c152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1bfff640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c15b180 0x7fac1c152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1bfff640 1 -- 172.21.15.12:0/507983219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1c15d210 msgr2=0x7fac1c1f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1bfff640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1c15d210 0x7fac1c1f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1bfff640 1 -- 172.21.15.12:0/507983219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009b30 con 0x7fac1c1540f0 2023-11-28T21:35:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac20c08640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1c15d210 0x7fac1c1f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:35:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1b7fe640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c15b180 0x7fac1c152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:35:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac1bfff640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c1540f0 0x7fac1c1f3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00e8b0 tx=0x7fac0c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac18ff9640 1 -- 172.21.15.12:0/507983219 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c00c940 con 0x7fac1c1540f0 2023-11-28T21:35:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac18ff9640 1 -- 172.21.15.12:0/507983219 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c00caa0 con 0x7fac1c1540f0 2023-11-28T21:35:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac2266b640 1 -- 172.21.15.12:0/507983219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c151c60 con 0x7fac1c1540f0 2023-11-28T21:35:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac18ff9640 1 -- 172.21.15.12:0/507983219 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c00ce30 con 0x7fac1c1540f0 2023-11-28T21:35:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac2266b640 1 -- 172.21.15.12:0/507983219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c1f2e00 con 0x7fac1c1540f0 2023-11-28T21:35:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac2266b640 1 -- 172.21.15.12:0/507983219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1c1f2fc0 con 0x7fac1c1540f0 2023-11-28T21:35:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.636+0000 7fac18ff9640 1 -- 172.21.15.12:0/507983219 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fac0c013470 con 0x7fac1c1540f0 2023-11-28T21:35:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.640+0000 7fac18ff9640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac08066470 0x7fac08068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.640+0000 7fac18ff9640 1 -- 172.21.15.12:0/507983219 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fac0c084320 con 0x7fac1c1540f0 2023-11-28T21:35:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.640+0000 7fac1b7fe640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac08066470 0x7fac08068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:05.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.640+0000 7fac1b7fe640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac08066470 0x7fac08068930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac1c144b70 tx=0x7fac10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:05.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.644+0000 7fac18ff9640 1 -- 172.21.15.12:0/507983219 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fac0c0514a0 con 0x7fac1c1540f0 2023-11-28T21:35:05.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.832+0000 7fac2266b640 1 -- 172.21.15.12:0/507983219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac1c1f3450 con 0x7fac1c1540f0 2023-11-28T21:35:05.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.832+0000 7fac18ff9640 1 -- 172.21.15.12:0/507983219 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6514 (secure 0 0 0) 0x7fac0c054ac0 con 0x7fac1c1540f0 2023-11-28T21:35:05.835 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:05.836 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:35:04.843297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:creating","state_seq":1,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:35:05.836 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 10 2023-11-28T21:35:05.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.832+0000 7fac2266b640 1 -- 172.21.15.12:0/507983219 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac08066470 msgr2=0x7fac08068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:05.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.832+0000 7fac2266b640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac08066470 0x7fac08068930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac1c144b70 tx=0x7fac10008040 comp rx=0 tx=0).stop 2023-11-28T21:35:05.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.832+0000 7fac2266b640 1 -- 172.21.15.12:0/507983219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c1540f0 msgr2=0x7fac1c1f3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:05.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.832+0000 7fac2266b640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c1540f0 0x7fac1c1f3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00e8b0 tx=0x7fac0c00ed70 comp rx=0 tx=0).stop 2023-11-28T21:35:05.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.836+0000 7fac2266b640 1 -- 172.21.15.12:0/507983219 shutdown_connections 2023-11-28T21:35:05.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.836+0000 7fac2266b640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac08066470 0x7fac08068930 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.836+0000 7fac2266b640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1c15d210 0x7fac1c1f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.836+0000 7fac2266b640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1c15b180 0x7fac1c152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.836+0000 7fac2266b640 1 --2- 172.21.15.12:0/507983219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1c1540f0 0x7fac1c1f3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:05.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.836+0000 7fac2266b640 1 -- 172.21.15.12:0/507983219 >> 172.21.15.12:0/507983219 conn(0x7fac1c066360 msgr2=0x7fac1c15c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:05.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.836+0000 7fac2266b640 1 -- 172.21.15.12:0/507983219 shutdown_connections 2023-11-28T21:35:05.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.836+0000 7fac2266b640 1 -- 172.21.15.12:0/507983219 wait complete. 2023-11-28T21:35:05.857 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:05.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.996+0000 7f32f47f8640 1 Processor -- start 2023-11-28T21:35:05.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.996+0000 7f32f47f8640 1 -- start start 2023-11-28T21:35:06.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.996+0000 7f32f47f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ec153ee0 0x7f32ec15a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.996+0000 7f32f47f8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f32ec06a5b0 0x7f32ec0c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.996+0000 7f32f47f8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32ec0c6350 con 0x7f32ec070bd0 2023-11-28T21:35:06.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.996+0000 7f32f47f8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ec0c6b90 con 0x7f32ec06a5b0 2023-11-28T21:35:06.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:05.996+0000 7f32f47f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec072410 con 0x7f32ec153ee0 2023-11-28T21:35:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.000+0000 7f32f2594640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ec153ee0 0x7f32ec15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.000+0000 7f32f1d93640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f32ec06a5b0 0x7f32ec0c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.000+0000 7f32f2d95640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f32ec070bd0 0x7f32ec071060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52614/0 (socket says 172.21.15.12:52614) 2023-11-28T21:35:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.000+0000 7f32f2d95640 1 -- 172.21.15.12:0/1929618296 learned_addr learned my addr 172.21.15.12:0/1929618296 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.000+0000 7f32f1d93640 1 -- 172.21.15.12:0/1929618296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ec153ee0 msgr2=0x7f32ec15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.000+0000 7f32f1d93640 1 --2- 172.21.15.12:0/1929618296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ec153ee0 0x7f32ec15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.000+0000 7f32f1d93640 1 -- 172.21.15.12:0/1929618296 >> v1:172.21.15.144:6789/0 conn(0x7f32ec070bd0 legacy=0x7f32ec071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.000+0000 7f32f1d93640 1 -- 172.21.15.12:0/1929618296 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec0c65d0 con 0x7f32ec06a5b0 2023-11-28T21:35:06.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.000+0000 7f32f1d93640 1 --2- 172.21.15.12:0/1929618296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ec06a5b0 0x7f32ec0c7a80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0097a0 tx=0x7f32dc01ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db1974765bf7befc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:06.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f1592640 1 -- 172.21.15.12:0/1929618296 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0a8950 con 0x7f32ec06a5b0 2023-11-28T21:35:06.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f1592640 1 -- 172.21.15.12:0/1929618296 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc0a8ab0 con 0x7f32ec06a5b0 2023-11-28T21:35:06.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f1592640 1 -- 172.21.15.12:0/1929618296 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0a8e40 con 0x7f32ec06a5b0 2023-11-28T21:35:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 -- 172.21.15.12:0/1929618296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ec06a5b0 msgr2=0x7f32ec0c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 --2- 172.21.15.12:0/1929618296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ec06a5b0 0x7f32ec0c7a80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0097a0 tx=0x7f32dc01ec50 comp rx=0 tx=0).stop 2023-11-28T21:35:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 -- 172.21.15.12:0/1929618296 shutdown_connections 2023-11-28T21:35:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 --2- 172.21.15.12:0/1929618296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ec06a5b0 0x7f32ec0c7a80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 --2- 172.21.15.12:0/1929618296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ec153ee0 0x7f32ec15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 -- 172.21.15.12:0/1929618296 >> 172.21.15.12:0/1929618296 conn(0x7f32ec0c8450 msgr2=0x7f32ec0c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 -- 172.21.15.12:0/1929618296 shutdown_connections 2023-11-28T21:35:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 -- 172.21.15.12:0/1929618296 wait complete. 2023-11-28T21:35:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 Processor -- start 2023-11-28T21:35:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 -- start start 2023-11-28T21:35:06.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ec06a5b0 0x7f32ec149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ec070bd0 0x7f32ec1e4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ec153ee0 0x7f32ec1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec0c4530 con 0x7f32ec070bd0 2023-11-28T21:35:06.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec0c3080 con 0x7f32ec153ee0 2023-11-28T21:35:06.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f47f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1e3220 con 0x7f32ec06a5b0 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f1d93640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ec070bd0 0x7f32ec1e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f1d93640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ec070bd0 0x7f32ec1e4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37960/0 (socket says 172.21.15.12:37960) 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f1d93640 1 -- 172.21.15.12:0/1565255873 learned_addr learned my addr 172.21.15.12:0/1565255873 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.004+0000 7f32f2594640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ec06a5b0 0x7f32ec149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f2d95640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ec153ee0 0x7f32ec1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f1d93640 1 -- 172.21.15.12:0/1565255873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ec06a5b0 msgr2=0x7f32ec149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f1d93640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ec06a5b0 0x7f32ec149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f1d93640 1 -- 172.21.15.12:0/1565255873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ec153ee0 msgr2=0x7f32ec1514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f1d93640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ec153ee0 0x7f32ec1514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f1d93640 1 -- 172.21.15.12:0/1565255873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc0093b0 con 0x7f32ec070bd0 2023-11-28T21:35:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f1d93640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ec070bd0 0x7f32ec1e4070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f32dc000c10 tx=0x7f32dc01b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:06.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32e2ffd640 1 -- 172.21.15.12:0/1565255873 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b1de0 con 0x7f32ec070bd0 2023-11-28T21:35:06.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32e2ffd640 1 -- 172.21.15.12:0/1565255873 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc006dc0 con 0x7f32ec070bd0 2023-11-28T21:35:06.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32e2ffd640 1 -- 172.21.15.12:0/1565255873 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc005be0 con 0x7f32ec070bd0 2023-11-28T21:35:06.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f47f8640 1 -- 172.21.15.12:0/1565255873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec1e3500 con 0x7f32ec070bd0 2023-11-28T21:35:06.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f47f8640 1 -- 172.21.15.12:0/1565255873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec153660 con 0x7f32ec070bd0 2023-11-28T21:35:06.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f47f8640 1 -- 172.21.15.12:0/1565255873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ec074f60 con 0x7f32ec070bd0 2023-11-28T21:35:06.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32e2ffd640 1 -- 172.21.15.12:0/1565255873 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f32dc0b7040 con 0x7f32ec070bd0 2023-11-28T21:35:06.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32e2ffd640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32d00664a0 0x7f32d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32e2ffd640 1 -- 172.21.15.12:0/1565255873 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f32dc121250 con 0x7f32ec070bd0 2023-11-28T21:35:06.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.008+0000 7f32f2594640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32d00664a0 0x7f32d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.012+0000 7f32f2594640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32d00664a0 0x7f32d0068960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009af0 tx=0x7f32e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:06.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.012+0000 7f32e2ffd640 1 -- 172.21.15.12:0/1565255873 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f32dc0ee340 con 0x7f32ec070bd0 2023-11-28T21:35:06.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.172+0000 7f32f47f8640 1 -- 172.21.15.12:0/1565255873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32ec15a710 con 0x7f32ec070bd0 2023-11-28T21:35:06.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.172+0000 7f32e2ffd640 1 -- 172.21.15.12:0/1565255873 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7f32dc0f1960 con 0x7f32ec070bd0 2023-11-28T21:35:06.176 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:06.176 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 -- 172.21.15.12:0/1565255873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32d00664a0 msgr2=0x7f32d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32d00664a0 0x7f32d0068960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009af0 tx=0x7f32e8008040 comp rx=0 tx=0).stop 2023-11-28T21:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 -- 172.21.15.12:0/1565255873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ec070bd0 msgr2=0x7f32ec1e4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ec070bd0 0x7f32ec1e4070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f32dc000c10 tx=0x7f32dc01b9f0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 -- 172.21.15.12:0/1565255873 shutdown_connections 2023-11-28T21:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32d00664a0 0x7f32d0068960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ec153ee0 0x7f32ec1514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32ec070bd0 0x7f32ec1e4070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 --2- 172.21.15.12:0/1565255873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ec06a5b0 0x7f32ec149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 -- 172.21.15.12:0/1565255873 >> 172.21.15.12:0/1565255873 conn(0x7f32ec0c8450 msgr2=0x7f32ec073ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 -- 172.21.15.12:0/1565255873 shutdown_connections 2023-11-28T21:35:06.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.176+0000 7f32f47f8640 1 -- 172.21.15.12:0/1565255873 wait complete. 2023-11-28T21:35:06.197 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T21:35:06.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:35:06.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd7edf5640 1 Processor -- start 2023-11-28T21:35:06.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd7edf5640 1 -- start start 2023-11-28T21:35:06.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd7edf5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efd780c52f0 0x7efd780c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd7edf5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7815e270 con 0x7efd7815d320 2023-11-28T21:35:06.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd7edf5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd780c4e60 con 0x7efd78154100 2023-11-28T21:35:06.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd7edf5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7815d740 con 0x7efd780c52f0 2023-11-28T21:35:06.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd77fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efd7815d320 0x7efd7806a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36302/0 (socket says 172.21.15.12:36302) 2023-11-28T21:35:06.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd77fff640 1 -- 172.21.15.12:0/2319099245 learned_addr learned my addr 172.21.15.12:0/2319099245 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:06.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd7d392640 1 --2- 172.21.15.12:0/2319099245 >> v2:172.21.15.144:3300/0 conn(0x7efd780c52f0 0x7efd780c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902638139 0 0) 0x7efd7815e270 con 0x7efd7815d320 2023-11-28T21:35:06.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd68003530 con 0x7efd7815d320 2023-11-28T21:35:06.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3348448381 0 0) 0x7efd68003530 con 0x7efd7815d320 2023-11-28T21:35:06.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 >> v2:172.21.15.144:3300/0 conn(0x7efd780c52f0 msgr2=0x7efd780c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:06.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 --2- 172.21.15.12:0/2319099245 >> v2:172.21.15.144:3300/0 conn(0x7efd780c52f0 0x7efd780c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 >> v1:172.21.15.144:6789/0 conn(0x7efd78154100 legacy=0x7efd780c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd781544e0 con 0x7efd7815d320 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7efd64001680 con 0x7efd7815d320 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 >> v1:172.21.15.12:6789/0 conn(0x7efd7815d320 legacy=0x7efd7806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd680039f0 0x7efd68003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd680047d0 0x7efd68004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd68005190 0x7efd6800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7815d740 con 0x7efd680047d0 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6800c0c0 con 0x7efd68005190 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6800c280 con 0x7efd680039f0 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd77fff640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd680047d0 0x7efd68004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd7d392640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd68005190 0x7efd6800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd7cb91640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd680039f0 0x7efd68003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd77fff640 1 -- 172.21.15.12:0/2319099245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd680039f0 msgr2=0x7efd68003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:06.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd77fff640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd680039f0 0x7efd68003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd77fff640 1 -- 172.21.15.12:0/2319099245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd68005190 msgr2=0x7efd6800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:06.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd77fff640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd68005190 0x7efd6800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd7d392640 1 -- 172.21.15.12:0/2319099245 reap_dead start 2023-11-28T21:35:06.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.340+0000 7efd77fff640 1 -- 172.21.15.12:0/2319099245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68003530 con 0x7efd680047d0 2023-11-28T21:35:06.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd77fff640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd680047d0 0x7efd68004c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd6400c240 tx=0x7efd6401f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846506c277abcf12 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:06.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64005ae0 con 0x7efd680047d0 2023-11-28T21:35:06.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd640ab070 con 0x7efd680047d0 2023-11-28T21:35:06.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd777fe640 1 -- 172.21.15.12:0/2319099245 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64007070 con 0x7efd680047d0 2023-11-28T21:35:06.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd7edf5640 1 -- 172.21.15.12:0/2319099245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd680047d0 msgr2=0x7efd68004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd7edf5640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd680047d0 0x7efd68004c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd6400c240 tx=0x7efd6401f330 comp rx=0 tx=0).stop 2023-11-28T21:35:06.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd7edf5640 1 -- 172.21.15.12:0/2319099245 shutdown_connections 2023-11-28T21:35:06.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd7edf5640 1 --2- 172.21.15.12:0/2319099245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd680047d0 0x7efd68004c50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd7edf5640 1 -- 172.21.15.12:0/2319099245 >> 172.21.15.12:0/2319099245 conn(0x7efd7806c690 msgr2=0x7efd78066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:06.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd7edf5640 1 -- 172.21.15.12:0/2319099245 shutdown_connections 2023-11-28T21:35:06.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.344+0000 7efd7edf5640 1 -- 172.21.15.12:0/2319099245 wait complete. 2023-11-28T21:35:06.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7edf5640 1 Processor -- start 2023-11-28T21:35:06.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7edf5640 1 -- start start 2023-11-28T21:35:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7edf5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680047d0 0x7efd78148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7edf5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd780cc740 0x7efd781529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7edf5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd78144d10 0x7efd78152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7edf5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd781f3270 con 0x7efd780cc740 2023-11-28T21:35:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7edf5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd781524d0 con 0x7efd680047d0 2023-11-28T21:35:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7edf5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd78150e50 con 0x7efd78144d10 2023-11-28T21:35:06.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd77fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd780cc740 0x7efd781529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd77fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd780cc740 0x7efd781529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37980/0 (socket says 172.21.15.12:37980) 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd77fff640 1 -- 172.21.15.12:0/3526187031 learned_addr learned my addr 172.21.15.12:0/3526187031 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7cb91640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680047d0 0x7efd78148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7d392640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd78144d10 0x7efd78152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd77fff640 1 -- 172.21.15.12:0/3526187031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd78144d10 msgr2=0x7efd78152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd77fff640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd78144d10 0x7efd78152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd77fff640 1 -- 172.21.15.12:0/3526187031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680047d0 msgr2=0x7efd78148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd77fff640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680047d0 0x7efd78148570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd77fff640 1 -- 172.21.15.12:0/3526187031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd78151130 con 0x7efd780cc740 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7d392640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd78144d10 0x7efd78152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:06.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd77fff640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd780cc740 0x7efd781529d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efd6401f160 tx=0x7efd68005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:06.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd757fa640 1 -- 172.21.15.12:0/3526187031 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64001680 con 0x7efd780cc740 2023-11-28T21:35:06.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd757fa640 1 -- 172.21.15.12:0/3526187031 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd64007070 con 0x7efd780cc740 2023-11-28T21:35:06.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd757fa640 1 -- 172.21.15.12:0/3526187031 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64007440 con 0x7efd780cc740 2023-11-28T21:35:06.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.348+0000 7efd7edf5640 1 -- 172.21.15.12:0/3526187031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd78151410 con 0x7efd780cc740 2023-11-28T21:35:06.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.352+0000 7efd7edf5640 1 -- 172.21.15.12:0/3526187031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd780c96a0 con 0x7efd780cc740 2023-11-28T21:35:06.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.352+0000 7efd7edf5640 1 -- 172.21.15.12:0/3526187031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd78067c40 con 0x7efd780cc740 2023-11-28T21:35:06.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.352+0000 7efd757fa640 1 -- 172.21.15.12:0/3526187031 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7efd640b9030 con 0x7efd780cc740 2023-11-28T21:35:06.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.352+0000 7efd757fa640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 0x7efd5c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.352+0000 7efd757fa640 1 -- 172.21.15.12:0/3526187031 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efd640b7070 con 0x7efd780cc740 2023-11-28T21:35:06.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.352+0000 7efd7cb91640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 0x7efd5c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.352+0000 7efd7cb91640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 0x7efd5c068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd6c007fd0 tx=0x7efd6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:06.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.356+0000 7efd757fa640 1 -- 172.21.15.12:0/3526187031 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efd640ee5b0 con 0x7efd780cc740 2023-11-28T21:35:06.388 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:35:06.392 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:35:06.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.512+0000 7efd7edf5640 1 -- 172.21.15.12:0/3526187031 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd7815df90 con 0x7efd5c0664a0 2023-11-28T21:35:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.524+0000 7efd757fa640 1 -- 172.21.15.12:0/3526187031 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345285 (secure 0 0 0) 0x7efd6c001c50 con 0x7efd5c0664a0 2023-11-28T21:35:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:06.535 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:35:06.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 -- 172.21.15.12:0/3526187031 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 msgr2=0x7efd5c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 0x7efd5c068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd6c007fd0 tx=0x7efd6c009040 comp rx=0 tx=0).stop 2023-11-28T21:35:06.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 -- 172.21.15.12:0/3526187031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd780cc740 msgr2=0x7efd781529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd780cc740 0x7efd781529d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efd6401f160 tx=0x7efd68005800 comp rx=0 tx=0).stop 2023-11-28T21:35:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 -- 172.21.15.12:0/3526187031 shutdown_connections 2023-11-28T21:35:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 0x7efd5c068960 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd78144d10 0x7efd78152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd780cc740 0x7efd781529d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 --2- 172.21.15.12:0/3526187031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680047d0 0x7efd78148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 -- 172.21.15.12:0/3526187031 >> 172.21.15.12:0/3526187031 conn(0x7efd7806c690 msgr2=0x7efd780cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 -- 172.21.15.12:0/3526187031 shutdown_connections 2023-11-28T21:35:06.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.528+0000 7efd7edf5640 1 -- 172.21.15.12:0/3526187031 wait complete. 2023-11-28T21:35:06.551 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-11-28T21:35:05.822267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218356,"kb_used_data":2172,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756364,"statfs":{"total":773094113280,"available":772870516736,"internally_reserved":0,"allocated":2224128,"data_stored":1987331,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003155"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343838+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:34:42.343838+0000","last_peered":"2023-11-28T21:34:42.343838+0000","last_clean":"2023-11-28T21:34:42.343838+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:34:42.343838+0000","last_undegraded":"2023-11-28T21:34:42.343838+0000","last_fullsized":"2023-11-28T21:34:42.343838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:46.167511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341374+0000","last_change":"2023-11-28T21:34:37.335656+0000","last_active":"2023-11-28T21:34:41.341374+0000","last_peered":"2023-11-28T21:34:41.341374+0000","last_clean":"2023-11-28T21:34:41.341374+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:34:41.341374+0000","last_undegraded":"2023-11-28T21:34:41.341374+0000","last_fullsized":"2023-11-28T21:34:41.341374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:25:49.707378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359842+0000","last_change":"2023-11-28T21:34:41.483856+0000","last_active":"2023-11-28T21:34:42.359842+0000","last_peered":"2023-11-28T21:34:42.359842+0000","last_clean":"2023-11-28T21:34:42.359842+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:34:42.359842+0000","last_undegraded":"2023-11-28T21:34:42.359842+0000","last_fullsized":"2023-11-28T21:34:42.359842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:50:56.005578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.340464+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:34:42.340464+0000","last_peered":"2023-11-28T21:34:42.340464+0000","last_clean":"2023-11-28T21:34:42.340464+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:34:42.340464+0000","last_undegraded":"2023-11-28T21:34:42.340464+0000","last_fullsized":"2023-11-28T21:34:42.340464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344725+0000","last_change":"2023-11-28T21:34:37.333649+0000","last_active":"2023-11-28T21:34:41.344725+0000","last_peered":"2023-11-28T21:34:41.344725+0000","last_clean":"2023-11-28T21:34:41.344725+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:34:41.344725+0000","last_undegraded":"2023-11-28T21:34:41.344725+0000","last_fullsized":"2023-11-28T21:34:41.344725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:21:42.041590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594292+0000","last_change":"2023-11-28T21:34:41.388232+0000","last_active":"2023-11-28T21:34:42.594292+0000","last_peered":"2023-11-28T21:34:42.594292+0000","last_clean":"2023-11-28T21:34:42.594292+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:34:42.594292+0000","last_undegraded":"2023-11-28T21:34:42.594292+0000","last_fullsized":"2023-11-28T21:34:42.594292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:22.960012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.345785+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:34:42.345785+0000","last_peered":"2023-11-28T21:34:42.345785+0000","last_clean":"2023-11-28T21:34:42.345785+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:34:42.345785+0000","last_undegraded":"2023-11-28T21:34:42.345785+0000","last_fullsized":"2023-11-28T21:34:42.345785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342908+0000","last_change":"2023-11-28T21:34:37.348126+0000","last_active":"2023-11-28T21:34:41.342908+0000","last_peered":"2023-11-28T21:34:41.342908+0000","last_clean":"2023-11-28T21:34:41.342908+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:34:41.342908+0000","last_undegraded":"2023-11-28T21:34:41.342908+0000","last_fullsized":"2023-11-28T21:34:41.342908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:47.982143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.596939+0000","last_change":"2023-11-28T21:34:41.388539+0000","last_active":"2023-11-28T21:34:42.596939+0000","last_peered":"2023-11-28T21:34:42.596939+0000","last_clean":"2023-11-28T21:34:42.596939+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:34:42.596939+0000","last_undegraded":"2023-11-28T21:34:42.596939+0000","last_fullsized":"2023-11-28T21:34:42.596939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343904+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:34:42.343904+0000","last_peered":"2023-11-28T21:34:42.343904+0000","last_clean":"2023-11-28T21:34:42.343904+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:34:42.343904+0000","last_undegraded":"2023-11-28T21:34:42.343904+0000","last_fullsized":"2023-11-28T21:34:42.343904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346931+0000","last_change":"2023-11-28T21:34:37.337266+0000","last_active":"2023-11-28T21:34:41.346931+0000","last_peered":"2023-11-28T21:34:41.346931+0000","last_clean":"2023-11-28T21:34:41.346931+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:34:41.346931+0000","last_undegraded":"2023-11-28T21:34:41.346931+0000","last_fullsized":"2023-11-28T21:34:41.346931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:32:48.848989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595154+0000","last_change":"2023-11-28T21:34:41.372957+0000","last_active":"2023-11-28T21:34:42.595154+0000","last_peered":"2023-11-28T21:34:42.595154+0000","last_clean":"2023-11-28T21:34:42.595154+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:34:42.595154+0000","last_undegraded":"2023-11-28T21:34:42.595154+0000","last_fullsized":"2023-11-28T21:34:42.595154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367244+0000","last_change":"2023-11-28T21:34:37.353174+0000","last_active":"2023-11-28T21:34:41.367244+0000","last_peered":"2023-11-28T21:34:41.367244+0000","last_clean":"2023-11-28T21:34:41.367244+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:34:41.367244+0000","last_undegraded":"2023-11-28T21:34:41.367244+0000","last_fullsized":"2023-11-28T21:34:41.367244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:32.966564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.345795+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:34:42.345795+0000","last_peered":"2023-11-28T21:34:42.345795+0000","last_clean":"2023-11-28T21:34:42.345795+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:34:42.345795+0000","last_undegraded":"2023-11-28T21:34:42.345795+0000","last_fullsized":"2023-11-28T21:34:42.345795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.599114+0000","last_change":"2023-11-28T21:34:41.364738+0000","last_active":"2023-11-28T21:34:42.599114+0000","last_peered":"2023-11-28T21:34:42.599114+0000","last_clean":"2023-11-28T21:34:42.599114+0000","last_became_active":"2023-11-28T21:34:41.362862+0000","last_became_peered":"2023-11-28T21:34:41.362862+0000","last_unstale":"2023-11-28T21:34:42.599114+0000","last_undegraded":"2023-11-28T21:34:42.599114+0000","last_fullsized":"2023-11-28T21:34:42.599114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:13.830214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339933+0000","last_change":"2023-11-28T21:34:37.313495+0000","last_active":"2023-11-28T21:34:41.339933+0000","last_peered":"2023-11-28T21:34:41.339933+0000","last_clean":"2023-11-28T21:34:41.339933+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:34:41.339933+0000","last_undegraded":"2023-11-28T21:34:41.339933+0000","last_fullsized":"2023-11-28T21:34:41.339933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:32:50.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345994+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:34:41.345994+0000","last_peered":"2023-11-28T21:34:41.345994+0000","last_clean":"2023-11-28T21:34:41.345994+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:34:41.345994+0000","last_undegraded":"2023-11-28T21:34:41.345994+0000","last_fullsized":"2023-11-28T21:34:41.345994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:53.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597596+0000","last_change":"2023-11-28T21:34:41.367580+0000","last_active":"2023-11-28T21:34:42.597596+0000","last_peered":"2023-11-28T21:34:42.597596+0000","last_clean":"2023-11-28T21:34:42.597596+0000","last_became_active":"2023-11-28T21:34:41.366442+0000","last_became_peered":"2023-11-28T21:34:41.366442+0000","last_unstale":"2023-11-28T21:34:42.597596+0000","last_undegraded":"2023-11-28T21:34:42.597596+0000","last_fullsized":"2023-11-28T21:34:42.597596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339648+0000","last_change":"2023-11-28T21:34:37.326569+0000","last_active":"2023-11-28T21:34:41.339648+0000","last_peered":"2023-11-28T21:34:41.339648+0000","last_clean":"2023-11-28T21:34:41.339648+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:34:41.339648+0000","last_undegraded":"2023-11-28T21:34:41.339648+0000","last_fullsized":"2023-11-28T21:34:41.339648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:39:14.264355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343240+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:34:41.343240+0000","last_peered":"2023-11-28T21:34:41.343240+0000","last_clean":"2023-11-28T21:34:41.343240+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:34:41.343240+0000","last_undegraded":"2023-11-28T21:34:41.343240+0000","last_fullsized":"2023-11-28T21:34:41.343240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593262+0000","last_change":"2023-11-28T21:34:41.483661+0000","last_active":"2023-11-28T21:34:42.593262+0000","last_peered":"2023-11-28T21:34:42.593262+0000","last_clean":"2023-11-28T21:34:42.593262+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:34:42.593262+0000","last_undegraded":"2023-11-28T21:34:42.593262+0000","last_fullsized":"2023-11-28T21:34:42.593262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.347011+0000","last_change":"2023-11-28T21:34:37.336922+0000","last_active":"2023-11-28T21:34:41.347011+0000","last_peered":"2023-11-28T21:34:41.347011+0000","last_clean":"2023-11-28T21:34:41.347011+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:34:41.347011+0000","last_undegraded":"2023-11-28T21:34:41.347011+0000","last_fullsized":"2023-11-28T21:34:41.347011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:45.663475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.469299+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:34:41.469299+0000","last_peered":"2023-11-28T21:34:41.469299+0000","last_clean":"2023-11-28T21:34:41.469299+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:34:41.469299+0000","last_undegraded":"2023-11-28T21:34:41.469299+0000","last_fullsized":"2023-11-28T21:34:41.469299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.599296+0000","last_change":"2023-11-28T21:34:41.382756+0000","last_active":"2023-11-28T21:34:42.599296+0000","last_peered":"2023-11-28T21:34:42.599296+0000","last_clean":"2023-11-28T21:34:42.599296+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:34:42.599296+0000","last_undegraded":"2023-11-28T21:34:42.599296+0000","last_fullsized":"2023-11-28T21:34:42.599296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:07.402513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344639+0000","last_change":"2023-11-28T21:34:37.320446+0000","last_active":"2023-11-28T21:34:41.344639+0000","last_peered":"2023-11-28T21:34:41.344639+0000","last_clean":"2023-11-28T21:34:41.344639+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:34:41.344639+0000","last_undegraded":"2023-11-28T21:34:41.344639+0000","last_fullsized":"2023-11-28T21:34:41.344639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:33.404441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.346048+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:34:42.346048+0000","last_peered":"2023-11-28T21:34:42.346048+0000","last_clean":"2023-11-28T21:34:42.346048+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:34:42.346048+0000","last_undegraded":"2023-11-28T21:34:42.346048+0000","last_fullsized":"2023-11-28T21:34:42.346048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595588+0000","last_change":"2023-11-28T21:34:41.485107+0000","last_active":"2023-11-28T21:34:42.595588+0000","last_peered":"2023-11-28T21:34:42.595588+0000","last_clean":"2023-11-28T21:34:42.595588+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:34:42.595588+0000","last_undegraded":"2023-11-28T21:34:42.595588+0000","last_fullsized":"2023-11-28T21:34:42.595588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343566+0000","last_change":"2023-11-28T21:34:37.345104+0000","last_active":"2023-11-28T21:34:41.343566+0000","last_peered":"2023-11-28T21:34:41.343566+0000","last_clean":"2023-11-28T21:34:41.343566+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:34:41.343566+0000","last_undegraded":"2023-11-28T21:34:41.343566+0000","last_fullsized":"2023-11-28T21:34:41.343566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:25:16.125179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593312+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:34:42.593312+0000","last_peered":"2023-11-28T21:34:42.593312+0000","last_clean":"2023-11-28T21:34:42.593312+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:34:42.593312+0000","last_undegraded":"2023-11-28T21:34:42.593312+0000","last_fullsized":"2023-11-28T21:34:42.593312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:36.968220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359721+0000","last_change":"2023-11-28T21:34:41.475704+0000","last_active":"2023-11-28T21:34:42.359721+0000","last_peered":"2023-11-28T21:34:42.359721+0000","last_clean":"2023-11-28T21:34:42.359721+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:34:42.359721+0000","last_undegraded":"2023-11-28T21:34:42.359721+0000","last_fullsized":"2023-11-28T21:34:42.359721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:01:34.494307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341552+0000","last_change":"2023-11-28T21:34:37.330831+0000","last_active":"2023-11-28T21:34:41.341552+0000","last_peered":"2023-11-28T21:34:41.341552+0000","last_clean":"2023-11-28T21:34:41.341552+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:34:41.341552+0000","last_undegraded":"2023-11-28T21:34:41.341552+0000","last_fullsized":"2023-11-28T21:34:41.341552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:50:38.612360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471963+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:34:41.471963+0000","last_peered":"2023-11-28T21:34:41.471963+0000","last_clean":"2023-11-28T21:34:41.471963+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:34:41.471963+0000","last_undegraded":"2023-11-28T21:34:41.471963+0000","last_fullsized":"2023-11-28T21:34:41.471963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592137+0000","last_change":"2023-11-28T21:34:41.475481+0000","last_active":"2023-11-28T21:34:42.592137+0000","last_peered":"2023-11-28T21:34:42.592137+0000","last_clean":"2023-11-28T21:34:42.592137+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:34:42.592137+0000","last_undegraded":"2023-11-28T21:34:42.592137+0000","last_fullsized":"2023-11-28T21:34:42.592137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:22:18.635807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.347169+0000","last_change":"2023-11-28T21:34:37.337846+0000","last_active":"2023-11-28T21:34:41.347169+0000","last_peered":"2023-11-28T21:34:41.347169+0000","last_clean":"2023-11-28T21:34:41.347169+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:34:41.347169+0000","last_undegraded":"2023-11-28T21:34:41.347169+0000","last_fullsized":"2023-11-28T21:34:41.347169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:46:21.358603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342465+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:34:41.342465+0000","last_peered":"2023-11-28T21:34:41.342465+0000","last_clean":"2023-11-28T21:34:41.342465+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:34:41.342465+0000","last_undegraded":"2023-11-28T21:34:41.342465+0000","last_fullsized":"2023-11-28T21:34:41.342465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:08:15.270352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591609+0000","last_change":"2023-11-28T21:34:41.370410+0000","last_active":"2023-11-28T21:34:42.591609+0000","last_peered":"2023-11-28T21:34:42.591609+0000","last_clean":"2023-11-28T21:34:42.591609+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:34:42.591609+0000","last_undegraded":"2023-11-28T21:34:42.591609+0000","last_fullsized":"2023-11-28T21:34:42.591609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342823+0000","last_change":"2023-11-28T21:34:37.324608+0000","last_active":"2023-11-28T21:34:41.342823+0000","last_peered":"2023-11-28T21:34:41.342823+0000","last_clean":"2023-11-28T21:34:41.342823+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:34:41.342823+0000","last_undegraded":"2023-11-28T21:34:41.342823+0000","last_fullsized":"2023-11-28T21:34:41.342823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:13:53.438473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341590+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:34:41.341590+0000","last_peered":"2023-11-28T21:34:41.341590+0000","last_clean":"2023-11-28T21:34:41.341590+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:34:41.341590+0000","last_undegraded":"2023-11-28T21:34:41.341590+0000","last_fullsized":"2023-11-28T21:34:41.341590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592949+0000","last_change":"2023-11-28T21:34:41.380599+0000","last_active":"2023-11-28T21:34:42.592949+0000","last_peered":"2023-11-28T21:34:42.592949+0000","last_clean":"2023-11-28T21:34:42.592949+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:34:42.592949+0000","last_undegraded":"2023-11-28T21:34:42.592949+0000","last_fullsized":"2023-11-28T21:34:42.592949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342555+0000","last_change":"2023-11-28T21:34:37.330869+0000","last_active":"2023-11-28T21:34:41.342555+0000","last_peered":"2023-11-28T21:34:41.342555+0000","last_clean":"2023-11-28T21:34:41.342555+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:34:41.342555+0000","last_undegraded":"2023-11-28T21:34:41.342555+0000","last_fullsized":"2023-11-28T21:34:41.342555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:58:55.987235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.355516+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:34:41.355516+0000","last_peered":"2023-11-28T21:34:41.355516+0000","last_clean":"2023-11-28T21:34:41.355516+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:34:41.355516+0000","last_undegraded":"2023-11-28T21:34:41.355516+0000","last_fullsized":"2023-11-28T21:34:41.355516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593284+0000","last_change":"2023-11-28T21:34:41.371897+0000","last_active":"2023-11-28T21:34:42.593284+0000","last_peered":"2023-11-28T21:34:42.593284+0000","last_clean":"2023-11-28T21:34:42.593284+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:34:42.593284+0000","last_undegraded":"2023-11-28T21:34:42.593284+0000","last_fullsized":"2023-11-28T21:34:42.593284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343750+0000","last_change":"2023-11-28T21:34:37.319833+0000","last_active":"2023-11-28T21:34:41.343750+0000","last_peered":"2023-11-28T21:34:41.343750+0000","last_clean":"2023-11-28T21:34:41.343750+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:34:41.343750+0000","last_undegraded":"2023-11-28T21:34:41.343750+0000","last_fullsized":"2023-11-28T21:34:41.343750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:02:17.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.469428+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:34:41.469428+0000","last_peered":"2023-11-28T21:34:41.469428+0000","last_clean":"2023-11-28T21:34:41.469428+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:34:41.469428+0000","last_undegraded":"2023-11-28T21:34:41.469428+0000","last_fullsized":"2023-11-28T21:34:41.469428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592392+0000","last_change":"2023-11-28T21:34:41.475053+0000","last_active":"2023-11-28T21:34:42.592392+0000","last_peered":"2023-11-28T21:34:42.592392+0000","last_clean":"2023-11-28T21:34:42.592392+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:34:42.592392+0000","last_undegraded":"2023-11-28T21:34:42.592392+0000","last_fullsized":"2023-11-28T21:34:42.592392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:32.658001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340067+0000","last_change":"2023-11-28T21:34:37.358890+0000","last_active":"2023-11-28T21:34:41.340067+0000","last_peered":"2023-11-28T21:34:41.340067+0000","last_clean":"2023-11-28T21:34:41.340067+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:34:41.340067+0000","last_undegraded":"2023-11-28T21:34:41.340067+0000","last_fullsized":"2023-11-28T21:34:41.340067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:54.878141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340738+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:34:41.340738+0000","last_peered":"2023-11-28T21:34:41.340738+0000","last_clean":"2023-11-28T21:34:41.340738+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:34:41.340738+0000","last_undegraded":"2023-11-28T21:34:41.340738+0000","last_fullsized":"2023-11-28T21:34:41.340738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.469889+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:34:41.469889+0000","last_peered":"2023-11-28T21:34:41.469889+0000","last_clean":"2023-11-28T21:34:41.469889+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:34:41.469889+0000","last_undegraded":"2023-11-28T21:34:41.469889+0000","last_fullsized":"2023-11-28T21:34:41.469889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594727+0000","last_change":"2023-11-28T21:34:41.477522+0000","last_active":"2023-11-28T21:34:42.594727+0000","last_peered":"2023-11-28T21:34:42.594727+0000","last_clean":"2023-11-28T21:34:42.594727+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:34:42.594727+0000","last_undegraded":"2023-11-28T21:34:42.594727+0000","last_fullsized":"2023-11-28T21:34:42.594727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:36:36.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471469+0000","last_change":"2023-11-28T21:34:37.350441+0000","last_active":"2023-11-28T21:34:41.471469+0000","last_peered":"2023-11-28T21:34:41.471469+0000","last_clean":"2023-11-28T21:34:41.471469+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:34:41.471469+0000","last_undegraded":"2023-11-28T21:34:41.471469+0000","last_fullsized":"2023-11-28T21:34:41.471469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:54.775495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.355701+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:34:41.355701+0000","last_peered":"2023-11-28T21:34:41.355701+0000","last_clean":"2023-11-28T21:34:41.355701+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:34:41.355701+0000","last_undegraded":"2023-11-28T21:34:41.355701+0000","last_fullsized":"2023-11-28T21:34:41.355701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593850+0000","last_change":"2023-11-28T21:34:41.485154+0000","last_active":"2023-11-28T21:34:42.593850+0000","last_peered":"2023-11-28T21:34:42.593850+0000","last_clean":"2023-11-28T21:34:42.593850+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:34:42.593850+0000","last_undegraded":"2023-11-28T21:34:42.593850+0000","last_fullsized":"2023-11-28T21:34:42.593850+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:35:21.006172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343008+0000","last_change":"2023-11-28T21:34:37.345652+0000","last_active":"2023-11-28T21:34:41.343008+0000","last_peered":"2023-11-28T21:34:41.343008+0000","last_clean":"2023-11-28T21:34:41.343008+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:34:41.343008+0000","last_undegraded":"2023-11-28T21:34:41.343008+0000","last_fullsized":"2023-11-28T21:34:41.343008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:21:17.474200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343244+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:34:42.343244+0000","last_peered":"2023-11-28T21:34:42.343244+0000","last_clean":"2023-11-28T21:34:42.343244+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:34:42.343244+0000","last_undegraded":"2023-11-28T21:34:42.343244+0000","last_fullsized":"2023-11-28T21:34:42.343244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:48.073446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592358+0000","last_change":"2023-11-28T21:34:41.370764+0000","last_active":"2023-11-28T21:34:42.592358+0000","last_peered":"2023-11-28T21:34:42.592358+0000","last_clean":"2023-11-28T21:34:42.592358+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:34:42.592358+0000","last_undegraded":"2023-11-28T21:34:42.592358+0000","last_fullsized":"2023-11-28T21:34:42.592358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:06:42.297098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471917+0000","last_change":"2023-11-28T21:34:37.359589+0000","last_active":"2023-11-28T21:34:41.471917+0000","last_peered":"2023-11-28T21:34:41.471917+0000","last_clean":"2023-11-28T21:34:41.471917+0000","last_became_active":"2023-11-28T21:34:37.359042+0000","last_became_peered":"2023-11-28T21:34:37.359042+0000","last_unstale":"2023-11-28T21:34:41.471917+0000","last_undegraded":"2023-11-28T21:34:41.471917+0000","last_fullsized":"2023-11-28T21:34:41.471917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:27.020614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.339535+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:34:42.339535+0000","last_peered":"2023-11-28T21:34:42.339535+0000","last_clean":"2023-11-28T21:34:42.339535+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:34:42.339535+0000","last_undegraded":"2023-11-28T21:34:42.339535+0000","last_fullsized":"2023-11-28T21:34:42.339535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.362523+0000","last_change":"2023-11-28T21:34:41.477127+0000","last_active":"2023-11-28T21:34:42.362523+0000","last_peered":"2023-11-28T21:34:42.362523+0000","last_clean":"2023-11-28T21:34:42.362523+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:34:42.362523+0000","last_undegraded":"2023-11-28T21:34:42.362523+0000","last_fullsized":"2023-11-28T21:34:42.362523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:32.798817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471715+0000","last_change":"2023-11-28T21:34:37.355693+0000","last_active":"2023-11-28T21:34:41.471715+0000","last_peered":"2023-11-28T21:34:41.471715+0000","last_clean":"2023-11-28T21:34:41.471715+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:34:41.471715+0000","last_undegraded":"2023-11-28T21:34:41.471715+0000","last_fullsized":"2023-11-28T21:34:41.471715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:12.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.338222+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:34:42.338222+0000","last_peered":"2023-11-28T21:34:42.338222+0000","last_clean":"2023-11-28T21:34:42.338222+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:34:42.338222+0000","last_undegraded":"2023-11-28T21:34:42.338222+0000","last_fullsized":"2023-11-28T21:34:42.338222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594073+0000","last_change":"2023-11-28T21:34:41.382468+0000","last_active":"2023-11-28T21:34:42.594073+0000","last_peered":"2023-11-28T21:34:42.594073+0000","last_clean":"2023-11-28T21:34:42.594073+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:34:42.594073+0000","last_undegraded":"2023-11-28T21:34:42.594073+0000","last_fullsized":"2023-11-28T21:34:42.594073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:17.410943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470942+0000","last_change":"2023-11-28T21:34:37.353232+0000","last_active":"2023-11-28T21:34:41.470942+0000","last_peered":"2023-11-28T21:34:41.470942+0000","last_clean":"2023-11-28T21:34:41.470942+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:34:41.470942+0000","last_undegraded":"2023-11-28T21:34:41.470942+0000","last_fullsized":"2023-11-28T21:34:41.470942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:11.285378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.340568+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:34:42.340568+0000","last_peered":"2023-11-28T21:34:42.340568+0000","last_clean":"2023-11-28T21:34:42.340568+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:34:42.340568+0000","last_undegraded":"2023-11-28T21:34:42.340568+0000","last_fullsized":"2023-11-28T21:34:42.340568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592317+0000","last_change":"2023-11-28T21:34:41.385400+0000","last_active":"2023-11-28T21:34:42.592317+0000","last_peered":"2023-11-28T21:34:42.592317+0000","last_clean":"2023-11-28T21:34:42.592317+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:34:42.592317+0000","last_undegraded":"2023-11-28T21:34:42.592317+0000","last_fullsized":"2023-11-28T21:34:42.592317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:36.433439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365468+0000","last_change":"2023-11-28T21:34:37.333461+0000","last_active":"2023-11-28T21:34:41.365468+0000","last_peered":"2023-11-28T21:34:41.365468+0000","last_clean":"2023-11-28T21:34:41.365468+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:34:41.365468+0000","last_undegraded":"2023-11-28T21:34:41.365468+0000","last_fullsized":"2023-11-28T21:34:41.365468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:25.513524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.366026+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:34:41.366026+0000","last_peered":"2023-11-28T21:34:41.366026+0000","last_clean":"2023-11-28T21:34:41.366026+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:34:41.366026+0000","last_undegraded":"2023-11-28T21:34:41.366026+0000","last_fullsized":"2023-11-28T21:34:41.366026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361391+0000","last_change":"2023-11-28T21:34:41.479919+0000","last_active":"2023-11-28T21:34:42.361391+0000","last_peered":"2023-11-28T21:34:42.361391+0000","last_clean":"2023-11-28T21:34:42.361391+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:34:42.361391+0000","last_undegraded":"2023-11-28T21:34:42.361391+0000","last_fullsized":"2023-11-28T21:34:42.361391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:11.183117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471078+0000","last_change":"2023-11-28T21:34:37.355729+0000","last_active":"2023-11-28T21:34:41.471078+0000","last_peered":"2023-11-28T21:34:41.471078+0000","last_clean":"2023-11-28T21:34:41.471078+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:34:41.471078+0000","last_undegraded":"2023-11-28T21:34:41.471078+0000","last_fullsized":"2023-11-28T21:34:41.471078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:39.178378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341388+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:34:41.341388+0000","last_peered":"2023-11-28T21:34:41.341388+0000","last_clean":"2023-11-28T21:34:41.341388+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:34:41.341388+0000","last_undegraded":"2023-11-28T21:34:41.341388+0000","last_fullsized":"2023-11-28T21:34:41.341388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361306+0000","last_change":"2023-11-28T21:34:41.478483+0000","last_active":"2023-11-28T21:34:42.361306+0000","last_peered":"2023-11-28T21:34:42.361306+0000","last_clean":"2023-11-28T21:34:42.361306+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:34:42.361306+0000","last_undegraded":"2023-11-28T21:34:42.361306+0000","last_fullsized":"2023-11-28T21:34:42.361306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:25:54.342692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345020+0000","last_change":"2023-11-28T21:34:37.345476+0000","last_active":"2023-11-28T21:34:41.345020+0000","last_peered":"2023-11-28T21:34:41.345020+0000","last_clean":"2023-11-28T21:34:41.345020+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:34:41.345020+0000","last_undegraded":"2023-11-28T21:34:41.345020+0000","last_fullsized":"2023-11-28T21:34:41.345020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.660412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343340+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:34:42.343340+0000","last_peered":"2023-11-28T21:34:42.343340+0000","last_clean":"2023-11-28T21:34:42.343340+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:34:42.343340+0000","last_undegraded":"2023-11-28T21:34:42.343340+0000","last_fullsized":"2023-11-28T21:34:42.343340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:28.694184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363244+0000","last_change":"2023-11-28T21:34:41.477926+0000","last_active":"2023-11-28T21:34:42.363244+0000","last_peered":"2023-11-28T21:34:42.363244+0000","last_clean":"2023-11-28T21:34:42.363244+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:34:42.363244+0000","last_undegraded":"2023-11-28T21:34:42.363244+0000","last_fullsized":"2023-11-28T21:34:42.363244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:26.403506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471448+0000","last_change":"2023-11-28T21:34:37.343885+0000","last_active":"2023-11-28T21:34:41.471448+0000","last_peered":"2023-11-28T21:34:41.471448+0000","last_clean":"2023-11-28T21:34:41.471448+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:34:41.471448+0000","last_undegraded":"2023-11-28T21:34:41.471448+0000","last_fullsized":"2023-11-28T21:34:41.471448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:36.255834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:50.648988+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:34:50.648988+0000","last_peered":"2023-11-28T21:34:50.648988+0000","last_clean":"2023-11-28T21:34:50.648988+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:34:50.648988+0000","last_undegraded":"2023-11-28T21:34:50.648988+0000","last_fullsized":"2023-11-28T21:34:50.648988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591595+0000","last_change":"2023-11-28T21:34:41.380565+0000","last_active":"2023-11-28T21:34:42.591595+0000","last_peered":"2023-11-28T21:34:42.591595+0000","last_clean":"2023-11-28T21:34:42.591595+0000","last_became_active":"2023-11-28T21:34:41.380080+0000","last_became_peered":"2023-11-28T21:34:41.380080+0000","last_unstale":"2023-11-28T21:34:42.591595+0000","last_undegraded":"2023-11-28T21:34:42.591595+0000","last_fullsized":"2023-11-28T21:34:42.591595+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343524+0000","last_change":"2023-11-28T21:34:37.345635+0000","last_active":"2023-11-28T21:34:41.343524+0000","last_peered":"2023-11-28T21:34:41.343524+0000","last_clean":"2023-11-28T21:34:41.343524+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:34:41.343524+0000","last_undegraded":"2023-11-28T21:34:41.343524+0000","last_fullsized":"2023-11-28T21:34:41.343524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:52:12.144267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343474+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:34:42.343474+0000","last_peered":"2023-11-28T21:34:42.343474+0000","last_clean":"2023-11-28T21:34:42.343474+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:34:42.343474+0000","last_undegraded":"2023-11-28T21:34:42.343474+0000","last_fullsized":"2023-11-28T21:34:42.343474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:35.266650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597618+0000","last_change":"2023-11-28T21:34:41.365758+0000","last_active":"2023-11-28T21:34:42.597618+0000","last_peered":"2023-11-28T21:34:42.597618+0000","last_clean":"2023-11-28T21:34:42.597618+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:34:42.597618+0000","last_undegraded":"2023-11-28T21:34:42.597618+0000","last_fullsized":"2023-11-28T21:34:42.597618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344832+0000","last_change":"2023-11-28T21:34:37.333607+0000","last_active":"2023-11-28T21:34:41.344832+0000","last_peered":"2023-11-28T21:34:41.344832+0000","last_clean":"2023-11-28T21:34:41.344832+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:34:41.344832+0000","last_undegraded":"2023-11-28T21:34:41.344832+0000","last_fullsized":"2023-11-28T21:34:41.344832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:13:01.639908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342208+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:34:41.342208+0000","last_peered":"2023-11-28T21:34:41.342208+0000","last_clean":"2023-11-28T21:34:41.342208+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:34:41.342208+0000","last_undegraded":"2023-11-28T21:34:41.342208+0000","last_fullsized":"2023-11-28T21:34:41.342208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359893+0000","last_change":"2023-11-28T21:34:41.380413+0000","last_active":"2023-11-28T21:34:42.359893+0000","last_peered":"2023-11-28T21:34:42.359893+0000","last_clean":"2023-11-28T21:34:42.359893+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:34:42.359893+0000","last_undegraded":"2023-11-28T21:34:42.359893+0000","last_fullsized":"2023-11-28T21:34:42.359893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:20:28.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342001+0000","last_change":"2023-11-28T21:34:37.351009+0000","last_active":"2023-11-28T21:34:41.342001+0000","last_peered":"2023-11-28T21:34:41.342001+0000","last_clean":"2023-11-28T21:34:41.342001+0000","last_became_active":"2023-11-28T21:34:37.350321+0000","last_became_peered":"2023-11-28T21:34:37.350321+0000","last_unstale":"2023-11-28T21:34:41.342001+0000","last_undegraded":"2023-11-28T21:34:41.342001+0000","last_fullsized":"2023-11-28T21:34:41.342001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:27:42.022466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.368215+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:34:41.368215+0000","last_peered":"2023-11-28T21:34:41.368215+0000","last_clean":"2023-11-28T21:34:41.368215+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:34:41.368215+0000","last_undegraded":"2023-11-28T21:34:41.368215+0000","last_fullsized":"2023-11-28T21:34:41.368215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359889+0000","last_change":"2023-11-28T21:34:41.475185+0000","last_active":"2023-11-28T21:34:42.359889+0000","last_peered":"2023-11-28T21:34:42.359889+0000","last_clean":"2023-11-28T21:34:42.359889+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:34:42.359889+0000","last_undegraded":"2023-11-28T21:34:42.359889+0000","last_fullsized":"2023-11-28T21:34:42.359889+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:09.389881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592347+0000","last_change":"2023-11-28T21:34:41.479956+0000","last_active":"2023-11-28T21:34:42.592347+0000","last_peered":"2023-11-28T21:34:42.592347+0000","last_clean":"2023-11-28T21:34:42.592347+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:34:42.592347+0000","last_undegraded":"2023-11-28T21:34:42.592347+0000","last_fullsized":"2023-11-28T21:34:42.592347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:46.004852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343050+0000","last_change":"2023-11-28T21:34:37.317513+0000","last_active":"2023-11-28T21:34:41.343050+0000","last_peered":"2023-11-28T21:34:41.343050+0000","last_clean":"2023-11-28T21:34:41.343050+0000","last_became_active":"2023-11-28T21:34:37.317082+0000","last_became_peered":"2023-11-28T21:34:37.317082+0000","last_unstale":"2023-11-28T21:34:41.343050+0000","last_undegraded":"2023-11-28T21:34:41.343050+0000","last_fullsized":"2023-11-28T21:34:41.343050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:05:37.846078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346371+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:34:41.346371+0000","last_peered":"2023-11-28T21:34:41.346371+0000","last_clean":"2023-11-28T21:34:41.346371+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:34:41.346371+0000","last_undegraded":"2023-11-28T21:34:41.346371+0000","last_fullsized":"2023-11-28T21:34:41.346371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595858+0000","last_change":"2023-11-28T21:34:41.385969+0000","last_active":"2023-11-28T21:34:42.595858+0000","last_peered":"2023-11-28T21:34:42.595858+0000","last_clean":"2023-11-28T21:34:42.595858+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:34:42.595858+0000","last_undegraded":"2023-11-28T21:34:42.595858+0000","last_fullsized":"2023-11-28T21:34:42.595858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:01.692549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339578+0000","last_change":"2023-11-28T21:34:37.314087+0000","last_active":"2023-11-28T21:34:41.339578+0000","last_peered":"2023-11-28T21:34:41.339578+0000","last_clean":"2023-11-28T21:34:41.339578+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:34:41.339578+0000","last_undegraded":"2023-11-28T21:34:41.339578+0000","last_fullsized":"2023-11-28T21:34:41.339578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:25:11.912313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340387+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:34:41.340387+0000","last_peered":"2023-11-28T21:34:41.340387+0000","last_clean":"2023-11-28T21:34:41.340387+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:34:41.340387+0000","last_undegraded":"2023-11-28T21:34:41.340387+0000","last_fullsized":"2023-11-28T21:34:41.340387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343264+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:34:42.343264+0000","last_peered":"2023-11-28T21:34:42.343264+0000","last_clean":"2023-11-28T21:34:42.343264+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:34:42.343264+0000","last_undegraded":"2023-11-28T21:34:42.343264+0000","last_fullsized":"2023-11-28T21:34:42.343264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345191+0000","last_change":"2023-11-28T21:34:37.319801+0000","last_active":"2023-11-28T21:34:41.345191+0000","last_peered":"2023-11-28T21:34:41.345191+0000","last_clean":"2023-11-28T21:34:41.345191+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:34:41.345191+0000","last_undegraded":"2023-11-28T21:34:41.345191+0000","last_fullsized":"2023-11-28T21:34:41.345191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:40:13.251522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360747+0000","last_change":"2023-11-28T21:34:41.370523+0000","last_active":"2023-11-28T21:34:42.360747+0000","last_peered":"2023-11-28T21:34:42.360747+0000","last_clean":"2023-11-28T21:34:42.360747+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:34:42.360747+0000","last_undegraded":"2023-11-28T21:34:42.360747+0000","last_fullsized":"2023-11-28T21:34:42.360747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:17:36.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343058+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:34:42.343058+0000","last_peered":"2023-11-28T21:34:42.343058+0000","last_clean":"2023-11-28T21:34:42.343058+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:34:42.343058+0000","last_undegraded":"2023-11-28T21:34:42.343058+0000","last_fullsized":"2023-11-28T21:34:42.343058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342331+0000","last_change":"2023-11-28T21:34:37.328809+0000","last_active":"2023-11-28T21:34:41.342331+0000","last_peered":"2023-11-28T21:34:41.342331+0000","last_clean":"2023-11-28T21:34:41.342331+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:34:41.342331+0000","last_undegraded":"2023-11-28T21:34:41.342331+0000","last_fullsized":"2023-11-28T21:34:41.342331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:27.300090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361127+0000","last_change":"2023-11-28T21:34:41.363087+0000","last_active":"2023-11-28T21:34:42.361127+0000","last_peered":"2023-11-28T21:34:42.361127+0000","last_clean":"2023-11-28T21:34:42.361127+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:34:42.361127+0000","last_undegraded":"2023-11-28T21:34:42.361127+0000","last_fullsized":"2023-11-28T21:34:42.361127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:35.046132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365193+0000","last_change":"2023-11-28T21:34:37.333670+0000","last_active":"2023-11-28T21:34:41.365193+0000","last_peered":"2023-11-28T21:34:41.365193+0000","last_clean":"2023-11-28T21:34:41.365193+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:34:41.365193+0000","last_undegraded":"2023-11-28T21:34:41.365193+0000","last_fullsized":"2023-11-28T21:34:41.365193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:32.598014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.355458+0000","last_change":"2023-11-28T21:34:37.347101+0000","last_active":"2023-11-28T21:34:41.355458+0000","last_peered":"2023-11-28T21:34:41.355458+0000","last_clean":"2023-11-28T21:34:41.355458+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:34:41.355458+0000","last_undegraded":"2023-11-28T21:34:41.355458+0000","last_fullsized":"2023-11-28T21:34:41.355458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:39:43.186110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341187+0000","last_change":"2023-11-28T21:34:37.338576+0000","last_active":"2023-11-28T21:34:41.341187+0000","last_peered":"2023-11-28T21:34:41.341187+0000","last_clean":"2023-11-28T21:34:41.341187+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:34:41.341187+0000","last_undegraded":"2023-11-28T21:34:41.341187+0000","last_fullsized":"2023-11-28T21:34:41.341187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:53:02.356097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345097+0000","last_change":"2023-11-28T21:34:37.316270+0000","last_active":"2023-11-28T21:34:41.345097+0000","last_peered":"2023-11-28T21:34:41.345097+0000","last_clean":"2023-11-28T21:34:41.345097+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:34:41.345097+0000","last_undegraded":"2023-11-28T21:34:41.345097+0000","last_fullsized":"2023-11-28T21:34:41.345097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:43.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340024+0000","last_change":"2023-11-28T21:34:37.334973+0000","last_active":"2023-11-28T21:34:41.340024+0000","last_peered":"2023-11-28T21:34:41.340024+0000","last_clean":"2023-11-28T21:34:41.340024+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:34:41.340024+0000","last_undegraded":"2023-11-28T21:34:41.340024+0000","last_fullsized":"2023-11-28T21:34:41.340024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:55.050945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345807+0000","last_change":"2023-11-28T21:34:37.318155+0000","last_active":"2023-11-28T21:34:41.345807+0000","last_peered":"2023-11-28T21:34:41.345807+0000","last_clean":"2023-11-28T21:34:41.345807+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:34:41.345807+0000","last_undegraded":"2023-11-28T21:34:41.345807+0000","last_fullsized":"2023-11-28T21:34:41.345807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:29.840255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367255+0000","last_change":"2023-11-28T21:34:37.345758+0000","last_active":"2023-11-28T21:34:41.367255+0000","last_peered":"2023-11-28T21:34:41.367255+0000","last_clean":"2023-11-28T21:34:41.367255+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:34:41.367255+0000","last_undegraded":"2023-11-28T21:34:41.367255+0000","last_fullsized":"2023-11-28T21:34:41.367255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:26.444654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343483+0000","last_change":"2023-11-28T21:34:37.334989+0000","last_active":"2023-11-28T21:34:41.343483+0000","last_peered":"2023-11-28T21:34:41.343483+0000","last_clean":"2023-11-28T21:34:41.343483+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:34:41.343483+0000","last_undegraded":"2023-11-28T21:34:41.343483+0000","last_fullsized":"2023-11-28T21:34:41.343483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:36:57.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346163+0000","last_change":"2023-11-28T21:34:37.333075+0000","last_active":"2023-11-28T21:34:41.346163+0000","last_peered":"2023-11-28T21:34:41.346163+0000","last_clean":"2023-11-28T21:34:41.346163+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:34:41.346163+0000","last_undegraded":"2023-11-28T21:34:41.346163+0000","last_fullsized":"2023-11-28T21:34:41.346163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:50:43.748349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339854+0000","last_change":"2023-11-28T21:34:37.337230+0000","last_active":"2023-11-28T21:34:41.339854+0000","last_peered":"2023-11-28T21:34:41.339854+0000","last_clean":"2023-11-28T21:34:41.339854+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:34:41.339854+0000","last_undegraded":"2023-11-28T21:34:41.339854+0000","last_fullsized":"2023-11-28T21:34:41.339854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:48:45.994559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340945+0000","last_change":"2023-11-28T21:34:37.349894+0000","last_active":"2023-11-28T21:34:41.340945+0000","last_peered":"2023-11-28T21:34:41.340945+0000","last_clean":"2023-11-28T21:34:41.340945+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:34:41.340945+0000","last_undegraded":"2023-11-28T21:34:41.340945+0000","last_fullsized":"2023-11-28T21:34:41.340945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:10.293911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345003+0000","last_change":"2023-11-28T21:34:37.344269+0000","last_active":"2023-11-28T21:34:41.345003+0000","last_peered":"2023-11-28T21:34:41.345003+0000","last_clean":"2023-11-28T21:34:41.345003+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:34:41.345003+0000","last_undegraded":"2023-11-28T21:34:41.345003+0000","last_fullsized":"2023-11-28T21:34:41.345003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:03.204740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365122+0000","last_change":"2023-11-28T21:34:37.353059+0000","last_active":"2023-11-28T21:34:41.365122+0000","last_peered":"2023-11-28T21:34:41.365122+0000","last_clean":"2023-11-28T21:34:41.365122+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:34:41.365122+0000","last_undegraded":"2023-11-28T21:34:41.365122+0000","last_fullsized":"2023-11-28T21:34:41.365122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:45:12.640797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471801+0000","last_change":"2023-11-28T21:34:37.352965+0000","last_active":"2023-11-28T21:34:41.471801+0000","last_peered":"2023-11-28T21:34:41.471801+0000","last_clean":"2023-11-28T21:34:41.471801+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:34:41.471801+0000","last_undegraded":"2023-11-28T21:34:41.471801+0000","last_fullsized":"2023-11-28T21:34:41.471801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:30:42.087715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.356122+0000","last_change":"2023-11-28T21:34:37.336830+0000","last_active":"2023-11-28T21:34:41.356122+0000","last_peered":"2023-11-28T21:34:41.356122+0000","last_clean":"2023-11-28T21:34:41.356122+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:34:41.356122+0000","last_undegraded":"2023-11-28T21:34:41.356122+0000","last_fullsized":"2023-11-28T21:34:41.356122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:57.318311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341567+0000","last_change":"2023-11-28T21:34:37.349181+0000","last_active":"2023-11-28T21:34:41.341567+0000","last_peered":"2023-11-28T21:34:41.341567+0000","last_clean":"2023-11-28T21:34:41.341567+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:34:41.341567+0000","last_undegraded":"2023-11-28T21:34:41.341567+0000","last_fullsized":"2023-11-28T21:34:41.341567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:27:59.989299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345968+0000","last_change":"2023-11-28T21:34:37.336668+0000","last_active":"2023-11-28T21:34:41.345968+0000","last_peered":"2023-11-28T21:34:41.345968+0000","last_clean":"2023-11-28T21:34:41.345968+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:34:41.345968+0000","last_undegraded":"2023-11-28T21:34:41.345968+0000","last_fullsized":"2023-11-28T21:34:41.345968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:06.416717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343147+0000","last_change":"2023-11-28T21:34:37.332790+0000","last_active":"2023-11-28T21:34:41.343147+0000","last_peered":"2023-11-28T21:34:41.343147+0000","last_clean":"2023-11-28T21:34:41.343147+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:34:41.343147+0000","last_undegraded":"2023-11-28T21:34:41.343147+0000","last_fullsized":"2023-11-28T21:34:41.343147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:32:45.920387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344930+0000","last_change":"2023-11-28T21:34:37.320794+0000","last_active":"2023-11-28T21:34:41.344930+0000","last_peered":"2023-11-28T21:34:41.344930+0000","last_clean":"2023-11-28T21:34:41.344930+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:34:41.344930+0000","last_undegraded":"2023-11-28T21:34:41.344930+0000","last_fullsized":"2023-11-28T21:34:41.344930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:16:46.347683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339818+0000","last_change":"2023-11-28T21:34:37.337726+0000","last_active":"2023-11-28T21:34:41.339818+0000","last_peered":"2023-11-28T21:34:41.339818+0000","last_clean":"2023-11-28T21:34:41.339818+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:34:41.339818+0000","last_undegraded":"2023-11-28T21:34:41.339818+0000","last_fullsized":"2023-11-28T21:34:41.339818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:38:43.884668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343259+0000","last_change":"2023-11-28T21:34:37.334677+0000","last_active":"2023-11-28T21:34:41.343259+0000","last_peered":"2023-11-28T21:34:41.343259+0000","last_clean":"2023-11-28T21:34:41.343259+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:34:41.343259+0000","last_undegraded":"2023-11-28T21:34:41.343259+0000","last_fullsized":"2023-11-28T21:34:41.343259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:05:54.981899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.366695+0000","last_change":"2023-11-28T21:34:37.354861+0000","last_active":"2023-11-28T21:34:41.366695+0000","last_peered":"2023-11-28T21:34:41.366695+0000","last_clean":"2023-11-28T21:34:41.366695+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:34:41.366695+0000","last_undegraded":"2023-11-28T21:34:41.366695+0000","last_fullsized":"2023-11-28T21:34:41.366695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:30.602284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367055+0000","last_change":"2023-11-28T21:34:37.347675+0000","last_active":"2023-11-28T21:34:41.367055+0000","last_peered":"2023-11-28T21:34:41.367055+0000","last_clean":"2023-11-28T21:34:41.367055+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:34:41.367055+0000","last_undegraded":"2023-11-28T21:34:41.367055+0000","last_fullsized":"2023-11-28T21:34:41.367055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:51:38.803896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471861+0000","last_change":"2023-11-28T21:34:37.357735+0000","last_active":"2023-11-28T21:34:41.471861+0000","last_peered":"2023-11-28T21:34:41.471861+0000","last_clean":"2023-11-28T21:34:41.471861+0000","last_became_active":"2023-11-28T21:34:37.357173+0000","last_became_peered":"2023-11-28T21:34:37.357173+0000","last_unstale":"2023-11-28T21:34:41.471861+0000","last_undegraded":"2023-11-28T21:34:41.471861+0000","last_fullsized":"2023-11-28T21:34:41.471861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:38.660713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339546+0000","last_change":"2023-11-28T21:34:37.358851+0000","last_active":"2023-11-28T21:34:41.339546+0000","last_peered":"2023-11-28T21:34:41.339546+0000","last_clean":"2023-11-28T21:34:41.339546+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:34:41.339546+0000","last_undegraded":"2023-11-28T21:34:41.339546+0000","last_fullsized":"2023-11-28T21:34:41.339546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.378519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470714+0000","last_change":"2023-11-28T21:34:37.343972+0000","last_active":"2023-11-28T21:34:41.470714+0000","last_peered":"2023-11-28T21:34:41.470714+0000","last_clean":"2023-11-28T21:34:41.470714+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:34:41.470714+0000","last_undegraded":"2023-11-28T21:34:41.470714+0000","last_fullsized":"2023-11-28T21:34:41.470714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:56:00.591888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340667+0000","last_change":"2023-11-28T21:34:37.346299+0000","last_active":"2023-11-28T21:34:41.340667+0000","last_peered":"2023-11-28T21:34:41.340667+0000","last_clean":"2023-11-28T21:34:41.340667+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:34:41.340667+0000","last_undegraded":"2023-11-28T21:34:41.340667+0000","last_fullsized":"2023-11-28T21:34:41.340667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:06.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470739+0000","last_change":"2023-11-28T21:34:37.343410+0000","last_active":"2023-11-28T21:34:41.470739+0000","last_peered":"2023-11-28T21:34:41.470739+0000","last_clean":"2023-11-28T21:34:41.470739+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:34:41.470739+0000","last_undegraded":"2023-11-28T21:34:41.470739+0000","last_fullsized":"2023-11-28T21:34:41.470739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:29:45.070478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342784+0000","last_change":"2023-11-28T21:34:37.348270+0000","last_active":"2023-11-28T21:34:41.342784+0000","last_peered":"2023-11-28T21:34:41.342784+0000","last_clean":"2023-11-28T21:34:41.342784+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:34:41.342784+0000","last_undegraded":"2023-11-28T21:34:41.342784+0000","last_fullsized":"2023-11-28T21:34:41.342784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:36:10.938478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344314+0000","last_change":"2023-11-28T21:34:37.330786+0000","last_active":"2023-11-28T21:34:41.344314+0000","last_peered":"2023-11-28T21:34:41.344314+0000","last_clean":"2023-11-28T21:34:41.344314+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:34:41.344314+0000","last_undegraded":"2023-11-28T21:34:41.344314+0000","last_fullsized":"2023-11-28T21:34:41.344314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:52:36.738867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346251+0000","last_change":"2023-11-28T21:34:37.318725+0000","last_active":"2023-11-28T21:34:41.346251+0000","last_peered":"2023-11-28T21:34:41.346251+0000","last_clean":"2023-11-28T21:34:41.346251+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:34:41.346251+0000","last_undegraded":"2023-11-28T21:34:41.346251+0000","last_fullsized":"2023-11-28T21:34:41.346251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:49:21.775066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471764+0000","last_change":"2023-11-28T21:34:37.349829+0000","last_active":"2023-11-28T21:34:41.471764+0000","last_peered":"2023-11-28T21:34:41.471764+0000","last_clean":"2023-11-28T21:34:41.471764+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:34:41.471764+0000","last_undegraded":"2023-11-28T21:34:41.471764+0000","last_fullsized":"2023-11-28T21:34:41.471764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:07.752862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":690742,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":99952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":689779,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:35:06.557 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:35:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.736+0000 7f53f0e91640 1 Processor -- start 2023-11-28T21:35:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.736+0000 7f53f0e91640 1 -- start start 2023-11-28T21:35:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.736+0000 7f53f0e91640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f53ec153ee0 0x7f53ec15a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.736+0000 7f53f0e91640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f53ec070bd0 0x7f53ec071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53f0e91640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ec0c6350 con 0x7f53ec06a5b0 2023-11-28T21:35:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53f0e91640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec0c6b90 con 0x7f53ec153ee0 2023-11-28T21:35:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53f0e91640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec072410 con 0x7f53ec070bd0 2023-11-28T21:35:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53ea59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f53ec153ee0 0x7f53ec15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53ea59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f53ec153ee0 0x7f53ec15a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37992/0 (socket says 172.21.15.12:37992) 2023-11-28T21:35:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53ea59c640 1 -- 172.21.15.12:0/897223268 learned_addr learned my addr 172.21.15.12:0/897223268 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53e3fff640 1 --1- 172.21.15.12:0/897223268 >> v1:172.21.15.12:6789/0 conn(0x7f53ec06a5b0 0x7f53ec0c7a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36312/0 (socket says 172.21.15.12:36312) 2023-11-28T21:35:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53ead9d640 1 --2- 172.21.15.12:0/897223268 >> v2:172.21.15.144:3300/0 conn(0x7f53ec070bd0 0x7f53ec071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53ea59c640 1 -- 172.21.15.12:0/897223268 >> v2:172.21.15.144:3300/0 conn(0x7f53ec070bd0 msgr2=0x7f53ec071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53ea59c640 1 --2- 172.21.15.12:0/897223268 >> v2:172.21.15.144:3300/0 conn(0x7f53ec070bd0 0x7f53ec071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53ea59c640 1 -- 172.21.15.12:0/897223268 >> v1:172.21.15.12:6789/0 conn(0x7f53ec06a5b0 legacy=0x7f53ec0c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53e9d9b640 1 -- 172.21.15.12:0/897223268 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332849120 0 0) 0x7f53ec0c6350 con 0x7f53ec06a5b0 2023-11-28T21:35:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53ea59c640 1 -- 172.21.15.12:0/897223268 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0c65d0 con 0x7f53ec153ee0 2023-11-28T21:35:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53ea59c640 1 --2- 172.21.15.12:0/897223268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec153ee0 0x7f53ec15a680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53ec066b90 tx=0x7f53dc01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab3655244b74e0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53e9d9b640 1 -- 172.21.15.12:0/897223268 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0a89f0 con 0x7f53ec153ee0 2023-11-28T21:35:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53e9d9b640 1 -- 172.21.15.12:0/897223268 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc0a8b50 con 0x7f53ec153ee0 2023-11-28T21:35:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.740+0000 7f53e9d9b640 1 -- 172.21.15.12:0/897223268 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0b0440 con 0x7f53ec153ee0 2023-11-28T21:35:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 -- 172.21.15.12:0/897223268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec153ee0 msgr2=0x7f53ec15a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 --2- 172.21.15.12:0/897223268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec153ee0 0x7f53ec15a680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53ec066b90 tx=0x7f53dc01ecf0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 -- 172.21.15.12:0/897223268 shutdown_connections 2023-11-28T21:35:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 --2- 172.21.15.12:0/897223268 >> v2:172.21.15.144:3300/0 conn(0x7f53ec070bd0 0x7f53ec071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 --2- 172.21.15.12:0/897223268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec153ee0 0x7f53ec15a680 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 -- 172.21.15.12:0/897223268 >> 172.21.15.12:0/897223268 conn(0x7f53ec0c8450 msgr2=0x7f53ec0c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 -- 172.21.15.12:0/897223268 shutdown_connections 2023-11-28T21:35:06.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 -- 172.21.15.12:0/897223268 wait complete. 2023-11-28T21:35:06.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 Processor -- start 2023-11-28T21:35:06.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 -- start start 2023-11-28T21:35:06.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec06a5b0 0x7f53ec1ef840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53ec070bd0 0x7f53ec1e5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53ec153ee0 0x7f53ec158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec158050 con 0x7f53ec06a5b0 2023-11-28T21:35:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec157a20 con 0x7f53ec070bd0 2023-11-28T21:35:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53f0e91640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1efd80 con 0x7f53ec153ee0 2023-11-28T21:35:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53ea59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec06a5b0 0x7f53ec1ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53ea59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec06a5b0 0x7f53ec1ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38002/0 (socket says 172.21.15.12:38002) 2023-11-28T21:35:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53ea59c640 1 -- 172.21.15.12:0/4122193941 learned_addr learned my addr 172.21.15.12:0/4122193941 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53ead9d640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53ec153ee0 0x7f53ec158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.744+0000 7f53e3fff640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53ec070bd0 0x7f53ec1e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53ea59c640 1 -- 172.21.15.12:0/4122193941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53ec153ee0 msgr2=0x7f53ec158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53ea59c640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53ec153ee0 0x7f53ec158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53ea59c640 1 -- 172.21.15.12:0/4122193941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53ec070bd0 msgr2=0x7f53ec1e5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53ea59c640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53ec070bd0 0x7f53ec1e5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53ea59c640 1 -- 172.21.15.12:0/4122193941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc0094d0 con 0x7f53ec06a5b0 2023-11-28T21:35:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53ea59c640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec06a5b0 0x7f53ec1ef840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0b3040 tx=0x7f53dc01ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:06.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53e2ffd640 1 -- 172.21.15.12:0/4122193941 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0ba600 con 0x7f53ec06a5b0 2023-11-28T21:35:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53e2ffd640 1 -- 172.21.15.12:0/4122193941 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc0ba760 con 0x7f53ec06a5b0 2023-11-28T21:35:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53e2ffd640 1 -- 172.21.15.12:0/4122193941 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0bab10 con 0x7f53ec06a5b0 2023-11-28T21:35:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53f0e91640 1 -- 172.21.15.12:0/4122193941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec1f0010 con 0x7f53ec06a5b0 2023-11-28T21:35:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53f0e91640 1 -- 172.21.15.12:0/4122193941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec1f05d0 con 0x7f53ec06a5b0 2023-11-28T21:35:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53f0e91640 1 -- 172.21.15.12:0/4122193941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec1f07c0 con 0x7f53ec06a5b0 2023-11-28T21:35:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53e2ffd640 1 -- 172.21.15.12:0/4122193941 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f53dc0bac70 con 0x7f53ec06a5b0 2023-11-28T21:35:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53e2ffd640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53cc066470 0x7f53cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53e2ffd640 1 -- 172.21.15.12:0/4122193941 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f53dc122840 con 0x7f53ec06a5b0 2023-11-28T21:35:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.748+0000 7f53e3fff640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53cc066470 0x7f53cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.752+0000 7f53e3fff640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53cc066470 0x7f53cc068930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f53d4007fd0 tx=0x7f53d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.752+0000 7f53e2ffd640 1 -- 172.21.15.12:0/4122193941 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f53dc0ef8a0 con 0x7f53ec06a5b0 2023-11-28T21:35:06.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.912+0000 7f53f0e91640 1 -- 172.21.15.12:0/4122193941 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53ec15a160 con 0x7f53cc066470 2023-11-28T21:35:06.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.920+0000 7f53e2ffd640 1 -- 172.21.15.12:0/4122193941 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345285 (secure 0 0 0) 0x7f53d4001c80 con 0x7f53cc066470 2023-11-28T21:35:06.925 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:06.928 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:35:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 -- 172.21.15.12:0/4122193941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53cc066470 msgr2=0x7f53cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53cc066470 0x7f53cc068930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f53d4007fd0 tx=0x7f53d4009040 comp rx=0 tx=0).stop 2023-11-28T21:35:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 -- 172.21.15.12:0/4122193941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec06a5b0 msgr2=0x7f53ec1ef840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec06a5b0 0x7f53ec1ef840 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0b3040 tx=0x7f53dc01ba90 comp rx=0 tx=0).stop 2023-11-28T21:35:06.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 -- 172.21.15.12:0/4122193941 shutdown_connections 2023-11-28T21:35:06.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53cc066470 0x7f53cc068930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53ec153ee0 0x7f53ec158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53ec070bd0 0x7f53ec1e5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 --2- 172.21.15.12:0/4122193941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53ec06a5b0 0x7f53ec1ef840 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 -- 172.21.15.12:0/4122193941 >> 172.21.15.12:0/4122193941 conn(0x7f53ec0c8450 msgr2=0x7f53ec06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 -- 172.21.15.12:0/4122193941 shutdown_connections 2023-11-28T21:35:06.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:06.924+0000 7f53f0e91640 1 -- 172.21.15.12:0/4122193941 wait complete. 2023-11-28T21:35:06.946 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-11-28T21:35:05.822267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218356,"kb_used_data":2172,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756364,"statfs":{"total":773094113280,"available":772870516736,"internally_reserved":0,"allocated":2224128,"data_stored":1987331,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003155"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343838+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:34:42.343838+0000","last_peered":"2023-11-28T21:34:42.343838+0000","last_clean":"2023-11-28T21:34:42.343838+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:34:42.343838+0000","last_undegraded":"2023-11-28T21:34:42.343838+0000","last_fullsized":"2023-11-28T21:34:42.343838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:46.167511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341374+0000","last_change":"2023-11-28T21:34:37.335656+0000","last_active":"2023-11-28T21:34:41.341374+0000","last_peered":"2023-11-28T21:34:41.341374+0000","last_clean":"2023-11-28T21:34:41.341374+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:34:41.341374+0000","last_undegraded":"2023-11-28T21:34:41.341374+0000","last_fullsized":"2023-11-28T21:34:41.341374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:25:49.707378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359842+0000","last_change":"2023-11-28T21:34:41.483856+0000","last_active":"2023-11-28T21:34:42.359842+0000","last_peered":"2023-11-28T21:34:42.359842+0000","last_clean":"2023-11-28T21:34:42.359842+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:34:42.359842+0000","last_undegraded":"2023-11-28T21:34:42.359842+0000","last_fullsized":"2023-11-28T21:34:42.359842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:50:56.005578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.340464+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:34:42.340464+0000","last_peered":"2023-11-28T21:34:42.340464+0000","last_clean":"2023-11-28T21:34:42.340464+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:34:42.340464+0000","last_undegraded":"2023-11-28T21:34:42.340464+0000","last_fullsized":"2023-11-28T21:34:42.340464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344725+0000","last_change":"2023-11-28T21:34:37.333649+0000","last_active":"2023-11-28T21:34:41.344725+0000","last_peered":"2023-11-28T21:34:41.344725+0000","last_clean":"2023-11-28T21:34:41.344725+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:34:41.344725+0000","last_undegraded":"2023-11-28T21:34:41.344725+0000","last_fullsized":"2023-11-28T21:34:41.344725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:21:42.041590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594292+0000","last_change":"2023-11-28T21:34:41.388232+0000","last_active":"2023-11-28T21:34:42.594292+0000","last_peered":"2023-11-28T21:34:42.594292+0000","last_clean":"2023-11-28T21:34:42.594292+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:34:42.594292+0000","last_undegraded":"2023-11-28T21:34:42.594292+0000","last_fullsized":"2023-11-28T21:34:42.594292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:22.960012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.345785+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:34:42.345785+0000","last_peered":"2023-11-28T21:34:42.345785+0000","last_clean":"2023-11-28T21:34:42.345785+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:34:42.345785+0000","last_undegraded":"2023-11-28T21:34:42.345785+0000","last_fullsized":"2023-11-28T21:34:42.345785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342908+0000","last_change":"2023-11-28T21:34:37.348126+0000","last_active":"2023-11-28T21:34:41.342908+0000","last_peered":"2023-11-28T21:34:41.342908+0000","last_clean":"2023-11-28T21:34:41.342908+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:34:41.342908+0000","last_undegraded":"2023-11-28T21:34:41.342908+0000","last_fullsized":"2023-11-28T21:34:41.342908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:47.982143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.596939+0000","last_change":"2023-11-28T21:34:41.388539+0000","last_active":"2023-11-28T21:34:42.596939+0000","last_peered":"2023-11-28T21:34:42.596939+0000","last_clean":"2023-11-28T21:34:42.596939+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:34:42.596939+0000","last_undegraded":"2023-11-28T21:34:42.596939+0000","last_fullsized":"2023-11-28T21:34:42.596939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343904+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:34:42.343904+0000","last_peered":"2023-11-28T21:34:42.343904+0000","last_clean":"2023-11-28T21:34:42.343904+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:34:42.343904+0000","last_undegraded":"2023-11-28T21:34:42.343904+0000","last_fullsized":"2023-11-28T21:34:42.343904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346931+0000","last_change":"2023-11-28T21:34:37.337266+0000","last_active":"2023-11-28T21:34:41.346931+0000","last_peered":"2023-11-28T21:34:41.346931+0000","last_clean":"2023-11-28T21:34:41.346931+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:34:41.346931+0000","last_undegraded":"2023-11-28T21:34:41.346931+0000","last_fullsized":"2023-11-28T21:34:41.346931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:32:48.848989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595154+0000","last_change":"2023-11-28T21:34:41.372957+0000","last_active":"2023-11-28T21:34:42.595154+0000","last_peered":"2023-11-28T21:34:42.595154+0000","last_clean":"2023-11-28T21:34:42.595154+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:34:42.595154+0000","last_undegraded":"2023-11-28T21:34:42.595154+0000","last_fullsized":"2023-11-28T21:34:42.595154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367244+0000","last_change":"2023-11-28T21:34:37.353174+0000","last_active":"2023-11-28T21:34:41.367244+0000","last_peered":"2023-11-28T21:34:41.367244+0000","last_clean":"2023-11-28T21:34:41.367244+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:34:41.367244+0000","last_undegraded":"2023-11-28T21:34:41.367244+0000","last_fullsized":"2023-11-28T21:34:41.367244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:32.966564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.345795+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:34:42.345795+0000","last_peered":"2023-11-28T21:34:42.345795+0000","last_clean":"2023-11-28T21:34:42.345795+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:34:42.345795+0000","last_undegraded":"2023-11-28T21:34:42.345795+0000","last_fullsized":"2023-11-28T21:34:42.345795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.599114+0000","last_change":"2023-11-28T21:34:41.364738+0000","last_active":"2023-11-28T21:34:42.599114+0000","last_peered":"2023-11-28T21:34:42.599114+0000","last_clean":"2023-11-28T21:34:42.599114+0000","last_became_active":"2023-11-28T21:34:41.362862+0000","last_became_peered":"2023-11-28T21:34:41.362862+0000","last_unstale":"2023-11-28T21:34:42.599114+0000","last_undegraded":"2023-11-28T21:34:42.599114+0000","last_fullsized":"2023-11-28T21:34:42.599114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:13.830214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339933+0000","last_change":"2023-11-28T21:34:37.313495+0000","last_active":"2023-11-28T21:34:41.339933+0000","last_peered":"2023-11-28T21:34:41.339933+0000","last_clean":"2023-11-28T21:34:41.339933+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:34:41.339933+0000","last_undegraded":"2023-11-28T21:34:41.339933+0000","last_fullsized":"2023-11-28T21:34:41.339933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:32:50.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345994+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:34:41.345994+0000","last_peered":"2023-11-28T21:34:41.345994+0000","last_clean":"2023-11-28T21:34:41.345994+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:34:41.345994+0000","last_undegraded":"2023-11-28T21:34:41.345994+0000","last_fullsized":"2023-11-28T21:34:41.345994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:53.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597596+0000","last_change":"2023-11-28T21:34:41.367580+0000","last_active":"2023-11-28T21:34:42.597596+0000","last_peered":"2023-11-28T21:34:42.597596+0000","last_clean":"2023-11-28T21:34:42.597596+0000","last_became_active":"2023-11-28T21:34:41.366442+0000","last_became_peered":"2023-11-28T21:34:41.366442+0000","last_unstale":"2023-11-28T21:34:42.597596+0000","last_undegraded":"2023-11-28T21:34:42.597596+0000","last_fullsized":"2023-11-28T21:34:42.597596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339648+0000","last_change":"2023-11-28T21:34:37.326569+0000","last_active":"2023-11-28T21:34:41.339648+0000","last_peered":"2023-11-28T21:34:41.339648+0000","last_clean":"2023-11-28T21:34:41.339648+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:34:41.339648+0000","last_undegraded":"2023-11-28T21:34:41.339648+0000","last_fullsized":"2023-11-28T21:34:41.339648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:39:14.264355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343240+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:34:41.343240+0000","last_peered":"2023-11-28T21:34:41.343240+0000","last_clean":"2023-11-28T21:34:41.343240+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:34:41.343240+0000","last_undegraded":"2023-11-28T21:34:41.343240+0000","last_fullsized":"2023-11-28T21:34:41.343240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593262+0000","last_change":"2023-11-28T21:34:41.483661+0000","last_active":"2023-11-28T21:34:42.593262+0000","last_peered":"2023-11-28T21:34:42.593262+0000","last_clean":"2023-11-28T21:34:42.593262+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:34:42.593262+0000","last_undegraded":"2023-11-28T21:34:42.593262+0000","last_fullsized":"2023-11-28T21:34:42.593262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.347011+0000","last_change":"2023-11-28T21:34:37.336922+0000","last_active":"2023-11-28T21:34:41.347011+0000","last_peered":"2023-11-28T21:34:41.347011+0000","last_clean":"2023-11-28T21:34:41.347011+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:34:41.347011+0000","last_undegraded":"2023-11-28T21:34:41.347011+0000","last_fullsized":"2023-11-28T21:34:41.347011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:45.663475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.469299+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:34:41.469299+0000","last_peered":"2023-11-28T21:34:41.469299+0000","last_clean":"2023-11-28T21:34:41.469299+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:34:41.469299+0000","last_undegraded":"2023-11-28T21:34:41.469299+0000","last_fullsized":"2023-11-28T21:34:41.469299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.599296+0000","last_change":"2023-11-28T21:34:41.382756+0000","last_active":"2023-11-28T21:34:42.599296+0000","last_peered":"2023-11-28T21:34:42.599296+0000","last_clean":"2023-11-28T21:34:42.599296+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:34:42.599296+0000","last_undegraded":"2023-11-28T21:34:42.599296+0000","last_fullsized":"2023-11-28T21:34:42.599296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:07.402513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344639+0000","last_change":"2023-11-28T21:34:37.320446+0000","last_active":"2023-11-28T21:34:41.344639+0000","last_peered":"2023-11-28T21:34:41.344639+0000","last_clean":"2023-11-28T21:34:41.344639+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:34:41.344639+0000","last_undegraded":"2023-11-28T21:34:41.344639+0000","last_fullsized":"2023-11-28T21:34:41.344639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:33.404441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.346048+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:34:42.346048+0000","last_peered":"2023-11-28T21:34:42.346048+0000","last_clean":"2023-11-28T21:34:42.346048+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:34:42.346048+0000","last_undegraded":"2023-11-28T21:34:42.346048+0000","last_fullsized":"2023-11-28T21:34:42.346048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595588+0000","last_change":"2023-11-28T21:34:41.485107+0000","last_active":"2023-11-28T21:34:42.595588+0000","last_peered":"2023-11-28T21:34:42.595588+0000","last_clean":"2023-11-28T21:34:42.595588+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:34:42.595588+0000","last_undegraded":"2023-11-28T21:34:42.595588+0000","last_fullsized":"2023-11-28T21:34:42.595588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343566+0000","last_change":"2023-11-28T21:34:37.345104+0000","last_active":"2023-11-28T21:34:41.343566+0000","last_peered":"2023-11-28T21:34:41.343566+0000","last_clean":"2023-11-28T21:34:41.343566+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:34:41.343566+0000","last_undegraded":"2023-11-28T21:34:41.343566+0000","last_fullsized":"2023-11-28T21:34:41.343566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:25:16.125179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593312+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:34:42.593312+0000","last_peered":"2023-11-28T21:34:42.593312+0000","last_clean":"2023-11-28T21:34:42.593312+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:34:42.593312+0000","last_undegraded":"2023-11-28T21:34:42.593312+0000","last_fullsized":"2023-11-28T21:34:42.593312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:36.968220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359721+0000","last_change":"2023-11-28T21:34:41.475704+0000","last_active":"2023-11-28T21:34:42.359721+0000","last_peered":"2023-11-28T21:34:42.359721+0000","last_clean":"2023-11-28T21:34:42.359721+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:34:42.359721+0000","last_undegraded":"2023-11-28T21:34:42.359721+0000","last_fullsized":"2023-11-28T21:34:42.359721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:01:34.494307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341552+0000","last_change":"2023-11-28T21:34:37.330831+0000","last_active":"2023-11-28T21:34:41.341552+0000","last_peered":"2023-11-28T21:34:41.341552+0000","last_clean":"2023-11-28T21:34:41.341552+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:34:41.341552+0000","last_undegraded":"2023-11-28T21:34:41.341552+0000","last_fullsized":"2023-11-28T21:34:41.341552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:50:38.612360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471963+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:34:41.471963+0000","last_peered":"2023-11-28T21:34:41.471963+0000","last_clean":"2023-11-28T21:34:41.471963+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:34:41.471963+0000","last_undegraded":"2023-11-28T21:34:41.471963+0000","last_fullsized":"2023-11-28T21:34:41.471963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592137+0000","last_change":"2023-11-28T21:34:41.475481+0000","last_active":"2023-11-28T21:34:42.592137+0000","last_peered":"2023-11-28T21:34:42.592137+0000","last_clean":"2023-11-28T21:34:42.592137+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:34:42.592137+0000","last_undegraded":"2023-11-28T21:34:42.592137+0000","last_fullsized":"2023-11-28T21:34:42.592137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:22:18.635807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.347169+0000","last_change":"2023-11-28T21:34:37.337846+0000","last_active":"2023-11-28T21:34:41.347169+0000","last_peered":"2023-11-28T21:34:41.347169+0000","last_clean":"2023-11-28T21:34:41.347169+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:34:41.347169+0000","last_undegraded":"2023-11-28T21:34:41.347169+0000","last_fullsized":"2023-11-28T21:34:41.347169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:46:21.358603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342465+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:34:41.342465+0000","last_peered":"2023-11-28T21:34:41.342465+0000","last_clean":"2023-11-28T21:34:41.342465+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:34:41.342465+0000","last_undegraded":"2023-11-28T21:34:41.342465+0000","last_fullsized":"2023-11-28T21:34:41.342465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:08:15.270352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591609+0000","last_change":"2023-11-28T21:34:41.370410+0000","last_active":"2023-11-28T21:34:42.591609+0000","last_peered":"2023-11-28T21:34:42.591609+0000","last_clean":"2023-11-28T21:34:42.591609+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:34:42.591609+0000","last_undegraded":"2023-11-28T21:34:42.591609+0000","last_fullsized":"2023-11-28T21:34:42.591609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342823+0000","last_change":"2023-11-28T21:34:37.324608+0000","last_active":"2023-11-28T21:34:41.342823+0000","last_peered":"2023-11-28T21:34:41.342823+0000","last_clean":"2023-11-28T21:34:41.342823+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:34:41.342823+0000","last_undegraded":"2023-11-28T21:34:41.342823+0000","last_fullsized":"2023-11-28T21:34:41.342823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:13:53.438473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341590+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:34:41.341590+0000","last_peered":"2023-11-28T21:34:41.341590+0000","last_clean":"2023-11-28T21:34:41.341590+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:34:41.341590+0000","last_undegraded":"2023-11-28T21:34:41.341590+0000","last_fullsized":"2023-11-28T21:34:41.341590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592949+0000","last_change":"2023-11-28T21:34:41.380599+0000","last_active":"2023-11-28T21:34:42.592949+0000","last_peered":"2023-11-28T21:34:42.592949+0000","last_clean":"2023-11-28T21:34:42.592949+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:34:42.592949+0000","last_undegraded":"2023-11-28T21:34:42.592949+0000","last_fullsized":"2023-11-28T21:34:42.592949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342555+0000","last_change":"2023-11-28T21:34:37.330869+0000","last_active":"2023-11-28T21:34:41.342555+0000","last_peered":"2023-11-28T21:34:41.342555+0000","last_clean":"2023-11-28T21:34:41.342555+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:34:41.342555+0000","last_undegraded":"2023-11-28T21:34:41.342555+0000","last_fullsized":"2023-11-28T21:34:41.342555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:58:55.987235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.355516+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:34:41.355516+0000","last_peered":"2023-11-28T21:34:41.355516+0000","last_clean":"2023-11-28T21:34:41.355516+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:34:41.355516+0000","last_undegraded":"2023-11-28T21:34:41.355516+0000","last_fullsized":"2023-11-28T21:34:41.355516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593284+0000","last_change":"2023-11-28T21:34:41.371897+0000","last_active":"2023-11-28T21:34:42.593284+0000","last_peered":"2023-11-28T21:34:42.593284+0000","last_clean":"2023-11-28T21:34:42.593284+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:34:42.593284+0000","last_undegraded":"2023-11-28T21:34:42.593284+0000","last_fullsized":"2023-11-28T21:34:42.593284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343750+0000","last_change":"2023-11-28T21:34:37.319833+0000","last_active":"2023-11-28T21:34:41.343750+0000","last_peered":"2023-11-28T21:34:41.343750+0000","last_clean":"2023-11-28T21:34:41.343750+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:34:41.343750+0000","last_undegraded":"2023-11-28T21:34:41.343750+0000","last_fullsized":"2023-11-28T21:34:41.343750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:02:17.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.469428+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:34:41.469428+0000","last_peered":"2023-11-28T21:34:41.469428+0000","last_clean":"2023-11-28T21:34:41.469428+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:34:41.469428+0000","last_undegraded":"2023-11-28T21:34:41.469428+0000","last_fullsized":"2023-11-28T21:34:41.469428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592392+0000","last_change":"2023-11-28T21:34:41.475053+0000","last_active":"2023-11-28T21:34:42.592392+0000","last_peered":"2023-11-28T21:34:42.592392+0000","last_clean":"2023-11-28T21:34:42.592392+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:34:42.592392+0000","last_undegraded":"2023-11-28T21:34:42.592392+0000","last_fullsized":"2023-11-28T21:34:42.592392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:32.658001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340067+0000","last_change":"2023-11-28T21:34:37.358890+0000","last_active":"2023-11-28T21:34:41.340067+0000","last_peered":"2023-11-28T21:34:41.340067+0000","last_clean":"2023-11-28T21:34:41.340067+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:34:41.340067+0000","last_undegraded":"2023-11-28T21:34:41.340067+0000","last_fullsized":"2023-11-28T21:34:41.340067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:54.878141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340738+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:34:41.340738+0000","last_peered":"2023-11-28T21:34:41.340738+0000","last_clean":"2023-11-28T21:34:41.340738+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:34:41.340738+0000","last_undegraded":"2023-11-28T21:34:41.340738+0000","last_fullsized":"2023-11-28T21:34:41.340738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.469889+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:34:41.469889+0000","last_peered":"2023-11-28T21:34:41.469889+0000","last_clean":"2023-11-28T21:34:41.469889+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:34:41.469889+0000","last_undegraded":"2023-11-28T21:34:41.469889+0000","last_fullsized":"2023-11-28T21:34:41.469889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594727+0000","last_change":"2023-11-28T21:34:41.477522+0000","last_active":"2023-11-28T21:34:42.594727+0000","last_peered":"2023-11-28T21:34:42.594727+0000","last_clean":"2023-11-28T21:34:42.594727+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:34:42.594727+0000","last_undegraded":"2023-11-28T21:34:42.594727+0000","last_fullsized":"2023-11-28T21:34:42.594727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:36:36.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471469+0000","last_change":"2023-11-28T21:34:37.350441+0000","last_active":"2023-11-28T21:34:41.471469+0000","last_peered":"2023-11-28T21:34:41.471469+0000","last_clean":"2023-11-28T21:34:41.471469+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:34:41.471469+0000","last_undegraded":"2023-11-28T21:34:41.471469+0000","last_fullsized":"2023-11-28T21:34:41.471469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:54.775495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.355701+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:34:41.355701+0000","last_peered":"2023-11-28T21:34:41.355701+0000","last_clean":"2023-11-28T21:34:41.355701+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:34:41.355701+0000","last_undegraded":"2023-11-28T21:34:41.355701+0000","last_fullsized":"2023-11-28T21:34:41.355701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593850+0000","last_change":"2023-11-28T21:34:41.485154+0000","last_active":"2023-11-28T21:34:42.593850+0000","last_peered":"2023-11-28T21:34:42.593850+0000","last_clean":"2023-11-28T21:34:42.593850+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:34:42.593850+0000","last_undegraded":"2023-11-28T21:34:42.593850+0000","last_fullsized":"2023-11-28T21:34:42.593850+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:35:21.006172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343008+0000","last_change":"2023-11-28T21:34:37.345652+0000","last_active":"2023-11-28T21:34:41.343008+0000","last_peered":"2023-11-28T21:34:41.343008+0000","last_clean":"2023-11-28T21:34:41.343008+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:34:41.343008+0000","last_undegraded":"2023-11-28T21:34:41.343008+0000","last_fullsized":"2023-11-28T21:34:41.343008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:21:17.474200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343244+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:34:42.343244+0000","last_peered":"2023-11-28T21:34:42.343244+0000","last_clean":"2023-11-28T21:34:42.343244+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:34:42.343244+0000","last_undegraded":"2023-11-28T21:34:42.343244+0000","last_fullsized":"2023-11-28T21:34:42.343244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:48.073446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592358+0000","last_change":"2023-11-28T21:34:41.370764+0000","last_active":"2023-11-28T21:34:42.592358+0000","last_peered":"2023-11-28T21:34:42.592358+0000","last_clean":"2023-11-28T21:34:42.592358+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:34:42.592358+0000","last_undegraded":"2023-11-28T21:34:42.592358+0000","last_fullsized":"2023-11-28T21:34:42.592358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:06:42.297098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471917+0000","last_change":"2023-11-28T21:34:37.359589+0000","last_active":"2023-11-28T21:34:41.471917+0000","last_peered":"2023-11-28T21:34:41.471917+0000","last_clean":"2023-11-28T21:34:41.471917+0000","last_became_active":"2023-11-28T21:34:37.359042+0000","last_became_peered":"2023-11-28T21:34:37.359042+0000","last_unstale":"2023-11-28T21:34:41.471917+0000","last_undegraded":"2023-11-28T21:34:41.471917+0000","last_fullsized":"2023-11-28T21:34:41.471917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:27.020614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.339535+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:34:42.339535+0000","last_peered":"2023-11-28T21:34:42.339535+0000","last_clean":"2023-11-28T21:34:42.339535+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:34:42.339535+0000","last_undegraded":"2023-11-28T21:34:42.339535+0000","last_fullsized":"2023-11-28T21:34:42.339535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.362523+0000","last_change":"2023-11-28T21:34:41.477127+0000","last_active":"2023-11-28T21:34:42.362523+0000","last_peered":"2023-11-28T21:34:42.362523+0000","last_clean":"2023-11-28T21:34:42.362523+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:34:42.362523+0000","last_undegraded":"2023-11-28T21:34:42.362523+0000","last_fullsized":"2023-11-28T21:34:42.362523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:32.798817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471715+0000","last_change":"2023-11-28T21:34:37.355693+0000","last_active":"2023-11-28T21:34:41.471715+0000","last_peered":"2023-11-28T21:34:41.471715+0000","last_clean":"2023-11-28T21:34:41.471715+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:34:41.471715+0000","last_undegraded":"2023-11-28T21:34:41.471715+0000","last_fullsized":"2023-11-28T21:34:41.471715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:12.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.338222+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:34:42.338222+0000","last_peered":"2023-11-28T21:34:42.338222+0000","last_clean":"2023-11-28T21:34:42.338222+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:34:42.338222+0000","last_undegraded":"2023-11-28T21:34:42.338222+0000","last_fullsized":"2023-11-28T21:34:42.338222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594073+0000","last_change":"2023-11-28T21:34:41.382468+0000","last_active":"2023-11-28T21:34:42.594073+0000","last_peered":"2023-11-28T21:34:42.594073+0000","last_clean":"2023-11-28T21:34:42.594073+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:34:42.594073+0000","last_undegraded":"2023-11-28T21:34:42.594073+0000","last_fullsized":"2023-11-28T21:34:42.594073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:17.410943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470942+0000","last_change":"2023-11-28T21:34:37.353232+0000","last_active":"2023-11-28T21:34:41.470942+0000","last_peered":"2023-11-28T21:34:41.470942+0000","last_clean":"2023-11-28T21:34:41.470942+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:34:41.470942+0000","last_undegraded":"2023-11-28T21:34:41.470942+0000","last_fullsized":"2023-11-28T21:34:41.470942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:11.285378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.340568+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:34:42.340568+0000","last_peered":"2023-11-28T21:34:42.340568+0000","last_clean":"2023-11-28T21:34:42.340568+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:34:42.340568+0000","last_undegraded":"2023-11-28T21:34:42.340568+0000","last_fullsized":"2023-11-28T21:34:42.340568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592317+0000","last_change":"2023-11-28T21:34:41.385400+0000","last_active":"2023-11-28T21:34:42.592317+0000","last_peered":"2023-11-28T21:34:42.592317+0000","last_clean":"2023-11-28T21:34:42.592317+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:34:42.592317+0000","last_undegraded":"2023-11-28T21:34:42.592317+0000","last_fullsized":"2023-11-28T21:34:42.592317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:36.433439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365468+0000","last_change":"2023-11-28T21:34:37.333461+0000","last_active":"2023-11-28T21:34:41.365468+0000","last_peered":"2023-11-28T21:34:41.365468+0000","last_clean":"2023-11-28T21:34:41.365468+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:34:41.365468+0000","last_undegraded":"2023-11-28T21:34:41.365468+0000","last_fullsized":"2023-11-28T21:34:41.365468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:25.513524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.366026+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:34:41.366026+0000","last_peered":"2023-11-28T21:34:41.366026+0000","last_clean":"2023-11-28T21:34:41.366026+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:34:41.366026+0000","last_undegraded":"2023-11-28T21:34:41.366026+0000","last_fullsized":"2023-11-28T21:34:41.366026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361391+0000","last_change":"2023-11-28T21:34:41.479919+0000","last_active":"2023-11-28T21:34:42.361391+0000","last_peered":"2023-11-28T21:34:42.361391+0000","last_clean":"2023-11-28T21:34:42.361391+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:34:42.361391+0000","last_undegraded":"2023-11-28T21:34:42.361391+0000","last_fullsized":"2023-11-28T21:34:42.361391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:11.183117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471078+0000","last_change":"2023-11-28T21:34:37.355729+0000","last_active":"2023-11-28T21:34:41.471078+0000","last_peered":"2023-11-28T21:34:41.471078+0000","last_clean":"2023-11-28T21:34:41.471078+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:34:41.471078+0000","last_undegraded":"2023-11-28T21:34:41.471078+0000","last_fullsized":"2023-11-28T21:34:41.471078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:39.178378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341388+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:34:41.341388+0000","last_peered":"2023-11-28T21:34:41.341388+0000","last_clean":"2023-11-28T21:34:41.341388+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:34:41.341388+0000","last_undegraded":"2023-11-28T21:34:41.341388+0000","last_fullsized":"2023-11-28T21:34:41.341388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361306+0000","last_change":"2023-11-28T21:34:41.478483+0000","last_active":"2023-11-28T21:34:42.361306+0000","last_peered":"2023-11-28T21:34:42.361306+0000","last_clean":"2023-11-28T21:34:42.361306+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:34:42.361306+0000","last_undegraded":"2023-11-28T21:34:42.361306+0000","last_fullsized":"2023-11-28T21:34:42.361306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:25:54.342692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345020+0000","last_change":"2023-11-28T21:34:37.345476+0000","last_active":"2023-11-28T21:34:41.345020+0000","last_peered":"2023-11-28T21:34:41.345020+0000","last_clean":"2023-11-28T21:34:41.345020+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:34:41.345020+0000","last_undegraded":"2023-11-28T21:34:41.345020+0000","last_fullsized":"2023-11-28T21:34:41.345020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.660412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343340+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:34:42.343340+0000","last_peered":"2023-11-28T21:34:42.343340+0000","last_clean":"2023-11-28T21:34:42.343340+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:34:42.343340+0000","last_undegraded":"2023-11-28T21:34:42.343340+0000","last_fullsized":"2023-11-28T21:34:42.343340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:28.694184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363244+0000","last_change":"2023-11-28T21:34:41.477926+0000","last_active":"2023-11-28T21:34:42.363244+0000","last_peered":"2023-11-28T21:34:42.363244+0000","last_clean":"2023-11-28T21:34:42.363244+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:34:42.363244+0000","last_undegraded":"2023-11-28T21:34:42.363244+0000","last_fullsized":"2023-11-28T21:34:42.363244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:26.403506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471448+0000","last_change":"2023-11-28T21:34:37.343885+0000","last_active":"2023-11-28T21:34:41.471448+0000","last_peered":"2023-11-28T21:34:41.471448+0000","last_clean":"2023-11-28T21:34:41.471448+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:34:41.471448+0000","last_undegraded":"2023-11-28T21:34:41.471448+0000","last_fullsized":"2023-11-28T21:34:41.471448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:36.255834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:50.648988+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:34:50.648988+0000","last_peered":"2023-11-28T21:34:50.648988+0000","last_clean":"2023-11-28T21:34:50.648988+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:34:50.648988+0000","last_undegraded":"2023-11-28T21:34:50.648988+0000","last_fullsized":"2023-11-28T21:34:50.648988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591595+0000","last_change":"2023-11-28T21:34:41.380565+0000","last_active":"2023-11-28T21:34:42.591595+0000","last_peered":"2023-11-28T21:34:42.591595+0000","last_clean":"2023-11-28T21:34:42.591595+0000","last_became_active":"2023-11-28T21:34:41.380080+0000","last_became_peered":"2023-11-28T21:34:41.380080+0000","last_unstale":"2023-11-28T21:34:42.591595+0000","last_undegraded":"2023-11-28T21:34:42.591595+0000","last_fullsized":"2023-11-28T21:34:42.591595+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343524+0000","last_change":"2023-11-28T21:34:37.345635+0000","last_active":"2023-11-28T21:34:41.343524+0000","last_peered":"2023-11-28T21:34:41.343524+0000","last_clean":"2023-11-28T21:34:41.343524+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:34:41.343524+0000","last_undegraded":"2023-11-28T21:34:41.343524+0000","last_fullsized":"2023-11-28T21:34:41.343524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:52:12.144267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343474+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:34:42.343474+0000","last_peered":"2023-11-28T21:34:42.343474+0000","last_clean":"2023-11-28T21:34:42.343474+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:34:42.343474+0000","last_undegraded":"2023-11-28T21:34:42.343474+0000","last_fullsized":"2023-11-28T21:34:42.343474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:35.266650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597618+0000","last_change":"2023-11-28T21:34:41.365758+0000","last_active":"2023-11-28T21:34:42.597618+0000","last_peered":"2023-11-28T21:34:42.597618+0000","last_clean":"2023-11-28T21:34:42.597618+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:34:42.597618+0000","last_undegraded":"2023-11-28T21:34:42.597618+0000","last_fullsized":"2023-11-28T21:34:42.597618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344832+0000","last_change":"2023-11-28T21:34:37.333607+0000","last_active":"2023-11-28T21:34:41.344832+0000","last_peered":"2023-11-28T21:34:41.344832+0000","last_clean":"2023-11-28T21:34:41.344832+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:34:41.344832+0000","last_undegraded":"2023-11-28T21:34:41.344832+0000","last_fullsized":"2023-11-28T21:34:41.344832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:13:01.639908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342208+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:34:41.342208+0000","last_peered":"2023-11-28T21:34:41.342208+0000","last_clean":"2023-11-28T21:34:41.342208+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:34:41.342208+0000","last_undegraded":"2023-11-28T21:34:41.342208+0000","last_fullsized":"2023-11-28T21:34:41.342208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359893+0000","last_change":"2023-11-28T21:34:41.380413+0000","last_active":"2023-11-28T21:34:42.359893+0000","last_peered":"2023-11-28T21:34:42.359893+0000","last_clean":"2023-11-28T21:34:42.359893+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:34:42.359893+0000","last_undegraded":"2023-11-28T21:34:42.359893+0000","last_fullsized":"2023-11-28T21:34:42.359893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:20:28.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342001+0000","last_change":"2023-11-28T21:34:37.351009+0000","last_active":"2023-11-28T21:34:41.342001+0000","last_peered":"2023-11-28T21:34:41.342001+0000","last_clean":"2023-11-28T21:34:41.342001+0000","last_became_active":"2023-11-28T21:34:37.350321+0000","last_became_peered":"2023-11-28T21:34:37.350321+0000","last_unstale":"2023-11-28T21:34:41.342001+0000","last_undegraded":"2023-11-28T21:34:41.342001+0000","last_fullsized":"2023-11-28T21:34:41.342001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:27:42.022466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.368215+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:34:41.368215+0000","last_peered":"2023-11-28T21:34:41.368215+0000","last_clean":"2023-11-28T21:34:41.368215+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:34:41.368215+0000","last_undegraded":"2023-11-28T21:34:41.368215+0000","last_fullsized":"2023-11-28T21:34:41.368215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359889+0000","last_change":"2023-11-28T21:34:41.475185+0000","last_active":"2023-11-28T21:34:42.359889+0000","last_peered":"2023-11-28T21:34:42.359889+0000","last_clean":"2023-11-28T21:34:42.359889+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:34:42.359889+0000","last_undegraded":"2023-11-28T21:34:42.359889+0000","last_fullsized":"2023-11-28T21:34:42.359889+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:09.389881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592347+0000","last_change":"2023-11-28T21:34:41.479956+0000","last_active":"2023-11-28T21:34:42.592347+0000","last_peered":"2023-11-28T21:34:42.592347+0000","last_clean":"2023-11-28T21:34:42.592347+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:34:42.592347+0000","last_undegraded":"2023-11-28T21:34:42.592347+0000","last_fullsized":"2023-11-28T21:34:42.592347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:46.004852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343050+0000","last_change":"2023-11-28T21:34:37.317513+0000","last_active":"2023-11-28T21:34:41.343050+0000","last_peered":"2023-11-28T21:34:41.343050+0000","last_clean":"2023-11-28T21:34:41.343050+0000","last_became_active":"2023-11-28T21:34:37.317082+0000","last_became_peered":"2023-11-28T21:34:37.317082+0000","last_unstale":"2023-11-28T21:34:41.343050+0000","last_undegraded":"2023-11-28T21:34:41.343050+0000","last_fullsized":"2023-11-28T21:34:41.343050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:05:37.846078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346371+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:34:41.346371+0000","last_peered":"2023-11-28T21:34:41.346371+0000","last_clean":"2023-11-28T21:34:41.346371+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:34:41.346371+0000","last_undegraded":"2023-11-28T21:34:41.346371+0000","last_fullsized":"2023-11-28T21:34:41.346371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595858+0000","last_change":"2023-11-28T21:34:41.385969+0000","last_active":"2023-11-28T21:34:42.595858+0000","last_peered":"2023-11-28T21:34:42.595858+0000","last_clean":"2023-11-28T21:34:42.595858+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:34:42.595858+0000","last_undegraded":"2023-11-28T21:34:42.595858+0000","last_fullsized":"2023-11-28T21:34:42.595858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:01.692549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339578+0000","last_change":"2023-11-28T21:34:37.314087+0000","last_active":"2023-11-28T21:34:41.339578+0000","last_peered":"2023-11-28T21:34:41.339578+0000","last_clean":"2023-11-28T21:34:41.339578+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:34:41.339578+0000","last_undegraded":"2023-11-28T21:34:41.339578+0000","last_fullsized":"2023-11-28T21:34:41.339578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:25:11.912313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340387+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:34:41.340387+0000","last_peered":"2023-11-28T21:34:41.340387+0000","last_clean":"2023-11-28T21:34:41.340387+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:34:41.340387+0000","last_undegraded":"2023-11-28T21:34:41.340387+0000","last_fullsized":"2023-11-28T21:34:41.340387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343264+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:34:42.343264+0000","last_peered":"2023-11-28T21:34:42.343264+0000","last_clean":"2023-11-28T21:34:42.343264+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:34:42.343264+0000","last_undegraded":"2023-11-28T21:34:42.343264+0000","last_fullsized":"2023-11-28T21:34:42.343264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345191+0000","last_change":"2023-11-28T21:34:37.319801+0000","last_active":"2023-11-28T21:34:41.345191+0000","last_peered":"2023-11-28T21:34:41.345191+0000","last_clean":"2023-11-28T21:34:41.345191+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:34:41.345191+0000","last_undegraded":"2023-11-28T21:34:41.345191+0000","last_fullsized":"2023-11-28T21:34:41.345191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:40:13.251522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360747+0000","last_change":"2023-11-28T21:34:41.370523+0000","last_active":"2023-11-28T21:34:42.360747+0000","last_peered":"2023-11-28T21:34:42.360747+0000","last_clean":"2023-11-28T21:34:42.360747+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:34:42.360747+0000","last_undegraded":"2023-11-28T21:34:42.360747+0000","last_fullsized":"2023-11-28T21:34:42.360747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:17:36.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343058+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:34:42.343058+0000","last_peered":"2023-11-28T21:34:42.343058+0000","last_clean":"2023-11-28T21:34:42.343058+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:34:42.343058+0000","last_undegraded":"2023-11-28T21:34:42.343058+0000","last_fullsized":"2023-11-28T21:34:42.343058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342331+0000","last_change":"2023-11-28T21:34:37.328809+0000","last_active":"2023-11-28T21:34:41.342331+0000","last_peered":"2023-11-28T21:34:41.342331+0000","last_clean":"2023-11-28T21:34:41.342331+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:34:41.342331+0000","last_undegraded":"2023-11-28T21:34:41.342331+0000","last_fullsized":"2023-11-28T21:34:41.342331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:27.300090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361127+0000","last_change":"2023-11-28T21:34:41.363087+0000","last_active":"2023-11-28T21:34:42.361127+0000","last_peered":"2023-11-28T21:34:42.361127+0000","last_clean":"2023-11-28T21:34:42.361127+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:34:42.361127+0000","last_undegraded":"2023-11-28T21:34:42.361127+0000","last_fullsized":"2023-11-28T21:34:42.361127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:35.046132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365193+0000","last_change":"2023-11-28T21:34:37.333670+0000","last_active":"2023-11-28T21:34:41.365193+0000","last_peered":"2023-11-28T21:34:41.365193+0000","last_clean":"2023-11-28T21:34:41.365193+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:34:41.365193+0000","last_undegraded":"2023-11-28T21:34:41.365193+0000","last_fullsized":"2023-11-28T21:34:41.365193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:32.598014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.355458+0000","last_change":"2023-11-28T21:34:37.347101+0000","last_active":"2023-11-28T21:34:41.355458+0000","last_peered":"2023-11-28T21:34:41.355458+0000","last_clean":"2023-11-28T21:34:41.355458+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:34:41.355458+0000","last_undegraded":"2023-11-28T21:34:41.355458+0000","last_fullsized":"2023-11-28T21:34:41.355458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:39:43.186110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341187+0000","last_change":"2023-11-28T21:34:37.338576+0000","last_active":"2023-11-28T21:34:41.341187+0000","last_peered":"2023-11-28T21:34:41.341187+0000","last_clean":"2023-11-28T21:34:41.341187+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:34:41.341187+0000","last_undegraded":"2023-11-28T21:34:41.341187+0000","last_fullsized":"2023-11-28T21:34:41.341187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:53:02.356097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345097+0000","last_change":"2023-11-28T21:34:37.316270+0000","last_active":"2023-11-28T21:34:41.345097+0000","last_peered":"2023-11-28T21:34:41.345097+0000","last_clean":"2023-11-28T21:34:41.345097+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:34:41.345097+0000","last_undegraded":"2023-11-28T21:34:41.345097+0000","last_fullsized":"2023-11-28T21:34:41.345097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:43.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340024+0000","last_change":"2023-11-28T21:34:37.334973+0000","last_active":"2023-11-28T21:34:41.340024+0000","last_peered":"2023-11-28T21:34:41.340024+0000","last_clean":"2023-11-28T21:34:41.340024+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:34:41.340024+0000","last_undegraded":"2023-11-28T21:34:41.340024+0000","last_fullsized":"2023-11-28T21:34:41.340024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:55.050945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345807+0000","last_change":"2023-11-28T21:34:37.318155+0000","last_active":"2023-11-28T21:34:41.345807+0000","last_peered":"2023-11-28T21:34:41.345807+0000","last_clean":"2023-11-28T21:34:41.345807+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:34:41.345807+0000","last_undegraded":"2023-11-28T21:34:41.345807+0000","last_fullsized":"2023-11-28T21:34:41.345807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:29.840255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367255+0000","last_change":"2023-11-28T21:34:37.345758+0000","last_active":"2023-11-28T21:34:41.367255+0000","last_peered":"2023-11-28T21:34:41.367255+0000","last_clean":"2023-11-28T21:34:41.367255+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:34:41.367255+0000","last_undegraded":"2023-11-28T21:34:41.367255+0000","last_fullsized":"2023-11-28T21:34:41.367255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:26.444654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343483+0000","last_change":"2023-11-28T21:34:37.334989+0000","last_active":"2023-11-28T21:34:41.343483+0000","last_peered":"2023-11-28T21:34:41.343483+0000","last_clean":"2023-11-28T21:34:41.343483+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:34:41.343483+0000","last_undegraded":"2023-11-28T21:34:41.343483+0000","last_fullsized":"2023-11-28T21:34:41.343483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:36:57.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346163+0000","last_change":"2023-11-28T21:34:37.333075+0000","last_active":"2023-11-28T21:34:41.346163+0000","last_peered":"2023-11-28T21:34:41.346163+0000","last_clean":"2023-11-28T21:34:41.346163+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:34:41.346163+0000","last_undegraded":"2023-11-28T21:34:41.346163+0000","last_fullsized":"2023-11-28T21:34:41.346163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:50:43.748349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339854+0000","last_change":"2023-11-28T21:34:37.337230+0000","last_active":"2023-11-28T21:34:41.339854+0000","last_peered":"2023-11-28T21:34:41.339854+0000","last_clean":"2023-11-28T21:34:41.339854+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:34:41.339854+0000","last_undegraded":"2023-11-28T21:34:41.339854+0000","last_fullsized":"2023-11-28T21:34:41.339854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:48:45.994559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340945+0000","last_change":"2023-11-28T21:34:37.349894+0000","last_active":"2023-11-28T21:34:41.340945+0000","last_peered":"2023-11-28T21:34:41.340945+0000","last_clean":"2023-11-28T21:34:41.340945+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:34:41.340945+0000","last_undegraded":"2023-11-28T21:34:41.340945+0000","last_fullsized":"2023-11-28T21:34:41.340945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:10.293911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345003+0000","last_change":"2023-11-28T21:34:37.344269+0000","last_active":"2023-11-28T21:34:41.345003+0000","last_peered":"2023-11-28T21:34:41.345003+0000","last_clean":"2023-11-28T21:34:41.345003+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:34:41.345003+0000","last_undegraded":"2023-11-28T21:34:41.345003+0000","last_fullsized":"2023-11-28T21:34:41.345003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:03.204740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365122+0000","last_change":"2023-11-28T21:34:37.353059+0000","last_active":"2023-11-28T21:34:41.365122+0000","last_peered":"2023-11-28T21:34:41.365122+0000","last_clean":"2023-11-28T21:34:41.365122+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:34:41.365122+0000","last_undegraded":"2023-11-28T21:34:41.365122+0000","last_fullsized":"2023-11-28T21:34:41.365122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:45:12.640797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471801+0000","last_change":"2023-11-28T21:34:37.352965+0000","last_active":"2023-11-28T21:34:41.471801+0000","last_peered":"2023-11-28T21:34:41.471801+0000","last_clean":"2023-11-28T21:34:41.471801+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:34:41.471801+0000","last_undegraded":"2023-11-28T21:34:41.471801+0000","last_fullsized":"2023-11-28T21:34:41.471801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:30:42.087715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.356122+0000","last_change":"2023-11-28T21:34:37.336830+0000","last_active":"2023-11-28T21:34:41.356122+0000","last_peered":"2023-11-28T21:34:41.356122+0000","last_clean":"2023-11-28T21:34:41.356122+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:34:41.356122+0000","last_undegraded":"2023-11-28T21:34:41.356122+0000","last_fullsized":"2023-11-28T21:34:41.356122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:57.318311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341567+0000","last_change":"2023-11-28T21:34:37.349181+0000","last_active":"2023-11-28T21:34:41.341567+0000","last_peered":"2023-11-28T21:34:41.341567+0000","last_clean":"2023-11-28T21:34:41.341567+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:34:41.341567+0000","last_undegraded":"2023-11-28T21:34:41.341567+0000","last_fullsized":"2023-11-28T21:34:41.341567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:27:59.989299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345968+0000","last_change":"2023-11-28T21:34:37.336668+0000","last_active":"2023-11-28T21:34:41.345968+0000","last_peered":"2023-11-28T21:34:41.345968+0000","last_clean":"2023-11-28T21:34:41.345968+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:34:41.345968+0000","last_undegraded":"2023-11-28T21:34:41.345968+0000","last_fullsized":"2023-11-28T21:34:41.345968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:06.416717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343147+0000","last_change":"2023-11-28T21:34:37.332790+0000","last_active":"2023-11-28T21:34:41.343147+0000","last_peered":"2023-11-28T21:34:41.343147+0000","last_clean":"2023-11-28T21:34:41.343147+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:34:41.343147+0000","last_undegraded":"2023-11-28T21:34:41.343147+0000","last_fullsized":"2023-11-28T21:34:41.343147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:32:45.920387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344930+0000","last_change":"2023-11-28T21:34:37.320794+0000","last_active":"2023-11-28T21:34:41.344930+0000","last_peered":"2023-11-28T21:34:41.344930+0000","last_clean":"2023-11-28T21:34:41.344930+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:34:41.344930+0000","last_undegraded":"2023-11-28T21:34:41.344930+0000","last_fullsized":"2023-11-28T21:34:41.344930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:16:46.347683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339818+0000","last_change":"2023-11-28T21:34:37.337726+0000","last_active":"2023-11-28T21:34:41.339818+0000","last_peered":"2023-11-28T21:34:41.339818+0000","last_clean":"2023-11-28T21:34:41.339818+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:34:41.339818+0000","last_undegraded":"2023-11-28T21:34:41.339818+0000","last_fullsized":"2023-11-28T21:34:41.339818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:38:43.884668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343259+0000","last_change":"2023-11-28T21:34:37.334677+0000","last_active":"2023-11-28T21:34:41.343259+0000","last_peered":"2023-11-28T21:34:41.343259+0000","last_clean":"2023-11-28T21:34:41.343259+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:34:41.343259+0000","last_undegraded":"2023-11-28T21:34:41.343259+0000","last_fullsized":"2023-11-28T21:34:41.343259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:05:54.981899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.366695+0000","last_change":"2023-11-28T21:34:37.354861+0000","last_active":"2023-11-28T21:34:41.366695+0000","last_peered":"2023-11-28T21:34:41.366695+0000","last_clean":"2023-11-28T21:34:41.366695+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:34:41.366695+0000","last_undegraded":"2023-11-28T21:34:41.366695+0000","last_fullsized":"2023-11-28T21:34:41.366695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:30.602284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367055+0000","last_change":"2023-11-28T21:34:37.347675+0000","last_active":"2023-11-28T21:34:41.367055+0000","last_peered":"2023-11-28T21:34:41.367055+0000","last_clean":"2023-11-28T21:34:41.367055+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:34:41.367055+0000","last_undegraded":"2023-11-28T21:34:41.367055+0000","last_fullsized":"2023-11-28T21:34:41.367055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:51:38.803896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471861+0000","last_change":"2023-11-28T21:34:37.357735+0000","last_active":"2023-11-28T21:34:41.471861+0000","last_peered":"2023-11-28T21:34:41.471861+0000","last_clean":"2023-11-28T21:34:41.471861+0000","last_became_active":"2023-11-28T21:34:37.357173+0000","last_became_peered":"2023-11-28T21:34:37.357173+0000","last_unstale":"2023-11-28T21:34:41.471861+0000","last_undegraded":"2023-11-28T21:34:41.471861+0000","last_fullsized":"2023-11-28T21:34:41.471861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:38.660713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339546+0000","last_change":"2023-11-28T21:34:37.358851+0000","last_active":"2023-11-28T21:34:41.339546+0000","last_peered":"2023-11-28T21:34:41.339546+0000","last_clean":"2023-11-28T21:34:41.339546+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:34:41.339546+0000","last_undegraded":"2023-11-28T21:34:41.339546+0000","last_fullsized":"2023-11-28T21:34:41.339546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.378519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470714+0000","last_change":"2023-11-28T21:34:37.343972+0000","last_active":"2023-11-28T21:34:41.470714+0000","last_peered":"2023-11-28T21:34:41.470714+0000","last_clean":"2023-11-28T21:34:41.470714+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:34:41.470714+0000","last_undegraded":"2023-11-28T21:34:41.470714+0000","last_fullsized":"2023-11-28T21:34:41.470714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:56:00.591888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340667+0000","last_change":"2023-11-28T21:34:37.346299+0000","last_active":"2023-11-28T21:34:41.340667+0000","last_peered":"2023-11-28T21:34:41.340667+0000","last_clean":"2023-11-28T21:34:41.340667+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:34:41.340667+0000","last_undegraded":"2023-11-28T21:34:41.340667+0000","last_fullsized":"2023-11-28T21:34:41.340667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:06.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470739+0000","last_change":"2023-11-28T21:34:37.343410+0000","last_active":"2023-11-28T21:34:41.470739+0000","last_peered":"2023-11-28T21:34:41.470739+0000","last_clean":"2023-11-28T21:34:41.470739+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:34:41.470739+0000","last_undegraded":"2023-11-28T21:34:41.470739+0000","last_fullsized":"2023-11-28T21:34:41.470739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:29:45.070478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342784+0000","last_change":"2023-11-28T21:34:37.348270+0000","last_active":"2023-11-28T21:34:41.342784+0000","last_peered":"2023-11-28T21:34:41.342784+0000","last_clean":"2023-11-28T21:34:41.342784+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:34:41.342784+0000","last_undegraded":"2023-11-28T21:34:41.342784+0000","last_fullsized":"2023-11-28T21:34:41.342784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:36:10.938478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344314+0000","last_change":"2023-11-28T21:34:37.330786+0000","last_active":"2023-11-28T21:34:41.344314+0000","last_peered":"2023-11-28T21:34:41.344314+0000","last_clean":"2023-11-28T21:34:41.344314+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:34:41.344314+0000","last_undegraded":"2023-11-28T21:34:41.344314+0000","last_fullsized":"2023-11-28T21:34:41.344314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:52:36.738867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346251+0000","last_change":"2023-11-28T21:34:37.318725+0000","last_active":"2023-11-28T21:34:41.346251+0000","last_peered":"2023-11-28T21:34:41.346251+0000","last_clean":"2023-11-28T21:34:41.346251+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:34:41.346251+0000","last_undegraded":"2023-11-28T21:34:41.346251+0000","last_fullsized":"2023-11-28T21:34:41.346251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:49:21.775066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471764+0000","last_change":"2023-11-28T21:34:37.349829+0000","last_active":"2023-11-28T21:34:41.471764+0000","last_peered":"2023-11-28T21:34:41.471764+0000","last_clean":"2023-11-28T21:34:41.471764+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:34:41.471764+0000","last_undegraded":"2023-11-28T21:34:41.471764+0000","last_fullsized":"2023-11-28T21:34:41.471764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:07.752862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":690742,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":99952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":689779,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:35:06.954 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T21:35:06.954 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T21:35:06.955 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T21:35:06.955 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T21:35:06.955 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T21:35:07.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.132+0000 7fc9e86c9640 1 Processor -- start 2023-11-28T21:35:07.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.132+0000 7fc9e86c9640 1 -- start start 2023-11-28T21:35:07.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.132+0000 7fc9e86c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9e0151ed0 0x7fc9e00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.132+0000 7fc9e86c9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc9e006e300 0x7fc9e0072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.132+0000 7fc9e86c9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e0157b70 con 0x7fc9e0151af0 2023-11-28T21:35:07.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.132+0000 7fc9e86c9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e00707f0 con 0x7fc9e006e300 2023-11-28T21:35:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.132+0000 7fc9e86c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00c7b90 con 0x7fc9e0151ed0 2023-11-28T21:35:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e6c66640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc9e006e300 0x7fc9e0072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e6465640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc9e0151af0 0x7fc9e00c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36322/0 (socket says 172.21.15.12:36322) 2023-11-28T21:35:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e6465640 1 -- 172.21.15.12:0/4257703879 learned_addr learned my addr 172.21.15.12:0/4257703879 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:07.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e5c64640 1 --2- 172.21.15.12:0/4257703879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9e0151ed0 0x7fc9e00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e6c66640 1 -- 172.21.15.12:0/4257703879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9e0151ed0 msgr2=0x7fc9e00c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:07.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e6c66640 1 --2- 172.21.15.12:0/4257703879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9e0151ed0 0x7fc9e00c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e6c66640 1 -- 172.21.15.12:0/4257703879 >> v1:172.21.15.12:6789/0 conn(0x7fc9e0151af0 legacy=0x7fc9e00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e6c66640 1 -- 172.21.15.12:0/4257703879 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e0157e50 con 0x7fc9e006e300 2023-11-28T21:35:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e5463640 1 -- 172.21.15.12:0/4257703879 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215071390 0 0) 0x7fc9e0157b70 con 0x7fc9e0151af0 2023-11-28T21:35:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e6c66640 1 --2- 172.21.15.12:0/4257703879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 0x7fc9e0072070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009d90 tx=0x7fc9d801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb2647c48ff81e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e5463640 1 -- 172.21.15.12:0/4257703879 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80a5810 con 0x7fc9e006e300 2023-11-28T21:35:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e5463640 1 -- 172.21.15.12:0/4257703879 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d80a5970 con 0x7fc9e006e300 2023-11-28T21:35:07.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.136+0000 7fc9e5463640 1 -- 172.21.15.12:0/4257703879 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80a5d00 con 0x7fc9e006e300 2023-11-28T21:35:07.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 -- 172.21.15.12:0/4257703879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 msgr2=0x7fc9e0072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 --2- 172.21.15.12:0/4257703879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 0x7fc9e0072070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009d90 tx=0x7fc9d801c950 comp rx=0 tx=0).stop 2023-11-28T21:35:07.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 -- 172.21.15.12:0/4257703879 shutdown_connections 2023-11-28T21:35:07.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 --2- 172.21.15.12:0/4257703879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 0x7fc9e0072070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 --2- 172.21.15.12:0/4257703879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9e0151ed0 0x7fc9e00c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 -- 172.21.15.12:0/4257703879 >> 172.21.15.12:0/4257703879 conn(0x7fc9e0066360 msgr2=0x7fc9e0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:07.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 -- 172.21.15.12:0/4257703879 shutdown_connections 2023-11-28T21:35:07.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 -- 172.21.15.12:0/4257703879 wait complete. 2023-11-28T21:35:07.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 Processor -- start 2023-11-28T21:35:07.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 -- start start 2023-11-28T21:35:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 0x7fc9e0144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9e0151af0 0x7fc9e0140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e01f3ab0 0x7fc9e0140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00c90c0 con 0x7fc9e006e300 2023-11-28T21:35:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00c9850 con 0x7fc9e01f3ab0 2023-11-28T21:35:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e86c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e0145490 con 0x7fc9e0151af0 2023-11-28T21:35:07.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e6465640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 0x7fc9e0144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e6465640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 0x7fc9e0144f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38008/0 (socket says 172.21.15.12:38008) 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e6465640 1 -- 172.21.15.12:0/2605225399 learned_addr learned my addr 172.21.15.12:0/2605225399 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e6c66640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e01f3ab0 0x7fc9e0140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e5c64640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9e0151af0 0x7fc9e0140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e6465640 1 -- 172.21.15.12:0/2605225399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9e0151af0 msgr2=0x7fc9e0140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e6465640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9e0151af0 0x7fc9e0140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e6465640 1 -- 172.21.15.12:0/2605225399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e01f3ab0 msgr2=0x7fc9e0140d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e6465640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e01f3ab0 0x7fc9e0140d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e6465640 1 -- 172.21.15.12:0/2605225399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d8009a70 con 0x7fc9e006e300 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.140+0000 7fc9e6c66640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e01f3ab0 0x7fc9e0140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9e6465640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 0x7fc9e0144f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0011f0 tx=0x7fc9dc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:07.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9d6ffd640 1 -- 172.21.15.12:0/2605225399 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0074e0 con 0x7fc9e006e300 2023-11-28T21:35:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9d6ffd640 1 -- 172.21.15.12:0/2605225399 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc007640 con 0x7fc9e006e300 2023-11-28T21:35:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9e86c9640 1 -- 172.21.15.12:0/2605225399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e0143970 con 0x7fc9e006e300 2023-11-28T21:35:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9d6ffd640 1 -- 172.21.15.12:0/2605225399 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc004a10 con 0x7fc9e006e300 2023-11-28T21:35:07.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9e86c9640 1 -- 172.21.15.12:0/2605225399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e00c9570 con 0x7fc9e006e300 2023-11-28T21:35:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9e86c9640 1 -- 172.21.15.12:0/2605225399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e0074f60 con 0x7fc9e006e300 2023-11-28T21:35:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9d6ffd640 1 -- 172.21.15.12:0/2605225399 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc9dc004b70 con 0x7fc9e006e300 2023-11-28T21:35:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9d6ffd640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9c40664a0 0x7fc9c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9d6ffd640 1 -- 172.21.15.12:0/2605225399 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc9dc0835b0 con 0x7fc9e006e300 2023-11-28T21:35:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.144+0000 7fc9e5c64640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9c40664a0 0x7fc9c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.148+0000 7fc9e5c64640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9c40664a0 0x7fc9c4068960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00706b0 tx=0x7fc9d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:07.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.148+0000 7fc9d6ffd640 1 -- 172.21.15.12:0/2605225399 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc9dc050770 con 0x7fc9e006e300 2023-11-28T21:35:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.348+0000 7fc9e86c9640 1 -- 172.21.15.12:0/2605225399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc9e0072c10 con 0x7fc9e006e300 2023-11-28T21:35:07.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.348+0000 7fc9d6ffd640 1 -- 172.21.15.12:0/2605225399 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164431 (secure 0 0 0) 0x7fc9dc053d90 con 0x7fc9e006e300 2023-11-28T21:35:07.354 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 -- 172.21.15.12:0/2605225399 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9c40664a0 msgr2=0x7fc9c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9c40664a0 0x7fc9c4068960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00706b0 tx=0x7fc9d0008040 comp rx=0 tx=0).stop 2023-11-28T21:35:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 -- 172.21.15.12:0/2605225399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 msgr2=0x7fc9e0144f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 0x7fc9e0144f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0011f0 tx=0x7fc9dc00c9b0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 -- 172.21.15.12:0/2605225399 shutdown_connections 2023-11-28T21:35:07.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9c40664a0 0x7fc9c4068960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e01f3ab0 0x7fc9e0140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9e0151af0 0x7fc9e0140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 --2- 172.21.15.12:0/2605225399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9e006e300 0x7fc9e0144f50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 -- 172.21.15.12:0/2605225399 >> 172.21.15.12:0/2605225399 conn(0x7fc9e0066360 msgr2=0x7fc9e006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 -- 172.21.15.12:0/2605225399 shutdown_connections 2023-11-28T21:35:07.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.352+0000 7fc9e86c9640 1 -- 172.21.15.12:0/2605225399 wait complete. 2023-11-28T21:35:07.375 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":3598850103},{"type":"v1","addr":"172.21.15.12:6833","nonce":3598850103}]},"active_addr":"172.21.15.12:6833/3598850103","active_change":"2023-11-28T21:34:23.802177+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4117,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3036912756}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":403772775}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3767264280}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":4099257050}]}]} 2023-11-28T21:35:07.381 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T21:35:07.381 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T21:35:07.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:07.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fbb65640 1 Processor -- start 2023-11-28T21:35:07.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fbb65640 1 -- start start 2023-11-28T21:35:07.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fbb65640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3f4154100 0x7fd3f415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fbb65640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd3f40c9fd0 0x7fd3f406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fbb65640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd3f40ca3f0 0x7fd3f406aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fbb65640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f415e270 con 0x7fd3f40ca3f0 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fbb65640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f40c4e60 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fbb65640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f41544e0 con 0x7fd3f4154100 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fa102640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd3f40ca3f0 0x7fd3f406aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fa102640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd3f40ca3f0 0x7fd3f406aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38022/0 (socket says 172.21.15.12:38022) 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fa102640 1 -- 172.21.15.12:0/1149035441 learned_addr learned my addr 172.21.15.12:0/1149035441 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3f9901640 1 --2- 172.21.15.12:0/1149035441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3f4154100 0x7fd3f415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3f9100640 1 --2- 172.21.15.12:0/1149035441 >> v2:172.21.15.144:3300/0 conn(0x7fd3f40c9fd0 0x7fd3f406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fa102640 1 -- 172.21.15.12:0/1149035441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3f4154100 msgr2=0x7fd3f415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fa102640 1 --2- 172.21.15.12:0/1149035441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3f4154100 0x7fd3f415d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fa102640 1 -- 172.21.15.12:0/1149035441 >> v2:172.21.15.144:3300/0 conn(0x7fd3f40c9fd0 msgr2=0x7fd3f406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:07.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fa102640 1 --2- 172.21.15.12:0/1149035441 >> v2:172.21.15.144:3300/0 conn(0x7fd3f40c9fd0 0x7fd3f406a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.524+0000 7fd3fa102640 1 -- 172.21.15.12:0/1149035441 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f415d9d0 con 0x7fd3f40ca3f0 2023-11-28T21:35:07.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3fa102640 1 --2- 172.21.15.12:0/1149035441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40ca3f0 0x7fd3f406aaf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009e50 tx=0x7fd3ec01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc70ce46d267cf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3f88ff640 1 -- 172.21.15.12:0/1149035441 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec0a5820 con 0x7fd3f40ca3f0 2023-11-28T21:35:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3f88ff640 1 -- 172.21.15.12:0/1149035441 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ec0a5980 con 0x7fd3f40ca3f0 2023-11-28T21:35:07.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3f88ff640 1 -- 172.21.15.12:0/1149035441 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec0a5d10 con 0x7fd3f40ca3f0 2023-11-28T21:35:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3fbb65640 1 -- 172.21.15.12:0/1149035441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40ca3f0 msgr2=0x7fd3f406aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3fbb65640 1 --2- 172.21.15.12:0/1149035441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40ca3f0 0x7fd3f406aaf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009e50 tx=0x7fd3ec01c9a0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3fbb65640 1 -- 172.21.15.12:0/1149035441 shutdown_connections 2023-11-28T21:35:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3fbb65640 1 --2- 172.21.15.12:0/1149035441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40ca3f0 0x7fd3f406aaf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3fbb65640 1 --2- 172.21.15.12:0/1149035441 >> v2:172.21.15.144:3300/0 conn(0x7fd3f40c9fd0 0x7fd3f406a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3fbb65640 1 --2- 172.21.15.12:0/1149035441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3f4154100 0x7fd3f415d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3fbb65640 1 -- 172.21.15.12:0/1149035441 >> 172.21.15.12:0/1149035441 conn(0x7fd3f406c690 msgr2=0x7fd3f4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3fbb65640 1 -- 172.21.15.12:0/1149035441 shutdown_connections 2023-11-28T21:35:07.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.528+0000 7fd3fbb65640 1 -- 172.21.15.12:0/1149035441 wait complete. 2023-11-28T21:35:07.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 Processor -- start 2023-11-28T21:35:07.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 -- start start 2023-11-28T21:35:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40c9fd0 0x7fd3f40c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3f40ca3f0 0x7fd3f41546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3f4154100 0x7fd3f415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f41f31c0 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f40c8c10 con 0x7fd3f4154100 2023-11-28T21:35:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f41f2910 con 0x7fd3f40ca3f0 2023-11-28T21:35:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3f9901640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40c9fd0 0x7fd3f40c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3f9901640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40c9fd0 0x7fd3f40c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38036/0 (socket says 172.21.15.12:38036) 2023-11-28T21:35:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3f9901640 1 -- 172.21.15.12:0/2360313130 learned_addr learned my addr 172.21.15.12:0/2360313130 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fa102640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3f4154100 0x7fd3f415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3f9100640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3f40ca3f0 0x7fd3f41546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3f9901640 1 -- 172.21.15.12:0/2360313130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3f40ca3f0 msgr2=0x7fd3f41546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3f9901640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3f40ca3f0 0x7fd3f41546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3f9901640 1 -- 172.21.15.12:0/2360313130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3f4154100 msgr2=0x7fd3f415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3f9901640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3f4154100 0x7fd3f415c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3f9901640 1 -- 172.21.15.12:0/2360313130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec009b30 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3f9901640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40c9fd0 0x7fd3f40c6520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000d5f0 tx=0x7fd3f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:07.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3ea7fc640 1 -- 172.21.15.12:0/2360313130 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f00125f0 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3ea7fc640 1 -- 172.21.15.12:0/2360313130 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f0012750 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3ea7fc640 1 -- 172.21.15.12:0/2360313130 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0012ae0 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 -- 172.21.15.12:0/2360313130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f415a640 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 -- 172.21.15.12:0/2360313130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f40c6bd0 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.532+0000 7fd3fbb65640 1 -- 172.21.15.12:0/2360313130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f41f3430 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.536+0000 7fd3ea7fc640 1 -- 172.21.15.12:0/2360313130 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd3f0012c40 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.536+0000 7fd3ea7fc640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3d8066470 0x7fd3d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.536+0000 7fd3ea7fc640 1 -- 172.21.15.12:0/2360313130 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd3f0084700 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.536+0000 7fd3f9100640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3d8066470 0x7fd3d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.536+0000 7fd3f9100640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3d8066470 0x7fd3d8068930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3f415aec0 tx=0x7fd3e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:07.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.540+0000 7fd3ea7fc640 1 -- 172.21.15.12:0/2360313130 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd3f00517f0 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 -- 172.21.15.12:0/2360313130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3f4067c40 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3ea7fc640 1 -- 172.21.15.12:0/2360313130 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7fd3f0007380 con 0x7fd3f40c9fd0 2023-11-28T21:35:07.702 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:07.703 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 -- 172.21.15.12:0/2360313130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3d8066470 msgr2=0x7fd3d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3d8066470 0x7fd3d8068930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd3f415aec0 tx=0x7fd3e4008040 comp rx=0 tx=0).stop 2023-11-28T21:35:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 -- 172.21.15.12:0/2360313130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40c9fd0 msgr2=0x7fd3f40c6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40c9fd0 0x7fd3f40c6520 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000d5f0 tx=0x7fd3f000dab0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 -- 172.21.15.12:0/2360313130 shutdown_connections 2023-11-28T21:35:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3d8066470 0x7fd3d8068930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3f4154100 0x7fd3f415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3f40ca3f0 0x7fd3f41546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 --2- 172.21.15.12:0/2360313130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3f40c9fd0 0x7fd3f40c6520 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 -- 172.21.15.12:0/2360313130 >> 172.21.15.12:0/2360313130 conn(0x7fd3f406c690 msgr2=0x7fd3f406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:07.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 -- 172.21.15.12:0/2360313130 shutdown_connections 2023-11-28T21:35:07.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.700+0000 7fd3fbb65640 1 -- 172.21.15.12:0/2360313130 wait complete. 2023-11-28T21:35:07.724 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T21:35:07.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:07.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.864+0000 7efdd2980640 1 Processor -- start 2023-11-28T21:35:07.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.864+0000 7efdd2980640 1 -- start start 2023-11-28T21:35:07.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdd2980640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efdcc1540f0 0x7efdcc0c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdd2980640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efdcc15d210 0x7efdcc15d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdd2980640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdcc06d2a0 con 0x7efdcc15b180 2023-11-28T21:35:07.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdd2980640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc06e200 con 0x7efdcc1540f0 2023-11-28T21:35:07.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdd2980640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdcc06d580 con 0x7efdcc15d210 2023-11-28T21:35:07.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdcbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efdcc1540f0 0x7efdcc0c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdcbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efdcc1540f0 0x7efdcc0c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38038/0 (socket says 172.21.15.12:38038) 2023-11-28T21:35:07.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdcbfff640 1 -- 172.21.15.12:0/1014688799 learned_addr learned my addr 172.21.15.12:0/1014688799 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:07.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdd0f1d640 1 --2- 172.21.15.12:0/1014688799 >> v2:172.21.15.144:3300/0 conn(0x7efdcc15d210 0x7efdcc15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdcbfff640 1 -- 172.21.15.12:0/1014688799 >> v2:172.21.15.144:3300/0 conn(0x7efdcc15d210 msgr2=0x7efdcc15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:07.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdcbfff640 1 --2- 172.21.15.12:0/1014688799 >> v2:172.21.15.144:3300/0 conn(0x7efdcc15d210 0x7efdcc15d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdcbfff640 1 -- 172.21.15.12:0/1014688799 >> v1:172.21.15.144:6789/0 conn(0x7efdcc15b180 legacy=0x7efdcc15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.868+0000 7efdcbfff640 1 -- 172.21.15.12:0/1014688799 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc06d860 con 0x7efdcc1540f0 2023-11-28T21:35:07.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdcbfff640 1 --2- 172.21.15.12:0/1014688799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc1540f0 0x7efdcc0c3be0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0c9170 tx=0x7efdbc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba46a34ffe333b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:07.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdcaffd640 1 -- 172.21.15.12:0/1014688799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0ac640 con 0x7efdcc1540f0 2023-11-28T21:35:07.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdcaffd640 1 -- 172.21.15.12:0/1014688799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc0ac7a0 con 0x7efdcc1540f0 2023-11-28T21:35:07.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdcaffd640 1 -- 172.21.15.12:0/1014688799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0acb30 con 0x7efdcc1540f0 2023-11-28T21:35:07.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdd2980640 1 -- 172.21.15.12:0/1014688799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc1540f0 msgr2=0x7efdcc0c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdd2980640 1 --2- 172.21.15.12:0/1014688799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc1540f0 0x7efdcc0c3be0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0c9170 tx=0x7efdbc01c4d0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdd2980640 1 -- 172.21.15.12:0/1014688799 shutdown_connections 2023-11-28T21:35:07.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdd2980640 1 --2- 172.21.15.12:0/1014688799 >> v2:172.21.15.144:3300/0 conn(0x7efdcc15d210 0x7efdcc15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdd2980640 1 --2- 172.21.15.12:0/1014688799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc1540f0 0x7efdcc0c3be0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdd2980640 1 -- 172.21.15.12:0/1014688799 >> 172.21.15.12:0/1014688799 conn(0x7efdcc066360 msgr2=0x7efdcc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:07.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdd2980640 1 -- 172.21.15.12:0/1014688799 shutdown_connections 2023-11-28T21:35:07.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdd2980640 1 -- 172.21.15.12:0/1014688799 wait complete. 2023-11-28T21:35:07.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdd2980640 1 Processor -- start 2023-11-28T21:35:07.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.872+0000 7efdd2980640 1 -- start start 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd2980640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdcc1540f0 0x7efdcc145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd2980640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdcc15b180 0x7efdcc072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd2980640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc15d210 0x7efdcc1f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd2980640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc0caec0 con 0x7efdcc15d210 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd2980640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc146070 con 0x7efdcc1540f0 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd2980640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc0ca340 con 0x7efdcc15b180 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd0f1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc15d210 0x7efdcc1f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd0f1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc15d210 0x7efdcc1f6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38052/0 (socket says 172.21.15.12:38052) 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd0f1d640 1 -- 172.21.15.12:0/205381921 learned_addr learned my addr 172.21.15.12:0/205381921 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdcb7fe640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdcc15b180 0x7efdcc072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdcbfff640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdcc1540f0 0x7efdcc145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd0f1d640 1 -- 172.21.15.12:0/205381921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdcc15b180 msgr2=0x7efdcc072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd0f1d640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdcc15b180 0x7efdcc072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd0f1d640 1 -- 172.21.15.12:0/205381921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdcc1540f0 msgr2=0x7efdcc145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd0f1d640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdcc1540f0 0x7efdcc145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd0f1d640 1 -- 172.21.15.12:0/205381921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc0094d0 con 0x7efdcc15d210 2023-11-28T21:35:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd0f1d640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc15d210 0x7efdcc1f6230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efdc4007e60 tx=0x7efdc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdc8ff9640 1 -- 172.21.15.12:0/205381921 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4014420 con 0x7efdcc15d210 2023-11-28T21:35:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdc8ff9640 1 -- 172.21.15.12:0/205381921 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc40145a0 con 0x7efdcc15d210 2023-11-28T21:35:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdc8ff9640 1 -- 172.21.15.12:0/205381921 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4014950 con 0x7efdcc15d210 2023-11-28T21:35:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd2980640 1 -- 172.21.15.12:0/205381921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdcc144c90 con 0x7efdcc15d210 2023-11-28T21:35:07.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd2980640 1 -- 172.21.15.12:0/205381921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc0ca870 con 0x7efdcc15d210 2023-11-28T21:35:07.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdd2980640 1 -- 172.21.15.12:0/205381921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdcc1f5f50 con 0x7efdcc15d210 2023-11-28T21:35:07.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.876+0000 7efdc8ff9640 1 -- 172.21.15.12:0/205381921 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7efdc4014ab0 con 0x7efdcc15d210 2023-11-28T21:35:07.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.880+0000 7efdc8ff9640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efdb80664a0 0x7efdb8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:07.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.880+0000 7efdc8ff9640 1 -- 172.21.15.12:0/205381921 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efdc4084870 con 0x7efdcc15d210 2023-11-28T21:35:07.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.880+0000 7efdcbfff640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efdb80664a0 0x7efdb8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:07.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.880+0000 7efdcbfff640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efdb80664a0 0x7efdb8068960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0ae000 tx=0x7efdbc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:07.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:07.884+0000 7efdc8ff9640 1 -- 172.21.15.12:0/205381921 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efdcc1f5f50 con 0x7efdcc15d210 2023-11-28T21:35:08.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.040+0000 7efdd2980640 1 -- 172.21.15.12:0/205381921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdcc1f5f50 con 0x7efdcc15d210 2023-11-28T21:35:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdc8ff9640 1 -- 172.21.15.12:0/205381921 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7efdcc1f5f50 con 0x7efdcc15d210 2023-11-28T21:35:08.046 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:08.046 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 -- 172.21.15.12:0/205381921 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efdb80664a0 msgr2=0x7efdb8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efdb80664a0 0x7efdb8068960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0ae000 tx=0x7efdbc008040 comp rx=0 tx=0).stop 2023-11-28T21:35:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 -- 172.21.15.12:0/205381921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc15d210 msgr2=0x7efdcc1f6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc15d210 0x7efdcc1f6230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efdc4007e60 tx=0x7efdc400e4b0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 -- 172.21.15.12:0/205381921 shutdown_connections 2023-11-28T21:35:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efdb80664a0 0x7efdb8068960 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdcc15d210 0x7efdcc1f6230 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efdcc15b180 0x7efdcc072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 --2- 172.21.15.12:0/205381921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efdcc1540f0 0x7efdcc145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 -- 172.21.15.12:0/205381921 >> 172.21.15.12:0/205381921 conn(0x7efdcc066360 msgr2=0x7efdcc14f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 -- 172.21.15.12:0/205381921 shutdown_connections 2023-11-28T21:35:08.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.044+0000 7efdd2980640 1 -- 172.21.15.12:0/205381921 wait complete. 2023-11-28T21:35:08.068 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T21:35:08.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T21:35:08.070 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T21:35:08.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T21:35:08.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T21:35:08.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T21:35:08.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T21:35:08.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T21:35:08.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.328+0000 7f4ad43c4640 1 Processor -- start 2023-11-28T21:35:08.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.328+0000 7f4ad43c4640 1 -- start start 2023-11-28T21:35:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad2160640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f4acc158290 0x7f4acc06ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52630/0 (socket says 172.21.15.12:52630) 2023-11-28T21:35:08.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad43c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 0x7f4acc161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad2160640 1 -- 172.21.15.12:0/588834226 learned_addr learned my addr 172.21.15.12:0/588834226 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad2961640 1 --2- 172.21.15.12:0/588834226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 0x7f4acc161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad43c4640 1 -- 172.21.15.12:0/588834226 --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4acc06a9c0 con 0x7f4acc158670 2023-11-28T21:35:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad43c4640 1 -- 172.21.15.12:0/588834226 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4acc0c5650 con 0x7f4acc158290 2023-11-28T21:35:08.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad43c4640 1 -- 172.21.15.12:0/588834226 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc162300 con 0x7f4acc06b560 2023-11-28T21:35:08.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad115e640 1 -- 172.21.15.12:0/588834226 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841314409 0 0) 0x7f4acc0c5650 con 0x7f4acc158290 2023-11-28T21:35:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad115e640 1 -- 172.21.15.12:0/588834226 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ab8003530 con 0x7f4acc158290 2023-11-28T21:35:08.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.336+0000 7fcd3c83c640 1 Processor -- start 2023-11-28T21:35:08.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad2961640 1 -- 172.21.15.12:0/588834226 >> v1:172.21.15.144:6789/0 conn(0x7f4acc158290 legacy=0x7f4acc06ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad2961640 1 -- 172.21.15.12:0/588834226 >> v1:172.21.15.12:6789/0 conn(0x7f4acc158670 legacy=0x7f4acc1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.332+0000 7f4ad2961640 1 -- 172.21.15.12:0/588834226 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc0c6700 con 0x7f4acc06b560 2023-11-28T21:35:08.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.336+0000 7fcd3c83c640 1 -- start start 2023-11-28T21:35:08.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd3c83c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcd380c6ec0 0x7fcd380c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd377fe640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcd38151d60 0x7fcd380c67b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52642/0 (socket says 172.21.15.12:52642) 2023-11-28T21:35:08.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd36ffd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcd380c6ec0 0x7fcd380c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd36ffd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcd380c6ec0 0x7fcd380c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:39472/0 (socket says 172.21.15.12:39472) 2023-11-28T21:35:08.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4ad2961640 1 --2- 172.21.15.12:0/588834226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 0x7f4acc161dc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0097f0 tx=0x7f4abc01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a84aedf07d6d92d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd37fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcd380c8df0 0x7fcd38162320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36340/0 (socket says 172.21.15.12:36340) 2023-11-28T21:35:08.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4a91d96640 1 Processor -- start 2023-11-28T21:35:08.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fb4056fe640 1 Processor -- start 2023-11-28T21:35:08.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd377fe640 1 -- 172.21.15.12:0/1645606840 learned_addr learned my addr 172.21.15.12:0/1645606840 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd3c83c640 1 -- 172.21.15.12:0/1645606840 --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd380c4750 con 0x7fcd380c8df0 2023-11-28T21:35:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd3c83c640 1 -- 172.21.15.12:0/1645606840 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3815bad0 con 0x7fcd38151d60 2023-11-28T21:35:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd3c83c640 1 -- 172.21.15.12:0/1645606840 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd38152140 con 0x7fcd380c6ec0 2023-11-28T21:35:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd367fc640 1 -- 172.21.15.12:0/1645606840 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155491139 0 0) 0x7fcd3815bad0 con 0x7fcd38151d60 2023-11-28T21:35:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd367fc640 1 -- 172.21.15.12:0/1645606840 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd20003530 con 0x7fcd38151d60 2023-11-28T21:35:08.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4a91d96640 1 -- start start 2023-11-28T21:35:08.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4ad115e640 1 -- 172.21.15.12:0/588834226 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0ac620 con 0x7f4acc06b560 2023-11-28T21:35:08.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4ad115e640 1 -- 172.21.15.12:0/588834226 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc0ac780 con 0x7f4acc06b560 2023-11-28T21:35:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4ad115e640 1 -- 172.21.15.12:0/588834226 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0ac620 con 0x7f4acc06b560 2023-11-28T21:35:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4ad43c4640 1 -- 172.21.15.12:0/588834226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 msgr2=0x7f4acc161dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4ad43c4640 1 --2- 172.21.15.12:0/588834226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 0x7f4acc161dc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0097f0 tx=0x7f4abc01c3e0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4ad43c4640 1 -- 172.21.15.12:0/588834226 shutdown_connections 2023-11-28T21:35:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4ad43c4640 1 --2- 172.21.15.12:0/588834226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 0x7f4acc161dc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4ad43c4640 1 -- 172.21.15.12:0/588834226 >> 172.21.15.12:0/588834226 conn(0x7f4acc06d4c0 msgr2=0x7f4acc06d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4ad43c4640 1 -- 172.21.15.12:0/588834226 shutdown_connections 2023-11-28T21:35:08.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4ad43c4640 1 -- 172.21.15.12:0/588834226 wait complete. 2023-11-28T21:35:08.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fb4056fe640 1 -- start start 2023-11-28T21:35:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb4056fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80dad20 0x7fb3f8006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb4056fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb3f8006580 0x7fb3f8034c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb4056fe640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f8006a10 con 0x7fb3f80d4e70 2023-11-28T21:35:08.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb4056fe640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f8006c80 con 0x7fb3f8006580 2023-11-28T21:35:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb4056fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8006f60 con 0x7fb3f80dad20 2023-11-28T21:35:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3feffd640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb3f80d4e70 0x7fb3f804b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52648/0 (socket says 172.21.15.12:52648) 2023-11-28T21:35:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3feffd640 1 -- 172.21.15.12:0/610192538 learned_addr learned my addr 172.21.15.12:0/610192538 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3fe7fc640 1 --2- 172.21.15.12:0/610192538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80dad20 0x7fb3f8006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3fffff640 1 --2- 172.21.15.12:0/610192538 >> v2:172.21.15.144:3300/0 conn(0x7fb3f8006580 0x7fb3f8034c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3fdffb640 1 -- 172.21.15.12:0/610192538 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147937855 0 0) 0x7fb3f8006a10 con 0x7fb3f80d4e70 2023-11-28T21:35:08.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3fdffb640 1 -- 172.21.15.12:0/610192538 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e8005740 con 0x7fb3f80d4e70 2023-11-28T21:35:08.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd36ffd640 1 -- 172.21.15.12:0/1645606840 >> v1:172.21.15.144:6789/0 conn(0x7fcd38151d60 legacy=0x7fcd380c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd36ffd640 1 -- 172.21.15.12:0/1645606840 >> v1:172.21.15.12:6789/0 conn(0x7fcd380c8df0 legacy=0x7fcd38162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7fcd36ffd640 1 -- 172.21.15.12:0/1645606840 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd380722e0 con 0x7fcd380c6ec0 2023-11-28T21:35:08.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad43c4640 1 Processor -- start 2023-11-28T21:35:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad43c4640 1 -- start start 2023-11-28T21:35:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4a91d96640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a8c06b260 0x7f4a8c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4a91d96640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c15a1c0 con 0x7f4a8c066e80 2023-11-28T21:35:08.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4a91d96640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8c150420 con 0x7f4a8c0c5bb0 2023-11-28T21:35:08.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.340+0000 7f4a91d96640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8c0c6460 con 0x7f4a8c06b260 2023-11-28T21:35:08.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a91595640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f4a8c0c5bb0 0x7f4a8c06a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52644/0 (socket says 172.21.15.12:52644) 2023-11-28T21:35:08.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a91595640 1 -- 172.21.15.12:0/1807837826 learned_addr learned my addr 172.21.15.12:0/1807837826 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012987503 0 0) 0x7f4a8c150420 con 0x7f4a8c0c5bb0 2023-11-28T21:35:08.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a74003530 con 0x7f4a8c0c5bb0 2023-11-28T21:35:08.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 355969436 0 0) 0x7f4a74003530 con 0x7f4a8c0c5bb0 2023-11-28T21:35:08.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 >> v2:172.21.15.12:3300/0 conn(0x7f4a8c06b260 msgr2=0x7f4a8c159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 --2- 172.21.15.12:0/1807837826 >> v2:172.21.15.12:3300/0 conn(0x7f4a8c06b260 0x7f4a8c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 >> v1:172.21.15.12:6789/0 conn(0x7f4a8c066e80 legacy=0x7f4a8c14fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c0c6740 con 0x7f4a8c0c5bb0 2023-11-28T21:35:08.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f4a7c001660 con 0x7f4a8c0c5bb0 2023-11-28T21:35:08.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 >> v1:172.21.15.144:6789/0 conn(0x7f4a8c0c5bb0 legacy=0x7f4a8c06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 --2- 172.21.15.12:0/1807837826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a740039d0 0x7f4a74003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 --2- 172.21.15.12:0/1807837826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a740047d0 0x7f4a74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 --2- 172.21.15.12:0/1807837826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a74005370 0x7f4a7401c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c0c6460 con 0x7f4a740047d0 2023-11-28T21:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7401d2b0 con 0x7f4a74005370 2023-11-28T21:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7401d470 con 0x7f4a740039d0 2023-11-28T21:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a91595640 1 --2- 172.21.15.12:0/1807837826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a74005370 0x7f4a7401c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a91595640 1 -- 172.21.15.12:0/1807837826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a740039d0 msgr2=0x7f4a74003e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T21:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a91595640 1 --2- 172.21.15.12:0/1807837826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a740039d0 0x7f4a74003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a91595640 1 -- 172.21.15.12:0/1807837826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a740047d0 msgr2=0x7f4a74004c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T21:35:08.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a91595640 1 --2- 172.21.15.12:0/1807837826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a740047d0 0x7f4a74004c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7f4a91595640 1 -- 172.21.15.12:0/1807837826 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a74003530 con 0x7f4a74005370 2023-11-28T21:35:08.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3fe7fc640 1 -- 172.21.15.12:0/610192538 >> v2:172.21.15.144:3300/0 conn(0x7fb3f8006580 msgr2=0x7fb3f8034c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3fe7fc640 1 --2- 172.21.15.12:0/610192538 >> v2:172.21.15.144:3300/0 conn(0x7fb3f8006580 0x7fb3f8034c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3fe7fc640 1 -- 172.21.15.12:0/610192538 >> v1:172.21.15.144:6789/0 conn(0x7fb3f80d4e70 legacy=0x7fb3f804b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3fe7fc640 1 -- 172.21.15.12:0/610192538 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f8007240 con 0x7fb3f80dad20 2023-11-28T21:35:08.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.344+0000 7fb3fdffb640 1 -- 172.21.15.12:0/610192538 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 459229634 0 0) 0x7fb3e8005740 con 0x7fb3f80d4e70 2023-11-28T21:35:08.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7fb3fffff640 1 --2- 172.21.15.12:0/610192538 >> v2:172.21.15.144:3300/0 conn(0x7fb3f8006580 0x7fb3f8034c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:08.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7fcd36ffd640 1 --2- 172.21.15.12:0/1645606840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380c6ec0 0x7fcd380c8310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0097c0 tx=0x7fcd2c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e26b572060b3020 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7fcd367fc640 1 -- 172.21.15.12:0/1645606840 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0ab600 con 0x7fcd380c6ec0 2023-11-28T21:35:08.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7fcd367fc640 1 -- 172.21.15.12:0/1645606840 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c0a7050 con 0x7fcd380c6ec0 2023-11-28T21:35:08.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad43c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 0x7f4acc159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7fcd367fc640 1 -- 172.21.15.12:0/1645606840 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0ab9c0 con 0x7fcd380c6ec0 2023-11-28T21:35:08.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad43c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4acc158290 0x7f4acc1472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad2160640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 0x7f4acc159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad2160640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 0x7f4acc159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56374/0 (socket says 172.21.15.12:56374) 2023-11-28T21:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad2160640 1 -- 172.21.15.12:0/2696692095 learned_addr learned my addr 172.21.15.12:0/2696692095 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad43c4640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4acc158670 0x7f4acc147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc147d50 con 0x7f4acc158290 2023-11-28T21:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc154b50 con 0x7f4acc158670 2023-11-28T21:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc154e30 con 0x7f4acc06b560 2023-11-28T21:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad195f640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4acc158290 0x7f4acc1472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7f4ad2961640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4acc158670 0x7f4acc147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fbe8e596640 1 Processor -- start 2023-11-28T21:35:08.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7fcd3c83c640 1 -- 172.21.15.12:0/1645606840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380c6ec0 msgr2=0x7fcd380c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7fcd3c83c640 1 --2- 172.21.15.12:0/1645606840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380c6ec0 0x7fcd380c8310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0097c0 tx=0x7fcd2c01c4a0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7fcd3c83c640 1 -- 172.21.15.12:0/1645606840 shutdown_connections 2023-11-28T21:35:08.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a91595640 1 -- 172.21.15.12:0/1807837826 reap_dead start 2023-11-28T21:35:08.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fbe8e596640 1 -- start start 2023-11-28T21:35:08.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7fcd3c83c640 1 --2- 172.21.15.12:0/1645606840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380c6ec0 0x7fcd380c8310 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.348+0000 7fcd3c83c640 1 -- 172.21.15.12:0/1645606840 >> 172.21.15.12:0/1645606840 conn(0x7fcd3806abe0 msgr2=0x7fcd38066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fcd3c83c640 1 -- 172.21.15.12:0/1645606840 shutdown_connections 2023-11-28T21:35:08.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fcd3c83c640 1 -- 172.21.15.12:0/1645606840 wait complete. 2023-11-28T21:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ad2160640 1 -- 172.21.15.12:0/2696692095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4acc158670 msgr2=0x7f4acc147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ad2160640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4acc158670 0x7f4acc147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ad2160640 1 -- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4acc158290 msgr2=0x7f4acc1472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ad2160640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4acc158290 0x7f4acc1472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ad2160640 1 -- 172.21.15.12:0/2696692095 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc009420 con 0x7f4acc06b560 2023-11-28T21:35:08.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a91595640 1 --2- 172.21.15.12:0/1807837826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a74005370 0x7f4a7401c9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c007f60 tx=0x7f4a7c03eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9cc364f74d5d160 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fb3fe7fc640 1 --2- 172.21.15.12:0/610192538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80dad20 0x7fb3f8006040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400a860 tx=0x7fb3f401ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d6bd935d1541d80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ad2160640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 0x7f4acc159e40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400cbf0 tx=0x7f4ac400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c0c7820 con 0x7f4a74005370 2023-11-28T21:35:08.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a8b7fe640 1 -- 172.21.15.12:0/1807837826 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c0c79a0 con 0x7f4a74005370 2023-11-28T21:35:08.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fb3fdffb640 1 -- 172.21.15.12:0/610192538 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0004ce0 con 0x7fb3f80dad20 2023-11-28T21:35:08.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ac2ffd640 1 -- 172.21.15.12:0/2696692095 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400ad50 con 0x7f4acc06b560 2023-11-28T21:35:08.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fb3fdffb640 1 -- 172.21.15.12:0/610192538 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f40a8980 con 0x7fb3f80dad20 2023-11-28T21:35:08.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fb3fdffb640 1 -- 172.21.15.12:0/610192538 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40a8cf0 con 0x7fb3f80dad20 2023-11-28T21:35:08.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fbe8e596640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbe880c7300 0x7fbe88156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ac2ffd640 1 -- 172.21.15.12:0/2696692095 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac4012400 con 0x7f4acc06b560 2023-11-28T21:35:08.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ac2ffd640 1 -- 172.21.15.12:0/2696692095 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4012790 con 0x7f4acc06b560 2023-11-28T21:35:08.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc159640 con 0x7f4acc06b560 2023-11-28T21:35:08.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc0c25a0 con 0x7f4acc06b560 2023-11-28T21:35:08.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a91d96640 1 -- 172.21.15.12:0/1807837826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a74005370 msgr2=0x7f4a7401c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a91d96640 1 --2- 172.21.15.12:0/1807837826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a74005370 0x7f4a7401c9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c007f60 tx=0x7f4a7c03eaf0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fbe8e596640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8806b7c0 con 0x7fbe880c26b0 2023-11-28T21:35:08.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fbe8e596640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8806a8f0 con 0x7fbe880c6f20 2023-11-28T21:35:08.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fbe8e596640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8806abb0 con 0x7fbe880c7300 2023-11-28T21:35:08.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f02d31e6640 1 Processor -- start 2023-11-28T21:35:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a91d96640 1 -- 172.21.15.12:0/1807837826 shutdown_connections 2023-11-28T21:35:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fb4056fe640 1 -- 172.21.15.12:0/610192538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80dad20 msgr2=0x7fb3f8006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fb4056fe640 1 --2- 172.21.15.12:0/610192538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80dad20 0x7fb3f8006040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400a860 tx=0x7fb3f401ee50 comp rx=0 tx=0).stop 2023-11-28T21:35:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a91d96640 1 --2- 172.21.15.12:0/1807837826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a74005370 0x7f4a7401c9e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a91d96640 1 -- 172.21.15.12:0/1807837826 >> 172.21.15.12:0/1807837826 conn(0x7f4a8c066360 msgr2=0x7f4a8c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f7671534640 1 Processor -- start 2023-11-28T21:35:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fbe8d594640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fbe880c6f20 0x7fbe88158170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52662/0 (socket says 172.21.15.12:52662) 2023-11-28T21:35:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fbe8d594640 1 -- 172.21.15.12:0/1688252758 learned_addr learned my addr 172.21.15.12:0/1688252758 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f02d31e6640 1 -- start start 2023-11-28T21:35:08.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fbe8dd95640 1 --2- 172.21.15.12:0/1688252758 >> v2:172.21.15.12:3300/0 conn(0x7fbe880c7300 0x7fbe88156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a91d96640 1 -- 172.21.15.12:0/1807837826 shutdown_connections 2023-11-28T21:35:08.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4a91d96640 1 -- 172.21.15.12:0/1807837826 wait complete. 2023-11-28T21:35:08.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549517585 0 0) 0x7fbe8806a8f0 con 0x7fbe880c6f20 2023-11-28T21:35:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4acc072f20 con 0x7f4acc06b560 2023-11-28T21:35:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe74003530 con 0x7fbe880c6f20 2023-11-28T21:35:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f46f8a31640 1 Processor -- start 2023-11-28T21:35:08.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006711350 0 0) 0x7fbe74003530 con 0x7fbe880c6f20 2023-11-28T21:35:08.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d31e6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f02cc158290 0x7f02cc06aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d31e6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02cc158670 0x7f02cc06b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d31e6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02cc0c6cc0 con 0x7f02cc0c7520 2023-11-28T21:35:08.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d31e6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02cc0c56a0 con 0x7f02cc158290 2023-11-28T21:35:08.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d31e6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc162580 con 0x7f02cc158670 2023-11-28T21:35:08.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 >> v2:172.21.15.12:3300/0 conn(0x7fbe880c7300 msgr2=0x7fbe88156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/1688252758 >> v2:172.21.15.12:3300/0 conn(0x7fbe880c7300 0x7fbe88156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 >> v1:172.21.15.12:6789/0 conn(0x7fbe880c26b0 legacy=0x7fbe88150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d1783640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f02cc0c7520 0x7f02cc1616b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36368/0 (socket says 172.21.15.12:36368) 2023-11-28T21:35:08.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d1783640 1 -- 172.21.15.12:0/447323066 learned_addr learned my addr 172.21.15.12:0/447323066 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.352+0000 7f7671534640 1 -- start start 2023-11-28T21:35:08.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f7671534640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f766c0c6d50 0x7f766c06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f7671534640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c0c6100 0x7f766c0c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f7671534640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766c0728b0 con 0x7f766c1582c0 2023-11-28T21:35:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f7671534640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766c0c4c50 con 0x7f766c0c6d50 2023-11-28T21:35:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f7671534640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c0c5c70 con 0x7f766c0c6100 2023-11-28T21:35:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d0f82640 1 --2- 172.21.15.12:0/447323066 >> v2:172.21.15.144:3300/0 conn(0x7f02cc158290 0x7f02cc06aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8806ae90 con 0x7fbe880c6f20 2023-11-28T21:35:08.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fbe84001680 con 0x7fbe880c6f20 2023-11-28T21:35:08.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 >> v1:172.21.15.144:6789/0 conn(0x7fbe880c6f20 legacy=0x7fbe88158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe74003980 0x7fbe74003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe740047d0 0x7fbe74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f4a91d96640 1 Processor -- start 2023-11-28T21:35:08.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f7670d33640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c0c6100 0x7f766c0c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74005190 0x7fbe7400fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8806abb0 con 0x7fbe74005190 2023-11-28T21:35:08.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe740103f0 con 0x7fbe740047d0 2023-11-28T21:35:08.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe740105b0 con 0x7fbe74003980 2023-11-28T21:35:08.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe8dd95640 1 --2- 172.21.15.12:0/1688252758 >> v2:172.21.15.12:3300/0 conn(0x7fbe880c7300 0x7fbe88156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:08.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe8d594640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe74003980 0x7fbe74003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02cbfff640 1 --2- 172.21.15.12:0/447323066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02cc158670 0x7f02cc06b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02cb7fe640 1 -- 172.21.15.12:0/447323066 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890536027 0 0) 0x7f02cc0c6cc0 con 0x7f02cc0c7520 2023-11-28T21:35:08.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f766bfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f766c1582c0 0x7f766c06a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36382/0 (socket says 172.21.15.12:36382) 2023-11-28T21:35:08.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f766bfff640 1 -- 172.21.15.12:0/2667788351 learned_addr learned my addr 172.21.15.12:0/2667788351 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f4a91d96640 1 -- start start 2023-11-28T21:35:08.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe8cd93640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe740047d0 0x7fbe74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f46f8a31640 1 -- start start 2023-11-28T21:35:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f46f8a31640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f46f41582c0 0x7f46f406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f46f8a31640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40c6100 0x7f46f40c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f46f8a31640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f40728b0 con 0x7f46f40c6d50 2023-11-28T21:35:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f46f8a31640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f40c4c50 con 0x7f46f41582c0 2023-11-28T21:35:08.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f46f8a31640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40c5c70 con 0x7f46f40c6100 2023-11-28T21:35:08.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02cb7fe640 1 -- 172.21.15.12:0/447323066 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02bc003530 con 0x7f02cc0c7520 2023-11-28T21:35:08.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fcd3c83c640 1 Processor -- start 2023-11-28T21:35:08.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fcd3c83c640 1 -- start start 2023-11-28T21:35:08.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fcd3c83c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380c6ec0 0x7fcd380cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fbe8dd95640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74005190 0x7fbe7400fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f46f259c640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f46f40c6d50 0x7f46f406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52674/0 (socket says 172.21.15.12:52674) 2023-11-28T21:35:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f46f259c640 1 -- 172.21.15.12:0/971762842 learned_addr learned my addr 172.21.15.12:0/971762842 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f46f359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40c6100 0x7f46f40c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fbe8d594640 1 -- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe740047d0 msgr2=0x7fbe74004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fbe8d594640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe740047d0 0x7fbe74004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fbe8d594640 1 -- 172.21.15.12:0/1688252758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74005190 msgr2=0x7fbe7400fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fbe8d594640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74005190 0x7fbe7400fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fbe8d594640 1 -- 172.21.15.12:0/1688252758 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe74003530 con 0x7fbe74003980 2023-11-28T21:35:08.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f46f2d9d640 1 --2- 172.21.15.12:0/971762842 >> v2:172.21.15.12:3300/0 conn(0x7f46f41582c0 0x7f46f406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d0f82640 1 -- 172.21.15.12:0/447323066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02cc158670 msgr2=0x7f02cc06b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d0f82640 1 --2- 172.21.15.12:0/447323066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02cc158670 0x7f02cc06b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d0f82640 1 -- 172.21.15.12:0/447323066 >> v1:172.21.15.12:6789/0 conn(0x7f02cc0c7520 legacy=0x7f02cc1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7f02d0f82640 1 -- 172.21.15.12:0/447323066 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc0c6700 con 0x7f02cc158290 2023-11-28T21:35:08.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.356+0000 7fcd3c83c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380c8df0 0x7fcd380c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd377fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380c6ec0 0x7fcd380cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd377fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380c6ec0 0x7fcd380cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56422/0 (socket says 172.21.15.12:56422) 2023-11-28T21:35:08.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd377fe640 1 -- 172.21.15.12:0/2743914015 learned_addr learned my addr 172.21.15.12:0/2743914015 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f766affd640 1 -- 172.21.15.12:0/2667788351 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125166054 0 0) 0x7f766c0728b0 con 0x7f766c1582c0 2023-11-28T21:35:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f766b7fe640 1 --2- 172.21.15.12:0/2667788351 >> v2:172.21.15.12:3300/0 conn(0x7f766c0c6d50 0x7f766c06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f7670d33640 1 -- 172.21.15.12:0/2667788351 >> v2:172.21.15.12:3300/0 conn(0x7f766c0c6d50 msgr2=0x7f766c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f7670d33640 1 --2- 172.21.15.12:0/2667788351 >> v2:172.21.15.12:3300/0 conn(0x7f766c0c6d50 0x7f766c06b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f7670d33640 1 -- 172.21.15.12:0/2667788351 >> v1:172.21.15.12:6789/0 conn(0x7f766c1582c0 legacy=0x7f766c06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f7670d33640 1 -- 172.21.15.12:0/2667788351 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c0c6a70 con 0x7f766c0c6100 2023-11-28T21:35:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fbe8dd95640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe74005190 0x7fbe7400fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:35:08.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fbe8cd93640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe740047d0 0x7fbe74004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fbe8dd95640 1 -- 172.21.15.12:0/1688252758 reap_dead start 2023-11-28T21:35:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f46f1d9b640 1 -- 172.21.15.12:0/971762842 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261318560 0 0) 0x7f46f40728b0 con 0x7f46f40c6d50 2023-11-28T21:35:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f46f1d9b640 1 -- 172.21.15.12:0/971762842 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46dc003530 con 0x7f46f40c6d50 2023-11-28T21:35:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f46f359e640 1 -- 172.21.15.12:0/971762842 >> v2:172.21.15.12:3300/0 conn(0x7f46f41582c0 msgr2=0x7f46f406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f46f359e640 1 --2- 172.21.15.12:0/971762842 >> v2:172.21.15.12:3300/0 conn(0x7f46f41582c0 0x7f46f406b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f46f359e640 1 -- 172.21.15.12:0/971762842 >> v1:172.21.15.144:6789/0 conn(0x7f46f40c6d50 legacy=0x7f46f406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f46f359e640 1 -- 172.21.15.12:0/971762842 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40c6a70 con 0x7f46f40c6100 2023-11-28T21:35:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd36ffd640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380c8df0 0x7fcd380c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd3c83c640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd38151d60 0x7fcd3806d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd3c83c640 1 -- 172.21.15.12:0/2743914015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380cb330 con 0x7fcd38151d60 2023-11-28T21:35:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd3c83c640 1 -- 172.21.15.12:0/2743914015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380c2740 con 0x7fcd380c8df0 2023-11-28T21:35:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd3c83c640 1 -- 172.21.15.12:0/2743914015 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380cb510 con 0x7fcd380c6ec0 2023-11-28T21:35:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd37fff640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd38151d60 0x7fcd3806d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f46f2d9d640 1 --2- 172.21.15.12:0/971762842 >> v2:172.21.15.12:3300/0 conn(0x7f46f41582c0 0x7f46f406b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:08.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd36ffd640 1 -- 172.21.15.12:0/2743914015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380c6ec0 msgr2=0x7fcd380cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd36ffd640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380c6ec0 0x7fcd380cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd36ffd640 1 -- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd38151d60 msgr2=0x7fcd3806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd36ffd640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd38151d60 0x7fcd3806d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd36ffd640 1 -- 172.21.15.12:0/2743914015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c009420 con 0x7fcd380c8df0 2023-11-28T21:35:08.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd377fe640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380c6ec0 0x7fcd380cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:08.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a91d96640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a8c158990 0x7f4a8c1ec560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a91d96640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a8c158d70 0x7f4a8c150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a91d96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8c142940 0x7f4a8c1ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a91d96640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c143200 con 0x7f4a8c142940 2023-11-28T21:35:08.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a91d96640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1eb0f0 con 0x7f4a8c158d70 2023-11-28T21:35:08.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a91d96640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c1ef110 con 0x7f4a8c158990 2023-11-28T21:35:08.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fb4056fe640 1 -- 172.21.15.12:0/610192538 shutdown_connections 2023-11-28T21:35:08.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fb4056fe640 1 --2- 172.21.15.12:0/610192538 >> v2:172.21.15.144:3300/0 conn(0x7fb3f8006580 0x7fb3f8034c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fb4056fe640 1 --2- 172.21.15.12:0/610192538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80dad20 0x7fb3f8006040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fb4056fe640 1 -- 172.21.15.12:0/610192538 >> 172.21.15.12:0/610192538 conn(0x7fb3f8050330 msgr2=0x7fb3f8050740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4ac2ffd640 1 -- 172.21.15.12:0/2696692095 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4ac4007070 con 0x7f4acc06b560 2023-11-28T21:35:08.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a90d94640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a8c158990 0x7f4a8c1ec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a90d94640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a8c158990 0x7f4a8c1ec560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56430/0 (socket says 172.21.15.12:56430) 2023-11-28T21:35:08.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a90d94640 1 -- 172.21.15.12:0/886103099 learned_addr learned my addr 172.21.15.12:0/886103099 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a8bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a8c158d70 0x7f4a8c150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fb4056fe640 1 -- 172.21.15.12:0/610192538 shutdown_connections 2023-11-28T21:35:08.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fb4056fe640 1 -- 172.21.15.12:0/610192538 wait complete. 2023-11-28T21:35:08.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a91595640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8c142940 0x7f4a8c1ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7fcd36ffd640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380c8df0 0x7fcd380c3200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0ab000 tx=0x7fcd2c019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a90d94640 1 -- 172.21.15.12:0/886103099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a8c158d70 msgr2=0x7f4a8c150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a90d94640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a8c158d70 0x7f4a8c150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a90d94640 1 -- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8c142940 msgr2=0x7f4a8c1ebbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a90d94640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a8c142940 0x7f4a8c1ebbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a90d94640 1 -- 172.21.15.12:0/886103099 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c1ef3f0 con 0x7f4a8c158990 2023-11-28T21:35:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.360+0000 7f4a90d94640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a8c158990 0x7f4a8c1ec560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400b990 tx=0x7f4a8400be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7fcd1ffff640 1 -- 172.21.15.12:0/2743914015 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0b4400 con 0x7fcd380c8df0 2023-11-28T21:35:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7fcd1ffff640 1 -- 172.21.15.12:0/2743914015 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c0b4580 con 0x7fcd380c8df0 2023-11-28T21:35:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7fcd1ffff640 1 -- 172.21.15.12:0/2743914015 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0b4930 con 0x7fcd380c8df0 2023-11-28T21:35:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7fcd3c83c640 1 -- 172.21.15.12:0/2743914015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd380cb7f0 con 0x7fcd380c8df0 2023-11-28T21:35:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7fcd3c83c640 1 -- 172.21.15.12:0/2743914015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd380c2ce0 con 0x7fcd380c8df0 2023-11-28T21:35:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4a897fa640 1 -- 172.21.15.12:0/886103099 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8400cba0 con 0x7f4a8c158990 2023-11-28T21:35:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4a897fa640 1 -- 172.21.15.12:0/886103099 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8400cd00 con 0x7f4a8c158990 2023-11-28T21:35:08.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4a91d96640 1 -- 172.21.15.12:0/886103099 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8c1eb3b0 con 0x7f4a8c158990 2023-11-28T21:35:08.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4a91d96640 1 -- 172.21.15.12:0/886103099 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8c149cf0 con 0x7f4a8c158990 2023-11-28T21:35:08.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4a897fa640 1 -- 172.21.15.12:0/886103099 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84014680 con 0x7f4a8c158990 2023-11-28T21:35:08.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f02d0f82640 1 --2- 172.21.15.12:0/447323066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc158290 0x7f02cc06aee0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f02c00097c0 tx=0x7f02c001c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e916a917f917ea2a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4ac2ffd640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4ab0066470 0x7f4ab0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4ac2ffd640 1 -- 172.21.15.12:0/2696692095 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4ac40841a0 con 0x7f4acc06b560 2023-11-28T21:35:08.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4ac2ffd640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] conn(0x7f4ab006fab0 0x7f4ab0071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4ac2ffd640 1 -- 172.21.15.12:0/2696692095 --> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ab00725c0 con 0x7f4ab006fab0 2023-11-28T21:35:08.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4ac2ffd640 1 -- 172.21.15.12:0/2696692095 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ac40867c0 con 0x7f4acc06b560 2023-11-28T21:35:08.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7fb4056fe640 1 Processor -- start 2023-11-28T21:35:08.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4ad195f640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4ab0066470 0x7f4ab0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f46f359e640 1 --2- 172.21.15.12:0/971762842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40c6100 0x7f46f40c6530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f46e40097c0 tx=0x7f46e401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=452263fe951d7094 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f02cb7fe640 1 -- 172.21.15.12:0/447323066 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c00ab5f0 con 0x7f02cc158290 2023-11-28T21:35:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f02cb7fe640 1 -- 172.21.15.12:0/447323066 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c00ab750 con 0x7f02cc158290 2023-11-28T21:35:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f02cb7fe640 1 -- 172.21.15.12:0/447323066 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c00abae0 con 0x7f02cc158290 2023-11-28T21:35:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4ad2961640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] conn(0x7f4ab006fab0 0x7f4ab0071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4a72ffd640 1 -- 172.21.15.12:0/886103099 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a8c159cf0 con 0x7f4a8c158990 2023-11-28T21:35:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f7670d33640 1 --2- 172.21.15.12:0/2667788351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c0c6100 0x7f766c0c6530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f765c0097c0 tx=0x7f765c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a669dd5046deae4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7fcd1dffb640 1 -- 172.21.15.12:0/2743914015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd10000fd0 con 0x7fcd380c8df0 2023-11-28T21:35:08.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4ad195f640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4ab0066470 0x7f4ab0068930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0cb0f0 tx=0x7f4ac8009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f46f1d9b640 1 -- 172.21.15.12:0/971762842 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40ab600 con 0x7f46f40c6100 2023-11-28T21:35:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f02d31e6640 1 -- 172.21.15.12:0/447323066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc158290 msgr2=0x7f02cc06aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f02d31e6640 1 --2- 172.21.15.12:0/447323066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc158290 0x7f02cc06aee0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f02c00097c0 tx=0x7f02c001c3f0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7fbe8d594640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe74003980 0x7fbe74003d60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400c190 tx=0x7fbe8401f370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8648e84e1ff565d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f46f1d9b640 1 -- 172.21.15.12:0/971762842 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e40a7050 con 0x7f46f40c6100 2023-11-28T21:35:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f46f1d9b640 1 -- 172.21.15.12:0/971762842 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40ab980 con 0x7f46f40c6100 2023-11-28T21:35:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7fb4056fe640 1 -- start start 2023-11-28T21:35:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f02d31e6640 1 -- 172.21.15.12:0/447323066 shutdown_connections 2023-11-28T21:35:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f02d31e6640 1 --2- 172.21.15.12:0/447323066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02cc158670 0x7f02cc06b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f02d31e6640 1 --2- 172.21.15.12:0/447323066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc158290 0x7f02cc06aee0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f02d31e6640 1 -- 172.21.15.12:0/447323066 >> 172.21.15.12:0/447323066 conn(0x7f02cc06d4c0 msgr2=0x7f02cc06d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f4ad2961640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] conn(0x7f4ab006fab0 0x7f4ab0071ef0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.364+0000 7f766affd640 1 -- 172.21.15.12:0/2667788351 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c0ab600 con 0x7f766c0c6100 2023-11-28T21:35:08.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84005ae0 con 0x7fbe74003980 2023-11-28T21:35:08.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe840ab070 con 0x7fbe74003980 2023-11-28T21:35:08.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fbe7ffff640 1 -- 172.21.15.12:0/1688252758 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84007070 con 0x7fbe74003980 2023-11-28T21:35:08.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f766affd640 1 -- 172.21.15.12:0/2667788351 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c0a7050 con 0x7f766c0c6100 2023-11-28T21:35:08.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f766affd640 1 -- 172.21.15.12:0/2667788351 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c0ab980 con 0x7f766c0c6100 2023-11-28T21:35:08.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f02d31e6640 1 -- 172.21.15.12:0/447323066 shutdown_connections 2023-11-28T21:35:08.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f02d31e6640 1 -- 172.21.15.12:0/447323066 wait complete. 2023-11-28T21:35:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f46f8a31640 1 -- 172.21.15.12:0/971762842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40c6100 msgr2=0x7f46f40c6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f46f8a31640 1 --2- 172.21.15.12:0/971762842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40c6100 0x7f46f40c6530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f46e40097c0 tx=0x7f46e401c3f0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb4056fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3f8006580 0x7fb3f80ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb4056fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80d4e70 0x7fb3f80cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb4056fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f80dad20 0x7fb3f80cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb4056fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8033460 con 0x7fb3f80dad20 2023-11-28T21:35:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb4056fe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f80cf600 con 0x7fb3f8006580 2023-11-28T21:35:08.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb4056fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f80cf2d0 con 0x7fb3f80d4e70 2023-11-28T21:35:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f46f8a31640 1 -- 172.21.15.12:0/971762842 shutdown_connections 2023-11-28T21:35:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3fffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f80dad20 0x7fb3f80cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f46f8a31640 1 --2- 172.21.15.12:0/971762842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f40c6100 0x7f46f40c6530 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f46f8a31640 1 --2- 172.21.15.12:0/971762842 >> v2:172.21.15.12:3300/0 conn(0x7f46f41582c0 0x7f46f406b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fbe8e596640 1 -- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe74003980 msgr2=0x7fbe74003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fbe8e596640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe74003980 0x7fbe74003d60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400c190 tx=0x7fbe8401f370 comp rx=0 tx=0).stop 2023-11-28T21:35:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f46f8a31640 1 -- 172.21.15.12:0/971762842 >> 172.21.15.12:0/971762842 conn(0x7f46f4066360 msgr2=0x7f46f4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fcd1ffff640 1 -- 172.21.15.12:0/2743914015 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fcd2c0b2040 con 0x7fcd380c8df0 2023-11-28T21:35:08.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f46f8a31640 1 -- 172.21.15.12:0/971762842 shutdown_connections 2023-11-28T21:35:08.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f4ac2ffd640 1 -- 172.21.15.12:0/2696692095 <== osd.0 v2:172.21.15.12:6816/3923148634 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f4ab00725c0 con 0x7f4ab006fab0 2023-11-28T21:35:08.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f46f8a31640 1 -- 172.21.15.12:0/971762842 wait complete. 2023-11-28T21:35:08.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f4a897fa640 1 -- 172.21.15.12:0/886103099 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4a8401d3b0 con 0x7f4a8c158990 2023-11-28T21:35:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3fe7fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80d4e70 0x7fb3f80cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3fffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f80dad20 0x7fb3f80cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38124/0 (socket says 172.21.15.12:38124) 2023-11-28T21:35:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3fffff640 1 -- 172.21.15.12:0/1070452844 learned_addr learned my addr 172.21.15.12:0/1070452844 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3feffd640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3f8006580 0x7fb3f80ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fcd1ffff640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcd240664a0 0x7fcd24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f4a897fa640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4a78066460 0x7f4a78068920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f4a897fa640 1 -- 172.21.15.12:0/886103099 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4a840846d0 con 0x7f4a8c158990 2023-11-28T21:35:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f4a897fa640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] conn(0x7f4a7806f8b0 0x7f4a78071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f4a897fa640 1 -- 172.21.15.12:0/886103099 --> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a780723c0 con 0x7f4a7806f8b0 2023-11-28T21:35:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f4a897fa640 1 -- 172.21.15.12:0/886103099 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a84084b20 con 0x7f4a8c158990 2023-11-28T21:35:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3fe7fc640 1 -- 172.21.15.12:0/1070452844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3f8006580 msgr2=0x7fb3f80ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3fe7fc640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3f8006580 0x7fb3f80ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3fe7fc640 1 -- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f80dad20 msgr2=0x7fb3f80cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3fe7fc640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f80dad20 0x7fb3f80cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3fe7fc640 1 -- 172.21.15.12:0/1070452844 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f400a4d0 con 0x7fb3f80d4e70 2023-11-28T21:35:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fb3fffff640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3f80dad20 0x7fb3f80cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f7671534640 1 -- 172.21.15.12:0/2667788351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c0c6100 msgr2=0x7f766c0c6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f7671534640 1 --2- 172.21.15.12:0/2667788351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c0c6100 0x7f766c0c6530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f765c0097c0 tx=0x7f765c01c410 comp rx=0 tx=0).stop 2023-11-28T21:35:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fbe8e596640 1 -- 172.21.15.12:0/1688252758 shutdown_connections 2023-11-28T21:35:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fbe8e596640 1 --2- 172.21.15.12:0/1688252758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe74003980 0x7fbe74003d60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fbe8e596640 1 -- 172.21.15.12:0/1688252758 >> 172.21.15.12:0/1688252758 conn(0x7fbe88066360 msgr2=0x7fbe88066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fbe8e596640 1 -- 172.21.15.12:0/1688252758 shutdown_connections 2023-11-28T21:35:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fcd377fe640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcd240664a0 0x7fcd24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f4a8bfff640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4a78066460 0x7f4a78068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7f4a91595640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] conn(0x7f4a7806f8b0 0x7f4a78071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fcd1ffff640 1 -- 172.21.15.12:0/2743914015 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcd2c0ef3a0 con 0x7fcd380c8df0 2023-11-28T21:35:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fcd1ffff640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] conn(0x7fcd240706b0 0x7fcd24072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.368+0000 7fbe8e596640 1 -- 172.21.15.12:0/1688252758 wait complete. 2023-11-28T21:35:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f8a31640 1 Processor -- start 2023-11-28T21:35:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d31e6640 1 Processor -- start 2023-11-28T21:35:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f4a91595640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] conn(0x7f4a7806f8b0 0x7f4a78071cf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d31e6640 1 -- start start 2023-11-28T21:35:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fb3fe7fc640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80d4e70 0x7fb3f80cbcb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80070a0 tx=0x7fb3f401bbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f7671534640 1 -- 172.21.15.12:0/2667788351 shutdown_connections 2023-11-28T21:35:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f7671534640 1 --2- 172.21.15.12:0/2667788351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c0c6100 0x7f766c0c6530 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f7671534640 1 --2- 172.21.15.12:0/2667788351 >> v2:172.21.15.12:3300/0 conn(0x7f766c0c6d50 0x7f766c06b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f7671534640 1 -- 172.21.15.12:0/2667788351 >> 172.21.15.12:0/2667788351 conn(0x7f766c066360 msgr2=0x7f766c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fcd37fff640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] conn(0x7fcd240706b0 0x7fcd24072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fcd377fe640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcd240664a0 0x7fcd24068960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd30002240 tx=0x7fcd30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f8a31640 1 -- start start 2023-11-28T21:35:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f8a31640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f40c6100 0x7f46f4161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f8a31640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f40c6d50 0x7f46f4162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f8a31640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f41582c0 0x7f46f414aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f8a31640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f415fd30 con 0x7f46f40c6100 2023-11-28T21:35:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f8a31640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4160830 con 0x7f46f40c6d50 2023-11-28T21:35:08.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fcd1ffff640 1 -- 172.21.15.12:0/2743914015 --> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd24073220 con 0x7fcd240706b0 2023-11-28T21:35:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f4a8bfff640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4a78066460 0x7f4a78068920 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c153480 tx=0x7f4a800090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f8a31640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f415f2e0 con 0x7f46f41582c0 2023-11-28T21:35:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fcd1ffff640 1 -- 172.21.15.12:0/2743914015 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd2c0eea70 con 0x7fcd380c8df0 2023-11-28T21:35:08.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f40c6100 0x7f46f4161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f4a897fa640 1 -- 172.21.15.12:0/886103099 <== osd.2 v2:172.21.15.12:6800/2573246535 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f4a780723c0 con 0x7f4a7806f8b0 2023-11-28T21:35:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f40c6100 0x7f46f4161c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38138/0 (socket says 172.21.15.12:38138) 2023-11-28T21:35:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f40c6d50 0x7f46f4162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f2d9d640 1 -- 172.21.15.12:0/1702824053 learned_addr learned my addr 172.21.15.12:0/1702824053 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f359e640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f41582c0 0x7f46f414aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d31e6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02cc0c7520 0x7f02cc14b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d31e6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02cc158290 0x7f02cc1482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d31e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc158670 0x7f02cc148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d31e6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc15d0e0 con 0x7f02cc0c7520 2023-11-28T21:35:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d31e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc151e90 con 0x7f02cc158670 2023-11-28T21:35:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d31e6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc15d3a0 con 0x7f02cc158290 2023-11-28T21:35:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d0f82640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02cc0c7520 0x7f02cc14b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d0f82640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02cc0c7520 0x7f02cc14b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38146/0 (socket says 172.21.15.12:38146) 2023-11-28T21:35:08.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d0f82640 1 -- 172.21.15.12:0/1405731619 learned_addr learned my addr 172.21.15.12:0/1405731619 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fbe8e596640 1 Processor -- start 2023-11-28T21:35:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f2d9d640 1 -- 172.21.15.12:0/1702824053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f41582c0 msgr2=0x7f46f414aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f2d9d640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f41582c0 0x7f46f414aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f2d9d640 1 -- 172.21.15.12:0/1702824053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f40c6d50 msgr2=0x7f46f4162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f2d9d640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f40c6d50 0x7f46f4162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f46f2d9d640 1 -- 172.21.15.12:0/1702824053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4009420 con 0x7f46f40c6100 2023-11-28T21:35:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1070452844 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40ba950 con 0x7fb3f80d4e70 2023-11-28T21:35:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1070452844 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f40baab0 con 0x7fb3f80d4e70 2023-11-28T21:35:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fcd37fff640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] conn(0x7fcd240706b0 0x7fcd24072b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1070452844 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40bae40 con 0x7fb3f80d4e70 2023-11-28T21:35:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fb4056fe640 1 -- 172.21.15.12:0/1070452844 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f80def30 con 0x7fb3f80d4e70 2023-11-28T21:35:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fb4056fe640 1 -- 172.21.15.12:0/1070452844 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f80d59b0 con 0x7fb3f80d4e70 2023-11-28T21:35:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f7671534640 1 -- 172.21.15.12:0/2667788351 shutdown_connections 2023-11-28T21:35:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f7671534640 1 -- 172.21.15.12:0/2667788351 wait complete. 2023-11-28T21:35:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7f02d1783640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc158670 0x7f02cc148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fcd1ffff640 1 -- 172.21.15.12:0/2743914015 <== osd.1 v2:172.21.15.12:6802/1716423303 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fcd24073220 con 0x7fcd240706b0 2023-11-28T21:35:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.372+0000 7fbe8e596640 1 -- start start 2023-11-28T21:35:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02d1783640 1 -- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02cc158290 msgr2=0x7f02cc1482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02d1783640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02cc158290 0x7f02cc1482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02d1783640 1 -- 172.21.15.12:0/1405731619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02cc0c7520 msgr2=0x7f02cc14b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02d1783640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02cc0c7520 0x7f02cc14b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02d1783640 1 -- 172.21.15.12:0/1405731619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0009420 con 0x7f02cc158670 2023-11-28T21:35:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f46f2d9d640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f40c6100 0x7f46f4161c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f46ec007f20 tx=0x7f46ec00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02d1783640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc158670 0x7f02cc148820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f02c400ca50 tx=0x7f02c400cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f46db7fe640 1 -- 172.21.15.12:0/1702824053 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec00dee0 con 0x7f46f40c6100 2023-11-28T21:35:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f46db7fe640 1 -- 172.21.15.12:0/1702824053 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec00f040 con 0x7f46f40c6100 2023-11-28T21:35:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f46db7fe640 1 -- 172.21.15.12:0/1702824053 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec003dc0 con 0x7f46f40c6100 2023-11-28T21:35:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fbe8e596640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe74003980 0x7fbe88146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02c97fa640 1 -- 172.21.15.12:0/1405731619 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c400abb0 con 0x7f02cc158670 2023-11-28T21:35:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02c97fa640 1 -- 172.21.15.12:0/1405731619 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c400ad10 con 0x7f02cc158670 2023-11-28T21:35:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fb3e57fa640 1 -- 172.21.15.12:0/1070452844 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3d8000fd0 con 0x7fb3f80d4e70 2023-11-28T21:35:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02c97fa640 1 -- 172.21.15.12:0/1405731619 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4012640 con 0x7f02cc158670 2023-11-28T21:35:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02d31e6640 1 -- 172.21.15.12:0/1405731619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc162350 con 0x7f02cc158670 2023-11-28T21:35:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fbe8e596640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe8814e1d0 0x7fbe88154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fbe8e596640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe88154f70 0x7fbe881edf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fbe8e596640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88152b40 con 0x7fbe88154f70 2023-11-28T21:35:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fbe8e596640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88148af0 con 0x7fbe74003980 2023-11-28T21:35:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fbe8e596640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88152230 con 0x7fbe8814e1d0 2023-11-28T21:35:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02d31e6640 1 -- 172.21.15.12:0/1405731619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc15d960 con 0x7f02cc158670 2023-11-28T21:35:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fbe8d594640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe74003980 0x7fbe88146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f7671534640 1 Processor -- start 2023-11-28T21:35:08.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f7671534640 1 -- start start 2023-11-28T21:35:08.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fbe8d594640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe74003980 0x7fbe88146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:39586/0 (socket says 172.21.15.12:39586) 2023-11-28T21:35:08.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fbe8d594640 1 -- 172.21.15.12:0/3910919114 learned_addr learned my addr 172.21.15.12:0/3910919114 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7fbe8dd95640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe88154f70 0x7fbe881edf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7fbe8dd95640 1 -- 172.21.15.12:0/3910919114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe8814e1d0 msgr2=0x7fbe88154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7fbe8dd95640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe8814e1d0 0x7fbe88154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7fbe8dd95640 1 -- 172.21.15.12:0/3910919114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe74003980 msgr2=0x7fbe88146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7fbe8dd95640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe74003980 0x7fbe88146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7fbe8dd95640 1 -- 172.21.15.12:0/3910919114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe88155420 con 0x7fbe88154f70 2023-11-28T21:35:08.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7fbe8dd95640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe88154f70 0x7fbe881edf00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe88073170 tx=0x7fbe78010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f415b900 con 0x7f46f40c6100 2023-11-28T21:35:08.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f415b340 con 0x7f46f40c6100 2023-11-28T21:35:08.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f46f40c7970 con 0x7f46f40c6100 2023-11-28T21:35:08.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.376+0000 7f02b6ffd640 1 -- 172.21.15.12:0/1405731619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02ac000fd0 con 0x7f02cc158670 2023-11-28T21:35:08.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f7671534640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f766c0c6100 0x7f766c0c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f7671534640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f766c0c6d50 0x7f766c148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f7671534640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c1582c0 0x7f766c15cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f7671534640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c158e00 con 0x7f766c0c6100 2023-11-28T21:35:08.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f7671534640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766c14a710 con 0x7f766c0c6d50 2023-11-28T21:35:08.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f7671534640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c149e70 con 0x7f766c1582c0 2023-11-28T21:35:08.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f7670d33640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c1582c0 0x7f766c15cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f766b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f766c0c6d50 0x7f766c148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f7670d33640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c1582c0 0x7f766c15cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56474/0 (socket says 172.21.15.12:56474) 2023-11-28T21:35:08.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f7670d33640 1 -- 172.21.15.12:0/512594531 learned_addr learned my addr 172.21.15.12:0/512594531 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1070452844 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb3f40b1550 con 0x7fb3f80d4e70 2023-11-28T21:35:08.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f02c97fa640 1 -- 172.21.15.12:0/1405731619 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f02c4007070 con 0x7f02cc158670 2023-11-28T21:35:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.380+0000 7f766bfff640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f766c0c6100 0x7f766c0c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7670d33640 1 -- 172.21.15.12:0/512594531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f766c0c6d50 msgr2=0x7f766c148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7670d33640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f766c0c6d50 0x7f766c148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7670d33640 1 -- 172.21.15.12:0/512594531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f766c0c6100 msgr2=0x7f766c0c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7670d33640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f766c0c6100 0x7f766c0c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7670d33640 1 -- 172.21.15.12:0/512594531 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c009420 con 0x7f766c1582c0 2023-11-28T21:35:08.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f02c97fa640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02b0066540 0x7f02b0068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fb3e77fe640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb3ec0664c0 0x7fb3ec068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1070452844 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb3f40c0030 con 0x7fb3f80d4e70 2023-11-28T21:35:08.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fb3e77fe640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] conn(0x7fb3ec06fb00 0x7fb3ec071f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1070452844 --> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3ec072610 con 0x7fb3ec06fb00 2023-11-28T21:35:08.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1070452844 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3f4126060 con 0x7fb3f80d4e70 2023-11-28T21:35:08.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7670d33640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c1582c0 0x7f766c15cfe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f765c01a040 tx=0x7f765c0191b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3910919114 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78004230 con 0x7fbe88154f70 2023-11-28T21:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fb3fffff640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] conn(0x7fb3ec06fb00 0x7fb3ec071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f02d0f82640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02b0066540 0x7f02b0068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f02c97fa640 1 -- 172.21.15.12:0/1405731619 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f02c4084050 con 0x7f02cc158670 2023-11-28T21:35:08.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3910919114 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe78004390 con 0x7fbe88154f70 2023-11-28T21:35:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3910919114 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800ea70 con 0x7fbe88154f70 2023-11-28T21:35:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fb3feffd640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb3ec0664c0 0x7fb3ec068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7668ff9640 1 -- 172.21.15.12:0/512594531 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c0b4400 con 0x7f766c1582c0 2023-11-28T21:35:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7668ff9640 1 -- 172.21.15.12:0/512594531 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c0b4560 con 0x7f766c1582c0 2023-11-28T21:35:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7668ff9640 1 -- 172.21.15.12:0/512594531 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c0b48f0 con 0x7f766c1582c0 2023-11-28T21:35:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766c14a150 con 0x7f766c1582c0 2023-11-28T21:35:08.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766c148930 con 0x7f766c1582c0 2023-11-28T21:35:08.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe88150cb0 con 0x7fbe88154f70 2023-11-28T21:35:08.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f02c97fa640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] conn(0x7f02b006fb80 0x7f02b0071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f02c97fa640 1 -- 172.21.15.12:0/1405731619 --> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02b0072690 con 0x7f02b006fb80 2023-11-28T21:35:08.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f02c97fa640 1 -- 172.21.15.12:0/1405731619 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02c4086670 con 0x7f02cc158670 2023-11-28T21:35:08.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7f46db7fe640 1 -- 172.21.15.12:0/1702824053 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f46ec004030 con 0x7f46f40c6100 2023-11-28T21:35:08.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7fb3feffd640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb3ec0664c0 0x7fb3ec068980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0004850 tx=0x7fb3f000b220 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f766c0c7970 con 0x7f766c1582c0 2023-11-28T21:35:08.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.384+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe88152480 con 0x7fbe88154f70 2023-11-28T21:35:08.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f46db7fe640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46e00664a0 0x7f46e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f02cbfff640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] conn(0x7f02b006fb80 0x7f02b0071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f02d0f82640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02b0066540 0x7f02b0068a00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f02c00097c0 tx=0x7f02c00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7fb3fffff640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] conn(0x7fb3ec06fb00 0x7fb3ec071f40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f46db7fe640 1 -- 172.21.15.12:0/1702824053 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f46ec084ee0 con 0x7f46f40c6100 2023-11-28T21:35:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f46f259c640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46e00664a0 0x7f46e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f46db7fe640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] conn(0x7f46e006fa60 0x7f46e0071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f46db7fe640 1 -- 172.21.15.12:0/1702824053 --> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46e0072570 con 0x7f46e006fa60 2023-11-28T21:35:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f46db7fe640 1 -- 172.21.15.12:0/1702824053 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ec085330 con 0x7f46f40c6100 2023-11-28T21:35:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f46f359e640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] conn(0x7f46e006fa60 0x7f46e0071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe880c7950 con 0x7fbe88154f70 2023-11-28T21:35:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f02cbfff640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] conn(0x7f02b006fb80 0x7f02b0071fc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f46f359e640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] conn(0x7f46e006fa60 0x7f46e0071ea0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1070452844 <== osd.3 v2:172.21.15.12:6824/1952033258 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb3ec072610 con 0x7fb3ec06fb00 2023-11-28T21:35:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f46f259c640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46e00664a0 0x7f46e0068960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f46f4067530 tx=0x7f46e8007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.388+0000 7f02c97fa640 1 -- 172.21.15.12:0/1405731619 <== osd.7 v2:172.21.15.144:6808/3855981118 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f02b0072690 con 0x7f02b006fb80 2023-11-28T21:35:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7f7668ff9640 1 -- 172.21.15.12:0/512594531 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f765c0b2040 con 0x7f766c1582c0 2023-11-28T21:35:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7f46db7fe640 1 -- 172.21.15.12:0/1702824053 <== osd.6 v2:172.21.15.144:6824/2995192634 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f46e0072570 con 0x7f46e006fa60 2023-11-28T21:35:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3910919114 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fbe7800ebd0 con 0x7fbe88154f70 2023-11-28T21:35:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7f7668ff9640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76580664f0 0x7f76580689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7f766bfff640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76580664f0 0x7f76580689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7f7668ff9640 1 -- 172.21.15.12:0/512594531 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f765c0ef450 con 0x7f766c1582c0 2023-11-28T21:35:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7f7668ff9640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] conn(0x7f7658070850 0x7f7658072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7fbe7dffb640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbe6c066570 0x7fbe6c068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3910919114 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbe78084070 con 0x7fbe88154f70 2023-11-28T21:35:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7fbe7dffb640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] conn(0x7fbe6c06fbb0 0x7fbe6c071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.392+0000 7f7668ff9640 1 -- 172.21.15.12:0/512594531 --> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76580733c0 con 0x7f7658070850 2023-11-28T21:35:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f7668ff9640 1 -- 172.21.15.12:0/512594531 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f765c0ef220 con 0x7f766c1582c0 2023-11-28T21:35:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f766bfff640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76580664f0 0x7f76580689b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f76640012f0 tx=0x7f7664001690 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f766b7fe640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] conn(0x7f7658070850 0x7f7658072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7fbe8d594640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbe6c066570 0x7fbe6c068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3910919114 --> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe6c0726c0 con 0x7fbe6c06fbb0 2023-11-28T21:35:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3910919114 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe78084420 con 0x7fbe88154f70 2023-11-28T21:35:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7fbe8cd93640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] conn(0x7fbe6c06fbb0 0x7fbe6c071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 --> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4acc0c36a0 con 0x7f4ab006fab0 2023-11-28T21:35:08.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7fbe8cd93640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] conn(0x7fbe6c06fbb0 0x7fbe6c071ff0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ac2ffd640 1 -- 172.21.15.12:0/2696692095 <== osd.0 v2:172.21.15.12:6816/3923148634 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4acc0c36a0 con 0x7f4ab006fab0 2023-11-28T21:35:08.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f766b7fe640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] conn(0x7f7658070850 0x7f7658072cf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] conn(0x7f4ab006fab0 msgr2=0x7f4ab0071ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad43c4640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] conn(0x7f4ab006fab0 0x7f4ab0071ef0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7fbe8d594640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbe6c066570 0x7fbe6c068a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe88073210 tx=0x7fbe840aa000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4ab0066470 msgr2=0x7f4ab0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad43c4640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4ab0066470 0x7f4ab0068930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0cb0f0 tx=0x7f4ac8009060 comp rx=0 tx=0).stop 2023-11-28T21:35:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 msgr2=0x7f4acc159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad43c4640 1 --2- 172.21.15.12:0/2696692095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4acc06b560 0x7f4acc159e40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400cbf0 tx=0x7f4ac400a430 comp rx=0 tx=0).stop 2023-11-28T21:35:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad2961640 1 -- 172.21.15.12:0/2696692095 reap_dead start 2023-11-28T21:35:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 shutdown_connections 2023-11-28T21:35:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 >> 172.21.15.12:0/2696692095 conn(0x7f4acc06d4c0 msgr2=0x7f4acc161800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.396+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 shutdown_connections 2023-11-28T21:35:08.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.400+0000 7f4ad43c4640 1 -- 172.21.15.12:0/2696692095 wait complete. 2023-11-28T21:35:08.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.400+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3910919114 <== osd.5 v2:172.21.15.144:6816/2546756774 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fbe6c0726c0 con 0x7fbe6c06fbb0 2023-11-28T21:35:08.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.400+0000 7f7668ff9640 1 -- 172.21.15.12:0/512594531 <== osd.4 v2:172.21.15.144:6800/3927315436 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f76580733c0 con 0x7f7658070850 2023-11-28T21:35:08.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a72ffd640 1 -- 172.21.15.12:0/886103099 --> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4a8c0c6d00 con 0x7f4a7806f8b0 2023-11-28T21:35:08.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a897fa640 1 -- 172.21.15.12:0/886103099 <== osd.2 v2:172.21.15.12:6800/2573246535 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4a780723c0 con 0x7f4a7806f8b0 2023-11-28T21:35:08.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7fcd1dffb640 1 -- 172.21.15.12:0/2743914015 --> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcd10002c90 con 0x7fcd240706b0 2023-11-28T21:35:08.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a72ffd640 1 -- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] conn(0x7f4a7806f8b0 msgr2=0x7f4a78071cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a72ffd640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] conn(0x7f4a7806f8b0 0x7f4a78071cf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a72ffd640 1 -- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4a78066460 msgr2=0x7f4a78068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a72ffd640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4a78066460 0x7f4a78068920 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c153480 tx=0x7f4a800090e0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a72ffd640 1 -- 172.21.15.12:0/886103099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a8c158990 msgr2=0x7f4a8c1ec560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a72ffd640 1 --2- 172.21.15.12:0/886103099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a8c158990 0x7f4a8c1ec560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400b990 tx=0x7f4a8400be50 comp rx=0 tx=0).stop 2023-11-28T21:35:08.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a91595640 1 -- 172.21.15.12:0/886103099 reap_dead start 2023-11-28T21:35:08.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a72ffd640 1 -- 172.21.15.12:0/886103099 shutdown_connections 2023-11-28T21:35:08.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.404+0000 7f4a72ffd640 1 -- 172.21.15.12:0/886103099 >> 172.21.15.12:0/886103099 conn(0x7f4a8c066360 msgr2=0x7f4a8c066740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.408+0000 7f4a72ffd640 1 -- 172.21.15.12:0/886103099 shutdown_connections 2023-11-28T21:35:08.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.408+0000 7f4a72ffd640 1 -- 172.21.15.12:0/886103099 wait complete. 2023-11-28T21:35:08.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 -- 172.21.15.12:0/1405731619 --> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f02ac002cc0 con 0x7f02b006fb80 2023-11-28T21:35:08.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02c97fa640 1 -- 172.21.15.12:0/1405731619 <== osd.7 v2:172.21.15.144:6808/3855981118 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f02ac002cc0 con 0x7f02b006fb80 2023-11-28T21:35:08.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 -- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] conn(0x7f02b006fb80 msgr2=0x7f02b0071fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] conn(0x7f02b006fb80 0x7f02b0071fc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 -- 172.21.15.12:0/1405731619 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02b0066540 msgr2=0x7f02b0068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02b0066540 0x7f02b0068a00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f02c00097c0 tx=0x7f02c00a7000 comp rx=0 tx=0).stop 2023-11-28T21:35:08.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 -- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc158670 msgr2=0x7f02cc148820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 --2- 172.21.15.12:0/1405731619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02cc158670 0x7f02cc148820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f02c400ca50 tx=0x7f02c400cf10 comp rx=0 tx=0).stop 2023-11-28T21:35:08.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02d1783640 1 -- 172.21.15.12:0/1405731619 reap_dead start 2023-11-28T21:35:08.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 -- 172.21.15.12:0/1405731619 shutdown_connections 2023-11-28T21:35:08.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 -- 172.21.15.12:0/1405731619 >> 172.21.15.12:0/1405731619 conn(0x7f02cc06d4c0 msgr2=0x7f02cc156160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 -- 172.21.15.12:0/1405731619 shutdown_connections 2023-11-28T21:35:08.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.416+0000 7f02b6ffd640 1 -- 172.21.15.12:0/1405731619 wait complete. 2023-11-28T21:35:08.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.420+0000 7fb3e57fa640 1 -- 172.21.15.12:0/1070452844 --> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb3d8002cc0 con 0x7fb3ec06fb00 2023-11-28T21:35:08.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.420+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 --> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f46f406b200 con 0x7f46e006fa60 2023-11-28T21:35:08.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.420+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1070452844 <== osd.3 v2:172.21.15.12:6824/1952033258 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb3d8002cc0 con 0x7fb3ec06fb00 2023-11-28T21:35:08.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.420+0000 7f46db7fe640 1 -- 172.21.15.12:0/1702824053 <== osd.6 v2:172.21.15.144:6824/2995192634 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f46e40ab520 con 0x7f46e006fa60 2023-11-28T21:35:08.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.420+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 >> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] conn(0x7f46e006fa60 msgr2=0x7f46e0071ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.420+0000 7f46f8a31640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] conn(0x7f46e006fa60 0x7f46e0071ea0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3e57fa640 1 -- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] conn(0x7fb3ec06fb00 msgr2=0x7fb3ec071f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3e57fa640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] conn(0x7fb3ec06fb00 0x7fb3ec071f40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46e00664a0 msgr2=0x7f46e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f46f8a31640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46e00664a0 0x7f46e0068960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f46f4067530 tx=0x7f46e8007d40 comp rx=0 tx=0).stop 2023-11-28T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f40c6100 msgr2=0x7f46f4161c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f46f8a31640 1 --2- 172.21.15.12:0/1702824053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f40c6100 0x7f46f4161c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f46ec007f20 tx=0x7f46ec00d560 comp rx=0 tx=0).stop 2023-11-28T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3e57fa640 1 -- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb3ec0664c0 msgr2=0x7fb3ec068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3e57fa640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb3ec0664c0 0x7fb3ec068980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0004850 tx=0x7fb3f000b220 comp rx=0 tx=0).stop 2023-11-28T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3e57fa640 1 -- 172.21.15.12:0/1070452844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80d4e70 msgr2=0x7fb3f80cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3e57fa640 1 --2- 172.21.15.12:0/1070452844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80d4e70 0x7fb3f80cbcb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80070a0 tx=0x7fb3f401bbf0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3fffff640 1 -- 172.21.15.12:0/1070452844 reap_dead start 2023-11-28T21:35:08.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f46f359e640 1 -- 172.21.15.12:0/1702824053 reap_dead start 2023-11-28T21:35:08.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3e57fa640 1 -- 172.21.15.12:0/1070452844 shutdown_connections 2023-11-28T21:35:08.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3e57fa640 1 -- 172.21.15.12:0/1070452844 >> 172.21.15.12:0/1070452844 conn(0x7fb3f8050330 msgr2=0x7fb3f80d8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 shutdown_connections 2023-11-28T21:35:08.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 >> 172.21.15.12:0/1702824053 conn(0x7f46f4066360 msgr2=0x7f46f4157860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 shutdown_connections 2023-11-28T21:35:08.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3e57fa640 1 -- 172.21.15.12:0/1070452844 shutdown_connections 2023-11-28T21:35:08.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f46f8a31640 1 -- 172.21.15.12:0/1702824053 wait complete. 2023-11-28T21:35:08.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fb3e57fa640 1 -- 172.21.15.12:0/1070452844 wait complete. 2023-11-28T21:35:08.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 --> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe8806a5b0 con 0x7fbe6c06fbb0 2023-11-28T21:35:08.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 --> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f766c06b200 con 0x7f7658070850 2023-11-28T21:35:08.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3910919114 <== osd.5 v2:172.21.15.144:6816/2546756774 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbe6c0726c0 con 0x7fbe6c06fbb0 2023-11-28T21:35:08.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.424+0000 7f7668ff9640 1 -- 172.21.15.12:0/512594531 <== osd.4 v2:172.21.15.144:6800/3927315436 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7660007610 con 0x7f7658070850 2023-11-28T21:35:08.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 >> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] conn(0x7fbe6c06fbb0 msgr2=0x7fbe6c071ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8e596640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] conn(0x7fbe6c06fbb0 0x7fbe6c071ff0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 >> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] conn(0x7f7658070850 msgr2=0x7f7658072cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbe6c066570 msgr2=0x7fbe6c068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8e596640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbe6c066570 0x7fbe6c068a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe88073210 tx=0x7fbe840aa000 comp rx=0 tx=0).stop 2023-11-28T21:35:08.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe88154f70 msgr2=0x7fbe881edf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8e596640 1 --2- 172.21.15.12:0/3910919114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe88154f70 0x7fbe881edf00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe88073170 tx=0x7fbe78010c20 comp rx=0 tx=0).stop 2023-11-28T21:35:08.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7671534640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] conn(0x7f7658070850 0x7f7658072cf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8dd95640 1 -- 172.21.15.12:0/3910919114 reap_dead start 2023-11-28T21:35:08.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 shutdown_connections 2023-11-28T21:35:08.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 >> 172.21.15.12:0/3910919114 conn(0x7fbe88066360 msgr2=0x7fbe881597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 shutdown_connections 2023-11-28T21:35:08.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7fbe8e596640 1 -- 172.21.15.12:0/3910919114 wait complete. 2023-11-28T21:35:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76580664f0 msgr2=0x7f76580689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7671534640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76580664f0 0x7f76580689b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f76640012f0 tx=0x7f7664001690 comp rx=0 tx=0).stop 2023-11-28T21:35:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c1582c0 msgr2=0x7f766c15cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7671534640 1 --2- 172.21.15.12:0/512594531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f766c1582c0 0x7f766c15cfe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f765c01a040 tx=0x7f765c0191b0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7670d33640 1 -- 172.21.15.12:0/512594531 reap_dead start 2023-11-28T21:35:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 shutdown_connections 2023-11-28T21:35:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 >> 172.21.15.12:0/512594531 conn(0x7f766c066360 msgr2=0x7f766c1576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 shutdown_connections 2023-11-28T21:35:08.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.428+0000 7f7671534640 1 -- 172.21.15.12:0/512594531 wait complete. 2023-11-28T21:35:08.466 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-11-28T21:35:08.469 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T21:35:08.470 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-11-28T21:35:08.470 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T21:35:08.471 INFO:teuthology.orchestra.run.smithi012.stdout:55834574859 2023-11-28T21:35:08.472 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T21:35:08.472 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-11-28T21:35:08.473 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T21:35:08.473 INFO:teuthology.orchestra.run.smithi012.stdout:55834574859 2023-11-28T21:35:08.474 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T21:35:08.474 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-11-28T21:35:08.475 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T21:35:08.475 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-11-28T21:35:08.476 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T21:35:08.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.692+0000 7f4b47879640 1 Processor -- start 2023-11-28T21:35:08.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b47879640 1 -- start start 2023-11-28T21:35:08.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b47879640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4b40154100 0x7f4b4015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b47879640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b4006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b47879640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4b400ca3f0 0x7f4b4006aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b47879640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4015e270 con 0x7f4b40154100 2023-11-28T21:35:08.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b47879640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b400c4e60 con 0x7f4b400ca3f0 2023-11-28T21:35:08.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b47879640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b401544e0 con 0x7f4b400c9fd0 2023-11-28T21:35:08.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b44e14640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b4006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b44e14640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b4006a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56488/0 (socket says 172.21.15.12:56488) 2023-11-28T21:35:08.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b44e14640 1 -- 172.21.15.12:0/2231512294 learned_addr learned my addr 172.21.15.12:0/2231512294 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b45615640 1 --2- 172.21.15.12:0/2231512294 >> v2:172.21.15.12:3300/0 conn(0x7f4b40154100 0x7f4b4015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b44e14640 1 -- 172.21.15.12:0/2231512294 >> v2:172.21.15.144:3300/0 conn(0x7f4b400ca3f0 msgr2=0x7f4b4006aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b45e16640 1 --2- 172.21.15.12:0/2231512294 >> v2:172.21.15.144:3300/0 conn(0x7f4b400ca3f0 0x7f4b4006aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b44e14640 1 --2- 172.21.15.12:0/2231512294 >> v2:172.21.15.144:3300/0 conn(0x7f4b400ca3f0 0x7f4b4006aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b44e14640 1 -- 172.21.15.12:0/2231512294 >> v2:172.21.15.12:3300/0 conn(0x7f4b40154100 msgr2=0x7f4b4015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b44e14640 1 --2- 172.21.15.12:0/2231512294 >> v2:172.21.15.12:3300/0 conn(0x7f4b40154100 0x7f4b4015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.696+0000 7f4b44e14640 1 -- 172.21.15.12:0/2231512294 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4015d9d0 con 0x7f4b400c9fd0 2023-11-28T21:35:08.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.700+0000 7f4b44e14640 1 --2- 172.21.15.12:0/2231512294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b4006a5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009870 tx=0x7f4b3001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d219a41e5a687451 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b37fff640 1 -- 172.21.15.12:0/2231512294 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b300aa5f0 con 0x7f4b400c9fd0 2023-11-28T21:35:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b37fff640 1 -- 172.21.15.12:0/2231512294 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b300aa750 con 0x7f4b400c9fd0 2023-11-28T21:35:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b37fff640 1 -- 172.21.15.12:0/2231512294 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b300aaae0 con 0x7f4b400c9fd0 2023-11-28T21:35:08.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b47879640 1 -- 172.21.15.12:0/2231512294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 msgr2=0x7f4b4006a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b47879640 1 --2- 172.21.15.12:0/2231512294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b4006a5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009870 tx=0x7f4b3001c4d0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b47879640 1 -- 172.21.15.12:0/2231512294 shutdown_connections 2023-11-28T21:35:08.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b47879640 1 --2- 172.21.15.12:0/2231512294 >> v2:172.21.15.144:3300/0 conn(0x7f4b400ca3f0 0x7f4b4006aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b47879640 1 --2- 172.21.15.12:0/2231512294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b4006a5b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b47879640 1 --2- 172.21.15.12:0/2231512294 >> v2:172.21.15.12:3300/0 conn(0x7f4b40154100 0x7f4b4015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b47879640 1 -- 172.21.15.12:0/2231512294 >> 172.21.15.12:0/2231512294 conn(0x7f4b4006c690 msgr2=0x7f4b40066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.704+0000 7f4b47879640 1 -- 172.21.15.12:0/2231512294 shutdown_connections 2023-11-28T21:35:08.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7f4b47879640 1 -- 172.21.15.12:0/2231512294 wait complete. 2023-11-28T21:35:08.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1ffff640 1 -- 172.21.15.12:0/2743914015 <== osd.1 v2:172.21.15.12:6802/1716423303 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcd10002c90 con 0x7fcd240706b0 2023-11-28T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1dffb640 1 -- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] conn(0x7fcd240706b0 msgr2=0x7fcd24072b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1dffb640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] conn(0x7fcd240706b0 0x7fcd24072b50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1dffb640 1 -- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcd240664a0 msgr2=0x7fcd24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1dffb640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcd240664a0 0x7fcd24068960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd30002240 tx=0x7fcd30009040 comp rx=0 tx=0).stop 2023-11-28T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1dffb640 1 -- 172.21.15.12:0/2743914015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380c8df0 msgr2=0x7fcd380c3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1dffb640 1 --2- 172.21.15.12:0/2743914015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380c8df0 0x7fcd380c3200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0ab000 tx=0x7fcd2c019240 comp rx=0 tx=0).stop 2023-11-28T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7f4b47879640 1 Processor -- start 2023-11-28T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd37fff640 1 -- 172.21.15.12:0/2743914015 reap_dead start 2023-11-28T21:35:08.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1dffb640 1 -- 172.21.15.12:0/2743914015 shutdown_connections 2023-11-28T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1dffb640 1 -- 172.21.15.12:0/2743914015 >> 172.21.15.12:0/2743914015 conn(0x7fcd3806abe0 msgr2=0x7fcd3814ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1dffb640 1 -- 172.21.15.12:0/2743914015 shutdown_connections 2023-11-28T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7f4b47879640 1 -- start start 2023-11-28T21:35:08.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.708+0000 7fcd1dffb640 1 -- 172.21.15.12:0/2743914015 wait complete. 2023-11-28T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b47879640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b400c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b47879640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b400ca3f0 0x7f4b4015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b47879640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b40154100 0x7f4b400c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b47879640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b401f2440 con 0x7f4b400ca3f0 2023-11-28T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b47879640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40155f90 con 0x7f4b40154100 2023-11-28T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b47879640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400c86d0 con 0x7f4b400c9fd0 2023-11-28T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b44e14640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b400ca3f0 0x7f4b4015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b45615640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b400c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b45615640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b400c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56498/0 (socket says 172.21.15.12:56498) 2023-11-28T21:35:08.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b44e14640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b400ca3f0 0x7f4b4015c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38202/0 (socket says 172.21.15.12:38202) 2023-11-28T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b45615640 1 -- 172.21.15.12:0/2912880715 learned_addr learned my addr 172.21.15.12:0/2912880715 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b45e16640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b40154100 0x7f4b400c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b44e14640 1 -- 172.21.15.12:0/2912880715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 msgr2=0x7f4b400c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b44e14640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b400c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b44e14640 1 -- 172.21.15.12:0/2912880715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b40154100 msgr2=0x7f4b400c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b44e14640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b40154100 0x7f4b400c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.712+0000 7f4b44e14640 1 -- 172.21.15.12:0/2912880715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300094d0 con 0x7f4b400ca3f0 2023-11-28T21:35:08.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.716+0000 7f4b44e14640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b400ca3f0 0x7f4b4015c3b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4b300ab000 tx=0x7f4b3001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.716+0000 7f4b35ffb640 1 -- 172.21.15.12:0/2912880715 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b300aae90 con 0x7f4b400ca3f0 2023-11-28T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.716+0000 7f4b47879640 1 -- 172.21.15.12:0/2912880715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b401f2630 con 0x7f4b400ca3f0 2023-11-28T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.716+0000 7f4b47879640 1 -- 172.21.15.12:0/2912880715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b401f2bf0 con 0x7f4b400ca3f0 2023-11-28T21:35:08.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.716+0000 7f4b35ffb640 1 -- 172.21.15.12:0/2912880715 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b300aae90 con 0x7f4b400ca3f0 2023-11-28T21:35:08.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.716+0000 7f4b35ffb640 1 -- 172.21.15.12:0/2912880715 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b300bd430 con 0x7f4b400ca3f0 2023-11-28T21:35:08.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.720+0000 7f4b35ffb640 1 -- 172.21.15.12:0/2912880715 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4b300bd590 con 0x7f4b400ca3f0 2023-11-28T21:35:08.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.724+0000 7f4b47879640 1 -- 172.21.15.12:0/2912880715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b20005260 con 0x7f4b400ca3f0 2023-11-28T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.724+0000 7f4b35ffb640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4b240664d0 0x7f4b24068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.724+0000 7f4b35ffb640 1 -- 172.21.15.12:0/2912880715 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4b300a7070 con 0x7f4b400ca3f0 2023-11-28T21:35:08.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.736+0000 7f4b45615640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4b240664d0 0x7f4b24068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.736+0000 7f4b45615640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4b240664d0 0x7f4b24068990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009b80 tx=0x7f4b3c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.736+0000 7f4b35ffb640 1 -- 172.21.15.12:0/2912880715 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4b300eb2b0 con 0x7f4b400ca3f0 2023-11-28T21:35:08.751 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-11-28T21:35:08.751 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.752+0000 7f9978cbb640 1 Processor -- start 2023-11-28T21:35:08.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.752+0000 7f9978cbb640 1 -- start start 2023-11-28T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f9978cbb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c04ef70 0x7f996c050a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f9978cbb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f996c050230 0x7f996c0378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f997259c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f996c04fe50 0x7f996c0e39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36390/0 (socket says 172.21.15.12:36390) 2023-11-28T21:35:08.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f997259c640 1 -- 172.21.15.12:0/4267991843 learned_addr learned my addr 172.21.15.12:0/4267991843 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f9978cbb640 1 -- 172.21.15.12:0/4267991843 --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996c050830 con 0x7f996c04fe50 2023-11-28T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f9972d9d640 1 --2- 172.21.15.12:0/4267991843 >> v2:172.21.15.12:3300/0 conn(0x7f996c050230 0x7f996c0378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f9978cbb640 1 -- 172.21.15.12:0/4267991843 --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996c007ab0 con 0x7f996c050230 2023-11-28T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f9978cbb640 1 -- 172.21.15.12:0/4267991843 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c006c50 con 0x7f996c04ef70 2023-11-28T21:35:08.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857644005 0 0) 0x7f996c050830 con 0x7f996c04fe50 2023-11-28T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f995c003530 con 0x7f996c04fe50 2023-11-28T21:35:08.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f9971d9b640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c04ef70 0x7f996c050a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2430310789 0 0) 0x7f995c003530 con 0x7f996c04fe50 2023-11-28T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.756+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c04ef70 msgr2=0x7f996c050a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c04ef70 0x7f996c050a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 >> v2:172.21.15.12:3300/0 conn(0x7f996c050230 msgr2=0x7f996c0378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 --2- 172.21.15.12:0/4267991843 >> v2:172.21.15.12:3300/0 conn(0x7f996c050230 0x7f996c0378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c006f30 con 0x7f996c04fe50 2023-11-28T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f99640016d0 con 0x7f996c04fe50 2023-11-28T21:35:08.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 >> v1:172.21.15.12:6789/0 conn(0x7f996c04fe50 legacy=0x7f996c0e39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f995c003980 0x7f995c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c0047d0 0x7f995c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c005190 0x7f995c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c007ab0 con 0x7f995c005190 2023-11-28T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c006c50 con 0x7f995c0047d0 2023-11-28T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995c00c0e0 con 0x7f995c003980 2023-11-28T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f9971d9b640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c04ef70 0x7f996c050a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f9972d9d640 1 --2- 172.21.15.12:0/4267991843 >> v2:172.21.15.12:3300/0 conn(0x7f996c050230 0x7f996c0378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f9971d9b640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c0047d0 0x7f995c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f997259c640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f995c003980 0x7f995c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f9972d9d640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c005190 0x7f995c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f9971d9b640 1 -- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f995c003980 msgr2=0x7f995c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f9971d9b640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f995c003980 0x7f995c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f9971d9b640 1 -- 172.21.15.12:0/4267991843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c005190 msgr2=0x7f995c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f9971d9b640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c005190 0x7f995c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.760+0000 7f9971d9b640 1 -- 172.21.15.12:0/4267991843 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c003530 con 0x7f995c0047d0 2023-11-28T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.764+0000 7fc7ae46b640 1 Processor -- start 2023-11-28T21:35:08.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.764+0000 7fc7ae46b640 1 -- start start 2023-11-28T21:35:08.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.764+0000 7fe57edf8640 1 Processor -- start 2023-11-28T21:35:08.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.764+0000 7fc7ae46b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc7a806d1a0 0x7fc7a8156790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.764+0000 7fc7ae46b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d580 0x7fc7a814f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.764+0000 7fc7ae46b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc7a806b5c0 0x7fc7a814fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.764+0000 7fc7ae46b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a8156cd0 con 0x7fc7a806d1a0 2023-11-28T21:35:08.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.764+0000 7fe57edf8640 1 -- start start 2023-11-28T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.764+0000 7fc7ae46b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a80c5cc0 con 0x7fc7a806b5c0 2023-11-28T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.764+0000 7fc7ae46b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80c6e00 con 0x7fc7a806d580 2023-11-28T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7a7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc7a806d1a0 0x7fc7a8156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7a77fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d580 0x7fc7a814f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7aca08640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc7a806b5c0 0x7fc7a814fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7a7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc7a806d1a0 0x7fc7a8156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38232/0 (socket says 172.21.15.12:38232) 2023-11-28T21:35:08.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7a7fff640 1 -- 172.21.15.12:0/3110709888 learned_addr learned my addr 172.21.15.12:0/3110709888 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7f9971d9b640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c0047d0 0x7f995c004c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f996800a880 tx=0x7f996801c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e81bf305657c886 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57edf8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57814ff70 0x7fe578159c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57edf8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57815a1c0 con 0x7fe578066e80 2023-11-28T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57edf8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5780c6180 con 0x7fe57806b260 2023-11-28T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57edf8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780c6460 con 0x7fe57814ff70 2023-11-28T21:35:08.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7aca08640 1 -- 172.21.15.12:0/3110709888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d580 msgr2=0x7fc7a814f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7aca08640 1 --2- 172.21.15.12:0/3110709888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d580 0x7fc7a814f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7aca08640 1 -- 172.21.15.12:0/3110709888 >> v2:172.21.15.12:3300/0 conn(0x7fc7a806d1a0 msgr2=0x7fc7a8156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7a77fe640 1 --2- 172.21.15.12:0/3110709888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d580 0x7fc7a814f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57ddf6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57814ff70 0x7fe578159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57e5f7640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe578066e80 0x7fe578160600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36400/0 (socket says 172.21.15.12:36400) 2023-11-28T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57ddf6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57814ff70 0x7fe578159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56516/0 (socket says 172.21.15.12:56516) 2023-11-28T21:35:08.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57e5f7640 1 -- 172.21.15.12:0/2715427612 learned_addr learned my addr 172.21.15.12:0/2715427612 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99680ad680 con 0x7f995c0047d0 2023-11-28T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99680ad7e0 con 0x7f995c0047d0 2023-11-28T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7f997159a640 1 -- 172.21.15.12:0/4267991843 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99680ad680 con 0x7f995c0047d0 2023-11-28T21:35:08.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7aca08640 1 --2- 172.21.15.12:0/3110709888 >> v2:172.21.15.12:3300/0 conn(0x7fc7a806d1a0 0x7fc7a8156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fc7aca08640 1 -- 172.21.15.12:0/3110709888 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a80c5f10 con 0x7fc7a806b5c0 2023-11-28T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57ddf6640 1 -- 172.21.15.12:0/2715427612 >> v1:172.21.15.144:6789/0 conn(0x7fe57806b260 legacy=0x7fe57806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57ddf6640 1 -- 172.21.15.12:0/2715427612 >> v1:172.21.15.12:6789/0 conn(0x7fe578066e80 legacy=0x7fe578160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57cdf4640 1 -- 172.21.15.12:0/2715427612 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881479030 0 0) 0x7fe574001120 con 0x7fe578066e80 2023-11-28T21:35:08.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.768+0000 7fe57ddf6640 1 -- 172.21.15.12:0/2715427612 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5780c6740 con 0x7fe57814ff70 2023-11-28T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.772+0000 7f9978cbb640 1 -- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c0047d0 msgr2=0x7f995c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.772+0000 7f9978cbb640 1 --2- 172.21.15.12:0/4267991843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f995c0047d0 0x7f995c004c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f996800a880 tx=0x7f996801c430 comp rx=0 tx=0).stop 2023-11-28T21:35:08.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.772+0000 7f9972d9d640 1 -- 172.21.15.12:0/4267991843 reap_dead start 2023-11-28T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.772+0000 7f9978cbb640 1 -- 172.21.15.12:0/4267991843 shutdown_connections 2023-11-28T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.772+0000 7f9978cbb640 1 -- 172.21.15.12:0/4267991843 >> 172.21.15.12:0/4267991843 conn(0x7f996c0545f0 msgr2=0x7f996c0549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.772+0000 7f9978cbb640 1 -- 172.21.15.12:0/4267991843 shutdown_connections 2023-11-28T21:35:08.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.772+0000 7f9978cbb640 1 -- 172.21.15.12:0/4267991843 wait complete. 2023-11-28T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.772+0000 7fc7aca08640 1 --2- 172.21.15.12:0/3110709888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806b5c0 0x7fc7a814fe40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc7a806fb30 tx=0x7fc7a001f0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db175b2f486d1b74 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7a6ffd640 1 -- 172.21.15.12:0/3110709888 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00b0440 con 0x7fc7a806b5c0 2023-11-28T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7a6ffd640 1 -- 172.21.15.12:0/3110709888 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a00b05a0 con 0x7fc7a806b5c0 2023-11-28T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7a6ffd640 1 -- 172.21.15.12:0/3110709888 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00b0930 con 0x7fc7a806b5c0 2023-11-28T21:35:08.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fe57ddf6640 1 --2- 172.21.15.12:0/2715427612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57814ff70 0x7fe578159c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe5680097f0 tx=0x7fe56801ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abfaa5fc7966ee16 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f9978cbb640 1 Processor -- start 2023-11-28T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7ae46b640 1 -- 172.21.15.12:0/3110709888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806b5c0 msgr2=0x7fc7a814fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7ae46b640 1 --2- 172.21.15.12:0/3110709888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806b5c0 0x7fc7a814fe40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc7a806fb30 tx=0x7fc7a001f0f0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7ae46b640 1 -- 172.21.15.12:0/3110709888 shutdown_connections 2023-11-28T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7ae46b640 1 --2- 172.21.15.12:0/3110709888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806b5c0 0x7fc7a814fe40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7ae46b640 1 --2- 172.21.15.12:0/3110709888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d580 0x7fc7a814f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7ae46b640 1 --2- 172.21.15.12:0/3110709888 >> v2:172.21.15.12:3300/0 conn(0x7fc7a806d1a0 0x7fc7a8156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7ae46b640 1 -- 172.21.15.12:0/3110709888 >> 172.21.15.12:0/3110709888 conn(0x7fc7a8066360 msgr2=0x7fc7a8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fe57cdf4640 1 -- 172.21.15.12:0/2715427612 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680a8970 con 0x7fe57814ff70 2023-11-28T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fe57cdf4640 1 -- 172.21.15.12:0/2715427612 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5680a8ad0 con 0x7fe57814ff70 2023-11-28T21:35:08.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fe57cdf4640 1 -- 172.21.15.12:0/2715427612 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680a8e60 con 0x7fe57814ff70 2023-11-28T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f1e65d2a640 1 Processor -- start 2023-11-28T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f1e65d2a640 1 -- start start 2023-11-28T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f9978cbb640 1 -- start start 2023-11-28T21:35:08.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f9978cbb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c0047d0 0x7f996c00c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fe57edf8640 1 -- 172.21.15.12:0/2715427612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57814ff70 msgr2=0x7fe578159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fe57edf8640 1 --2- 172.21.15.12:0/2715427612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57814ff70 0x7fe578159c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe5680097f0 tx=0x7fe56801ec70 comp rx=0 tx=0).stop 2023-11-28T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f997259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c0047d0 0x7f996c00c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f997259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c0047d0 0x7f996c00c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38248/0 (socket says 172.21.15.12:38248) 2023-11-28T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f9978cbb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c00d570 0x7f996c048250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fe57edf8640 1 -- 172.21.15.12:0/2715427612 shutdown_connections 2023-11-28T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fe57edf8640 1 --2- 172.21.15.12:0/2715427612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57814ff70 0x7fe578159c80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fe57edf8640 1 -- 172.21.15.12:0/2715427612 >> 172.21.15.12:0/2715427612 conn(0x7fe578066360 msgr2=0x7fe578066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f997259c640 1 -- 172.21.15.12:0/2982796179 learned_addr learned my addr 172.21.15.12:0/2982796179 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f9978cbb640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c00d9b0 0x7f996c048790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c00ccb0 con 0x7f995c0047d0 2023-11-28T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c047dc0 con 0x7f996c00d570 2023-11-28T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1795f0 con 0x7f996c00d9b0 2023-11-28T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9971d9b640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c00d570 0x7f996c048250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9972d9d640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c00d9b0 0x7f996c048790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e65d2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e601540f0 0x7f1e600c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fea4b5df640 1 Processor -- start 2023-11-28T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fea4b5df640 1 -- start start 2023-11-28T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea4b5df640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fea44150160 0x7fea440c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea4b5df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea44159250 0x7fea441596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f8fde5aa640 1 Processor -- start 2023-11-28T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7f8fde5aa640 1 -- start start 2023-11-28T21:35:08.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e65d2a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1e6015d210 0x7f1e6015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9971d9b640 1 -- 172.21.15.12:0/2982796179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c00d9b0 msgr2=0x7f996c048790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9971d9b640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c00d9b0 0x7f996c048790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9971d9b640 1 -- 172.21.15.12:0/2982796179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c0047d0 msgr2=0x7f996c00c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9971d9b640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c0047d0 0x7f996c00c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9971d9b640 1 -- 172.21.15.12:0/2982796179 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c049df0 con 0x7f996c00d570 2023-11-28T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f997259c640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c0047d0 0x7f996c00c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7ae46b640 1 -- 172.21.15.12:0/3110709888 shutdown_connections 2023-11-28T21:35:08.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.776+0000 7fc7ae46b640 1 -- 172.21.15.12:0/3110709888 wait complete. 2023-11-28T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fc7ae46b640 1 Processor -- start 2023-11-28T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fde5aa640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fd80cb880 0x7f8fd80c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fde5aa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8fd80cbca0 0x7f8fd80c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e5effd640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f1e6015b180 0x7f1e6015b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52698/0 (socket says 172.21.15.12:52698) 2023-11-28T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e601540f0 0x7f1e600c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e601540f0 0x7f1e600c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56536/0 (socket says 172.21.15.12:56536) 2023-11-28T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e5effd640 1 -- 172.21.15.12:0/3172900093 learned_addr learned my addr 172.21.15.12:0/3172900093 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3172900093 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6006d2a0 con 0x7f1e6015b180 2023-11-28T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e5ffff640 1 --2- 172.21.15.12:0/3172900093 >> v2:172.21.15.12:3300/0 conn(0x7f1e6015d210 0x7f1e6015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea48b7a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fea44150160 0x7fea440c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea4937b640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fea4414fd80 0x7fea440c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52714/0 (socket says 172.21.15.12:52714) 2023-11-28T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea4937b640 1 -- 172.21.15.12:0/772685521 learned_addr learned my addr 172.21.15.12:0/772685521 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea49b7c640 1 --2- 172.21.15.12:0/772685521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea44159250 0x7fea441596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea4b5df640 1 -- 172.21.15.12:0/772685521 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4406cf40 con 0x7fea4414fd80 2023-11-28T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fd7fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8fd8153f40 0x7f8fd80ca9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52724/0 (socket says 172.21.15.12:52724) 2023-11-28T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3172900093 --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6006e200 con 0x7f1e6015d210 2023-11-28T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3172900093 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6006d580 con 0x7f1e601540f0 2023-11-28T21:35:08.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9971d9b640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c00d570 0x7f996c048250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99680ac000 tx=0x7f995c005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fd77fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fd80cb880 0x7f8fd80c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fd77fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8fd80cb880 0x7f8fd80c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:39668/0 (socket says 172.21.15.12:39668) 2023-11-28T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fdcb47640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8fd80cbca0 0x7f8fd80c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fdcb47640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8fd80cbca0 0x7f8fd80c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38258/0 (socket says 172.21.15.12:38258) 2023-11-28T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fd7fff640 1 -- 172.21.15.12:0/147031449 learned_addr learned my addr 172.21.15.12:0/147031449 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:08.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea4b5df640 1 -- 172.21.15.12:0/772685521 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea44072a80 con 0x7fea44150160 2023-11-28T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9962ffd640 1 -- 172.21.15.12:0/2982796179 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99680be070 con 0x7f996c00d570 2023-11-28T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9962ffd640 1 -- 172.21.15.12:0/2982796179 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99680add60 con 0x7f996c00d570 2023-11-28T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9962ffd640 1 -- 172.21.15.12:0/2982796179 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99680b6620 con 0x7f996c00d570 2023-11-28T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fde5aa640 1 -- 172.21.15.12:0/147031449 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd815e180 con 0x7f8fd8153f40 2023-11-28T21:35:08.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea4b5df640 1 -- 172.21.15.12:0/772685521 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea440c7c60 con 0x7fea44159250 2023-11-28T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fde5aa640 1 -- 172.21.15.12:0/147031449 --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd815d2b0 con 0x7f8fd80cbca0 2023-11-28T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e5f7fe640 1 -- 172.21.15.12:0/3172900093 >> v2:172.21.15.12:3300/0 conn(0x7f1e6015d210 msgr2=0x7f1e6015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fc7ae46b640 1 -- start start 2023-11-28T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e5f7fe640 1 --2- 172.21.15.12:0/3172900093 >> v2:172.21.15.12:3300/0 conn(0x7f1e6015d210 0x7f1e6015d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e5f7fe640 1 -- 172.21.15.12:0/3172900093 >> v1:172.21.15.144:6789/0 conn(0x7f1e6015b180 legacy=0x7f1e6015b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f1e5f7fe640 1 -- 172.21.15.12:0/3172900093 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6006d860 con 0x7f1e601540f0 2023-11-28T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fde5aa640 1 -- 172.21.15.12:0/147031449 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd815d570 con 0x7f8fd80cb880 2023-11-28T21:35:08.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c04a0b0 con 0x7f996c00d570 2023-11-28T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c04a670 con 0x7f996c00d570 2023-11-28T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fd6ffd640 1 -- 172.21.15.12:0/147031449 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466218651 0 0) 0x7f8fd815e180 con 0x7f8fd8153f40 2023-11-28T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7f8fd6ffd640 1 -- 172.21.15.12:0/147031449 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc0003530 con 0x7f8fd8153f40 2023-11-28T21:35:08.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7f8fdcb47640 1 -- 172.21.15.12:0/147031449 >> v2:172.21.15.144:3300/0 conn(0x7f8fd80cb880 msgr2=0x7f8fd80c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7f8fdcb47640 1 --2- 172.21.15.12:0/147031449 >> v2:172.21.15.144:3300/0 conn(0x7f8fd80cb880 0x7f8fd80c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7f8fdcb47640 1 -- 172.21.15.12:0/147031449 >> v1:172.21.15.144:6789/0 conn(0x7f8fd8153f40 legacy=0x7f8fd80ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7f8fdcb47640 1 -- 172.21.15.12:0/147031449 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd815d850 con 0x7f8fd80cbca0 2023-11-28T21:35:08.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c005370 con 0x7f996c00d570 2023-11-28T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fe57edf8640 1 -- 172.21.15.12:0/2715427612 shutdown_connections 2023-11-28T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fe57edf8640 1 -- 172.21.15.12:0/2715427612 wait complete. 2023-11-28T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7fe57edf8640 1 Processor -- start 2023-11-28T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7fe57edf8640 1 -- start start 2023-11-28T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea49b7c640 1 -- 172.21.15.12:0/772685521 >> v2:172.21.15.144:3300/0 conn(0x7fea44150160 msgr2=0x7fea440c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea49b7c640 1 --2- 172.21.15.12:0/772685521 >> v2:172.21.15.144:3300/0 conn(0x7fea44150160 0x7fea440c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea49b7c640 1 -- 172.21.15.12:0/772685521 >> v1:172.21.15.144:6789/0 conn(0x7fea4414fd80 legacy=0x7fea440c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea49b7c640 1 -- 172.21.15.12:0/772685521 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea440715b0 con 0x7fea44159250 2023-11-28T21:35:08.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.780+0000 7fea3bfff640 1 -- 172.21.15.12:0/772685521 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263967648 0 0) 0x7fea4406cf40 con 0x7fea4414fd80 2023-11-28T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7fc7ae46b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a806b5c0 0x7fc7a81490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7fc7ae46b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d1a0 0x7fc7a8200240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7fc7ae46b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806d580 0x7fc7a80722e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7f9962ffd640 1 -- 172.21.15.12:0/2982796179 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f99680b4040 con 0x7f996c00d570 2023-11-28T21:35:08.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7fc7ae46b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a8143d70 con 0x7fc7a806b5c0 2023-11-28T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7fc7ae46b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a81ffdb0 con 0x7fc7a806d580 2023-11-28T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.784+0000 7fc7ae46b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a81fcfe0 con 0x7fc7a806d1a0 2023-11-28T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7aca08640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806d580 0x7fc7a80722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7aca08640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806d580 0x7fc7a80722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:39676/0 (socket says 172.21.15.12:39676) 2023-11-28T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7aca08640 1 -- 172.21.15.12:0/1811264794 learned_addr learned my addr 172.21.15.12:0/1811264794 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f9962ffd640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 0x7f9954068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7a77fe640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d1a0 0x7fc7a8200240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7a7fff640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a806b5c0 0x7fc7a81490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f9962ffd640 1 -- 172.21.15.12:0/2982796179 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f996811d480 con 0x7f996c00d570 2023-11-28T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7aca08640 1 -- 172.21.15.12:0/1811264794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d1a0 msgr2=0x7fc7a8200240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7aca08640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d1a0 0x7fc7a8200240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7aca08640 1 -- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a806b5c0 msgr2=0x7fc7a81490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea49b7c640 1 --2- 172.21.15.12:0/772685521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea44159250 0x7fea441596c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009d90 tx=0x7fea3c01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b162d75e5aca5ff6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7aca08640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a806b5c0 0x7fc7a81490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7aca08640 1 -- 172.21.15.12:0/1811264794 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0009a70 con 0x7fc7a806d580 2023-11-28T21:35:08.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7a7fff640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a806b5c0 0x7fc7a81490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57edf8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe578066e80 0x7fe5781ecad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f1e5f7fe640 1 --2- 172.21.15.12:0/3172900093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e601540f0 0x7f1e600c3be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009d90 tx=0x7f1e5801c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e1051d62ab868 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f8fdcb47640 1 --2- 172.21.15.12:0/147031449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd80cbca0 0x7f8fd80c4be0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80cce50 tx=0x7f8fd001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5a24dfc8f11cb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea3bfff640 1 -- 172.21.15.12:0/772685521 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0af600 con 0x7fea44159250 2023-11-28T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f997259c640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 0x7f9954068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57ddf6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe578066e80 0x7fe5781ecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57edf8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57806b260 0x7fe578159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57ddf6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe578066e80 0x7fe5781ecad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38288/0 (socket says 172.21.15.12:38288) 2023-11-28T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fc7aca08640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806d580 0x7fc7a80722e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc7a001d7a0 tx=0x7fc7a001be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57edf8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe57814ff70 0x7fe5780720a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57ddf6640 1 -- 172.21.15.12:0/801682324 learned_addr learned my addr 172.21.15.12:0/801682324 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5781ed010 con 0x7fe578066e80 2023-11-28T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe578152280 con 0x7fe57814ff70 2023-11-28T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57814f740 con 0x7fe57806b260 2023-11-28T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57d5f5640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57806b260 0x7fe578159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57e5f7640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe57814ff70 0x7fe5780720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f8fd6ffd640 1 -- 172.21.15.12:0/147031449 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd00b0440 con 0x7f8fd80cbca0 2023-11-28T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f8fd6ffd640 1 -- 172.21.15.12:0/147031449 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd00b05a0 con 0x7f8fd80cbca0 2023-11-28T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f8fd6ffd640 1 -- 172.21.15.12:0/147031449 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd00b0930 con 0x7f8fd80cbca0 2023-11-28T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f1e5e7fc640 1 -- 172.21.15.12:0/3172900093 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580a58f0 con 0x7f1e601540f0 2023-11-28T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f1e5e7fc640 1 -- 172.21.15.12:0/3172900093 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e580a5a50 con 0x7f1e601540f0 2023-11-28T21:35:08.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f1e5e7fc640 1 -- 172.21.15.12:0/3172900093 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580a5de0 con 0x7f1e601540f0 2023-11-28T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea3bfff640 1 -- 172.21.15.12:0/772685521 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c0ab050 con 0x7fea44159250 2023-11-28T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea3bfff640 1 -- 172.21.15.12:0/772685521 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0af980 con 0x7fea44159250 2023-11-28T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea4b5df640 1 -- 172.21.15.12:0/772685521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea44159250 msgr2=0x7fea441596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea4b5df640 1 --2- 172.21.15.12:0/772685521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea44159250 0x7fea441596c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009d90 tx=0x7fea3c01f150 comp rx=0 tx=0).stop 2023-11-28T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f997259c640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 0x7f9954068930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9964001c50 tx=0x7f9964001e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57ddf6640 1 -- 172.21.15.12:0/801682324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57806b260 msgr2=0x7fe578159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57ddf6640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57806b260 0x7fe578159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57ddf6640 1 -- 172.21.15.12:0/801682324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe57814ff70 msgr2=0x7fe5780720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea4b5df640 1 -- 172.21.15.12:0/772685521 shutdown_connections 2023-11-28T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea4b5df640 1 --2- 172.21.15.12:0/772685521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea44159250 0x7fea441596c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea4b5df640 1 --2- 172.21.15.12:0/772685521 >> v2:172.21.15.144:3300/0 conn(0x7fea44150160 0x7fea440c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea4b5df640 1 -- 172.21.15.12:0/772685521 >> 172.21.15.12:0/772685521 conn(0x7fea44066360 msgr2=0x7fea44066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea4b5df640 1 -- 172.21.15.12:0/772685521 shutdown_connections 2023-11-28T21:35:08.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57ddf6640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe57814ff70 0x7fe5780720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57ddf6640 1 -- 172.21.15.12:0/801682324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe568009420 con 0x7fe578066e80 2023-11-28T21:35:08.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fe57e5f7640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe57814ff70 0x7fe5780720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:08.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7fea4b5df640 1 -- 172.21.15.12:0/772685521 wait complete. 2023-11-28T21:35:08.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f8fde5aa640 1 -- 172.21.15.12:0/147031449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd80cbca0 msgr2=0x7f8fd80c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f8fde5aa640 1 --2- 172.21.15.12:0/147031449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd80cbca0 0x7f8fd80c4be0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80cce50 tx=0x7f8fd001f130 comp rx=0 tx=0).stop 2023-11-28T21:35:08.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f8fde5aa640 1 -- 172.21.15.12:0/147031449 shutdown_connections 2023-11-28T21:35:08.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f8fde5aa640 1 --2- 172.21.15.12:0/147031449 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd80cbca0 0x7f8fd80c4be0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f8fde5aa640 1 --2- 172.21.15.12:0/147031449 >> v2:172.21.15.144:3300/0 conn(0x7f8fd80cb880 0x7f8fd80c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.788+0000 7f8fde5aa640 1 -- 172.21.15.12:0/147031449 >> 172.21.15.12:0/147031449 conn(0x7f8fd8071ce0 msgr2=0x7f8fd80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f8fde5aa640 1 -- 172.21.15.12:0/147031449 shutdown_connections 2023-11-28T21:35:08.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3172900093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e601540f0 msgr2=0x7f1e600c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f1e65d2a640 1 --2- 172.21.15.12:0/3172900093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e601540f0 0x7f1e600c3be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009d90 tx=0x7f1e5801c960 comp rx=0 tx=0).stop 2023-11-28T21:35:08.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3172900093 shutdown_connections 2023-11-28T21:35:08.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f1e65d2a640 1 --2- 172.21.15.12:0/3172900093 >> v2:172.21.15.12:3300/0 conn(0x7f1e6015d210 0x7f1e6015d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f1e65d2a640 1 --2- 172.21.15.12:0/3172900093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e601540f0 0x7f1e600c3be0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3172900093 >> 172.21.15.12:0/3172900093 conn(0x7f1e60066360 msgr2=0x7f1e60066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:08.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fe57ddf6640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe578066e80 0x7fe5781ecad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe56801c8a0 tx=0x7fe56801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3172900093 shutdown_connections 2023-11-28T21:35:08.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3172900093 wait complete. 2023-11-28T21:35:08.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fe56e7fc640 1 -- 172.21.15.12:0/801682324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680ba610 con 0x7fe578066e80 2023-11-28T21:35:08.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fe56e7fc640 1 -- 172.21.15.12:0/801682324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5680ba770 con 0x7fe578066e80 2023-11-28T21:35:08.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fe56e7fc640 1 -- 172.21.15.12:0/801682324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680bab20 con 0x7fe578066e80 2023-11-28T21:35:08.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f8fde5aa640 1 -- 172.21.15.12:0/147031449 wait complete. 2023-11-28T21:35:08.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fea4b5df640 1 Processor -- start 2023-11-28T21:35:08.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe578159fd0 con 0x7fe578066e80 2023-11-28T21:35:08.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe578070720 con 0x7fe578066e80 2023-11-28T21:35:08.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f8fde5aa640 1 Processor -- start 2023-11-28T21:35:08.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f8fde5aa640 1 -- start start 2023-11-28T21:35:08.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe578067c70 con 0x7fe578066e80 2023-11-28T21:35:08.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f8fde5aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd80cb880 0x7f8fd81ef570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7f8fde5aa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd8153f40 0x7f8fd81fbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fea4b5df640 1 -- start start 2023-11-28T21:35:08.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fea4b5df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4414fd80 0x7fea44143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fea4b5df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44150160 0x7fea4414a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fea4b5df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea44159250 0x7fea4414a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fea4b5df640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea440c19a0 con 0x7fea44150160 2023-11-28T21:35:08.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fea4b5df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea440c1e70 con 0x7fea44159250 2023-11-28T21:35:08.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fea4b5df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea440c1bd0 con 0x7fea4414fd80 2023-11-28T21:35:08.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fd7fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd80cb880 0x7f8fd81ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.792+0000 7fea4937b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4414fd80 0x7fea44143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea4937b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4414fd80 0x7fea44143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56576/0 (socket says 172.21.15.12:56576) 2023-11-28T21:35:08.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea4937b640 1 -- 172.21.15.12:0/3830737480 learned_addr learned my addr 172.21.15.12:0/3830737480 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fd7fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd80cb880 0x7f8fd81ef570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56588/0 (socket says 172.21.15.12:56588) 2023-11-28T21:35:08.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea48b7a640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44150160 0x7fea4414a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea4937b640 1 -- 172.21.15.12:0/3830737480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea44159250 msgr2=0x7fea4414a700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T21:35:08.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f1e65d2a640 1 Processor -- start 2023-11-28T21:35:08.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fd7fff640 1 -- 172.21.15.12:0/2928461208 learned_addr learned my addr 172.21.15.12:0/2928461208 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fd77fe640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd8153f40 0x7f8fd81fbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fde5aa640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd81fc4c0 0x7f8fd815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f1e65d2a640 1 -- start start 2023-11-28T21:35:08.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fdcb47640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd81fc4c0 0x7f8fd815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd80cdd40 con 0x7f8fd81fc4c0 2023-11-28T21:35:08.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd80cd730 con 0x7f8fd8153f40 2023-11-28T21:35:08.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd80b53f0 con 0x7f8fd80cb880 2023-11-28T21:35:08.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea4937b640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea44159250 0x7fea4414a700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea4937b640 1 -- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44150160 msgr2=0x7fea4414a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea4937b640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44150160 0x7fea4414a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea4937b640 1 -- 172.21.15.12:0/3830737480 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c009a70 con 0x7fea4414fd80 2023-11-28T21:35:08.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea48b7a640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44150160 0x7fea4414a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:08.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fd77fe640 1 -- 172.21.15.12:0/2928461208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd80cb880 msgr2=0x7f8fd81ef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fd77fe640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd80cb880 0x7f8fd81ef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fd77fe640 1 -- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd81fc4c0 msgr2=0x7f8fd815c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fd77fe640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd81fc4c0 0x7f8fd815c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fd77fe640 1 -- 172.21.15.12:0/2928461208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd0009a70 con 0x7f8fd8153f40 2023-11-28T21:35:08.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fdcb47640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd81fc4c0 0x7f8fd815c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:08.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea4937b640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4414fd80 0x7fea44143e80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c4d0 tx=0x7fea4000c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f1e65d2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e601540f0 0x7f1e601560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f1e65d2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e6015b180 0x7f1e60142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f1e65d2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6015d210 0x7f1e60149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea39ffb640 1 -- 172.21.15.12:0/3830737480 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40004d30 con 0x7fea4414fd80 2023-11-28T21:35:08.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea39ffb640 1 -- 172.21.15.12:0/3830737480 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea40004e90 con 0x7fea4414fd80 2023-11-28T21:35:08.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7fea39ffb640 1 -- 172.21.15.12:0/3830737480 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400072d0 con 0x7fea4414fd80 2023-11-28T21:35:08.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f8fd77fe640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd8153f40 0x7f8fd81fbf80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00d5f0 tx=0x7f8fcc00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f1e65d2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e60156730 con 0x7f1e601540f0 2023-11-28T21:35:08.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f1e65d2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601f7190 con 0x7f1e6015b180 2023-11-28T21:35:08.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f1e65d2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e601f7470 con 0x7f1e6015d210 2023-11-28T21:35:08.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e601540f0 0x7f1e601560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e601540f0 0x7f1e601560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38308/0 (socket says 172.21.15.12:38308) 2023-11-28T21:35:08.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e5f7fe640 1 -- 172.21.15.12:0/3594207906 learned_addr learned my addr 172.21.15.12:0/3594207906 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:08.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e5ffff640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6015d210 0x7f1e60149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.796+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e6015b180 0x7f1e60142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fea4b5df640 1 -- 172.21.15.12:0/3830737480 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea441f7ef0 con 0x7fea4414fd80 2023-11-28T21:35:08.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e5ffff640 1 -- 172.21.15.12:0/3594207906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e6015b180 msgr2=0x7f1e60142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e5ffff640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e6015b180 0x7f1e60142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fea4b5df640 1 -- 172.21.15.12:0/3830737480 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea44151eb0 con 0x7fea4414fd80 2023-11-28T21:35:08.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e5ffff640 1 -- 172.21.15.12:0/3594207906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e601540f0 msgr2=0x7f1e601560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:08.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e5ffff640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e601540f0 0x7f1e601560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:08.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e5ffff640 1 -- 172.21.15.12:0/3594207906 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e58009a70 con 0x7f1e6015d210 2023-11-28T21:35:08.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fe56e7fc640 1 -- 172.21.15.12:0/801682324 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe5680b7050 con 0x7fe578066e80 2023-11-28T21:35:08.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e5ffff640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6015d210 0x7f1e60149450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000d530 tx=0x7f1e5000d9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e47fff640 1 -- 172.21.15.12:0/3594207906 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e500125f0 con 0x7f1e6015d210 2023-11-28T21:35:08.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fea4b5df640 1 -- 172.21.15.12:0/3830737480 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea44074f60 con 0x7fea4414fd80 2023-11-28T21:35:08.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e47fff640 1 -- 172.21.15.12:0/3594207906 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e50012750 con 0x7f1e6015d210 2023-11-28T21:35:08.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e47fff640 1 -- 172.21.15.12:0/3594207906 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e50012ac0 con 0x7f1e6015d210 2023-11-28T21:35:08.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3594207906 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e601f7750 con 0x7f1e6015d210 2023-11-28T21:35:08.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3594207906 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e601f7d10 con 0x7f1e6015d210 2023-11-28T21:35:08.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fe56e7fc640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe55c066540 0x7fe55c068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f8fd4ff9640 1 -- 172.21.15.12:0/2928461208 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0125f0 con 0x7f8fd8153f40 2023-11-28T21:35:08.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fe56e7fc640 1 -- 172.21.15.12:0/801682324 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe568122980 con 0x7fe578066e80 2023-11-28T21:35:08.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3594207906 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e60074f60 con 0x7f1e6015d210 2023-11-28T21:35:08.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fe57d5f5640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe55c066540 0x7fe55c068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f9962ffd640 1 -- 172.21.15.12:0/2982796179 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f99680ea620 con 0x7f996c00d570 2023-11-28T21:35:08.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fc7a4ff9640 1 -- 172.21.15.12:0/1811264794 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00b0440 con 0x7fc7a806d580 2023-11-28T21:35:08.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fc7a4ff9640 1 -- 172.21.15.12:0/1811264794 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a00ba820 con 0x7fc7a806d580 2023-11-28T21:35:08.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fc7a4ff9640 1 -- 172.21.15.12:0/1811264794 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00babd0 con 0x7fc7a806d580 2023-11-28T21:35:08.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7fc7ae46b640 1 -- 172.21.15.12:0/1811264794 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a81fd2a0 con 0x7fc7a806d580 2023-11-28T21:35:08.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7fc7ae46b640 1 -- 172.21.15.12:0/1811264794 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a81442a0 con 0x7fc7a806d580 2023-11-28T21:35:08.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f8fd4ff9640 1 -- 172.21.15.12:0/2928461208 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc012750 con 0x7f8fd8153f40 2023-11-28T21:35:08.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.800+0000 7f8fd4ff9640 1 -- 172.21.15.12:0/2928461208 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc012b00 con 0x7f8fd8153f40 2023-11-28T21:35:08.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd80b56d0 con 0x7f8fd8153f40 2023-11-28T21:35:08.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd80b4b20 con 0x7f8fd8153f40 2023-11-28T21:35:08.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7fc7ae46b640 1 -- 172.21.15.12:0/1811264794 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a8144460 con 0x7fc7a806d580 2023-11-28T21:35:08.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd80cdeb0 con 0x7f8fd8153f40 2023-11-28T21:35:08.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7f1e47fff640 1 -- 172.21.15.12:0/3594207906 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1e50012c20 con 0x7f1e6015d210 2023-11-28T21:35:08.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7fe56e7fc640 1 -- 172.21.15.12:0/801682324 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe5680efa60 con 0x7fe578066e80 2023-11-28T21:35:08.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7fe57d5f5640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe55c066540 0x7fe55c068a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe578149f80 tx=0x7fe570003dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7f8fd4ff9640 1 -- 172.21.15.12:0/2928461208 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8fcc012c60 con 0x7f8fd8153f40 2023-11-28T21:35:08.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7f1e47fff640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1e4c066470 0x7f1e4c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7fc7a4ff9640 1 -- 172.21.15.12:0/1811264794 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc7a00b7050 con 0x7fc7a806d580 2023-11-28T21:35:08.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7f1e5f7fe640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1e4c066470 0x7f1e4c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7f8fd4ff9640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fc40664c0 0x7f8fc4068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.804+0000 7fea39ffb640 1 -- 172.21.15.12:0/3830737480 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fea40007430 con 0x7fea4414fd80 2023-11-28T21:35:08.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7fea39ffb640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea280664a0 0x7fea28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7fea39ffb640 1 -- 172.21.15.12:0/3830737480 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fea400836c0 con 0x7fea4414fd80 2023-11-28T21:35:08.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7fea48b7a640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea280664a0 0x7fea28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7f8fd4ff9640 1 -- 172.21.15.12:0/2928461208 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8fcc085100 con 0x7f8fd8153f40 2023-11-28T21:35:08.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7f8fd7fff640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fc40664c0 0x7f8fc4068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7fea48b7a640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea280664a0 0x7fea28068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea441466a0 tx=0x7fea34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7fc7a4ff9640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc794066540 0x7fc794068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:08.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7fc7a4ff9640 1 -- 172.21.15.12:0/1811264794 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc7a0123590 con 0x7fc7a806d580 2023-11-28T21:35:08.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7fc7a7fff640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc794066540 0x7fc794068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:08.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7f1e5f7fe640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1e4c066470 0x7f1e4c068930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1e58000f90 tx=0x7f1e580a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7f8fd7fff640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fc40664c0 0x7f8fc4068980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80011f0 tx=0x7f8fc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.808+0000 7fc7a7fff640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc794066540 0x7fc794068a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00c940 tx=0x7fc79c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:08.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.816+0000 7f1e47fff640 1 -- 172.21.15.12:0/3594207906 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1e50055ed0 con 0x7f1e6015d210 2023-11-28T21:35:08.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.816+0000 7fea39ffb640 1 -- 172.21.15.12:0/3830737480 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fea40050850 con 0x7fea4414fd80 2023-11-28T21:35:08.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.820+0000 7fc7a4ff9640 1 -- 172.21.15.12:0/1811264794 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc7a00f0790 con 0x7fc7a806d580 2023-11-28T21:35:08.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.828+0000 7f1e47fff640 1 -- 172.21.15.12:0/3594207906 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1e50055ca0 con 0x7f1e6015d210 2023-11-28T21:35:08.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:08.832+0000 7f8fd4ff9640 1 -- 172.21.15.12:0/2928461208 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8fcc052270 con 0x7f8fd8153f40 2023-11-28T21:35:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.016+0000 7fd508a88640 1 Processor -- start 2023-11-28T21:35:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.016+0000 7fd508a88640 1 -- start start 2023-11-28T21:35:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.016+0000 7fd508a88640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50406a810 0x7fd50406abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd508a88640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5041674d0 con 0x7fd50408c670 2023-11-28T21:35:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd508a88640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd504167770 con 0x7fd504164270 2023-11-28T21:35:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd508a88640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040c2d80 con 0x7fd50406a810 2023-11-28T21:35:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd503fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd504164270 0x7fd5040c3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52736/0 (socket says 172.21.15.12:52736) 2023-11-28T21:35:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd503fff640 1 -- 172.21.15.12:0/1799959701 learned_addr learned my addr 172.21.15.12:0/1799959701 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:09.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd5037fe640 1 --2- 172.21.15.12:0/1799959701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50406a810 0x7fd50406abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd5027fc640 1 -- 172.21.15.12:0/1799959701 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341613485 0 0) 0x7fd504167770 con 0x7fd504164270 2023-11-28T21:35:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd5027fc640 1 -- 172.21.15.12:0/1799959701 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4ec003530 con 0x7fd504164270 2023-11-28T21:35:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd5037fe640 1 -- 172.21.15.12:0/1799959701 >> v1:172.21.15.144:6789/0 conn(0x7fd504164270 legacy=0x7fd5040c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd5037fe640 1 -- 172.21.15.12:0/1799959701 >> v1:172.21.15.12:6789/0 conn(0x7fd50408c670 legacy=0x7fd504163b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:09.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.020+0000 7fd5037fe640 1 -- 172.21.15.12:0/1799959701 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5040c3060 con 0x7fd50406a810 2023-11-28T21:35:09.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.024+0000 7fd5037fe640 1 --2- 172.21.15.12:0/1799959701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50406a810 0x7fd50406abf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009dc0 tx=0x7fd4fc01f140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ed10a5ce0d24d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:09.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.024+0000 7fd5027fc640 1 -- 172.21.15.12:0/1799959701 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc0af600 con 0x7fd50406a810 2023-11-28T21:35:09.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.024+0000 7fd5027fc640 1 -- 172.21.15.12:0/1799959701 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc0ab050 con 0x7fd50406a810 2023-11-28T21:35:09.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.028+0000 7fd508a88640 1 -- 172.21.15.12:0/1799959701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50406a810 msgr2=0x7fd50406abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.028+0000 7fd508a88640 1 --2- 172.21.15.12:0/1799959701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50406a810 0x7fd50406abf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc009dc0 tx=0x7fd4fc01f140 comp rx=0 tx=0).stop 2023-11-28T21:35:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.028+0000 7fd508a88640 1 -- 172.21.15.12:0/1799959701 shutdown_connections 2023-11-28T21:35:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.028+0000 7fd508a88640 1 --2- 172.21.15.12:0/1799959701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50406a810 0x7fd50406abf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.028+0000 7fd508a88640 1 -- 172.21.15.12:0/1799959701 >> 172.21.15.12:0/1799959701 conn(0x7fd504066360 msgr2=0x7fd504066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.028+0000 7fd508a88640 1 -- 172.21.15.12:0/1799959701 shutdown_connections 2023-11-28T21:35:09.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.028+0000 7fd508a88640 1 -- 172.21.15.12:0/1799959701 wait complete. 2023-11-28T21:35:09.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.028+0000 7fd508a88640 1 Processor -- start 2023-11-28T21:35:09.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd508a88640 1 -- start start 2023-11-28T21:35:09.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd508a88640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd50406a810 0x7fd5040c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:09.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd508a88640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50408c670 0x7fd504156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:09.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd508a88640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd504164270 0x7fd504157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:09.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd5037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd50406a810 0x7fd5040c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:09.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd502ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50408c670 0x7fd504156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:09.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd503fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd504164270 0x7fd504157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:09.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd502ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50408c670 0x7fd504156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56630/0 (socket says 172.21.15.12:56630) 2023-11-28T21:35:09.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd502ffd640 1 -- 172.21.15.12:0/2793811873 learned_addr learned my addr 172.21.15.12:0/2793811873 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd508a88640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd504163140 con 0x7fd504164270 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd504156550 con 0x7fd50406a810 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd504163630 con 0x7fd50408c670 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd502ffd640 1 -- 172.21.15.12:0/2793811873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd50406a810 msgr2=0x7fd5040c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd502ffd640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd50406a810 0x7fd5040c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd502ffd640 1 -- 172.21.15.12:0/2793811873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd504164270 msgr2=0x7fd504157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd502ffd640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd504164270 0x7fd504157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd502ffd640 1 -- 172.21.15.12:0/2793811873 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc009a70 con 0x7fd50408c670 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.032+0000 7fd502ffd640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50408c670 0x7fd504156f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c410 tx=0x7fd4f400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.036+0000 7fd4ebfff640 1 -- 172.21.15.12:0/2793811873 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4015a80 con 0x7fd50408c670 2023-11-28T21:35:09.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.036+0000 7fd4ebfff640 1 -- 172.21.15.12:0/2793811873 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f4015be0 con 0x7fd50408c670 2023-11-28T21:35:09.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.036+0000 7fd4ebfff640 1 -- 172.21.15.12:0/2793811873 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4003600 con 0x7fd50408c670 2023-11-28T21:35:09.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.036+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd504156a90 con 0x7fd50408c670 2023-11-28T21:35:09.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.040+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd504156760 con 0x7fd50408c670 2023-11-28T21:35:09.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.040+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd504067c70 con 0x7fd50408c670 2023-11-28T21:35:09.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.044+0000 7fd4ebfff640 1 -- 172.21.15.12:0/2793811873 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd4f401b020 con 0x7fd50408c670 2023-11-28T21:35:09.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.044+0000 7fd4ebfff640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd4f00664c0 0x7fd4f0068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:09.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.044+0000 7fd4ebfff640 1 -- 172.21.15.12:0/2793811873 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd4f4029e20 con 0x7fd50408c670 2023-11-28T21:35:09.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.044+0000 7fd5037fe640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd4f00664c0 0x7fd4f0068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:09.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.044+0000 7fd5037fe640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd4f00664c0 0x7fd4f0068980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0b1000 tx=0x7fd4fc0aa000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:09.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.052+0000 7fd4ebfff640 1 -- 172.21.15.12:0/2793811873 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd4f4059a30 con 0x7fd50408c670 2023-11-28T21:35:09.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.072+0000 7f4b47879640 1 -- 172.21.15.12:0/2912880715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4b20005510 con 0x7f4b400ca3f0 2023-11-28T21:35:09.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.076+0000 7f4b35ffb640 1 -- 172.21.15.12:0/2912880715 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b300ee8d0 con 0x7f4b400ca3f0 2023-11-28T21:35:09.077 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-11-28T21:35:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.076+0000 7f4b47879640 1 -- 172.21.15.12:0/2912880715 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4b240664d0 msgr2=0x7f4b24068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.076+0000 7f4b47879640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4b240664d0 0x7f4b24068990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009b80 tx=0x7f4b3c009190 comp rx=0 tx=0).stop 2023-11-28T21:35:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.076+0000 7f4b47879640 1 -- 172.21.15.12:0/2912880715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b400ca3f0 msgr2=0x7f4b4015c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.076+0000 7f4b47879640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b400ca3f0 0x7f4b4015c3b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4b300ab000 tx=0x7f4b3001ad10 comp rx=0 tx=0).stop 2023-11-28T21:35:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.080+0000 7f4b47879640 1 -- 172.21.15.12:0/2912880715 shutdown_connections 2023-11-28T21:35:09.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.080+0000 7f4b47879640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4b240664d0 0x7f4b24068990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.080+0000 7f4b47879640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b40154100 0x7f4b400c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.080+0000 7f4b47879640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b400ca3f0 0x7f4b4015c3b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.080+0000 7f4b47879640 1 --2- 172.21.15.12:0/2912880715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b400c9fd0 0x7f4b400c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.080+0000 7f4b47879640 1 -- 172.21.15.12:0/2912880715 >> 172.21.15.12:0/2912880715 conn(0x7f4b4006c690 msgr2=0x7f4b401592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:09.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.080+0000 7f4b47879640 1 -- 172.21.15.12:0/2912880715 shutdown_connections 2023-11-28T21:35:09.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.080+0000 7f4b47879640 1 -- 172.21.15.12:0/2912880715 wait complete. 2023-11-28T21:35:09.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-11-28T21:35:09.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.124+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f996c047990 con 0x7f996c00d570 2023-11-28T21:35:09.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.124+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe578067c70 con 0x7fe578066e80 2023-11-28T21:35:09.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.124+0000 7fe56e7fc640 1 -- 172.21.15.12:0/801682324 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe5680b5070 con 0x7fe578066e80 2023-11-28T21:35:09.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.124+0000 7f9962ffd640 1 -- 172.21.15.12:0/2982796179 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f99680edc40 con 0x7f996c00d570 2023-11-28T21:35:09.131 INFO:teuthology.orchestra.run.smithi012.stdout:55834574856 2023-11-28T21:35:09.132 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-11-28T21:35:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe55c066540 msgr2=0x7fe55c068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe55c066540 0x7fe55c068a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe578149f80 tx=0x7fe570003dd0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 msgr2=0x7f9954068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 0x7f9954068930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9964001c50 tx=0x7f9964001e30 comp rx=0 tx=0).stop 2023-11-28T21:35:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c00d570 msgr2=0x7f996c048250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c00d570 0x7f996c048250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99680ac000 tx=0x7f995c005800 comp rx=0 tx=0).stop 2023-11-28T21:35:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe578066e80 msgr2=0x7fe5781ecad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe578066e80 0x7fe5781ecad0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe56801c8a0 tx=0x7fe56801ba10 comp rx=0 tx=0).stop 2023-11-28T21:35:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 shutdown_connections 2023-11-28T21:35:09.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe55c066540 0x7fe55c068a00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe57814ff70 0x7fe5780720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe57806b260 0x7fe578159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 --2- 172.21.15.12:0/801682324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe578066e80 0x7fe5781ecad0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 >> 172.21.15.12:0/801682324 conn(0x7fe578066360 msgr2=0x7fe5780c3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:09.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 shutdown_connections 2023-11-28T21:35:09.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 0x7f9954068930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c00d9b0 0x7f996c048790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c00d570 0x7f996c048250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 --2- 172.21.15.12:0/2982796179 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f995c0047d0 0x7f996c00c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 >> 172.21.15.12:0/2982796179 conn(0x7f996c0545f0 msgr2=0x7f996c0060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:09.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 shutdown_connections 2023-11-28T21:35:09.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 shutdown_connections 2023-11-28T21:35:09.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7f9978cbb640 1 -- 172.21.15.12:0/2982796179 wait complete. 2023-11-28T21:35:09.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.128+0000 7fe57edf8640 1 -- 172.21.15.12:0/801682324 wait complete. 2023-11-28T21:35:09.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.140+0000 7fea4b5df640 1 -- 172.21.15.12:0/3830737480 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fea4414bfc0 con 0x7fea4414fd80 2023-11-28T21:35:09.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.144+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8fd8074f60 con 0x7f8fd8153f40 2023-11-28T21:35:09.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.144+0000 7fea39ffb640 1 -- 172.21.15.12:0/3830737480 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fea40018020 con 0x7fea4414fd80 2023-11-28T21:35:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:55834574856 2023-11-28T21:35:09.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.144+0000 7f8fd4ff9640 1 -- 172.21.15.12:0/2928461208 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8fcc055890 con 0x7f8fd8153f40 2023-11-28T21:35:09.146 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-11-28T21:35:09.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 -- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea280664a0 msgr2=0x7fea28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea280664a0 0x7fea28068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea441466a0 tx=0x7fea34008040 comp rx=0 tx=0).stop 2023-11-28T21:35:09.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 -- 172.21.15.12:0/3830737480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4414fd80 msgr2=0x7fea44143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4414fd80 0x7fea44143e80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c4d0 tx=0x7fea4000c990 comp rx=0 tx=0).stop 2023-11-28T21:35:09.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fc40664c0 msgr2=0x7f8fc4068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fc40664c0 0x7f8fc4068980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80011f0 tx=0x7f8fc8009040 comp rx=0 tx=0).stop 2023-11-28T21:35:09.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd8153f40 msgr2=0x7f8fd81fbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd8153f40 0x7f8fd81fbf80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc00d5f0 tx=0x7f8fcc00dab0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 -- 172.21.15.12:0/3830737480 shutdown_connections 2023-11-28T21:35:09.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea280664a0 0x7fea28068960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea44159250 0x7fea4414a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44150160 0x7fea4414a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 --2- 172.21.15.12:0/3830737480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4414fd80 0x7fea44143e80 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c4d0 tx=0x7fea4000c990 comp rx=0 tx=0).stop 2023-11-28T21:35:09.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 -- 172.21.15.12:0/3830737480 >> 172.21.15.12:0/3830737480 conn(0x7fea44066360 msgr2=0x7fea44145c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:09.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 -- 172.21.15.12:0/3830737480 shutdown_connections 2023-11-28T21:35:09.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7fea4b5df640 1 -- 172.21.15.12:0/3830737480 wait complete. 2023-11-28T21:35:09.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 shutdown_connections 2023-11-28T21:35:09.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8fc40664c0 0x7f8fc4068980 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fd81fc4c0 0x7f8fd815c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd8153f40 0x7f8fd81fbf80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 --2- 172.21.15.12:0/2928461208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fd80cb880 0x7f8fd81ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 >> 172.21.15.12:0/2928461208 conn(0x7f8fd8071ce0 msgr2=0x7f8fd80c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:09.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 shutdown_connections 2023-11-28T21:35:09.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.148+0000 7f8fde5aa640 1 -- 172.21.15.12:0/2928461208 wait complete. 2023-11-28T21:35:09.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.156+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3594207906 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1e601f75b0 con 0x7f1e6015d210 2023-11-28T21:35:09.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7fc7ae46b640 1 -- 172.21.15.12:0/1811264794 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc7a8074f60 con 0x7fc7a806d580 2023-11-28T21:35:09.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e47fff640 1 -- 172.21.15.12:0/3594207906 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1e601f75b0 con 0x7f1e6015d210 2023-11-28T21:35:09.162 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-11-28T21:35:09.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3594207906 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1e4c066470 msgr2=0x7f1e4c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1e4c066470 0x7f1e4c068930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1e58000f90 tx=0x7f1e580a7070 comp rx=0 tx=0).stop 2023-11-28T21:35:09.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3594207906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6015d210 msgr2=0x7f1e60149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6015d210 0x7f1e60149450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000d530 tx=0x7f1e5000d9f0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3594207906 shutdown_connections 2023-11-28T21:35:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1e4c066470 0x7f1e4c068930 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1e58000f90 tx=0x7f1e580a7070 comp rx=0 tx=0).stop 2023-11-28T21:35:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e6015d210 0x7f1e60149450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e6015b180 0x7f1e60142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 --2- 172.21.15.12:0/3594207906 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e601540f0 0x7f1e601560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3594207906 >> 172.21.15.12:0/3594207906 conn(0x7f1e60066360 msgr2=0x7f1e6015b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:09.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7fc7a4ff9640 1 -- 172.21.15.12:0/1811264794 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc7a00b5090 con 0x7fc7a806d580 2023-11-28T21:35:09.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.160+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3594207906 shutdown_connections 2023-11-28T21:35:09.166 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-11-28T21:35:09.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.164+0000 7f1e65d2a640 1 -- 172.21.15.12:0/3594207906 wait complete. 2023-11-28T21:35:09.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2023-11-28T21:35:09.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2023-11-28T21:35:09.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.164+0000 7fc7ae46b640 1 -- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc794066540 msgr2=0x7fc794068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc794066540 0x7fc794068a00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00c940 tx=0x7fc79c008040 comp rx=0 tx=0).stop 2023-11-28T21:35:09.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 -- 172.21.15.12:0/1811264794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806d580 msgr2=0x7fc7a80722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806d580 0x7fc7a80722e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc7a001d7a0 tx=0x7fc7a001be90 comp rx=0 tx=0).stop 2023-11-28T21:35:09.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 -- 172.21.15.12:0/1811264794 shutdown_connections 2023-11-28T21:35:09.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc794066540 0x7fc794068a00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7a806d580 0x7fc7a80722e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7a806d1a0 0x7fc7a8200240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 --2- 172.21.15.12:0/1811264794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7a806b5c0 0x7fc7a81490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 -- 172.21.15.12:0/1811264794 >> 172.21.15.12:0/1811264794 conn(0x7fc7a8066360 msgr2=0x7fc7a81584f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:09.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 -- 172.21.15.12:0/1811264794 shutdown_connections 2023-11-28T21:35:09.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.168+0000 7fc7ae46b640 1 -- 172.21.15.12:0/1811264794 wait complete. 2023-11-28T21:35:09.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2023-11-28T21:35:09.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2023-11-28T21:35:09.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2023-11-28T21:35:09.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2023-11-28T21:35:09.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd504156c20 con 0x7fd50408c670 2023-11-28T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd4ebfff640 1 -- 172.21.15.12:0/2793811873 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd4f4023070 con 0x7fd50408c670 2023-11-28T21:35:09.275 INFO:teuthology.orchestra.run.smithi012.stdout:55834574856 2023-11-28T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd4f00664c0 msgr2=0x7fd4f0068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd4f00664c0 0x7fd4f0068980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0b1000 tx=0x7fd4fc0aa000 comp rx=0 tx=0).stop 2023-11-28T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50408c670 msgr2=0x7fd504156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50408c670 0x7fd504156f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c410 tx=0x7fd4f400c8d0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 shutdown_connections 2023-11-28T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd4f00664c0 0x7fd4f0068980 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd504164270 0x7fd504157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd50408c670 0x7fd504156f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 --2- 172.21.15.12:0/2793811873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd50406a810 0x7fd5040c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.272+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 >> 172.21.15.12:0/2793811873 conn(0x7fd504066360 msgr2=0x7fd5040c6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.276+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 shutdown_connections 2023-11-28T21:35:09.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:09.276+0000 7fd508a88640 1 -- 172.21.15.12:0/2793811873 wait complete. 2023-11-28T21:35:09.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2023-11-28T21:35:10.115 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T21:35:10.170 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T21:35:10.170 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T21:35:10.186 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T21:35:10.193 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T21:35:10.193 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T21:35:10.195 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T21:35:10.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.284+0000 7fafd09f5640 1 Processor -- start 2023-11-28T21:35:10.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.284+0000 7fafd09f5640 1 -- start start 2023-11-28T21:35:10.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.284+0000 7fafd09f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc154100 0x7fafcc15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.284+0000 7fafd09f5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fafcc0ca3f0 0x7fafcc0c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.284+0000 7fafd09f5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafcc15e270 con 0x7fafcc0c9fd0 2023-11-28T21:35:10.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafd09f5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafcc0c4e60 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafd09f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1544e0 con 0x7fafcc154100 2023-11-28T21:35:10.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafc9d9b640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fafcc0c9fd0 0x7fafcc06a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52752/0 (socket says 172.21.15.12:52752) 2023-11-28T21:35:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafc9d9b640 1 -- 172.21.15.12:0/3389052138 learned_addr learned my addr 172.21.15.12:0/3389052138 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafcad9d640 1 --2- 172.21.15.12:0/3389052138 >> v2:172.21.15.12:3300/0 conn(0x7fafcc0ca3f0 0x7fafcc0c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafc959a640 1 -- 172.21.15.12:0/3389052138 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449437541 0 0) 0x7fafcc15e270 con 0x7fafcc0c9fd0 2023-11-28T21:35:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafc959a640 1 -- 172.21.15.12:0/3389052138 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb4003530 con 0x7fafcc0c9fd0 2023-11-28T21:35:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafcad9d640 1 -- 172.21.15.12:0/3389052138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc154100 msgr2=0x7fafcc15d320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T21:35:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafcad9d640 1 --2- 172.21.15.12:0/3389052138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc154100 0x7fafcc15d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafcad9d640 1 -- 172.21.15.12:0/3389052138 >> v1:172.21.15.144:6789/0 conn(0x7fafcc0c9fd0 legacy=0x7fafcc06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.288+0000 7fafcad9d640 1 -- 172.21.15.12:0/3389052138 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc15d9d0 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.292+0000 7fafcad9d640 1 --2- 172.21.15.12:0/3389052138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafcc0ca3f0 0x7fafcc0c52f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009dc0 tx=0x7fafc401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c02e1619264e0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.297 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T21:35:10.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafc959a640 1 -- 172.21.15.12:0/3389052138 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc40a59e0 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafc959a640 1 -- 172.21.15.12:0/3389052138 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc40a5b40 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafc959a640 1 -- 172.21.15.12:0/3389052138 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc40ad440 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafd09f5640 1 -- 172.21.15.12:0/3389052138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafcc0ca3f0 msgr2=0x7fafcc0c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafd09f5640 1 --2- 172.21.15.12:0/3389052138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafcc0ca3f0 0x7fafcc0c52f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009dc0 tx=0x7fafc401c920 comp rx=0 tx=0).stop 2023-11-28T21:35:10.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafd09f5640 1 -- 172.21.15.12:0/3389052138 shutdown_connections 2023-11-28T21:35:10.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafd09f5640 1 --2- 172.21.15.12:0/3389052138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafcc0ca3f0 0x7fafcc0c52f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafd09f5640 1 --2- 172.21.15.12:0/3389052138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc154100 0x7fafcc15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafd09f5640 1 -- 172.21.15.12:0/3389052138 >> 172.21.15.12:0/3389052138 conn(0x7fafcc06c690 msgr2=0x7fafcc066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafd09f5640 1 -- 172.21.15.12:0/3389052138 shutdown_connections 2023-11-28T21:35:10.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.296+0000 7fafd09f5640 1 -- 172.21.15.12:0/3389052138 wait complete. 2023-11-28T21:35:10.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.300+0000 7fafd09f5640 1 Processor -- start 2023-11-28T21:35:10.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.300+0000 7fafd09f5640 1 -- start start 2023-11-28T21:35:10.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.300+0000 7fafd09f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafcc0c9fd0 0x7fafcc0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.300+0000 7fafd09f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc0ca3f0 0x7fafcc0c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.300+0000 7fafd09f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafcc154100 0x7fafcc156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.300+0000 7fafd09f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0c7750 con 0x7fafcc154100 2023-11-28T21:35:10.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.300+0000 7fafd09f5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc15d0a0 con 0x7fafcc0c9fd0 2023-11-28T21:35:10.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.300+0000 7fafd09f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0c7190 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.300+0000 7fafc9d9b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc0ca3f0 0x7fafcc0c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.300+0000 7fafca59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafcc0c9fd0 0x7fafcc0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafc9d9b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc0ca3f0 0x7fafcc0c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56648/0 (socket says 172.21.15.12:56648) 2023-11-28T21:35:10.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafc9d9b640 1 -- 172.21.15.12:0/816084503 learned_addr learned my addr 172.21.15.12:0/816084503 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafcad9d640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafcc154100 0x7fafcc156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafc9d9b640 1 -- 172.21.15.12:0/816084503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafcc0c9fd0 msgr2=0x7fafcc0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafc9d9b640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafcc0c9fd0 0x7fafcc0c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafc9d9b640 1 -- 172.21.15.12:0/816084503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafcc154100 msgr2=0x7fafcc156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafc9d9b640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafcc154100 0x7fafcc156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafc9d9b640 1 -- 172.21.15.12:0/816084503 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc4009a70 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafc9d9b640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc0ca3f0 0x7fafcc0c67c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c420 tx=0x7fafc000c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafbaffd640 1 -- 172.21.15.12:0/816084503 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0005cf0 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafbaffd640 1 -- 172.21.15.12:0/816084503 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc0005e50 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafbaffd640 1 -- 172.21.15.12:0/816084503 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00072d0 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafd09f5640 1 -- 172.21.15.12:0/816084503 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc0c7470 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.304+0000 7fafd09f5640 1 -- 172.21.15.12:0/816084503 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc156390 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.308+0000 7fafbaffd640 1 -- 172.21.15.12:0/816084503 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fafc0007430 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.308+0000 7fafbaffd640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fafac066470 0x7fafac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.308+0000 7fafbaffd640 1 -- 172.21.15.12:0/816084503 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fafc0083740 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.312+0000 7fafd09f5640 1 -- 172.21.15.12:0/816084503 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafcc067c40 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.316+0000 7fafca59c640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fafac066470 0x7fafac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.320+0000 7fafca59c640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fafac066470 0x7fafac068930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009890 tx=0x7fafbc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.324+0000 7fafbaffd640 1 -- 172.21.15.12:0/816084503 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fafc0054370 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.384+0000 7f38f6b9d640 1 Processor -- start 2023-11-28T21:35:10.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.384+0000 7f38f6b9d640 1 -- start start 2023-11-28T21:35:10.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7fe5d0db8640 1 Processor -- start 2023-11-28T21:35:10.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7fe5d0db8640 1 -- start start 2023-11-28T21:35:10.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7fe5d0db8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5cc158290 0x7fe5cc06ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7fe5d0db8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5cc06a9c0 con 0x7fe5cc158670 2023-11-28T21:35:10.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7fe5d0db8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5cc0c5650 con 0x7fe5cc0c7520 2023-11-28T21:35:10.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7fe5d0db8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5cc0c6700 con 0x7fe5cc158290 2023-11-28T21:35:10.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7f38f6b9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 0x7f38f01616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7f38f6b9d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f006a9c0 con 0x7f38f0158290 2023-11-28T21:35:10.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7f38f6b9d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f00c5650 con 0x7f38f006b560 2023-11-28T21:35:10.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7f38f6b9d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0162300 con 0x7f38f0158670 2023-11-28T21:35:10.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7f38f513a640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f38f006b560 0x7f38f0161bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52760/0 (socket says 172.21.15.12:52760) 2023-11-28T21:35:10.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7f38f513a640 1 -- 172.21.15.12:0/1915598385 learned_addr learned my addr 172.21.15.12:0/1915598385 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:10.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7f38effff640 1 --2- 172.21.15.12:0/1915598385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 0x7f38f01616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.392+0000 7fe5cad9d640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe5cc0c7520 0x7fe5cc1689e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52766/0 (socket says 172.21.15.12:52766) 2023-11-28T21:35:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.392+0000 7fe5ca59c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5cc158290 0x7fe5cc06ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.388+0000 7f38ef7fe640 1 -- 172.21.15.12:0/1915598385 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474842034 0 0) 0x7f38f00c5650 con 0x7f38f006b560 2023-11-28T21:35:10.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.392+0000 7fe5cad9d640 1 -- 172.21.15.12:0/550215590 learned_addr learned my addr 172.21.15.12:0/550215590 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:10.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.392+0000 7f38ef7fe640 1 -- 172.21.15.12:0/1915598385 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d8003530 con 0x7f38f006b560 2023-11-28T21:35:10.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.392+0000 7f38effff640 1 -- 172.21.15.12:0/1915598385 >> v1:172.21.15.144:6789/0 conn(0x7f38f006b560 legacy=0x7f38f0161bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.392+0000 7f38effff640 1 -- 172.21.15.12:0/1915598385 >> v1:172.21.15.12:6789/0 conn(0x7f38f0158290 legacy=0x7f38f006ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.392+0000 7f38effff640 1 -- 172.21.15.12:0/1915598385 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f00c6700 con 0x7f38f0158670 2023-11-28T21:35:10.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.396+0000 7fe5c959a640 1 -- 172.21.15.12:0/550215590 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672269040 0 0) 0x7fe5cc0c5650 con 0x7fe5cc0c7520 2023-11-28T21:35:10.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.396+0000 7fe5c959a640 1 -- 172.21.15.12:0/550215590 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5b4003530 con 0x7fe5cc0c7520 2023-11-28T21:35:10.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.396+0000 7fe5ca59c640 1 -- 172.21.15.12:0/550215590 >> v1:172.21.15.144:6789/0 conn(0x7fe5cc0c7520 legacy=0x7fe5cc1689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.396+0000 7f38effff640 1 --2- 172.21.15.12:0/1915598385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 0x7f38f01616b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009dc0 tx=0x7f38e801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b19aabc97cb8617 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.396+0000 7fe5ca59c640 1 -- 172.21.15.12:0/550215590 >> v1:172.21.15.12:6789/0 conn(0x7fe5cc158670 legacy=0x7fe5cc1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.396+0000 7fe5ca59c640 1 -- 172.21.15.12:0/550215590 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc162560 con 0x7fe5cc158290 2023-11-28T21:35:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.396+0000 7f38ef7fe640 1 -- 172.21.15.12:0/1915598385 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80a5930 con 0x7f38f0158670 2023-11-28T21:35:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.396+0000 7f38ef7fe640 1 -- 172.21.15.12:0/1915598385 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e80a5a90 con 0x7f38f0158670 2023-11-28T21:35:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.396+0000 7f38ef7fe640 1 -- 172.21.15.12:0/1915598385 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80a5e20 con 0x7f38f0158670 2023-11-28T21:35:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.400+0000 7f38f6b9d640 1 -- 172.21.15.12:0/1915598385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 msgr2=0x7f38f01616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.400+0000 7f38f6b9d640 1 --2- 172.21.15.12:0/1915598385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 0x7f38f01616b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009dc0 tx=0x7f38e801c890 comp rx=0 tx=0).stop 2023-11-28T21:35:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.400+0000 7f38f6b9d640 1 -- 172.21.15.12:0/1915598385 shutdown_connections 2023-11-28T21:35:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.400+0000 7f38f6b9d640 1 --2- 172.21.15.12:0/1915598385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 0x7f38f01616b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.400+0000 7f38f6b9d640 1 -- 172.21.15.12:0/1915598385 >> 172.21.15.12:0/1915598385 conn(0x7f38f006d4c0 msgr2=0x7f38f006d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.400+0000 7f38f6b9d640 1 -- 172.21.15.12:0/1915598385 shutdown_connections 2023-11-28T21:35:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.400+0000 7f38f6b9d640 1 -- 172.21.15.12:0/1915598385 wait complete. 2023-11-28T21:35:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.400+0000 7fe5ca59c640 1 --2- 172.21.15.12:0/550215590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158290 0x7fe5cc06ae50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0097c0 tx=0x7fe5bc01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d4d653d932f643a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7fe5c959a640 1 -- 172.21.15.12:0/550215590 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0ad680 con 0x7fe5cc158290 2023-11-28T21:35:10.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f6b9d640 1 Processor -- start 2023-11-28T21:35:10.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7fe5c959a640 1 -- 172.21.15.12:0/550215590 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0ad7e0 con 0x7fe5cc158290 2023-11-28T21:35:10.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7fe5c959a640 1 -- 172.21.15.12:0/550215590 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0adb70 con 0x7fe5cc158290 2023-11-28T21:35:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7fe5d0db8640 1 -- 172.21.15.12:0/550215590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158290 msgr2=0x7fe5cc06ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7fe5d0db8640 1 --2- 172.21.15.12:0/550215590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158290 0x7fe5cc06ae50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0097c0 tx=0x7fe5bc01c380 comp rx=0 tx=0).stop 2023-11-28T21:35:10.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7fe5d0db8640 1 -- 172.21.15.12:0/550215590 shutdown_connections 2023-11-28T21:35:10.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7fe5d0db8640 1 --2- 172.21.15.12:0/550215590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158290 0x7fe5cc06ae50 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0097c0 tx=0x7fe5bc01c380 comp rx=0 tx=0).stop 2023-11-28T21:35:10.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7fe5d0db8640 1 -- 172.21.15.12:0/550215590 >> 172.21.15.12:0/550215590 conn(0x7fe5cc06d4c0 msgr2=0x7fe5cc06d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f6b9d640 1 -- start start 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f6b9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f006b560 0x7f38f01548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f6b9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f0158290 0x7f38f0154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f6b9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 0x7f38f0145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f4939640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f006b560 0x7f38f01548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f6b9d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f006ffc0 con 0x7f38f0158290 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f6b9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f00c91c0 con 0x7f38f006b560 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f6b9d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f00cafb0 con 0x7f38f0158670 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f0158290 0x7f38f0154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f0158290 0x7f38f0154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38328/0 (socket says 172.21.15.12:38328) 2023-11-28T21:35:10.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38effff640 1 -- 172.21.15.12:0/2825521108 learned_addr learned my addr 172.21.15.12:0/2825521108 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f4939640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f006b560 0x7f38f01548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:39762/0 (socket says 172.21.15.12:39762) 2023-11-28T21:35:10.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7fe5d0db8640 1 -- 172.21.15.12:0/550215590 shutdown_connections 2023-11-28T21:35:10.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7fe5d0db8640 1 -- 172.21.15.12:0/550215590 wait complete. 2023-11-28T21:35:10.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f513a640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 0x7f38f0145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f4939640 1 -- 172.21.15.12:0/2825521108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 msgr2=0x7f38f0145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f4939640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 0x7f38f0145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f4939640 1 -- 172.21.15.12:0/2825521108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f0158290 msgr2=0x7f38f0154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f4939640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f0158290 0x7f38f0154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.404+0000 7f38f4939640 1 -- 172.21.15.12:0/2825521108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e8009a70 con 0x7f38f006b560 2023-11-28T21:35:10.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.408+0000 7f38f4939640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f006b560 0x7f38f01548e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38e0001ec0 tx=0x7f38e000c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.408+0000 7f38ed7fa640 1 -- 172.21.15.12:0/2825521108 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00052d0 con 0x7f38f006b560 2023-11-28T21:35:10.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.408+0000 7f38ed7fa640 1 -- 172.21.15.12:0/2825521108 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e0005430 con 0x7f38f006b560 2023-11-28T21:35:10.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.408+0000 7f38ed7fa640 1 -- 172.21.15.12:0/2825521108 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0003840 con 0x7f38f006b560 2023-11-28T21:35:10.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.408+0000 7f38f6b9d640 1 -- 172.21.15.12:0/2825521108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f0154170 con 0x7f38f006b560 2023-11-28T21:35:10.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.408+0000 7f38f6b9d640 1 -- 172.21.15.12:0/2825521108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f00caa40 con 0x7f38f006b560 2023-11-28T21:35:10.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.408+0000 7fe5d0db8640 1 Processor -- start 2023-11-28T21:35:10.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.408+0000 7fe5d0db8640 1 -- start start 2023-11-28T21:35:10.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.408+0000 7f38f6b9d640 1 -- 172.21.15.12:0/2825521108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f0067c70 con 0x7f38f006b560 2023-11-28T21:35:10.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5d0db8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc0c7520 0x7fe5cc154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5d0db8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158670 0x7fe5cc15fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5d0db8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc15b7b0 0x7fe5cc159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5ca59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc0c7520 0x7fe5cc154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5ca59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc0c7520 0x7fe5cc154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56682/0 (socket says 172.21.15.12:56682) 2023-11-28T21:35:10.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5c9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158670 0x7fe5cc15fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5c9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158670 0x7fe5cc15fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:39766/0 (socket says 172.21.15.12:39766) 2023-11-28T21:35:10.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5cad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc15b7b0 0x7fe5cc159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5cad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc15b7b0 0x7fe5cc159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38332/0 (socket says 172.21.15.12:38332) 2023-11-28T21:35:10.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5ca59c640 1 -- 172.21.15.12:0/1390430512 learned_addr learned my addr 172.21.15.12:0/1390430512 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc15ac20 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc159700 con 0x7fe5cc158670 2023-11-28T21:35:10.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc1599e0 con 0x7fe5cc0c7520 2023-11-28T21:35:10.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5cad9d640 1 -- 172.21.15.12:0/1390430512 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc0c7520 msgr2=0x7fe5cc154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5cad9d640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc0c7520 0x7fe5cc154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5cad9d640 1 -- 172.21.15.12:0/1390430512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158670 msgr2=0x7fe5cc15fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5cad9d640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158670 0x7fe5cc15fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7fe5cad9d640 1 -- 172.21.15.12:0/1390430512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc009420 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7fe5ca59c640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc0c7520 0x7fe5cc154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:35:10.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7fe5cad9d640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc15b7b0 0x7fe5cc159e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c390 tx=0x7fe5c000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7fe5c9d9b640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158670 0x7fe5cc15fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:35:10.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7fe5baffd640 1 -- 172.21.15.12:0/1390430512 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0005cf0 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7fe5baffd640 1 -- 172.21.15.12:0/1390430512 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c0005e50 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7fe5baffd640 1 -- 172.21.15.12:0/1390430512 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00072f0 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.412+0000 7f38ed7fa640 1 -- 172.21.15.12:0/2825521108 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f38e00039a0 con 0x7f38f006b560 2023-11-28T21:35:10.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7f38ed7fa640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f38dc0664f0 0x7f38dc0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5cc158b50 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7f38ed7fa640 1 -- 172.21.15.12:0/2825521108 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f38e0083ae0 con 0x7f38f006b560 2023-11-28T21:35:10.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc1590f0 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5cc067c70 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.416+0000 7f38effff640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f38dc0664f0 0x7f38dc0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.420+0000 7f38effff640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f38dc0664f0 0x7f38dc0689b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f38e8000f90 tx=0x7f38e80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.420+0000 7fe5baffd640 1 -- 172.21.15.12:0/1390430512 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe5c0007450 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.424+0000 7fe5baffd640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe5ac0664c0 0x7fe5ac068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.424+0000 7fe5ca59c640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe5ac0664c0 0x7fe5ac068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.424+0000 7fe5baffd640 1 -- 172.21.15.12:0/1390430512 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe5c0055100 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.424+0000 7f38ed7fa640 1 -- 172.21.15.12:0/2825521108 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f38e0050d70 con 0x7f38f006b560 2023-11-28T21:35:10.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.424+0000 7fe5ca59c640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe5ac0664c0 0x7fe5ac068980 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0b0040 tx=0x7fe5bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.428+0000 7fe5baffd640 1 -- 172.21.15.12:0/1390430512 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe5c00514d0 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.468+0000 7f5fabafd640 1 Processor -- start 2023-11-28T21:35:10.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.468+0000 7f5fabafd640 1 -- start start 2023-11-28T21:35:10.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.468+0000 7f5fabafd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5fa4158290 0x7f5fa406ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.468+0000 7f5fabafd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa406a9c0 con 0x7f5fa4158670 2023-11-28T21:35:10.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.468+0000 7f5fabafd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa40c5650 con 0x7f5fa40c7520 2023-11-28T21:35:10.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.468+0000 7f5fabafd640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa40c6700 con 0x7f5fa4158290 2023-11-28T21:35:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.472+0000 7f5fa9899640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5fa4158290 0x7f5fa406ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.472+0000 7f5fa9899640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5fa4158290 0x7f5fa406ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38342/0 (socket says 172.21.15.12:38342) 2023-11-28T21:35:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.472+0000 7f5fa9899640 1 -- 172.21.15.12:0/3795914090 learned_addr learned my addr 172.21.15.12:0/3795914090 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.472+0000 7f5fa9899640 1 -- 172.21.15.12:0/3795914090 >> v1:172.21.15.144:6789/0 conn(0x7f5fa40c7520 legacy=0x7f5fa41689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.472+0000 7f5fa9899640 1 -- 172.21.15.12:0/3795914090 >> v1:172.21.15.12:6789/0 conn(0x7f5fa4158670 legacy=0x7f5fa41616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.472+0000 7f5fa9899640 1 -- 172.21.15.12:0/3795914090 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa4162560 con 0x7f5fa4158290 2023-11-28T21:35:10.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.476+0000 7f13b8bd0640 1 Processor -- start 2023-11-28T21:35:10.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.476+0000 7f13b8bd0640 1 -- start start 2023-11-28T21:35:10.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.476+0000 7f5fa9899640 1 --2- 172.21.15.12:0/3795914090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa4158290 0x7f5fa406ae50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009870 tx=0x7f5f9401bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454490b69be085e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f5fa8897640 1 -- 172.21.15.12:0/3795914090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f940a56b0 con 0x7f5fa4158290 2023-11-28T21:35:10.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f5fa8897640 1 -- 172.21.15.12:0/3795914090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f940a5810 con 0x7f5fa4158290 2023-11-28T21:35:10.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f3ceab7d640 1 Processor -- start 2023-11-28T21:35:10.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f13b8bd0640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f13b406b260 0x7f13b4159c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f13b8bd0640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f13b40c5bb0 0x7f13b40c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f13b8bd0640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b415a1c0 con 0x7f13b414ff70 2023-11-28T21:35:10.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f13b8bd0640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b40c6180 con 0x7f13b40c5bb0 2023-11-28T21:35:10.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f13b8bd0640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b40c6460 con 0x7f13b406b260 2023-11-28T21:35:10.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f5fabafd640 1 -- 172.21.15.12:0/3795914090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa4158290 msgr2=0x7f5fa406ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f5fabafd640 1 --2- 172.21.15.12:0/3795914090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa4158290 0x7f5fa406ae50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009870 tx=0x7f5f9401bf00 comp rx=0 tx=0).stop 2023-11-28T21:35:10.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f3ceab7d640 1 -- start start 2023-11-28T21:35:10.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f5fabafd640 1 -- 172.21.15.12:0/3795914090 shutdown_connections 2023-11-28T21:35:10.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f5fabafd640 1 --2- 172.21.15.12:0/3795914090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa4158290 0x7f5fa406ae50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f5fabafd640 1 -- 172.21.15.12:0/3795914090 >> 172.21.15.12:0/3795914090 conn(0x7f5fa406d4c0 msgr2=0x7f5fa406d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f5fabafd640 1 -- 172.21.15.12:0/3795914090 shutdown_connections 2023-11-28T21:35:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f13b37fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f13b414ff70 0x7f13b406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36458/0 (socket says 172.21.15.12:36458) 2023-11-28T21:35:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f13b37fe640 1 -- 172.21.15.12:0/3195723467 learned_addr learned my addr 172.21.15.12:0/3195723467 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:10.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f5fabafd640 1 -- 172.21.15.12:0/3795914090 wait complete. 2023-11-28T21:35:10.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f3ceab7d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3ce41582c0 0x7f3ce406b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f3ceab7d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3ce40c6100 0x7f3ce40c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.480+0000 7f13b2ffd640 1 --2- 172.21.15.12:0/3195723467 >> v2:172.21.15.144:3300/0 conn(0x7f13b406b260 0x7f13b4159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7ff7494fe640 1 Processor -- start 2023-11-28T21:35:10.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f13b3fff640 1 --2- 172.21.15.12:0/3195723467 >> v2:172.21.15.12:3300/0 conn(0x7f13b40c5bb0 0x7f13b40c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7ff7494fe640 1 -- start start 2023-11-28T21:35:10.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f13b3fff640 1 -- 172.21.15.12:0/3195723467 >> v2:172.21.15.144:3300/0 conn(0x7f13b406b260 msgr2=0x7f13b4159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:10.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f13b3fff640 1 --2- 172.21.15.12:0/3195723467 >> v2:172.21.15.144:3300/0 conn(0x7f13b406b260 0x7f13b4159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f13b3fff640 1 -- 172.21.15.12:0/3195723467 >> v1:172.21.15.12:6789/0 conn(0x7f13b414ff70 legacy=0x7f13b406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f13b3fff640 1 -- 172.21.15.12:0/3195723467 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b40c6740 con 0x7f13b40c5bb0 2023-11-28T21:35:10.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f5fabafd640 1 Processor -- start 2023-11-28T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ceab7d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce40728b0 con 0x7f3ce40c6d50 2023-11-28T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ceab7d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce40c4c50 con 0x7f3ce40c6100 2023-11-28T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ceab7d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce40c5c70 con 0x7f3ce41582c0 2023-11-28T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce937a640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f3ce40c6d50 0x7f3ce406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52782/0 (socket says 172.21.15.12:52782) 2023-11-28T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce937a640 1 -- 172.21.15.12:0/783898989 learned_addr learned my addr 172.21.15.12:0/783898989 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3cea37c640 1 --2- 172.21.15.12:0/783898989 >> v2:172.21.15.12:3300/0 conn(0x7f3ce40c6100 0x7f3ce40c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce9b7b640 1 --2- 172.21.15.12:0/783898989 >> v2:172.21.15.144:3300/0 conn(0x7f3ce41582c0 0x7f3ce406b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087565398 0 0) 0x7f3ce40728b0 con 0x7f3ce40c6d50 2023-11-28T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd0003530 con 0x7f3ce40c6d50 2023-11-28T21:35:10.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7ff7494fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff73c0dd290 0x7ff73c0e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1429992371 0 0) 0x7f3cd0003530 con 0x7f3ce40c6d50 2023-11-28T21:35:10.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 >> v2:172.21.15.144:3300/0 conn(0x7f3ce41582c0 msgr2=0x7f3ce406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:10.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 --2- 172.21.15.12:0/783898989 >> v2:172.21.15.144:3300/0 conn(0x7f3ce41582c0 0x7f3ce406b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 >> v2:172.21.15.12:3300/0 conn(0x7f3ce40c6100 msgr2=0x7f3ce40c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:10.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 --2- 172.21.15.12:0/783898989 >> v2:172.21.15.12:3300/0 conn(0x7f3ce40c6100 0x7f3ce40c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce40c6a70 con 0x7f3ce40c6d50 2023-11-28T21:35:10.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f3ce0001680 con 0x7f3ce40c6d50 2023-11-28T21:35:10.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 >> v1:172.21.15.144:6789/0 conn(0x7f3ce40c6d50 legacy=0x7f3ce406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 --2- 172.21.15.12:0/783898989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 0x7f3cd0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 --2- 172.21.15.12:0/783898989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cd00047d0 0x7f3cd0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 --2- 172.21.15.12:0/783898989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd0005190 0x7f3cd000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce40c4c50 con 0x7f3cd0005190 2023-11-28T21:35:10.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce40c5c70 con 0x7f3cd00047d0 2023-11-28T21:35:10.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd000c060 con 0x7f3cd0003980 2023-11-28T21:35:10.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f5fabafd640 1 -- start start 2023-11-28T21:35:10.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7ff742ffd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff73c0dd290 0x7ff73c0e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7ff742ffd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff73c0dd290 0x7ff73c0e66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:39792/0 (socket says 172.21.15.12:39792) 2023-11-28T21:35:10.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7ff7427fc640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff73c0e71c0 0x7ff73c046270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:52790/0 (socket says 172.21.15.12:52790) 2023-11-28T21:35:10.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7ff742ffd640 1 -- 172.21.15.12:0/1689419691 learned_addr learned my addr 172.21.15.12:0/1689419691 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7ff7494fe640 1 -- 172.21.15.12:0/1689419691 --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c04c660 con 0x7ff73c0e6c00 2023-11-28T21:35:10.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7ff7494fe640 1 -- 172.21.15.12:0/1689419691 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c0454b0 con 0x7ff73c0e71c0 2023-11-28T21:35:10.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7ff7494fe640 1 -- 172.21.15.12:0/1689419691 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73c045790 con 0x7ff73c0dd290 2023-11-28T21:35:10.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.488+0000 7ff742ffd640 1 -- 172.21.15.12:0/1689419691 >> v1:172.21.15.144:6789/0 conn(0x7ff73c0e71c0 legacy=0x7ff73c046270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.488+0000 7ff742ffd640 1 -- 172.21.15.12:0/1689419691 >> v1:172.21.15.12:6789/0 conn(0x7ff73c0e6c00 legacy=0x7ff73c04ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:10.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.488+0000 7ff742ffd640 1 -- 172.21.15.12:0/1689419691 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c045a70 con 0x7ff73c0dd290 2023-11-28T21:35:10.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.484+0000 7f3ce937a640 1 --2- 172.21.15.12:0/783898989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cd00047d0 0x7f3cd0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.488+0000 7f3ce9b7b640 1 --2- 172.21.15.12:0/783898989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 0x7f3cd0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fabafd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa40c7520 0x7f5fa415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fabafd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa4158670 0x7f5fa415b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fabafd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fa415edb0 0x7f5fa4070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fabafd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa41f7a90 con 0x7f5fa4158670 2023-11-28T21:35:10.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fabafd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa415c7f0 con 0x7f5fa40c7520 2023-11-28T21:35:10.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fabafd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa41f7ca0 con 0x7f5fa415edb0 2023-11-28T21:35:10.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.488+0000 7f3ce9b7b640 1 -- 172.21.15.12:0/783898989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cd00047d0 msgr2=0x7f3cd0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:10.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.488+0000 7f3ce9b7b640 1 --2- 172.21.15.12:0/783898989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cd00047d0 0x7f3cd0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.488+0000 7f3ce9b7b640 1 -- 172.21.15.12:0/783898989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd0005190 msgr2=0x7f3cd000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:10.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.488+0000 7f3ce9b7b640 1 --2- 172.21.15.12:0/783898989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3cd0005190 0x7f3cd000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.488+0000 7f3ce9b7b640 1 -- 172.21.15.12:0/783898989 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce40c6a70 con 0x7f3cd0003980 2023-11-28T21:35:10.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f3cea37c640 1 -- 172.21.15.12:0/783898989 reap_dead start 2023-11-28T21:35:10.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fa9899640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa40c7520 0x7f5fa415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5faa09a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fa415edb0 0x7f5fa4070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fa9899640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa40c7520 0x7f5fa415bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:39818/0 (socket says 172.21.15.12:39818) 2023-11-28T21:35:10.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f3ce9b7b640 1 --2- 172.21.15.12:0/783898989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 0x7f3cd0003da0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009fc0 tx=0x7f3cdc01ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8bfc44a08e4ae6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fa9899640 1 -- 172.21.15.12:0/4244920334 learned_addr learned my addr 172.21.15.12:0/4244920334 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fa9098640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa4158670 0x7f5fa415b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7ff742ffd640 1 --2- 172.21.15.12:0/1689419691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c0dd290 0x7ff73c0e66c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff74407b290 tx=0x7ff74409a190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c3efd46c5d6bfb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5faa09a640 1 -- 172.21.15.12:0/4244920334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa40c7520 msgr2=0x7f5fa415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5faa09a640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa40c7520 0x7f5fa415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5faa09a640 1 -- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa4158670 msgr2=0x7f5fa415b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5faa09a640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa4158670 0x7f5fa415b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5faa09a640 1 -- 172.21.15.12:0/4244920334 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f940094d0 con 0x7f5fa415edb0 2023-11-28T21:35:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5fa9098640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa4158670 0x7f5fa415b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:10.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.492+0000 7f5faa09a640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fa415edb0 0x7f5fa4070d60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c430 tx=0x7f5fa000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b3fff640 1 --2- 172.21.15.12:0/3195723467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b40c5bb0 0x7f13b40c2ac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13b40c8cd0 tx=0x7f13a8021100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2e99c5b22c4a0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b27fc640 1 -- 172.21.15.12:0/3195723467 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a80b4400 con 0x7f13b40c5bb0 2023-11-28T21:35:10.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b27fc640 1 -- 172.21.15.12:0/3195723467 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a80ae070 con 0x7f13b40c5bb0 2023-11-28T21:35:10.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b27fc640 1 -- 172.21.15.12:0/3195723467 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a80b4780 con 0x7f13b40c5bb0 2023-11-28T21:35:10.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b8bd0640 1 -- 172.21.15.12:0/3195723467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b40c5bb0 msgr2=0x7f13b40c2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b8bd0640 1 --2- 172.21.15.12:0/3195723467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b40c5bb0 0x7f13b40c2ac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13b40c8cd0 tx=0x7f13a8021100 comp rx=0 tx=0).stop 2023-11-28T21:35:10.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b8bd0640 1 -- 172.21.15.12:0/3195723467 shutdown_connections 2023-11-28T21:35:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b8bd0640 1 --2- 172.21.15.12:0/3195723467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b40c5bb0 0x7f13b40c2ac0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b8bd0640 1 --2- 172.21.15.12:0/3195723467 >> v2:172.21.15.144:3300/0 conn(0x7f13b406b260 0x7f13b4159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b8bd0640 1 -- 172.21.15.12:0/3195723467 >> 172.21.15.12:0/3195723467 conn(0x7f13b4066360 msgr2=0x7f13b4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b8bd0640 1 -- 172.21.15.12:0/3195723467 shutdown_connections 2023-11-28T21:35:10.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f13b8bd0640 1 -- 172.21.15.12:0/3195723467 wait complete. 2023-11-28T21:35:10.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f5f9a7fc640 1 -- 172.21.15.12:0/4244920334 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00054e0 con 0x7f5fa415edb0 2023-11-28T21:35:10.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f5f9a7fc640 1 -- 172.21.15.12:0/4244920334 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa0003600 con 0x7f5fa415edb0 2023-11-28T21:35:10.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f5f9a7fc640 1 -- 172.21.15.12:0/4244920334 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00039d0 con 0x7f5fa415edb0 2023-11-28T21:35:10.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0a5c90 con 0x7f3cd0003980 2023-11-28T21:35:10.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.496+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc00f040 con 0x7f3cd0003980 2023-11-28T21:35:10.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f5fabafd640 1 -- 172.21.15.12:0/4244920334 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa41f7f80 con 0x7f5fa415edb0 2023-11-28T21:35:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f3ce8b79640 1 -- 172.21.15.12:0/783898989 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0ad680 con 0x7f3cd0003980 2023-11-28T21:35:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f3ceab7d640 1 -- 172.21.15.12:0/783898989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 msgr2=0x7f3cd0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f3ceab7d640 1 --2- 172.21.15.12:0/783898989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 0x7f3cd0003da0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009fc0 tx=0x7f3cdc01ccc0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7ff741ffb640 1 -- 172.21.15.12:0/1689419691 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744123e30 con 0x7ff73c0dd290 2023-11-28T21:35:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f3ceab7d640 1 -- 172.21.15.12:0/783898989 shutdown_connections 2023-11-28T21:35:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f3ceab7d640 1 --2- 172.21.15.12:0/783898989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 0x7f3cd0003da0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f3ceab7d640 1 -- 172.21.15.12:0/783898989 >> 172.21.15.12:0/783898989 conn(0x7f3ce4066360 msgr2=0x7f3ce4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f5fabafd640 1 -- 172.21.15.12:0/4244920334 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa41f8540 con 0x7f5fa415edb0 2023-11-28T21:35:10.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f3ceab7d640 1 -- 172.21.15.12:0/783898989 shutdown_connections 2023-11-28T21:35:10.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f5fabafd640 1 -- 172.21.15.12:0/4244920334 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa4067c70 con 0x7f5fa415edb0 2023-11-28T21:35:10.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7ff741ffb640 1 -- 172.21.15.12:0/1689419691 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74412a3b0 con 0x7ff73c0dd290 2023-11-28T21:35:10.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7ff741ffb640 1 -- 172.21.15.12:0/1689419691 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744123e30 con 0x7ff73c0dd290 2023-11-28T21:35:10.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7ff7494fe640 1 -- 172.21.15.12:0/1689419691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c0dd290 msgr2=0x7ff73c0e66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7ff7494fe640 1 --2- 172.21.15.12:0/1689419691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c0dd290 0x7ff73c0e66c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff74407b290 tx=0x7ff74409a190 comp rx=0 tx=0).stop 2023-11-28T21:35:10.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f3ceab7d640 1 -- 172.21.15.12:0/783898989 wait complete. 2023-11-28T21:35:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f13b8bd0640 1 Processor -- start 2023-11-28T21:35:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7ff7494fe640 1 -- 172.21.15.12:0/1689419691 shutdown_connections 2023-11-28T21:35:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7ff7494fe640 1 --2- 172.21.15.12:0/1689419691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c0dd290 0x7ff73c0e66c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7ff7494fe640 1 -- 172.21.15.12:0/1689419691 >> 172.21.15.12:0/1689419691 conn(0x7ff73c050160 msgr2=0x7ff73c050570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7f13b8bd0640 1 -- start start 2023-11-28T21:35:10.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7ff7494fe640 1 -- 172.21.15.12:0/1689419691 shutdown_connections 2023-11-28T21:35:10.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.500+0000 7ff7494fe640 1 -- 172.21.15.12:0/1689419691 wait complete. 2023-11-28T21:35:10.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f5f9a7fc640 1 -- 172.21.15.12:0/4244920334 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5fa0003b30 con 0x7f5fa415edb0 2023-11-28T21:35:10.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f3ceab7d640 1 Processor -- start 2023-11-28T21:35:10.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f3ceab7d640 1 -- start start 2023-11-28T21:35:10.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f5f9a7fc640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5f880664f0 0x7f5f880689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f5f9a7fc640 1 -- 172.21.15.12:0/4244920334 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5fa00545a0 con 0x7f5fa415edb0 2023-11-28T21:35:10.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7ff7494fe640 1 Processor -- start 2023-11-28T21:35:10.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b8bd0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13b406b260 0x7f13b4155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b8bd0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40c5bb0 0x7f13b40729d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7ff7494fe640 1 -- start start 2023-11-28T21:35:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b8bd0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b414ff70 0x7f13b4152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b8bd0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4072f10 con 0x7f13b414ff70 2023-11-28T21:35:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b8bd0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4071aa0 con 0x7f13b406b260 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b8bd0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b41f79f0 con 0x7f13b40c5bb0 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40c5bb0 0x7f13b40729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40c5bb0 0x7f13b40729d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56726/0 (socket says 172.21.15.12:56726) 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b2ffd640 1 -- 172.21.15.12:0/2107315094 learned_addr learned my addr 172.21.15.12:0/2107315094 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b3fff640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b414ff70 0x7f13b4152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b2ffd640 1 -- 172.21.15.12:0/2107315094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13b406b260 msgr2=0x7f13b4155cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b2ffd640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13b406b260 0x7f13b4155cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b2ffd640 1 -- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b414ff70 msgr2=0x7f13b4152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b2ffd640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b414ff70 0x7f13b4152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b2ffd640 1 -- 172.21.15.12:0/2107315094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a800b6b0 con 0x7f13b40c5bb0 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b3fff640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b414ff70 0x7f13b4152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:35:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b2ffd640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40c5bb0 0x7f13b40729d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009db0 tx=0x7f13ac00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f139bfff640 1 -- 172.21.15.12:0/2107315094 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac014480 con 0x7f13b40c5bb0 2023-11-28T21:35:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f139bfff640 1 -- 172.21.15.12:0/2107315094 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ac00f040 con 0x7f13b40c5bb0 2023-11-28T21:35:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f139bfff640 1 -- 172.21.15.12:0/2107315094 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac014800 con 0x7f13b40c5bb0 2023-11-28T21:35:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f13b8bd0640 1 -- 172.21.15.12:0/2107315094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b40701c0 con 0x7f13b40c5bb0 2023-11-28T21:35:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f13b8bd0640 1 -- 172.21.15.12:0/2107315094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b4145fb0 con 0x7f13b40c5bb0 2023-11-28T21:35:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f3ceab7d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 0x7f3ce406fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ceab7d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ce415bb90 0x7f3ce41f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.504+0000 7f5fa9899640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5f880664f0 0x7f5f880689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f5fa9899640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5f880664f0 0x7f5f880689b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f9401a5b0 tx=0x7f5f940a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ce9b7b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 0x7f3ce406fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ce9b7b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 0x7f3ce406fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56738/0 (socket says 172.21.15.12:56738) 2023-11-28T21:35:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ce937a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ce415bb90 0x7f3ce41f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ce937a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ce415bb90 0x7f3ce41f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38398/0 (socket says 172.21.15.12:38398) 2023-11-28T21:35:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ce9b7b640 1 -- 172.21.15.12:0/4047770117 learned_addr learned my addr 172.21.15.12:0/4047770117 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ceab7d640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ce4155b50 0x7f3ce41f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff7494fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff73c0dd290 0x7ff73c0e06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce41f4170 con 0x7f3ce415bb90 2023-11-28T21:35:10.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce4156020 con 0x7f3ce4155b50 2023-11-28T21:35:10.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce41f6d00 con 0x7f3cd0003980 2023-11-28T21:35:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f13b8bd0640 1 -- 172.21.15.12:0/2107315094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b4067c70 con 0x7f13b40c5bb0 2023-11-28T21:35:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff7494fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff73c0e6c00 0x7ff73c04f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3cea37c640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ce4155b50 0x7f3ce41f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff7494fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c0e71c0 0x7ff73c179190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff7494fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c17c430 con 0x7ff73c0dd290 2023-11-28T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff7494fe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c04a4d0 con 0x7ff73c0e71c0 2023-11-28T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff7494fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c17c6d0 con 0x7ff73c0e6c00 2023-11-28T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff742ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff73c0dd290 0x7ff73c0e06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff7427fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff73c0e6c00 0x7ff73c04f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff742ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff73c0dd290 0x7ff73c0e06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38410/0 (socket says 172.21.15.12:38410) 2023-11-28T21:35:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff742ffd640 1 -- 172.21.15.12:0/922296627 learned_addr learned my addr 172.21.15.12:0/922296627 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff7437fe640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c0e71c0 0x7ff73c179190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff742ffd640 1 -- 172.21.15.12:0/922296627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff73c0e6c00 msgr2=0x7ff73c04f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff742ffd640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff73c0e6c00 0x7ff73c04f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff742ffd640 1 -- 172.21.15.12:0/922296627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c0e71c0 msgr2=0x7ff73c179190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff742ffd640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c0e71c0 0x7ff73c179190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff742ffd640 1 -- 172.21.15.12:0/922296627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74404f7e0 con 0x7ff73c0dd290 2023-11-28T21:35:10.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3cea37c640 1 -- 172.21.15.12:0/4047770117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 msgr2=0x7f3ce406fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3cea37c640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 0x7f3ce406fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3cea37c640 1 -- 172.21.15.12:0/4047770117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ce415bb90 msgr2=0x7f3ce41f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3cea37c640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ce415bb90 0x7f3ce41f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3cea37c640 1 -- 172.21.15.12:0/4047770117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce41f6fe0 con 0x7f3ce4155b50 2023-11-28T21:35:10.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3ce937a640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ce415bb90 0x7f3ce41f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:10.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff742ffd640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff73c0dd290 0x7ff73c0e06b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff74402cef0 tx=0x7ff744098a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7f3cea37c640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ce4155b50 0x7f3ce41f3c30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4067010 tx=0x7f3cd40077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff72b7fe640 1 -- 172.21.15.12:0/922296627 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744123e30 con 0x7ff73c0dd290 2023-11-28T21:35:10.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff72b7fe640 1 -- 172.21.15.12:0/922296627 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74412ae70 con 0x7ff73c0dd290 2023-11-28T21:35:10.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff72b7fe640 1 -- 172.21.15.12:0/922296627 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74412b860 con 0x7ff73c0dd290 2023-11-28T21:35:10.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.508+0000 7ff7494fe640 1 -- 172.21.15.12:0/922296627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c17c9b0 con 0x7ff73c0dd290 2023-11-28T21:35:10.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.512+0000 7f3cda7fc640 1 -- 172.21.15.12:0/4047770117 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4004240 con 0x7f3ce4155b50 2023-11-28T21:35:10.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.512+0000 7f3cda7fc640 1 -- 172.21.15.12:0/4047770117 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd40043a0 con 0x7f3ce4155b50 2023-11-28T21:35:10.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.512+0000 7f3cda7fc640 1 -- 172.21.15.12:0/4047770117 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4003820 con 0x7f3ce4155b50 2023-11-28T21:35:10.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.512+0000 7ff7494fe640 1 -- 172.21.15.12:0/922296627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c17cf70 con 0x7ff73c0dd290 2023-11-28T21:35:10.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.512+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce41f72c0 con 0x7f3ce4155b50 2023-11-28T21:35:10.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.512+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce41f7880 con 0x7f3ce4155b50 2023-11-28T21:35:10.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.512+0000 7ff7494fe640 1 -- 172.21.15.12:0/922296627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c005370 con 0x7ff73c0dd290 2023-11-28T21:35:10.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.512+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce4067c70 con 0x7f3ce4155b50 2023-11-28T21:35:10.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.516+0000 7ff72b7fe640 1 -- 172.21.15.12:0/922296627 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff74412b9c0 con 0x7ff73c0dd290 2023-11-28T21:35:10.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.516+0000 7f3cda7fc640 1 -- 172.21.15.12:0/4047770117 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3cd4003980 con 0x7f3ce4155b50 2023-11-28T21:35:10.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.516+0000 7ff72b7fe640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7300664c0 0x7ff730068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.516+0000 7ff72b7fe640 1 -- 172.21.15.12:0/922296627 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff74419cbe0 con 0x7ff73c0dd290 2023-11-28T21:35:10.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.516+0000 7ff7427fc640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7300664c0 0x7ff730068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.516+0000 7f3cda7fc640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3cc8066470 0x7f3cc8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.516+0000 7f3cda7fc640 1 -- 172.21.15.12:0/4047770117 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3cd40836c0 con 0x7f3ce4155b50 2023-11-28T21:35:10.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.516+0000 7f3ce9b7b640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3cc8066470 0x7f3cc8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.516+0000 7ff7427fc640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7300664c0 0x7ff730068980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff73c1769e0 tx=0x7ff734007e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.520+0000 7f3ce9b7b640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3cc8066470 0x7f3cc8068930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc01cd00 tx=0x7f3cdc004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.520+0000 7f5f9a7fc640 1 -- 172.21.15.12:0/4244920334 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5fa0051070 con 0x7f5fa415edb0 2023-11-28T21:35:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.520+0000 7ff72b7fe640 1 -- 172.21.15.12:0/922296627 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff744169d70 con 0x7ff73c0dd290 2023-11-28T21:35:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.520+0000 7f139bfff640 1 -- 172.21.15.12:0/2107315094 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f13ac014960 con 0x7f13b40c5bb0 2023-11-28T21:35:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.524+0000 7f139bfff640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13a0066470 0x7f13a0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.524+0000 7f139bfff640 1 -- 172.21.15.12:0/2107315094 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f13ac085470 con 0x7f13b40c5bb0 2023-11-28T21:35:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.524+0000 7f13b37fe640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13a0066470 0x7f13a0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.528+0000 7f13b37fe640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13a0066470 0x7f13a0068930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13a400a710 tx=0x7f13a4005bb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.528+0000 7f3cda7fc640 1 -- 172.21.15.12:0/4047770117 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3cd40507d0 con 0x7f3ce4155b50 2023-11-28T21:35:10.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.536+0000 7f139bfff640 1 -- 172.21.15.12:0/2107315094 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f13ac0526e0 con 0x7f13b40c5bb0 2023-11-28T21:35:10.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.548+0000 7ff1a68ad640 1 Processor -- start 2023-11-28T21:35:10.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.548+0000 7ff1a68ad640 1 -- start start 2023-11-28T21:35:10.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a68ad640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff1a0153fb0 0x7ff1a00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a68ad640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a015a980 0x7ff1a00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a68ad640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a015d390 con 0x7ff1a00b3ad0 2023-11-28T21:35:10.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a68ad640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a00b2b60 con 0x7ff1a0153fb0 2023-11-28T21:35:10.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a68ad640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a015e1b0 con 0x7ff1a015a980 2023-11-28T21:35:10.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a50aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a015a980 0x7ff1a00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a50aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a015a980 0x7ff1a00caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:56758/0 (socket says 172.21.15.12:56758) 2023-11-28T21:35:10.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a50aa640 1 -- 172.21.15.12:0/3880116828 learned_addr learned my addr 172.21.15.12:0/3880116828 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a58ab640 1 --2- 172.21.15.12:0/3880116828 >> v2:172.21.15.12:3300/0 conn(0x7ff1a0153fb0 0x7ff1a00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a50aa640 1 -- 172.21.15.12:0/3880116828 >> v2:172.21.15.12:3300/0 conn(0x7ff1a0153fb0 msgr2=0x7ff1a00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:10.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a50aa640 1 --2- 172.21.15.12:0/3880116828 >> v2:172.21.15.12:3300/0 conn(0x7ff1a0153fb0 0x7ff1a00ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a50aa640 1 -- 172.21.15.12:0/3880116828 >> v1:172.21.15.12:6789/0 conn(0x7ff1a00b3ad0 legacy=0x7ff1a00b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.552+0000 7ff1a50aa640 1 -- 172.21.15.12:0/3880116828 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a015d630 con 0x7ff1a015a980 2023-11-28T21:35:10.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.556+0000 7ff1a58ab640 1 --2- 172.21.15.12:0/3880116828 >> v2:172.21.15.12:3300/0 conn(0x7ff1a0153fb0 0x7ff1a00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:10.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.560+0000 7ff1a50aa640 1 --2- 172.21.15.12:0/3880116828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a015a980 0x7ff1a00caa30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009870 tx=0x7ff19c01c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea647c832d030cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.560+0000 7ff1a48a9640 1 -- 172.21.15.12:0/3880116828 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0aa670 con 0x7ff1a015a980 2023-11-28T21:35:10.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.560+0000 7ff1a48a9640 1 -- 172.21.15.12:0/3880116828 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19c0aa7d0 con 0x7ff1a015a980 2023-11-28T21:35:10.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.560+0000 7ff1a48a9640 1 -- 172.21.15.12:0/3880116828 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0aab60 con 0x7ff1a015a980 2023-11-28T21:35:10.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.564+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3880116828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a015a980 msgr2=0x7ff1a00caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.564+0000 7ff1a68ad640 1 --2- 172.21.15.12:0/3880116828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a015a980 0x7ff1a00caa30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009870 tx=0x7ff19c01c550 comp rx=0 tx=0).stop 2023-11-28T21:35:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.568+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3880116828 shutdown_connections 2023-11-28T21:35:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.568+0000 7ff1a68ad640 1 --2- 172.21.15.12:0/3880116828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a015a980 0x7ff1a00caa30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.568+0000 7ff1a68ad640 1 --2- 172.21.15.12:0/3880116828 >> v2:172.21.15.12:3300/0 conn(0x7ff1a0153fb0 0x7ff1a00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.568+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3880116828 >> 172.21.15.12:0/3880116828 conn(0x7ff1a0071ce0 msgr2=0x7ff1a00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.568+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3880116828 shutdown_connections 2023-11-28T21:35:10.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.568+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3880116828 wait complete. 2023-11-28T21:35:10.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.568+0000 7ff1a68ad640 1 Processor -- start 2023-11-28T21:35:10.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.568+0000 7ff1a68ad640 1 -- start start 2023-11-28T21:35:10.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a68ad640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a00b3ad0 0x7ff1a00c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a68ad640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a0153fb0 0x7ff1a0145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a68ad640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a015a980 0x7ff1a0145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a68ad640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a00c6360 con 0x7ff1a0153fb0 2023-11-28T21:35:10.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a68ad640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a015b620 con 0x7ff1a015a980 2023-11-28T21:35:10.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a68ad640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a00c5b50 con 0x7ff1a00b3ad0 2023-11-28T21:35:10.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a50aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a0153fb0 0x7ff1a0145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a50aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a0153fb0 0x7ff1a0145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38428/0 (socket says 172.21.15.12:38428) 2023-11-28T21:35:10.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a50aa640 1 -- 172.21.15.12:0/3270915016 learned_addr learned my addr 172.21.15.12:0/3270915016 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:10.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a58ab640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a00b3ad0 0x7ff1a00c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a60ac640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a015a980 0x7ff1a0145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a60ac640 1 -- 172.21.15.12:0/3270915016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a00b3ad0 msgr2=0x7ff1a00c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a60ac640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a00b3ad0 0x7ff1a00c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a60ac640 1 -- 172.21.15.12:0/3270915016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a0153fb0 msgr2=0x7ff1a0145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a60ac640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a0153fb0 0x7ff1a0145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a60ac640 1 -- 172.21.15.12:0/3270915016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190009420 con 0x7ff1a015a980 2023-11-28T21:35:10.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.572+0000 7ff1a50aa640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a0153fb0 0x7ff1a0145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:10.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.576+0000 7ff1a60ac640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a015a980 0x7ff1a0145d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c700 tx=0x7ff19800cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.576+0000 7ff1967fc640 1 -- 172.21.15.12:0/3270915016 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198003b80 con 0x7ff1a015a980 2023-11-28T21:35:10.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.576+0000 7ff1967fc640 1 -- 172.21.15.12:0/3270915016 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff198003ce0 con 0x7ff1a015a980 2023-11-28T21:35:10.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.576+0000 7ff1967fc640 1 -- 172.21.15.12:0/3270915016 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19800aa30 con 0x7ff1a015a980 2023-11-28T21:35:10.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.576+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3270915016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c0094d0 con 0x7ff1a015a980 2023-11-28T21:35:10.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.576+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3270915016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a0145240 con 0x7ff1a015a980 2023-11-28T21:35:10.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.576+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3270915016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a0074f60 con 0x7ff1a015a980 2023-11-28T21:35:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.580+0000 7ff1967fc640 1 -- 172.21.15.12:0/3270915016 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff19800ab90 con 0x7ff1a015a980 2023-11-28T21:35:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.584+0000 7ff1967fc640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff184066470 0x7ff184068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.584+0000 7ff1967fc640 1 -- 172.21.15.12:0/3270915016 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff198084930 con 0x7ff1a015a980 2023-11-28T21:35:10.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.584+0000 7ff1a58ab640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff184066470 0x7ff184068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.584+0000 7ff1a58ab640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff184066470 0x7ff184068930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff190009cf0 tx=0x7ff190009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:10.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.592+0000 7ff1967fc640 1 -- 172.21.15.12:0/3270915016 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff198051b60 con 0x7ff1a015a980 2023-11-28T21:35:10.598 INFO:tasks.ceph.mds.f.smithi144.stderr:2023-11-28T21:35:10.595+0000 7fcaf2464640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T21:35:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.664+0000 7fafd09f5640 1 -- 172.21.15.12:0/816084503 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fafcc074f60 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.664+0000 7fafbaffd640 1 -- 172.21.15.12:0/816084503 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fafc0017030 con 0x7fafcc0ca3f0 2023-11-28T21:35:10.667 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-11-28T21:35:10.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 -- 172.21.15.12:0/816084503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fafac066470 msgr2=0x7fafac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fafac066470 0x7fafac068930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009890 tx=0x7fafbc008040 comp rx=0 tx=0).stop 2023-11-28T21:35:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 -- 172.21.15.12:0/816084503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc0ca3f0 msgr2=0x7fafcc0c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc0ca3f0 0x7fafcc0c67c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c420 tx=0x7fafc000c8e0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 -- 172.21.15.12:0/816084503 shutdown_connections 2023-11-28T21:35:10.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fafac066470 0x7fafac068930 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafcc154100 0x7fafcc156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafcc0ca3f0 0x7fafcc0c67c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 --2- 172.21.15.12:0/816084503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafcc0c9fd0 0x7fafcc0c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 -- 172.21.15.12:0/816084503 >> 172.21.15.12:0/816084503 conn(0x7fafcc06c690 msgr2=0x7fafcc147410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 -- 172.21.15.12:0/816084503 shutdown_connections 2023-11-28T21:35:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.668+0000 7fafd09f5640 1 -- 172.21.15.12:0/816084503 wait complete. 2023-11-28T21:35:10.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2023-11-28T21:35:10.704 DEBUG:teuthology.parallel:result is None 2023-11-28T21:35:10.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.732+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe5cc159b90 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5baffd640 1 -- 172.21.15.12:0/1390430512 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe5c0017070 con 0x7fe5cc15b7b0 2023-11-28T21:35:10.737 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-11-28T21:35:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe5ac0664c0 msgr2=0x7fe5ac068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe5ac0664c0 0x7fe5ac068980 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0b0040 tx=0x7fe5bc008040 comp rx=0 tx=0).stop 2023-11-28T21:35:10.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc15b7b0 msgr2=0x7fe5cc159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc15b7b0 0x7fe5cc159e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c390 tx=0x7fe5c000c850 comp rx=0 tx=0).stop 2023-11-28T21:35:10.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 shutdown_connections 2023-11-28T21:35:10.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe5ac0664c0 0x7fe5ac068980 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc15b7b0 0x7fe5cc159e70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc158670 0x7fe5cc15fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 --2- 172.21.15.12:0/1390430512 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc0c7520 0x7fe5cc154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 >> 172.21.15.12:0/1390430512 conn(0x7fe5cc06d4c0 msgr2=0x7fe5cc0c2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 shutdown_connections 2023-11-28T21:35:10.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.736+0000 7fe5d0db8640 1 -- 172.21.15.12:0/1390430512 wait complete. 2023-11-28T21:35:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.744+0000 7f38f6b9d640 1 -- 172.21.15.12:0/2825521108 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f38f0074f60 con 0x7f38f006b560 2023-11-28T21:35:10.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.744+0000 7f38ed7fa640 1 -- 172.21.15.12:0/2825521108 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38e0017020 con 0x7f38f006b560 2023-11-28T21:35:10.751 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-11-28T21:35:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 -- 172.21.15.12:0/2825521108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f38dc0664f0 msgr2=0x7f38dc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f38dc0664f0 0x7f38dc0689b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f38e8000f90 tx=0x7f38e80a7070 comp rx=0 tx=0).stop 2023-11-28T21:35:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 -- 172.21.15.12:0/2825521108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f006b560 msgr2=0x7f38f01548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f006b560 0x7f38f01548e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f38e0001ec0 tx=0x7f38e000c890 comp rx=0 tx=0).stop 2023-11-28T21:35:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 -- 172.21.15.12:0/2825521108 shutdown_connections 2023-11-28T21:35:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f38dc0664f0 0x7f38dc0689b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38f0158670 0x7f38f0145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38f0158290 0x7f38f0154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 --2- 172.21.15.12:0/2825521108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38f006b560 0x7f38f01548e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 -- 172.21.15.12:0/2825521108 >> 172.21.15.12:0/2825521108 conn(0x7f38f006d4c0 msgr2=0x7f38f00c3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.748+0000 7f38f6b9d640 1 -- 172.21.15.12:0/2825521108 shutdown_connections 2023-11-28T21:35:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.752+0000 7f38f6b9d640 1 -- 172.21.15.12:0/2825521108 wait complete. 2023-11-28T21:35:10.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2023-11-28T21:35:10.761 DEBUG:teuthology.parallel:result is None 2023-11-28T21:35:10.787 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2023-11-28T21:35:10.788 DEBUG:teuthology.parallel:result is None 2023-11-28T21:35:10.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 -- 172.21.15.12:0/4244920334 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5fa4074f60 con 0x7f5fa415edb0 2023-11-28T21:35:10.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5f9a7fc640 1 -- 172.21.15.12:0/4244920334 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5fa0017070 con 0x7f5fa415edb0 2023-11-28T21:35:10.826 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-11-28T21:35:10.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 -- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5f880664f0 msgr2=0x7f5f880689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5f880664f0 0x7f5f880689b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f9401a5b0 tx=0x7f5f940a6000 comp rx=0 tx=0).stop 2023-11-28T21:35:10.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 -- 172.21.15.12:0/4244920334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fa415edb0 msgr2=0x7f5fa4070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fa415edb0 0x7f5fa4070d60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000c430 tx=0x7f5fa000c8f0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 -- 172.21.15.12:0/4244920334 shutdown_connections 2023-11-28T21:35:10.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5f880664f0 0x7f5f880689b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fa415edb0 0x7f5fa4070d60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa4158670 0x7f5fa415b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 --2- 172.21.15.12:0/4244920334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fa40c7520 0x7f5fa415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 -- 172.21.15.12:0/4244920334 >> 172.21.15.12:0/4244920334 conn(0x7f5fa406d4c0 msgr2=0x7f5fa41614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.824+0000 7f5fabafd640 1 -- 172.21.15.12:0/4244920334 shutdown_connections 2023-11-28T21:35:10.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.828+0000 7f5fabafd640 1 -- 172.21.15.12:0/4244920334 wait complete. 2023-11-28T21:35:10.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.852+0000 7f13b8bd0640 1 -- 172.21.15.12:0/2107315094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f13b4074f60 con 0x7f13b40c5bb0 2023-11-28T21:35:10.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.852+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3270915016 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff1a00cb810 con 0x7ff1a015a980 2023-11-28T21:35:10.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.852+0000 7f139bfff640 1 -- 172.21.15.12:0/2107315094 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f13ac055d00 con 0x7f13b40c5bb0 2023-11-28T21:35:10.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.852+0000 7ff1967fc640 1 -- 172.21.15.12:0/3270915016 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff198055180 con 0x7ff1a015a980 2023-11-28T21:35:10.854 INFO:teuthology.orchestra.run.smithi012.stdout:55834574858 2023-11-28T21:35:10.855 INFO:teuthology.orchestra.run.smithi012.stdout:55834574857 2023-11-28T21:35:10.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-11-28T21:35:10.857 DEBUG:teuthology.parallel:result is None 2023-11-28T21:35:10.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 -- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13a0066470 msgr2=0x7f13a0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13a0066470 0x7f13a0068930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13a400a710 tx=0x7f13a4005bb0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 -- 172.21.15.12:0/2107315094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40c5bb0 msgr2=0x7f13b40729d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40c5bb0 0x7f13b40729d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009db0 tx=0x7f13ac00c490 comp rx=0 tx=0).stop 2023-11-28T21:35:10.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3270915016 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff184066470 msgr2=0x7ff184068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff184066470 0x7ff184068930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff190009cf0 tx=0x7ff190009190 comp rx=0 tx=0).stop 2023-11-28T21:35:10.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3270915016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a015a980 msgr2=0x7ff1a0145d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a015a980 0x7ff1a0145d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c700 tx=0x7ff19800cbc0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 -- 172.21.15.12:0/2107315094 shutdown_connections 2023-11-28T21:35:10.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13a0066470 0x7f13a0068930 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13b414ff70 0x7f13b4152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40c5bb0 0x7f13b40729d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 --2- 172.21.15.12:0/2107315094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13b406b260 0x7f13b4155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 -- 172.21.15.12:0/2107315094 >> 172.21.15.12:0/2107315094 conn(0x7f13b4066360 msgr2=0x7f13b4066740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3270915016 shutdown_connections 2023-11-28T21:35:10.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff184066470 0x7ff184068930 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a015a980 0x7ff1a0145d40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff1a0153fb0 0x7ff1a0145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 --2- 172.21.15.12:0/3270915016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1a00b3ad0 0x7ff1a00c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3270915016 >> 172.21.15.12:0/3270915016 conn(0x7ff1a0071ce0 msgr2=0x7ff1a00b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3270915016 shutdown_connections 2023-11-28T21:35:10.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7ff1a68ad640 1 -- 172.21.15.12:0/3270915016 wait complete. 2023-11-28T21:35:10.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 -- 172.21.15.12:0/2107315094 shutdown_connections 2023-11-28T21:35:10.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.856+0000 7f13b8bd0640 1 -- 172.21.15.12:0/2107315094 wait complete. 2023-11-28T21:35:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.860+0000 7ff7494fe640 1 -- 172.21.15.12:0/922296627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff73c178db0 con 0x7ff73c0dd290 2023-11-28T21:35:10.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.860+0000 7ff72b7fe640 1 -- 172.21.15.12:0/922296627 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff74413c090 con 0x7ff73c0dd290 2023-11-28T21:35:10.863 INFO:teuthology.orchestra.run.smithi012.stdout:55834574859 2023-11-28T21:35:10.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 -- 172.21.15.12:0/922296627 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7300664c0 msgr2=0x7ff730068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7300664c0 0x7ff730068980 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff73c1769e0 tx=0x7ff734007e50 comp rx=0 tx=0).stop 2023-11-28T21:35:10.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 -- 172.21.15.12:0/922296627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff73c0dd290 msgr2=0x7ff73c0e06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff73c0dd290 0x7ff73c0e06b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff74402cef0 tx=0x7ff744098a20 comp rx=0 tx=0).stop 2023-11-28T21:35:10.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 -- 172.21.15.12:0/922296627 shutdown_connections 2023-11-28T21:35:10.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7300664c0 0x7ff730068980 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff73c0e71c0 0x7ff73c179190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff73c0e6c00 0x7ff73c04f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 --2- 172.21.15.12:0/922296627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff73c0dd290 0x7ff73c0e06b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 -- 172.21.15.12:0/922296627 >> 172.21.15.12:0/922296627 conn(0x7ff73c050160 msgr2=0x7ff73c0066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 -- 172.21.15.12:0/922296627 shutdown_connections 2023-11-28T21:35:10.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7ff7494fe640 1 -- 172.21.15.12:0/922296627 wait complete. 2023-11-28T21:35:10.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.864+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3ce41f7a70 con 0x7f3ce4155b50 2023-11-28T21:35:10.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3cda7fc640 1 -- 172.21.15.12:0/4047770117 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3cd4018020 con 0x7f3ce4155b50 2023-11-28T21:35:10.869 INFO:teuthology.orchestra.run.smithi012.stdout:55834574859 2023-11-28T21:35:10.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3cc8066470 msgr2=0x7f3cc8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3cc8066470 0x7f3cc8068930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc01cd00 tx=0x7f3cdc004ac0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ce4155b50 msgr2=0x7f3ce41f3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:10.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ce4155b50 0x7f3ce41f3c30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4067010 tx=0x7f3cd40077e0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 shutdown_connections 2023-11-28T21:35:10.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3cc8066470 0x7f3cc8068930 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ce4155b50 0x7f3ce41f3c30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ce415bb90 0x7f3ce41f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 --2- 172.21.15.12:0/4047770117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cd0003980 0x7f3ce406fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:10.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 >> 172.21.15.12:0/4047770117 conn(0x7f3ce4066360 msgr2=0x7f3ce4156360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:10.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.868+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 shutdown_connections 2023-11-28T21:35:10.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:10.872+0000 7f3ceab7d640 1 -- 172.21.15.12:0/4047770117 wait complete. 2023-11-28T21:35:10.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2023-11-28T21:35:10.883 DEBUG:teuthology.parallel:result is None 2023-11-28T21:35:10.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2023-11-28T21:35:10.884 DEBUG:teuthology.parallel:result is None 2023-11-28T21:35:10.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2023-11-28T21:35:10.901 DEBUG:teuthology.parallel:result is None 2023-11-28T21:35:10.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2023-11-28T21:35:10.903 DEBUG:teuthology.parallel:result is None 2023-11-28T21:35:10.903 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T21:35:10.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:35:11.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.084+0000 7fb5c4fa7640 1 Processor -- start 2023-11-28T21:35:11.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.084+0000 7fb5c4fa7640 1 -- start start 2023-11-28T21:35:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.084+0000 7fb5c4fa7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb5c006b0a0 0x7fb5c00c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.084+0000 7fb5c4fa7640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c00c3070 con 0x7fb5c006b480 2023-11-28T21:35:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.084+0000 7fb5c4fa7640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c00c3330 con 0x7fb5c0150420 2023-11-28T21:35:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.084+0000 7fb5c4fa7640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c00c4290 con 0x7fb5c006b0a0 2023-11-28T21:35:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.088+0000 7fb5bf7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb5c006b0a0 0x7fb5c00c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.088+0000 7fb5beffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb5c006b480 0x7fb5c014fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36498/0 (socket says 172.21.15.12:36498) 2023-11-28T21:35:11.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.088+0000 7fb5beffd640 1 -- 172.21.15.12:0/787174987 learned_addr learned my addr 172.21.15.12:0/787174987 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:11.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.088+0000 7fb5bf7fe640 1 -- 172.21.15.12:0/787174987 >> v1:172.21.15.144:6789/0 conn(0x7fb5c0150420 legacy=0x7fb5c0160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.088+0000 7fb5bf7fe640 1 -- 172.21.15.12:0/787174987 >> v1:172.21.15.12:6789/0 conn(0x7fb5c006b480 legacy=0x7fb5c014fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.088+0000 7fb5bf7fe640 1 -- 172.21.15.12:0/787174987 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c00c3610 con 0x7fb5c006b0a0 2023-11-28T21:35:11.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.088+0000 7fb5bf7fe640 1 --2- 172.21.15.12:0/787174987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 0x7fb5c00c46b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00097c0 tx=0x7fb5b001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c0bc3e760c1422 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:11.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.088+0000 7fb5be7fc640 1 -- 172.21.15.12:0/787174987 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00ab650 con 0x7fb5c006b0a0 2023-11-28T21:35:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5be7fc640 1 -- 172.21.15.12:0/787174987 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b00ab7b0 con 0x7fb5c006b0a0 2023-11-28T21:35:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5be7fc640 1 -- 172.21.15.12:0/787174987 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00abb40 con 0x7fb5c006b0a0 2023-11-28T21:35:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/787174987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 msgr2=0x7fb5c00c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 --2- 172.21.15.12:0/787174987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 0x7fb5c00c46b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00097c0 tx=0x7fb5b001c450 comp rx=0 tx=0).stop 2023-11-28T21:35:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/787174987 shutdown_connections 2023-11-28T21:35:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 --2- 172.21.15.12:0/787174987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 0x7fb5c00c46b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/787174987 >> 172.21.15.12:0/787174987 conn(0x7fb5c0066360 msgr2=0x7fb5c0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/787174987 shutdown_connections 2023-11-28T21:35:11.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/787174987 wait complete. 2023-11-28T21:35:11.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 Processor -- start 2023-11-28T21:35:11.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 -- start start 2023-11-28T21:35:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 0x7fb5c01fb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c006b480 0x7fb5c014bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c0150420 0x7fb5c0151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c0150ca0 con 0x7fb5c006b0a0 2023-11-28T21:35:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c01fb690 con 0x7fb5c0150420 2023-11-28T21:35:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5c4fa7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c01f71d0 con 0x7fb5c006b480 2023-11-28T21:35:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5bf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 0x7fb5c01fb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5bf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 0x7fb5c01fb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38436/0 (socket says 172.21.15.12:38436) 2023-11-28T21:35:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5bf7fe640 1 -- 172.21.15.12:0/3704857902 learned_addr learned my addr 172.21.15.12:0/3704857902 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5beffd640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c006b480 0x7fb5c014bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.092+0000 7fb5bffff640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c0150420 0x7fb5c0151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5bf7fe640 1 -- 172.21.15.12:0/3704857902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c006b480 msgr2=0x7fb5c014bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5bf7fe640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c006b480 0x7fb5c014bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5bf7fe640 1 -- 172.21.15.12:0/3704857902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c0150420 msgr2=0x7fb5c0151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5bf7fe640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c0150420 0x7fb5c0151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5bf7fe640 1 -- 172.21.15.12:0/3704857902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b0009420 con 0x7fb5c006b0a0 2023-11-28T21:35:11.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5bffff640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c0150420 0x7fb5c0151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:11.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5bf7fe640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 0x7fb5c01fb150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb5b001a080 tx=0x7fb5b001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:11.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5a7fff640 1 -- 172.21.15.12:0/3704857902 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00b5810 con 0x7fb5c006b0a0 2023-11-28T21:35:11.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5a7fff640 1 -- 172.21.15.12:0/3704857902 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b00b5970 con 0x7fb5c006b0a0 2023-11-28T21:35:11.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5a7fff640 1 -- 172.21.15.12:0/3704857902 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b00b5d20 con 0x7fb5c006b0a0 2023-11-28T21:35:11.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/3704857902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c00c21a0 con 0x7fb5c006b0a0 2023-11-28T21:35:11.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/3704857902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c01f7730 con 0x7fb5c006b0a0 2023-11-28T21:35:11.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/3704857902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c0067c70 con 0x7fb5c006b0a0 2023-11-28T21:35:11.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5a7fff640 1 -- 172.21.15.12:0/3704857902 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb5b00c83f0 con 0x7fb5c006b0a0 2023-11-28T21:35:11.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5a7fff640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5ac0664c0 0x7fb5ac068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.096+0000 7fb5a7fff640 1 -- 172.21.15.12:0/3704857902 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb5b011f2e0 con 0x7fb5c006b0a0 2023-11-28T21:35:11.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.100+0000 7fb5beffd640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5ac0664c0 0x7fb5ac068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.100+0000 7fb5beffd640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5ac0664c0 0x7fb5ac068980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0072f80 tx=0x7fb5b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:11.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.100+0000 7fb5a7fff640 1 -- 172.21.15.12:0/3704857902 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb5b00ec3c0 con 0x7fb5c006b0a0 2023-11-28T21:35:11.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.256+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/3704857902 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5c00c5bb0 con 0x7fb5ac0664c0 2023-11-28T21:35:11.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.264+0000 7fb5a7fff640 1 -- 172.21.15.12:0/3704857902 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345437 (secure 0 0 0) 0x7fb5b4001ce0 con 0x7fb5ac0664c0 2023-11-28T21:35:11.272 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:11.275 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:35:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.268+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/3704857902 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5ac0664c0 msgr2=0x7fb5ac068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.268+0000 7fb5c4fa7640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5ac0664c0 0x7fb5ac068980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0072f80 tx=0x7fb5b4009040 comp rx=0 tx=0).stop 2023-11-28T21:35:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.268+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/3704857902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 msgr2=0x7fb5c01fb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.268+0000 7fb5c4fa7640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 0x7fb5c01fb150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb5b001a080 tx=0x7fb5b001ac90 comp rx=0 tx=0).stop 2023-11-28T21:35:11.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.268+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/3704857902 shutdown_connections 2023-11-28T21:35:11.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.268+0000 7fb5c4fa7640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5ac0664c0 0x7fb5ac068980 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.268+0000 7fb5c4fa7640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c0150420 0x7fb5c0151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.268+0000 7fb5c4fa7640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c006b480 0x7fb5c014bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.268+0000 7fb5c4fa7640 1 --2- 172.21.15.12:0/3704857902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c006b0a0 0x7fb5c01fb150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.268+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/3704857902 >> 172.21.15.12:0/3704857902 conn(0x7fb5c0066360 msgr2=0x7fb5c014c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:11.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.272+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/3704857902 shutdown_connections 2023-11-28T21:35:11.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.272+0000 7fb5c4fa7640 1 -- 172.21.15.12:0/3704857902 wait complete. 2023-11-28T21:35:11.293 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-11-28T21:35:09.823155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596413,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":246,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193,"ondisk_log_size":193,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218336,"kb_used_data":2180,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756384,"statfs":{"total":773094113280,"available":772870537216,"internally_reserved":0,"allocated":2232320,"data_stored":1988003,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1547,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002861"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343838+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:34:42.343838+0000","last_peered":"2023-11-28T21:34:42.343838+0000","last_clean":"2023-11-28T21:34:42.343838+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:34:42.343838+0000","last_undegraded":"2023-11-28T21:34:42.343838+0000","last_fullsized":"2023-11-28T21:34:42.343838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:46.167511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341374+0000","last_change":"2023-11-28T21:34:37.335656+0000","last_active":"2023-11-28T21:34:41.341374+0000","last_peered":"2023-11-28T21:34:41.341374+0000","last_clean":"2023-11-28T21:34:41.341374+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:34:41.341374+0000","last_undegraded":"2023-11-28T21:34:41.341374+0000","last_fullsized":"2023-11-28T21:34:41.341374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:25:49.707378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359842+0000","last_change":"2023-11-28T21:34:41.483856+0000","last_active":"2023-11-28T21:34:42.359842+0000","last_peered":"2023-11-28T21:34:42.359842+0000","last_clean":"2023-11-28T21:34:42.359842+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:34:42.359842+0000","last_undegraded":"2023-11-28T21:34:42.359842+0000","last_fullsized":"2023-11-28T21:34:42.359842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:50:56.005578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.340464+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:34:42.340464+0000","last_peered":"2023-11-28T21:34:42.340464+0000","last_clean":"2023-11-28T21:34:42.340464+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:34:42.340464+0000","last_undegraded":"2023-11-28T21:34:42.340464+0000","last_fullsized":"2023-11-28T21:34:42.340464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344725+0000","last_change":"2023-11-28T21:34:37.333649+0000","last_active":"2023-11-28T21:34:41.344725+0000","last_peered":"2023-11-28T21:34:41.344725+0000","last_clean":"2023-11-28T21:34:41.344725+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:34:41.344725+0000","last_undegraded":"2023-11-28T21:34:41.344725+0000","last_fullsized":"2023-11-28T21:34:41.344725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:21:42.041590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594292+0000","last_change":"2023-11-28T21:34:41.388232+0000","last_active":"2023-11-28T21:34:42.594292+0000","last_peered":"2023-11-28T21:34:42.594292+0000","last_clean":"2023-11-28T21:34:42.594292+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:34:42.594292+0000","last_undegraded":"2023-11-28T21:34:42.594292+0000","last_fullsized":"2023-11-28T21:34:42.594292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:22.960012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.345785+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:34:42.345785+0000","last_peered":"2023-11-28T21:34:42.345785+0000","last_clean":"2023-11-28T21:34:42.345785+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:34:42.345785+0000","last_undegraded":"2023-11-28T21:34:42.345785+0000","last_fullsized":"2023-11-28T21:34:42.345785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342908+0000","last_change":"2023-11-28T21:34:37.348126+0000","last_active":"2023-11-28T21:34:41.342908+0000","last_peered":"2023-11-28T21:34:41.342908+0000","last_clean":"2023-11-28T21:34:41.342908+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:34:41.342908+0000","last_undegraded":"2023-11-28T21:34:41.342908+0000","last_fullsized":"2023-11-28T21:34:41.342908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:47.982143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.596939+0000","last_change":"2023-11-28T21:34:41.388539+0000","last_active":"2023-11-28T21:34:42.596939+0000","last_peered":"2023-11-28T21:34:42.596939+0000","last_clean":"2023-11-28T21:34:42.596939+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:34:42.596939+0000","last_undegraded":"2023-11-28T21:34:42.596939+0000","last_fullsized":"2023-11-28T21:34:42.596939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343904+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:34:42.343904+0000","last_peered":"2023-11-28T21:34:42.343904+0000","last_clean":"2023-11-28T21:34:42.343904+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:34:42.343904+0000","last_undegraded":"2023-11-28T21:34:42.343904+0000","last_fullsized":"2023-11-28T21:34:42.343904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346931+0000","last_change":"2023-11-28T21:34:37.337266+0000","last_active":"2023-11-28T21:34:41.346931+0000","last_peered":"2023-11-28T21:34:41.346931+0000","last_clean":"2023-11-28T21:34:41.346931+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:34:41.346931+0000","last_undegraded":"2023-11-28T21:34:41.346931+0000","last_fullsized":"2023-11-28T21:34:41.346931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:32:48.848989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595154+0000","last_change":"2023-11-28T21:34:41.372957+0000","last_active":"2023-11-28T21:34:42.595154+0000","last_peered":"2023-11-28T21:34:42.595154+0000","last_clean":"2023-11-28T21:34:42.595154+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:34:42.595154+0000","last_undegraded":"2023-11-28T21:34:42.595154+0000","last_fullsized":"2023-11-28T21:34:42.595154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367244+0000","last_change":"2023-11-28T21:34:37.353174+0000","last_active":"2023-11-28T21:34:41.367244+0000","last_peered":"2023-11-28T21:34:41.367244+0000","last_clean":"2023-11-28T21:34:41.367244+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:34:41.367244+0000","last_undegraded":"2023-11-28T21:34:41.367244+0000","last_fullsized":"2023-11-28T21:34:41.367244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:32.966564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.304284+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:35:08.304284+0000","last_peered":"2023-11-28T21:35:08.304284+0000","last_clean":"2023-11-28T21:35:08.304284+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:35:08.304284+0000","last_undegraded":"2023-11-28T21:35:08.304284+0000","last_fullsized":"2023-11-28T21:35:08.304284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.599114+0000","last_change":"2023-11-28T21:34:41.364738+0000","last_active":"2023-11-28T21:34:42.599114+0000","last_peered":"2023-11-28T21:34:42.599114+0000","last_clean":"2023-11-28T21:34:42.599114+0000","last_became_active":"2023-11-28T21:34:41.362862+0000","last_became_peered":"2023-11-28T21:34:41.362862+0000","last_unstale":"2023-11-28T21:34:42.599114+0000","last_undegraded":"2023-11-28T21:34:42.599114+0000","last_fullsized":"2023-11-28T21:34:42.599114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:13.830214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339933+0000","last_change":"2023-11-28T21:34:37.313495+0000","last_active":"2023-11-28T21:34:41.339933+0000","last_peered":"2023-11-28T21:34:41.339933+0000","last_clean":"2023-11-28T21:34:41.339933+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:34:41.339933+0000","last_undegraded":"2023-11-28T21:34:41.339933+0000","last_fullsized":"2023-11-28T21:34:41.339933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:32:50.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345994+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:34:41.345994+0000","last_peered":"2023-11-28T21:34:41.345994+0000","last_clean":"2023-11-28T21:34:41.345994+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:34:41.345994+0000","last_undegraded":"2023-11-28T21:34:41.345994+0000","last_fullsized":"2023-11-28T21:34:41.345994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:53.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597596+0000","last_change":"2023-11-28T21:34:41.367580+0000","last_active":"2023-11-28T21:34:42.597596+0000","last_peered":"2023-11-28T21:34:42.597596+0000","last_clean":"2023-11-28T21:34:42.597596+0000","last_became_active":"2023-11-28T21:34:41.366442+0000","last_became_peered":"2023-11-28T21:34:41.366442+0000","last_unstale":"2023-11-28T21:34:42.597596+0000","last_undegraded":"2023-11-28T21:34:42.597596+0000","last_fullsized":"2023-11-28T21:34:42.597596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339648+0000","last_change":"2023-11-28T21:34:37.326569+0000","last_active":"2023-11-28T21:34:41.339648+0000","last_peered":"2023-11-28T21:34:41.339648+0000","last_clean":"2023-11-28T21:34:41.339648+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:34:41.339648+0000","last_undegraded":"2023-11-28T21:34:41.339648+0000","last_fullsized":"2023-11-28T21:34:41.339648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:39:14.264355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"24'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.334309+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:35:08.334309+0000","last_peered":"2023-11-28T21:35:08.334309+0000","last_clean":"2023-11-28T21:35:08.334309+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:35:08.334309+0000","last_undegraded":"2023-11-28T21:35:08.334309+0000","last_fullsized":"2023-11-28T21:35:08.334309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593262+0000","last_change":"2023-11-28T21:34:41.483661+0000","last_active":"2023-11-28T21:34:42.593262+0000","last_peered":"2023-11-28T21:34:42.593262+0000","last_clean":"2023-11-28T21:34:42.593262+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:34:42.593262+0000","last_undegraded":"2023-11-28T21:34:42.593262+0000","last_fullsized":"2023-11-28T21:34:42.593262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.347011+0000","last_change":"2023-11-28T21:34:37.336922+0000","last_active":"2023-11-28T21:34:41.347011+0000","last_peered":"2023-11-28T21:34:41.347011+0000","last_clean":"2023-11-28T21:34:41.347011+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:34:41.347011+0000","last_undegraded":"2023-11-28T21:34:41.347011+0000","last_fullsized":"2023-11-28T21:34:41.347011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:45.663475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"24'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.309635+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:35:08.309635+0000","last_peered":"2023-11-28T21:35:08.309635+0000","last_clean":"2023-11-28T21:35:08.309635+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:35:08.309635+0000","last_undegraded":"2023-11-28T21:35:08.309635+0000","last_fullsized":"2023-11-28T21:35:08.309635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.599296+0000","last_change":"2023-11-28T21:34:41.382756+0000","last_active":"2023-11-28T21:34:42.599296+0000","last_peered":"2023-11-28T21:34:42.599296+0000","last_clean":"2023-11-28T21:34:42.599296+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:34:42.599296+0000","last_undegraded":"2023-11-28T21:34:42.599296+0000","last_fullsized":"2023-11-28T21:34:42.599296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:07.402513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344639+0000","last_change":"2023-11-28T21:34:37.320446+0000","last_active":"2023-11-28T21:34:41.344639+0000","last_peered":"2023-11-28T21:34:41.344639+0000","last_clean":"2023-11-28T21:34:41.344639+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:34:41.344639+0000","last_undegraded":"2023-11-28T21:34:41.344639+0000","last_fullsized":"2023-11-28T21:34:41.344639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:33.404441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.329148+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:35:08.329148+0000","last_peered":"2023-11-28T21:35:08.329148+0000","last_clean":"2023-11-28T21:35:08.329148+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:35:08.329148+0000","last_undegraded":"2023-11-28T21:35:08.329148+0000","last_fullsized":"2023-11-28T21:35:08.329148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595588+0000","last_change":"2023-11-28T21:34:41.485107+0000","last_active":"2023-11-28T21:34:42.595588+0000","last_peered":"2023-11-28T21:34:42.595588+0000","last_clean":"2023-11-28T21:34:42.595588+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:34:42.595588+0000","last_undegraded":"2023-11-28T21:34:42.595588+0000","last_fullsized":"2023-11-28T21:34:42.595588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343566+0000","last_change":"2023-11-28T21:34:37.345104+0000","last_active":"2023-11-28T21:34:41.343566+0000","last_peered":"2023-11-28T21:34:41.343566+0000","last_clean":"2023-11-28T21:34:41.343566+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:34:41.343566+0000","last_undegraded":"2023-11-28T21:34:41.343566+0000","last_fullsized":"2023-11-28T21:34:41.343566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:25:16.125179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593312+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:34:42.593312+0000","last_peered":"2023-11-28T21:34:42.593312+0000","last_clean":"2023-11-28T21:34:42.593312+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:34:42.593312+0000","last_undegraded":"2023-11-28T21:34:42.593312+0000","last_fullsized":"2023-11-28T21:34:42.593312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:36.968220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359721+0000","last_change":"2023-11-28T21:34:41.475704+0000","last_active":"2023-11-28T21:34:42.359721+0000","last_peered":"2023-11-28T21:34:42.359721+0000","last_clean":"2023-11-28T21:34:42.359721+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:34:42.359721+0000","last_undegraded":"2023-11-28T21:34:42.359721+0000","last_fullsized":"2023-11-28T21:34:42.359721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:01:34.494307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341552+0000","last_change":"2023-11-28T21:34:37.330831+0000","last_active":"2023-11-28T21:34:41.341552+0000","last_peered":"2023-11-28T21:34:41.341552+0000","last_clean":"2023-11-28T21:34:41.341552+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:34:41.341552+0000","last_undegraded":"2023-11-28T21:34:41.341552+0000","last_fullsized":"2023-11-28T21:34:41.341552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:50:38.612360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.598662+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:34:42.598662+0000","last_peered":"2023-11-28T21:34:42.598662+0000","last_clean":"2023-11-28T21:34:42.598662+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:34:42.598662+0000","last_undegraded":"2023-11-28T21:34:42.598662+0000","last_fullsized":"2023-11-28T21:34:42.598662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592137+0000","last_change":"2023-11-28T21:34:41.475481+0000","last_active":"2023-11-28T21:34:42.592137+0000","last_peered":"2023-11-28T21:34:42.592137+0000","last_clean":"2023-11-28T21:34:42.592137+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:34:42.592137+0000","last_undegraded":"2023-11-28T21:34:42.592137+0000","last_fullsized":"2023-11-28T21:34:42.592137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:22:18.635807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.347169+0000","last_change":"2023-11-28T21:34:37.337846+0000","last_active":"2023-11-28T21:34:41.347169+0000","last_peered":"2023-11-28T21:34:41.347169+0000","last_clean":"2023-11-28T21:34:41.347169+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:34:41.347169+0000","last_undegraded":"2023-11-28T21:34:41.347169+0000","last_fullsized":"2023-11-28T21:34:41.347169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:46:21.358603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342465+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:34:41.342465+0000","last_peered":"2023-11-28T21:34:41.342465+0000","last_clean":"2023-11-28T21:34:41.342465+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:34:41.342465+0000","last_undegraded":"2023-11-28T21:34:41.342465+0000","last_fullsized":"2023-11-28T21:34:41.342465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:08:15.270352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591609+0000","last_change":"2023-11-28T21:34:41.370410+0000","last_active":"2023-11-28T21:34:42.591609+0000","last_peered":"2023-11-28T21:34:42.591609+0000","last_clean":"2023-11-28T21:34:42.591609+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:34:42.591609+0000","last_undegraded":"2023-11-28T21:34:42.591609+0000","last_fullsized":"2023-11-28T21:34:42.591609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342823+0000","last_change":"2023-11-28T21:34:37.324608+0000","last_active":"2023-11-28T21:34:41.342823+0000","last_peered":"2023-11-28T21:34:41.342823+0000","last_clean":"2023-11-28T21:34:41.342823+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:34:41.342823+0000","last_undegraded":"2023-11-28T21:34:41.342823+0000","last_fullsized":"2023-11-28T21:34:41.342823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:13:53.438473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341590+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:34:41.341590+0000","last_peered":"2023-11-28T21:34:41.341590+0000","last_clean":"2023-11-28T21:34:41.341590+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:34:41.341590+0000","last_undegraded":"2023-11-28T21:34:41.341590+0000","last_fullsized":"2023-11-28T21:34:41.341590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592949+0000","last_change":"2023-11-28T21:34:41.380599+0000","last_active":"2023-11-28T21:34:42.592949+0000","last_peered":"2023-11-28T21:34:42.592949+0000","last_clean":"2023-11-28T21:34:42.592949+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:34:42.592949+0000","last_undegraded":"2023-11-28T21:34:42.592949+0000","last_fullsized":"2023-11-28T21:34:42.592949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342555+0000","last_change":"2023-11-28T21:34:37.330869+0000","last_active":"2023-11-28T21:34:41.342555+0000","last_peered":"2023-11-28T21:34:41.342555+0000","last_clean":"2023-11-28T21:34:41.342555+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:34:41.342555+0000","last_undegraded":"2023-11-28T21:34:41.342555+0000","last_fullsized":"2023-11-28T21:34:41.342555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:58:55.987235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360360+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:34:42.360360+0000","last_peered":"2023-11-28T21:34:42.360360+0000","last_clean":"2023-11-28T21:34:42.360360+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:34:42.360360+0000","last_undegraded":"2023-11-28T21:34:42.360360+0000","last_fullsized":"2023-11-28T21:34:42.360360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593284+0000","last_change":"2023-11-28T21:34:41.371897+0000","last_active":"2023-11-28T21:34:42.593284+0000","last_peered":"2023-11-28T21:34:42.593284+0000","last_clean":"2023-11-28T21:34:42.593284+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:34:42.593284+0000","last_undegraded":"2023-11-28T21:34:42.593284+0000","last_fullsized":"2023-11-28T21:34:42.593284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343750+0000","last_change":"2023-11-28T21:34:37.319833+0000","last_active":"2023-11-28T21:34:41.343750+0000","last_peered":"2023-11-28T21:34:41.343750+0000","last_clean":"2023-11-28T21:34:41.343750+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:34:41.343750+0000","last_undegraded":"2023-11-28T21:34:41.343750+0000","last_fullsized":"2023-11-28T21:34:41.343750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:02:17.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595914+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:34:42.595914+0000","last_peered":"2023-11-28T21:34:42.595914+0000","last_clean":"2023-11-28T21:34:42.595914+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:34:42.595914+0000","last_undegraded":"2023-11-28T21:34:42.595914+0000","last_fullsized":"2023-11-28T21:34:42.595914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592392+0000","last_change":"2023-11-28T21:34:41.475053+0000","last_active":"2023-11-28T21:34:42.592392+0000","last_peered":"2023-11-28T21:34:42.592392+0000","last_clean":"2023-11-28T21:34:42.592392+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:34:42.592392+0000","last_undegraded":"2023-11-28T21:34:42.592392+0000","last_fullsized":"2023-11-28T21:34:42.592392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:32.658001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340067+0000","last_change":"2023-11-28T21:34:37.358890+0000","last_active":"2023-11-28T21:34:41.340067+0000","last_peered":"2023-11-28T21:34:41.340067+0000","last_clean":"2023-11-28T21:34:41.340067+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:34:41.340067+0000","last_undegraded":"2023-11-28T21:34:41.340067+0000","last_fullsized":"2023-11-28T21:34:41.340067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:54.878141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"24'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.332378+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:35:08.332378+0000","last_peered":"2023-11-28T21:35:08.332378+0000","last_clean":"2023-11-28T21:35:08.332378+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:35:08.332378+0000","last_undegraded":"2023-11-28T21:35:08.332378+0000","last_fullsized":"2023-11-28T21:35:08.332378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.469889+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:34:41.469889+0000","last_peered":"2023-11-28T21:34:41.469889+0000","last_clean":"2023-11-28T21:34:41.469889+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:34:41.469889+0000","last_undegraded":"2023-11-28T21:34:41.469889+0000","last_fullsized":"2023-11-28T21:34:41.469889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594727+0000","last_change":"2023-11-28T21:34:41.477522+0000","last_active":"2023-11-28T21:34:42.594727+0000","last_peered":"2023-11-28T21:34:42.594727+0000","last_clean":"2023-11-28T21:34:42.594727+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:34:42.594727+0000","last_undegraded":"2023-11-28T21:34:42.594727+0000","last_fullsized":"2023-11-28T21:34:42.594727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:36:36.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471469+0000","last_change":"2023-11-28T21:34:37.350441+0000","last_active":"2023-11-28T21:34:41.471469+0000","last_peered":"2023-11-28T21:34:41.471469+0000","last_clean":"2023-11-28T21:34:41.471469+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:34:41.471469+0000","last_undegraded":"2023-11-28T21:34:41.471469+0000","last_fullsized":"2023-11-28T21:34:41.471469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:54.775495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.355701+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:34:41.355701+0000","last_peered":"2023-11-28T21:34:41.355701+0000","last_clean":"2023-11-28T21:34:41.355701+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:34:41.355701+0000","last_undegraded":"2023-11-28T21:34:41.355701+0000","last_fullsized":"2023-11-28T21:34:41.355701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593850+0000","last_change":"2023-11-28T21:34:41.485154+0000","last_active":"2023-11-28T21:34:42.593850+0000","last_peered":"2023-11-28T21:34:42.593850+0000","last_clean":"2023-11-28T21:34:42.593850+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:34:42.593850+0000","last_undegraded":"2023-11-28T21:34:42.593850+0000","last_fullsized":"2023-11-28T21:34:42.593850+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:35:21.006172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343008+0000","last_change":"2023-11-28T21:34:37.345652+0000","last_active":"2023-11-28T21:34:41.343008+0000","last_peered":"2023-11-28T21:34:41.343008+0000","last_clean":"2023-11-28T21:34:41.343008+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:34:41.343008+0000","last_undegraded":"2023-11-28T21:34:41.343008+0000","last_fullsized":"2023-11-28T21:34:41.343008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:21:17.474200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343244+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:34:42.343244+0000","last_peered":"2023-11-28T21:34:42.343244+0000","last_clean":"2023-11-28T21:34:42.343244+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:34:42.343244+0000","last_undegraded":"2023-11-28T21:34:42.343244+0000","last_fullsized":"2023-11-28T21:34:42.343244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:48.073446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592358+0000","last_change":"2023-11-28T21:34:41.370764+0000","last_active":"2023-11-28T21:34:42.592358+0000","last_peered":"2023-11-28T21:34:42.592358+0000","last_clean":"2023-11-28T21:34:42.592358+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:34:42.592358+0000","last_undegraded":"2023-11-28T21:34:42.592358+0000","last_fullsized":"2023-11-28T21:34:42.592358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:06:42.297098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471917+0000","last_change":"2023-11-28T21:34:37.359589+0000","last_active":"2023-11-28T21:34:41.471917+0000","last_peered":"2023-11-28T21:34:41.471917+0000","last_clean":"2023-11-28T21:34:41.471917+0000","last_became_active":"2023-11-28T21:34:37.359042+0000","last_became_peered":"2023-11-28T21:34:37.359042+0000","last_unstale":"2023-11-28T21:34:41.471917+0000","last_undegraded":"2023-11-28T21:34:41.471917+0000","last_fullsized":"2023-11-28T21:34:41.471917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:27.020614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.339535+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:34:42.339535+0000","last_peered":"2023-11-28T21:34:42.339535+0000","last_clean":"2023-11-28T21:34:42.339535+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:34:42.339535+0000","last_undegraded":"2023-11-28T21:34:42.339535+0000","last_fullsized":"2023-11-28T21:34:42.339535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.362523+0000","last_change":"2023-11-28T21:34:41.477127+0000","last_active":"2023-11-28T21:34:42.362523+0000","last_peered":"2023-11-28T21:34:42.362523+0000","last_clean":"2023-11-28T21:34:42.362523+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:34:42.362523+0000","last_undegraded":"2023-11-28T21:34:42.362523+0000","last_fullsized":"2023-11-28T21:34:42.362523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:32.798817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471715+0000","last_change":"2023-11-28T21:34:37.355693+0000","last_active":"2023-11-28T21:34:41.471715+0000","last_peered":"2023-11-28T21:34:41.471715+0000","last_clean":"2023-11-28T21:34:41.471715+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:34:41.471715+0000","last_undegraded":"2023-11-28T21:34:41.471715+0000","last_fullsized":"2023-11-28T21:34:41.471715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:12.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.338222+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:34:42.338222+0000","last_peered":"2023-11-28T21:34:42.338222+0000","last_clean":"2023-11-28T21:34:42.338222+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:34:42.338222+0000","last_undegraded":"2023-11-28T21:34:42.338222+0000","last_fullsized":"2023-11-28T21:34:42.338222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594073+0000","last_change":"2023-11-28T21:34:41.382468+0000","last_active":"2023-11-28T21:34:42.594073+0000","last_peered":"2023-11-28T21:34:42.594073+0000","last_clean":"2023-11-28T21:34:42.594073+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:34:42.594073+0000","last_undegraded":"2023-11-28T21:34:42.594073+0000","last_fullsized":"2023-11-28T21:34:42.594073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:17.410943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470942+0000","last_change":"2023-11-28T21:34:37.353232+0000","last_active":"2023-11-28T21:34:41.470942+0000","last_peered":"2023-11-28T21:34:41.470942+0000","last_clean":"2023-11-28T21:34:41.470942+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:34:41.470942+0000","last_undegraded":"2023-11-28T21:34:41.470942+0000","last_fullsized":"2023-11-28T21:34:41.470942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:11.285378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'10","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.317627+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:35:08.317627+0000","last_peered":"2023-11-28T21:35:08.317627+0000","last_clean":"2023-11-28T21:35:08.317627+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:35:08.317627+0000","last_undegraded":"2023-11-28T21:35:08.317627+0000","last_fullsized":"2023-11-28T21:35:08.317627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592317+0000","last_change":"2023-11-28T21:34:41.385400+0000","last_active":"2023-11-28T21:34:42.592317+0000","last_peered":"2023-11-28T21:34:42.592317+0000","last_clean":"2023-11-28T21:34:42.592317+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:34:42.592317+0000","last_undegraded":"2023-11-28T21:34:42.592317+0000","last_fullsized":"2023-11-28T21:34:42.592317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:36.433439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365468+0000","last_change":"2023-11-28T21:34:37.333461+0000","last_active":"2023-11-28T21:34:41.365468+0000","last_peered":"2023-11-28T21:34:41.365468+0000","last_clean":"2023-11-28T21:34:41.365468+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:34:41.365468+0000","last_undegraded":"2023-11-28T21:34:41.365468+0000","last_fullsized":"2023-11-28T21:34:41.365468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:25.513524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.366026+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:34:41.366026+0000","last_peered":"2023-11-28T21:34:41.366026+0000","last_clean":"2023-11-28T21:34:41.366026+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:34:41.366026+0000","last_undegraded":"2023-11-28T21:34:41.366026+0000","last_fullsized":"2023-11-28T21:34:41.366026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361391+0000","last_change":"2023-11-28T21:34:41.479919+0000","last_active":"2023-11-28T21:34:42.361391+0000","last_peered":"2023-11-28T21:34:42.361391+0000","last_clean":"2023-11-28T21:34:42.361391+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:34:42.361391+0000","last_undegraded":"2023-11-28T21:34:42.361391+0000","last_fullsized":"2023-11-28T21:34:42.361391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:11.183117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471078+0000","last_change":"2023-11-28T21:34:37.355729+0000","last_active":"2023-11-28T21:34:41.471078+0000","last_peered":"2023-11-28T21:34:41.471078+0000","last_clean":"2023-11-28T21:34:41.471078+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:34:41.471078+0000","last_undegraded":"2023-11-28T21:34:41.471078+0000","last_fullsized":"2023-11-28T21:34:41.471078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:39.178378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"24'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.335897+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:35:08.335897+0000","last_peered":"2023-11-28T21:35:08.335897+0000","last_clean":"2023-11-28T21:35:08.335897+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:35:08.335897+0000","last_undegraded":"2023-11-28T21:35:08.335897+0000","last_fullsized":"2023-11-28T21:35:08.335897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361306+0000","last_change":"2023-11-28T21:34:41.478483+0000","last_active":"2023-11-28T21:34:42.361306+0000","last_peered":"2023-11-28T21:34:42.361306+0000","last_clean":"2023-11-28T21:34:42.361306+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:34:42.361306+0000","last_undegraded":"2023-11-28T21:34:42.361306+0000","last_fullsized":"2023-11-28T21:34:42.361306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:25:54.342692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345020+0000","last_change":"2023-11-28T21:34:37.345476+0000","last_active":"2023-11-28T21:34:41.345020+0000","last_peered":"2023-11-28T21:34:41.345020+0000","last_clean":"2023-11-28T21:34:41.345020+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:34:41.345020+0000","last_undegraded":"2023-11-28T21:34:41.345020+0000","last_fullsized":"2023-11-28T21:34:41.345020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.660412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.337504+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:35:08.337504+0000","last_peered":"2023-11-28T21:35:08.337504+0000","last_clean":"2023-11-28T21:35:08.337504+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:35:08.337504+0000","last_undegraded":"2023-11-28T21:35:08.337504+0000","last_fullsized":"2023-11-28T21:35:08.337504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:28.694184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363244+0000","last_change":"2023-11-28T21:34:41.477926+0000","last_active":"2023-11-28T21:34:42.363244+0000","last_peered":"2023-11-28T21:34:42.363244+0000","last_clean":"2023-11-28T21:34:42.363244+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:34:42.363244+0000","last_undegraded":"2023-11-28T21:34:42.363244+0000","last_fullsized":"2023-11-28T21:34:42.363244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:26.403506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471448+0000","last_change":"2023-11-28T21:34:37.343885+0000","last_active":"2023-11-28T21:34:41.471448+0000","last_peered":"2023-11-28T21:34:41.471448+0000","last_clean":"2023-11-28T21:34:41.471448+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:34:41.471448+0000","last_undegraded":"2023-11-28T21:34:41.471448+0000","last_fullsized":"2023-11-28T21:34:41.471448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:36.255834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'11","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.336181+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:35:08.336181+0000","last_peered":"2023-11-28T21:35:08.336181+0000","last_clean":"2023-11-28T21:35:08.336181+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:35:08.336181+0000","last_undegraded":"2023-11-28T21:35:08.336181+0000","last_fullsized":"2023-11-28T21:35:08.336181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591595+0000","last_change":"2023-11-28T21:34:41.380565+0000","last_active":"2023-11-28T21:34:42.591595+0000","last_peered":"2023-11-28T21:34:42.591595+0000","last_clean":"2023-11-28T21:34:42.591595+0000","last_became_active":"2023-11-28T21:34:41.380080+0000","last_became_peered":"2023-11-28T21:34:41.380080+0000","last_unstale":"2023-11-28T21:34:42.591595+0000","last_undegraded":"2023-11-28T21:34:42.591595+0000","last_fullsized":"2023-11-28T21:34:42.591595+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343524+0000","last_change":"2023-11-28T21:34:37.345635+0000","last_active":"2023-11-28T21:34:41.343524+0000","last_peered":"2023-11-28T21:34:41.343524+0000","last_clean":"2023-11-28T21:34:41.343524+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:34:41.343524+0000","last_undegraded":"2023-11-28T21:34:41.343524+0000","last_fullsized":"2023-11-28T21:34:41.343524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:52:12.144267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593063+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:34:42.593063+0000","last_peered":"2023-11-28T21:34:42.593063+0000","last_clean":"2023-11-28T21:34:42.593063+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:34:42.593063+0000","last_undegraded":"2023-11-28T21:34:42.593063+0000","last_fullsized":"2023-11-28T21:34:42.593063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:35.266650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597618+0000","last_change":"2023-11-28T21:34:41.365758+0000","last_active":"2023-11-28T21:34:42.597618+0000","last_peered":"2023-11-28T21:34:42.597618+0000","last_clean":"2023-11-28T21:34:42.597618+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:34:42.597618+0000","last_undegraded":"2023-11-28T21:34:42.597618+0000","last_fullsized":"2023-11-28T21:34:42.597618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344832+0000","last_change":"2023-11-28T21:34:37.333607+0000","last_active":"2023-11-28T21:34:41.344832+0000","last_peered":"2023-11-28T21:34:41.344832+0000","last_clean":"2023-11-28T21:34:41.344832+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:34:41.344832+0000","last_undegraded":"2023-11-28T21:34:41.344832+0000","last_fullsized":"2023-11-28T21:34:41.344832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:13:01.639908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"24'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.334144+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:35:08.334144+0000","last_peered":"2023-11-28T21:35:08.334144+0000","last_clean":"2023-11-28T21:35:08.334144+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:35:08.334144+0000","last_undegraded":"2023-11-28T21:35:08.334144+0000","last_fullsized":"2023-11-28T21:35:08.334144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359893+0000","last_change":"2023-11-28T21:34:41.380413+0000","last_active":"2023-11-28T21:34:42.359893+0000","last_peered":"2023-11-28T21:34:42.359893+0000","last_clean":"2023-11-28T21:34:42.359893+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:34:42.359893+0000","last_undegraded":"2023-11-28T21:34:42.359893+0000","last_fullsized":"2023-11-28T21:34:42.359893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:20:28.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342001+0000","last_change":"2023-11-28T21:34:37.351009+0000","last_active":"2023-11-28T21:34:41.342001+0000","last_peered":"2023-11-28T21:34:41.342001+0000","last_clean":"2023-11-28T21:34:41.342001+0000","last_became_active":"2023-11-28T21:34:37.350321+0000","last_became_peered":"2023-11-28T21:34:37.350321+0000","last_unstale":"2023-11-28T21:34:41.342001+0000","last_undegraded":"2023-11-28T21:34:41.342001+0000","last_fullsized":"2023-11-28T21:34:41.342001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:27:42.022466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"24'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.335647+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:35:08.335647+0000","last_peered":"2023-11-28T21:35:08.335647+0000","last_clean":"2023-11-28T21:35:08.335647+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:35:08.335647+0000","last_undegraded":"2023-11-28T21:35:08.335647+0000","last_fullsized":"2023-11-28T21:35:08.335647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359889+0000","last_change":"2023-11-28T21:34:41.475185+0000","last_active":"2023-11-28T21:34:42.359889+0000","last_peered":"2023-11-28T21:34:42.359889+0000","last_clean":"2023-11-28T21:34:42.359889+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:34:42.359889+0000","last_undegraded":"2023-11-28T21:34:42.359889+0000","last_fullsized":"2023-11-28T21:34:42.359889+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:09.389881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592347+0000","last_change":"2023-11-28T21:34:41.479956+0000","last_active":"2023-11-28T21:34:42.592347+0000","last_peered":"2023-11-28T21:34:42.592347+0000","last_clean":"2023-11-28T21:34:42.592347+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:34:42.592347+0000","last_undegraded":"2023-11-28T21:34:42.592347+0000","last_fullsized":"2023-11-28T21:34:42.592347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:46.004852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343050+0000","last_change":"2023-11-28T21:34:37.317513+0000","last_active":"2023-11-28T21:34:41.343050+0000","last_peered":"2023-11-28T21:34:41.343050+0000","last_clean":"2023-11-28T21:34:41.343050+0000","last_became_active":"2023-11-28T21:34:37.317082+0000","last_became_peered":"2023-11-28T21:34:37.317082+0000","last_unstale":"2023-11-28T21:34:41.343050+0000","last_undegraded":"2023-11-28T21:34:41.343050+0000","last_fullsized":"2023-11-28T21:34:41.343050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:05:37.846078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.362815+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:34:42.362815+0000","last_peered":"2023-11-28T21:34:42.362815+0000","last_clean":"2023-11-28T21:34:42.362815+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:34:42.362815+0000","last_undegraded":"2023-11-28T21:34:42.362815+0000","last_fullsized":"2023-11-28T21:34:42.362815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595858+0000","last_change":"2023-11-28T21:34:41.385969+0000","last_active":"2023-11-28T21:34:42.595858+0000","last_peered":"2023-11-28T21:34:42.595858+0000","last_clean":"2023-11-28T21:34:42.595858+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:34:42.595858+0000","last_undegraded":"2023-11-28T21:34:42.595858+0000","last_fullsized":"2023-11-28T21:34:42.595858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:01.692549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339578+0000","last_change":"2023-11-28T21:34:37.314087+0000","last_active":"2023-11-28T21:34:41.339578+0000","last_peered":"2023-11-28T21:34:41.339578+0000","last_clean":"2023-11-28T21:34:41.339578+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:34:41.339578+0000","last_undegraded":"2023-11-28T21:34:41.339578+0000","last_fullsized":"2023-11-28T21:34:41.339578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:25:11.912313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340387+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:34:41.340387+0000","last_peered":"2023-11-28T21:34:41.340387+0000","last_clean":"2023-11-28T21:34:41.340387+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:34:41.340387+0000","last_undegraded":"2023-11-28T21:34:41.340387+0000","last_fullsized":"2023-11-28T21:34:41.340387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343264+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:34:42.343264+0000","last_peered":"2023-11-28T21:34:42.343264+0000","last_clean":"2023-11-28T21:34:42.343264+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:34:42.343264+0000","last_undegraded":"2023-11-28T21:34:42.343264+0000","last_fullsized":"2023-11-28T21:34:42.343264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345191+0000","last_change":"2023-11-28T21:34:37.319801+0000","last_active":"2023-11-28T21:34:41.345191+0000","last_peered":"2023-11-28T21:34:41.345191+0000","last_clean":"2023-11-28T21:34:41.345191+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:34:41.345191+0000","last_undegraded":"2023-11-28T21:34:41.345191+0000","last_fullsized":"2023-11-28T21:34:41.345191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:40:13.251522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360747+0000","last_change":"2023-11-28T21:34:41.370523+0000","last_active":"2023-11-28T21:34:42.360747+0000","last_peered":"2023-11-28T21:34:42.360747+0000","last_clean":"2023-11-28T21:34:42.360747+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:34:42.360747+0000","last_undegraded":"2023-11-28T21:34:42.360747+0000","last_fullsized":"2023-11-28T21:34:42.360747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:17:36.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'5","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.334661+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:35:08.334661+0000","last_peered":"2023-11-28T21:35:08.334661+0000","last_clean":"2023-11-28T21:35:08.334661+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:35:08.334661+0000","last_undegraded":"2023-11-28T21:35:08.334661+0000","last_fullsized":"2023-11-28T21:35:08.334661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342331+0000","last_change":"2023-11-28T21:34:37.328809+0000","last_active":"2023-11-28T21:34:41.342331+0000","last_peered":"2023-11-28T21:34:41.342331+0000","last_clean":"2023-11-28T21:34:41.342331+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:34:41.342331+0000","last_undegraded":"2023-11-28T21:34:41.342331+0000","last_fullsized":"2023-11-28T21:34:41.342331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:27.300090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361127+0000","last_change":"2023-11-28T21:34:41.363087+0000","last_active":"2023-11-28T21:34:42.361127+0000","last_peered":"2023-11-28T21:34:42.361127+0000","last_clean":"2023-11-28T21:34:42.361127+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:34:42.361127+0000","last_undegraded":"2023-11-28T21:34:42.361127+0000","last_fullsized":"2023-11-28T21:34:42.361127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:35.046132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365193+0000","last_change":"2023-11-28T21:34:37.333670+0000","last_active":"2023-11-28T21:34:41.365193+0000","last_peered":"2023-11-28T21:34:41.365193+0000","last_clean":"2023-11-28T21:34:41.365193+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:34:41.365193+0000","last_undegraded":"2023-11-28T21:34:41.365193+0000","last_fullsized":"2023-11-28T21:34:41.365193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:32.598014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.355458+0000","last_change":"2023-11-28T21:34:37.347101+0000","last_active":"2023-11-28T21:34:41.355458+0000","last_peered":"2023-11-28T21:34:41.355458+0000","last_clean":"2023-11-28T21:34:41.355458+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:34:41.355458+0000","last_undegraded":"2023-11-28T21:34:41.355458+0000","last_fullsized":"2023-11-28T21:34:41.355458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:39:43.186110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341187+0000","last_change":"2023-11-28T21:34:37.338576+0000","last_active":"2023-11-28T21:34:41.341187+0000","last_peered":"2023-11-28T21:34:41.341187+0000","last_clean":"2023-11-28T21:34:41.341187+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:34:41.341187+0000","last_undegraded":"2023-11-28T21:34:41.341187+0000","last_fullsized":"2023-11-28T21:34:41.341187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:53:02.356097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345097+0000","last_change":"2023-11-28T21:34:37.316270+0000","last_active":"2023-11-28T21:34:41.345097+0000","last_peered":"2023-11-28T21:34:41.345097+0000","last_clean":"2023-11-28T21:34:41.345097+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:34:41.345097+0000","last_undegraded":"2023-11-28T21:34:41.345097+0000","last_fullsized":"2023-11-28T21:34:41.345097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:43.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340024+0000","last_change":"2023-11-28T21:34:37.334973+0000","last_active":"2023-11-28T21:34:41.340024+0000","last_peered":"2023-11-28T21:34:41.340024+0000","last_clean":"2023-11-28T21:34:41.340024+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:34:41.340024+0000","last_undegraded":"2023-11-28T21:34:41.340024+0000","last_fullsized":"2023-11-28T21:34:41.340024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:55.050945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345807+0000","last_change":"2023-11-28T21:34:37.318155+0000","last_active":"2023-11-28T21:34:41.345807+0000","last_peered":"2023-11-28T21:34:41.345807+0000","last_clean":"2023-11-28T21:34:41.345807+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:34:41.345807+0000","last_undegraded":"2023-11-28T21:34:41.345807+0000","last_fullsized":"2023-11-28T21:34:41.345807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:29.840255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367255+0000","last_change":"2023-11-28T21:34:37.345758+0000","last_active":"2023-11-28T21:34:41.367255+0000","last_peered":"2023-11-28T21:34:41.367255+0000","last_clean":"2023-11-28T21:34:41.367255+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:34:41.367255+0000","last_undegraded":"2023-11-28T21:34:41.367255+0000","last_fullsized":"2023-11-28T21:34:41.367255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:26.444654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343483+0000","last_change":"2023-11-28T21:34:37.334989+0000","last_active":"2023-11-28T21:34:41.343483+0000","last_peered":"2023-11-28T21:34:41.343483+0000","last_clean":"2023-11-28T21:34:41.343483+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:34:41.343483+0000","last_undegraded":"2023-11-28T21:34:41.343483+0000","last_fullsized":"2023-11-28T21:34:41.343483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:36:57.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346163+0000","last_change":"2023-11-28T21:34:37.333075+0000","last_active":"2023-11-28T21:34:41.346163+0000","last_peered":"2023-11-28T21:34:41.346163+0000","last_clean":"2023-11-28T21:34:41.346163+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:34:41.346163+0000","last_undegraded":"2023-11-28T21:34:41.346163+0000","last_fullsized":"2023-11-28T21:34:41.346163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:50:43.748349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339854+0000","last_change":"2023-11-28T21:34:37.337230+0000","last_active":"2023-11-28T21:34:41.339854+0000","last_peered":"2023-11-28T21:34:41.339854+0000","last_clean":"2023-11-28T21:34:41.339854+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:34:41.339854+0000","last_undegraded":"2023-11-28T21:34:41.339854+0000","last_fullsized":"2023-11-28T21:34:41.339854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:48:45.994559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340945+0000","last_change":"2023-11-28T21:34:37.349894+0000","last_active":"2023-11-28T21:34:41.340945+0000","last_peered":"2023-11-28T21:34:41.340945+0000","last_clean":"2023-11-28T21:34:41.340945+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:34:41.340945+0000","last_undegraded":"2023-11-28T21:34:41.340945+0000","last_fullsized":"2023-11-28T21:34:41.340945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:10.293911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345003+0000","last_change":"2023-11-28T21:34:37.344269+0000","last_active":"2023-11-28T21:34:41.345003+0000","last_peered":"2023-11-28T21:34:41.345003+0000","last_clean":"2023-11-28T21:34:41.345003+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:34:41.345003+0000","last_undegraded":"2023-11-28T21:34:41.345003+0000","last_fullsized":"2023-11-28T21:34:41.345003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:03.204740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365122+0000","last_change":"2023-11-28T21:34:37.353059+0000","last_active":"2023-11-28T21:34:41.365122+0000","last_peered":"2023-11-28T21:34:41.365122+0000","last_clean":"2023-11-28T21:34:41.365122+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:34:41.365122+0000","last_undegraded":"2023-11-28T21:34:41.365122+0000","last_fullsized":"2023-11-28T21:34:41.365122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:45:12.640797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471801+0000","last_change":"2023-11-28T21:34:37.352965+0000","last_active":"2023-11-28T21:34:41.471801+0000","last_peered":"2023-11-28T21:34:41.471801+0000","last_clean":"2023-11-28T21:34:41.471801+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:34:41.471801+0000","last_undegraded":"2023-11-28T21:34:41.471801+0000","last_fullsized":"2023-11-28T21:34:41.471801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:30:42.087715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.356122+0000","last_change":"2023-11-28T21:34:37.336830+0000","last_active":"2023-11-28T21:34:41.356122+0000","last_peered":"2023-11-28T21:34:41.356122+0000","last_clean":"2023-11-28T21:34:41.356122+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:34:41.356122+0000","last_undegraded":"2023-11-28T21:34:41.356122+0000","last_fullsized":"2023-11-28T21:34:41.356122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:57.318311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341567+0000","last_change":"2023-11-28T21:34:37.349181+0000","last_active":"2023-11-28T21:34:41.341567+0000","last_peered":"2023-11-28T21:34:41.341567+0000","last_clean":"2023-11-28T21:34:41.341567+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:34:41.341567+0000","last_undegraded":"2023-11-28T21:34:41.341567+0000","last_fullsized":"2023-11-28T21:34:41.341567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:27:59.989299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345968+0000","last_change":"2023-11-28T21:34:37.336668+0000","last_active":"2023-11-28T21:34:41.345968+0000","last_peered":"2023-11-28T21:34:41.345968+0000","last_clean":"2023-11-28T21:34:41.345968+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:34:41.345968+0000","last_undegraded":"2023-11-28T21:34:41.345968+0000","last_fullsized":"2023-11-28T21:34:41.345968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:06.416717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343147+0000","last_change":"2023-11-28T21:34:37.332790+0000","last_active":"2023-11-28T21:34:41.343147+0000","last_peered":"2023-11-28T21:34:41.343147+0000","last_clean":"2023-11-28T21:34:41.343147+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:34:41.343147+0000","last_undegraded":"2023-11-28T21:34:41.343147+0000","last_fullsized":"2023-11-28T21:34:41.343147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:32:45.920387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344930+0000","last_change":"2023-11-28T21:34:37.320794+0000","last_active":"2023-11-28T21:34:41.344930+0000","last_peered":"2023-11-28T21:34:41.344930+0000","last_clean":"2023-11-28T21:34:41.344930+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:34:41.344930+0000","last_undegraded":"2023-11-28T21:34:41.344930+0000","last_fullsized":"2023-11-28T21:34:41.344930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:16:46.347683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339818+0000","last_change":"2023-11-28T21:34:37.337726+0000","last_active":"2023-11-28T21:34:41.339818+0000","last_peered":"2023-11-28T21:34:41.339818+0000","last_clean":"2023-11-28T21:34:41.339818+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:34:41.339818+0000","last_undegraded":"2023-11-28T21:34:41.339818+0000","last_fullsized":"2023-11-28T21:34:41.339818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:38:43.884668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343259+0000","last_change":"2023-11-28T21:34:37.334677+0000","last_active":"2023-11-28T21:34:41.343259+0000","last_peered":"2023-11-28T21:34:41.343259+0000","last_clean":"2023-11-28T21:34:41.343259+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:34:41.343259+0000","last_undegraded":"2023-11-28T21:34:41.343259+0000","last_fullsized":"2023-11-28T21:34:41.343259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:05:54.981899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.366695+0000","last_change":"2023-11-28T21:34:37.354861+0000","last_active":"2023-11-28T21:34:41.366695+0000","last_peered":"2023-11-28T21:34:41.366695+0000","last_clean":"2023-11-28T21:34:41.366695+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:34:41.366695+0000","last_undegraded":"2023-11-28T21:34:41.366695+0000","last_fullsized":"2023-11-28T21:34:41.366695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:30.602284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367055+0000","last_change":"2023-11-28T21:34:37.347675+0000","last_active":"2023-11-28T21:34:41.367055+0000","last_peered":"2023-11-28T21:34:41.367055+0000","last_clean":"2023-11-28T21:34:41.367055+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:34:41.367055+0000","last_undegraded":"2023-11-28T21:34:41.367055+0000","last_fullsized":"2023-11-28T21:34:41.367055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:51:38.803896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471861+0000","last_change":"2023-11-28T21:34:37.357735+0000","last_active":"2023-11-28T21:34:41.471861+0000","last_peered":"2023-11-28T21:34:41.471861+0000","last_clean":"2023-11-28T21:34:41.471861+0000","last_became_active":"2023-11-28T21:34:37.357173+0000","last_became_peered":"2023-11-28T21:34:37.357173+0000","last_unstale":"2023-11-28T21:34:41.471861+0000","last_undegraded":"2023-11-28T21:34:41.471861+0000","last_fullsized":"2023-11-28T21:34:41.471861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:38.660713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339546+0000","last_change":"2023-11-28T21:34:37.358851+0000","last_active":"2023-11-28T21:34:41.339546+0000","last_peered":"2023-11-28T21:34:41.339546+0000","last_clean":"2023-11-28T21:34:41.339546+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:34:41.339546+0000","last_undegraded":"2023-11-28T21:34:41.339546+0000","last_fullsized":"2023-11-28T21:34:41.339546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.378519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470714+0000","last_change":"2023-11-28T21:34:37.343972+0000","last_active":"2023-11-28T21:34:41.470714+0000","last_peered":"2023-11-28T21:34:41.470714+0000","last_clean":"2023-11-28T21:34:41.470714+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:34:41.470714+0000","last_undegraded":"2023-11-28T21:34:41.470714+0000","last_fullsized":"2023-11-28T21:34:41.470714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:56:00.591888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340667+0000","last_change":"2023-11-28T21:34:37.346299+0000","last_active":"2023-11-28T21:34:41.340667+0000","last_peered":"2023-11-28T21:34:41.340667+0000","last_clean":"2023-11-28T21:34:41.340667+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:34:41.340667+0000","last_undegraded":"2023-11-28T21:34:41.340667+0000","last_fullsized":"2023-11-28T21:34:41.340667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:06.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470739+0000","last_change":"2023-11-28T21:34:37.343410+0000","last_active":"2023-11-28T21:34:41.470739+0000","last_peered":"2023-11-28T21:34:41.470739+0000","last_clean":"2023-11-28T21:34:41.470739+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:34:41.470739+0000","last_undegraded":"2023-11-28T21:34:41.470739+0000","last_fullsized":"2023-11-28T21:34:41.470739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:29:45.070478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342784+0000","last_change":"2023-11-28T21:34:37.348270+0000","last_active":"2023-11-28T21:34:41.342784+0000","last_peered":"2023-11-28T21:34:41.342784+0000","last_clean":"2023-11-28T21:34:41.342784+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:34:41.342784+0000","last_undegraded":"2023-11-28T21:34:41.342784+0000","last_fullsized":"2023-11-28T21:34:41.342784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:36:10.938478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344314+0000","last_change":"2023-11-28T21:34:37.330786+0000","last_active":"2023-11-28T21:34:41.344314+0000","last_peered":"2023-11-28T21:34:41.344314+0000","last_clean":"2023-11-28T21:34:41.344314+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:34:41.344314+0000","last_undegraded":"2023-11-28T21:34:41.344314+0000","last_fullsized":"2023-11-28T21:34:41.344314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:52:36.738867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346251+0000","last_change":"2023-11-28T21:34:37.318725+0000","last_active":"2023-11-28T21:34:41.346251+0000","last_peered":"2023-11-28T21:34:41.346251+0000","last_clean":"2023-11-28T21:34:41.346251+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:34:41.346251+0000","last_undegraded":"2023-11-28T21:34:41.346251+0000","last_fullsized":"2023-11-28T21:34:41.346251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:49:21.775066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471764+0000","last_change":"2023-11-28T21:34:37.349829+0000","last_active":"2023-11-28T21:34:41.471764+0000","last_peered":"2023-11-28T21:34:41.471764+0000","last_clean":"2023-11-28T21:34:41.471764+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:34:41.471764+0000","last_undegraded":"2023-11-28T21:34:41.471764+0000","last_fullsized":"2023-11-28T21:34:41.471764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:07.752862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6045,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":12090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":690742,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":100624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":689779,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:35:11.298 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:35:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.476+0000 7f0235516640 1 Processor -- start 2023-11-28T21:35:11.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.476+0000 7f0235516640 1 -- start start 2023-11-28T21:35:11.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.476+0000 7f0235516640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0230151ed0 0x7f02300c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.476+0000 7f0235516640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f023006e300 0x7f0230072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.476+0000 7f0235516640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0230157b70 con 0x7f0230151af0 2023-11-28T21:35:11.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.476+0000 7f0235516640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02300707f0 con 0x7f023006e300 2023-11-28T21:35:11.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.476+0000 7f0235516640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02300c7b90 con 0x7f0230151ed0 2023-11-28T21:35:11.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022effd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0230151af0 0x7f02300c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36512/0 (socket says 172.21.15.12:36512) 2023-11-28T21:35:11.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022effd640 1 -- 172.21.15.12:0/94749820 learned_addr learned my addr 172.21.15.12:0/94749820 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:11.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f0227fff640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0230151ed0 0x7f02300c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022f7fe640 1 --2- 172.21.15.12:0/94749820 >> v2:172.21.15.144:3300/0 conn(0x7f023006e300 0x7f0230072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144240756 0 0) 0x7f0230157b70 con 0x7f0230151af0 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f021c003530 con 0x7f0230151af0 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1940042334 0 0) 0x7f021c003530 con 0x7f0230151af0 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0230151ed0 msgr2=0x7f02300c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0230151ed0 0x7f02300c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 >> v2:172.21.15.144:3300/0 conn(0x7f023006e300 msgr2=0x7f0230072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 --2- 172.21.15.12:0/94749820 >> v2:172.21.15.144:3300/0 conn(0x7f023006e300 0x7f0230072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230157e50 con 0x7f0230151af0 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f0220001680 con 0x7f0230151af0 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 >> v1:172.21.15.12:6789/0 conn(0x7f0230151af0 legacy=0x7f02300c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f021c0039f0 0x7f021c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 0x7f021c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c004ee0 0x7f021c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02300707f0 con 0x7f021c0043a0 2023-11-28T21:35:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02300c7b90 con 0x7f021c004ee0 2023-11-28T21:35:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021c00bed0 con 0x7f021c0039f0 2023-11-28T21:35:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f0227fff640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 0x7f021c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022f7fe640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c004ee0 0x7f021c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f022effd640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f021c0039f0 0x7f021c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f0227fff640 1 -- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f021c0039f0 msgr2=0x7f021c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:11.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f0227fff640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f021c0039f0 0x7f021c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f0227fff640 1 -- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c004ee0 msgr2=0x7f021c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:11.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f0227fff640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c004ee0 0x7f021c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.480+0000 7f0227fff640 1 -- 172.21.15.12:0/94749820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c003530 con 0x7f021c0043a0 2023-11-28T21:35:11.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f0227fff640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 0x7f021c004870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0218009ab0 tx=0x7f021801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abd1c76a03b8ce4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:11.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02180a55f0 con 0x7f021c0043a0 2023-11-28T21:35:11.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02180a5750 con 0x7f021c0043a0 2023-11-28T21:35:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f022e7fc640 1 -- 172.21.15.12:0/94749820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02180a5b00 con 0x7f021c0043a0 2023-11-28T21:35:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f0235516640 1 -- 172.21.15.12:0/94749820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 msgr2=0x7f021c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f0235516640 1 --2- 172.21.15.12:0/94749820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 0x7f021c004870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0218009ab0 tx=0x7f021801ca00 comp rx=0 tx=0).stop 2023-11-28T21:35:11.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f022f7fe640 1 -- 172.21.15.12:0/94749820 reap_dead start 2023-11-28T21:35:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f0235516640 1 -- 172.21.15.12:0/94749820 shutdown_connections 2023-11-28T21:35:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f0235516640 1 -- 172.21.15.12:0/94749820 >> 172.21.15.12:0/94749820 conn(0x7f0230066360 msgr2=0x7f0230066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f0235516640 1 -- 172.21.15.12:0/94749820 shutdown_connections 2023-11-28T21:35:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f0235516640 1 -- 172.21.15.12:0/94749820 wait complete. 2023-11-28T21:35:11.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f0235516640 1 Processor -- start 2023-11-28T21:35:11.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.484+0000 7f0235516640 1 -- start start 2023-11-28T21:35:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f0235516640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 0x7f023014d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f0235516640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02301569e0 0x7f0230156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f0235516640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0230140e10 0x7f0230141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f0235516640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023014c5c0 con 0x7f021c0043a0 2023-11-28T21:35:11.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f0235516640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023014ddf0 con 0x7f02301569e0 2023-11-28T21:35:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f0235516640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0230157380 con 0x7f0230140e10 2023-11-28T21:35:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f022effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 0x7f023014d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f022effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 0x7f023014d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38448/0 (socket says 172.21.15.12:38448) 2023-11-28T21:35:11.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f022effd640 1 -- 172.21.15.12:0/4251554094 learned_addr learned my addr 172.21.15.12:0/4251554094 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f022f7fe640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0230140e10 0x7f0230141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f0227fff640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02301569e0 0x7f0230156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f022effd640 1 -- 172.21.15.12:0/4251554094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0230140e10 msgr2=0x7f0230141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f022effd640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0230140e10 0x7f0230141290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f022effd640 1 -- 172.21.15.12:0/4251554094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02301569e0 msgr2=0x7f0230156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f022effd640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02301569e0 0x7f0230156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f022effd640 1 -- 172.21.15.12:0/4251554094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230155bb0 con 0x7f021c0043a0 2023-11-28T21:35:11.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f022effd640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 0x7f023014d8b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0220011840 tx=0x7f0220011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:11.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f02277fe640 1 -- 172.21.15.12:0/4251554094 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022000fb60 con 0x7f021c0043a0 2023-11-28T21:35:11.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f02277fe640 1 -- 172.21.15.12:0/4251554094 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022000fd20 con 0x7f021c0043a0 2023-11-28T21:35:11.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f02277fe640 1 -- 172.21.15.12:0/4251554094 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0220015650 con 0x7f021c0043a0 2023-11-28T21:35:11.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f0235516640 1 -- 172.21.15.12:0/4251554094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02300c4460 con 0x7f021c0043a0 2023-11-28T21:35:11.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f0235516640 1 -- 172.21.15.12:0/4251554094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02301555f0 con 0x7f021c0043a0 2023-11-28T21:35:11.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.488+0000 7f0235516640 1 -- 172.21.15.12:0/4251554094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0230074f60 con 0x7f021c0043a0 2023-11-28T21:35:11.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.492+0000 7f02277fe640 1 -- 172.21.15.12:0/4251554094 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f02200157b0 con 0x7f021c0043a0 2023-11-28T21:35:11.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.492+0000 7f02277fe640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02100664c0 0x7f0210068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.492+0000 7f02277fe640 1 -- 172.21.15.12:0/4251554094 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0220087360 con 0x7f021c0043a0 2023-11-28T21:35:11.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.492+0000 7f0227fff640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02100664c0 0x7f0210068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.492+0000 7f0227fff640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02100664c0 0x7f0210068980 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f02180af000 tx=0x7f02180046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:11.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.496+0000 7f02277fe640 1 -- 172.21.15.12:0/4251554094 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0220054580 con 0x7f021c0043a0 2023-11-28T21:35:11.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.652+0000 7f0235516640 1 -- 172.21.15.12:0/4251554094 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02301586f0 con 0x7f02100664c0 2023-11-28T21:35:11.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.660+0000 7f02277fe640 1 -- 172.21.15.12:0/4251554094 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345437 (secure 0 0 0) 0x7f02180c1450 con 0x7f02100664c0 2023-11-28T21:35:11.667 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:11.669 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:35:11.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 -- 172.21.15.12:0/4251554094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02100664c0 msgr2=0x7f0210068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02100664c0 0x7f0210068980 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f02180af000 tx=0x7f02180046d0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 -- 172.21.15.12:0/4251554094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 msgr2=0x7f023014d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 0x7f023014d8b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0220011840 tx=0x7f0220011d00 comp rx=0 tx=0).stop 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 -- 172.21.15.12:0/4251554094 shutdown_connections 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02100664c0 0x7f0210068980 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0230140e10 0x7f0230141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02301569e0 0x7f0230156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 --2- 172.21.15.12:0/4251554094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c0043a0 0x7f023014d8b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 -- 172.21.15.12:0/4251554094 >> 172.21.15.12:0/4251554094 conn(0x7f0230066360 msgr2=0x7f023006ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 -- 172.21.15.12:0/4251554094 shutdown_connections 2023-11-28T21:35:11.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.664+0000 7f0235516640 1 -- 172.21.15.12:0/4251554094 wait complete. 2023-11-28T21:35:11.687 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-11-28T21:35:09.823155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596413,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":246,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193,"ondisk_log_size":193,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218336,"kb_used_data":2180,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756384,"statfs":{"total":773094113280,"available":772870537216,"internally_reserved":0,"allocated":2232320,"data_stored":1988003,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1547,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002861"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343838+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:34:42.343838+0000","last_peered":"2023-11-28T21:34:42.343838+0000","last_clean":"2023-11-28T21:34:42.343838+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:34:42.343838+0000","last_undegraded":"2023-11-28T21:34:42.343838+0000","last_fullsized":"2023-11-28T21:34:42.343838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:46.167511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341374+0000","last_change":"2023-11-28T21:34:37.335656+0000","last_active":"2023-11-28T21:34:41.341374+0000","last_peered":"2023-11-28T21:34:41.341374+0000","last_clean":"2023-11-28T21:34:41.341374+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:34:41.341374+0000","last_undegraded":"2023-11-28T21:34:41.341374+0000","last_fullsized":"2023-11-28T21:34:41.341374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:25:49.707378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359842+0000","last_change":"2023-11-28T21:34:41.483856+0000","last_active":"2023-11-28T21:34:42.359842+0000","last_peered":"2023-11-28T21:34:42.359842+0000","last_clean":"2023-11-28T21:34:42.359842+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:34:42.359842+0000","last_undegraded":"2023-11-28T21:34:42.359842+0000","last_fullsized":"2023-11-28T21:34:42.359842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:50:56.005578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.340464+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:34:42.340464+0000","last_peered":"2023-11-28T21:34:42.340464+0000","last_clean":"2023-11-28T21:34:42.340464+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:34:42.340464+0000","last_undegraded":"2023-11-28T21:34:42.340464+0000","last_fullsized":"2023-11-28T21:34:42.340464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344725+0000","last_change":"2023-11-28T21:34:37.333649+0000","last_active":"2023-11-28T21:34:41.344725+0000","last_peered":"2023-11-28T21:34:41.344725+0000","last_clean":"2023-11-28T21:34:41.344725+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:34:41.344725+0000","last_undegraded":"2023-11-28T21:34:41.344725+0000","last_fullsized":"2023-11-28T21:34:41.344725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:21:42.041590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594292+0000","last_change":"2023-11-28T21:34:41.388232+0000","last_active":"2023-11-28T21:34:42.594292+0000","last_peered":"2023-11-28T21:34:42.594292+0000","last_clean":"2023-11-28T21:34:42.594292+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:34:42.594292+0000","last_undegraded":"2023-11-28T21:34:42.594292+0000","last_fullsized":"2023-11-28T21:34:42.594292+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:22.960012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.345785+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:34:42.345785+0000","last_peered":"2023-11-28T21:34:42.345785+0000","last_clean":"2023-11-28T21:34:42.345785+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:34:42.345785+0000","last_undegraded":"2023-11-28T21:34:42.345785+0000","last_fullsized":"2023-11-28T21:34:42.345785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342908+0000","last_change":"2023-11-28T21:34:37.348126+0000","last_active":"2023-11-28T21:34:41.342908+0000","last_peered":"2023-11-28T21:34:41.342908+0000","last_clean":"2023-11-28T21:34:41.342908+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:34:41.342908+0000","last_undegraded":"2023-11-28T21:34:41.342908+0000","last_fullsized":"2023-11-28T21:34:41.342908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:47.982143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.596939+0000","last_change":"2023-11-28T21:34:41.388539+0000","last_active":"2023-11-28T21:34:42.596939+0000","last_peered":"2023-11-28T21:34:42.596939+0000","last_clean":"2023-11-28T21:34:42.596939+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:34:42.596939+0000","last_undegraded":"2023-11-28T21:34:42.596939+0000","last_fullsized":"2023-11-28T21:34:42.596939+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343904+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:34:42.343904+0000","last_peered":"2023-11-28T21:34:42.343904+0000","last_clean":"2023-11-28T21:34:42.343904+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:34:42.343904+0000","last_undegraded":"2023-11-28T21:34:42.343904+0000","last_fullsized":"2023-11-28T21:34:42.343904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346931+0000","last_change":"2023-11-28T21:34:37.337266+0000","last_active":"2023-11-28T21:34:41.346931+0000","last_peered":"2023-11-28T21:34:41.346931+0000","last_clean":"2023-11-28T21:34:41.346931+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:34:41.346931+0000","last_undegraded":"2023-11-28T21:34:41.346931+0000","last_fullsized":"2023-11-28T21:34:41.346931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:32:48.848989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595154+0000","last_change":"2023-11-28T21:34:41.372957+0000","last_active":"2023-11-28T21:34:42.595154+0000","last_peered":"2023-11-28T21:34:42.595154+0000","last_clean":"2023-11-28T21:34:42.595154+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:34:42.595154+0000","last_undegraded":"2023-11-28T21:34:42.595154+0000","last_fullsized":"2023-11-28T21:34:42.595154+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367244+0000","last_change":"2023-11-28T21:34:37.353174+0000","last_active":"2023-11-28T21:34:41.367244+0000","last_peered":"2023-11-28T21:34:41.367244+0000","last_clean":"2023-11-28T21:34:41.367244+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:34:41.367244+0000","last_undegraded":"2023-11-28T21:34:41.367244+0000","last_fullsized":"2023-11-28T21:34:41.367244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:32.966564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.304284+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:35:08.304284+0000","last_peered":"2023-11-28T21:35:08.304284+0000","last_clean":"2023-11-28T21:35:08.304284+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:35:08.304284+0000","last_undegraded":"2023-11-28T21:35:08.304284+0000","last_fullsized":"2023-11-28T21:35:08.304284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.599114+0000","last_change":"2023-11-28T21:34:41.364738+0000","last_active":"2023-11-28T21:34:42.599114+0000","last_peered":"2023-11-28T21:34:42.599114+0000","last_clean":"2023-11-28T21:34:42.599114+0000","last_became_active":"2023-11-28T21:34:41.362862+0000","last_became_peered":"2023-11-28T21:34:41.362862+0000","last_unstale":"2023-11-28T21:34:42.599114+0000","last_undegraded":"2023-11-28T21:34:42.599114+0000","last_fullsized":"2023-11-28T21:34:42.599114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:03:13.830214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339933+0000","last_change":"2023-11-28T21:34:37.313495+0000","last_active":"2023-11-28T21:34:41.339933+0000","last_peered":"2023-11-28T21:34:41.339933+0000","last_clean":"2023-11-28T21:34:41.339933+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:34:41.339933+0000","last_undegraded":"2023-11-28T21:34:41.339933+0000","last_fullsized":"2023-11-28T21:34:41.339933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:32:50.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345994+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:34:41.345994+0000","last_peered":"2023-11-28T21:34:41.345994+0000","last_clean":"2023-11-28T21:34:41.345994+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:34:41.345994+0000","last_undegraded":"2023-11-28T21:34:41.345994+0000","last_fullsized":"2023-11-28T21:34:41.345994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:53.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597596+0000","last_change":"2023-11-28T21:34:41.367580+0000","last_active":"2023-11-28T21:34:42.597596+0000","last_peered":"2023-11-28T21:34:42.597596+0000","last_clean":"2023-11-28T21:34:42.597596+0000","last_became_active":"2023-11-28T21:34:41.366442+0000","last_became_peered":"2023-11-28T21:34:41.366442+0000","last_unstale":"2023-11-28T21:34:42.597596+0000","last_undegraded":"2023-11-28T21:34:42.597596+0000","last_fullsized":"2023-11-28T21:34:42.597596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339648+0000","last_change":"2023-11-28T21:34:37.326569+0000","last_active":"2023-11-28T21:34:41.339648+0000","last_peered":"2023-11-28T21:34:41.339648+0000","last_clean":"2023-11-28T21:34:41.339648+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:34:41.339648+0000","last_undegraded":"2023-11-28T21:34:41.339648+0000","last_fullsized":"2023-11-28T21:34:41.339648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:39:14.264355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"24'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.334309+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:35:08.334309+0000","last_peered":"2023-11-28T21:35:08.334309+0000","last_clean":"2023-11-28T21:35:08.334309+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:35:08.334309+0000","last_undegraded":"2023-11-28T21:35:08.334309+0000","last_fullsized":"2023-11-28T21:35:08.334309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593262+0000","last_change":"2023-11-28T21:34:41.483661+0000","last_active":"2023-11-28T21:34:42.593262+0000","last_peered":"2023-11-28T21:34:42.593262+0000","last_clean":"2023-11-28T21:34:42.593262+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:34:42.593262+0000","last_undegraded":"2023-11-28T21:34:42.593262+0000","last_fullsized":"2023-11-28T21:34:42.593262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.347011+0000","last_change":"2023-11-28T21:34:37.336922+0000","last_active":"2023-11-28T21:34:41.347011+0000","last_peered":"2023-11-28T21:34:41.347011+0000","last_clean":"2023-11-28T21:34:41.347011+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:34:41.347011+0000","last_undegraded":"2023-11-28T21:34:41.347011+0000","last_fullsized":"2023-11-28T21:34:41.347011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:45.663475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"24'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.309635+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:35:08.309635+0000","last_peered":"2023-11-28T21:35:08.309635+0000","last_clean":"2023-11-28T21:35:08.309635+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:35:08.309635+0000","last_undegraded":"2023-11-28T21:35:08.309635+0000","last_fullsized":"2023-11-28T21:35:08.309635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.599296+0000","last_change":"2023-11-28T21:34:41.382756+0000","last_active":"2023-11-28T21:34:42.599296+0000","last_peered":"2023-11-28T21:34:42.599296+0000","last_clean":"2023-11-28T21:34:42.599296+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:34:42.599296+0000","last_undegraded":"2023-11-28T21:34:42.599296+0000","last_fullsized":"2023-11-28T21:34:42.599296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:07.402513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344639+0000","last_change":"2023-11-28T21:34:37.320446+0000","last_active":"2023-11-28T21:34:41.344639+0000","last_peered":"2023-11-28T21:34:41.344639+0000","last_clean":"2023-11-28T21:34:41.344639+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:34:41.344639+0000","last_undegraded":"2023-11-28T21:34:41.344639+0000","last_fullsized":"2023-11-28T21:34:41.344639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:33.404441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.329148+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:35:08.329148+0000","last_peered":"2023-11-28T21:35:08.329148+0000","last_clean":"2023-11-28T21:35:08.329148+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:35:08.329148+0000","last_undegraded":"2023-11-28T21:35:08.329148+0000","last_fullsized":"2023-11-28T21:35:08.329148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595588+0000","last_change":"2023-11-28T21:34:41.485107+0000","last_active":"2023-11-28T21:34:42.595588+0000","last_peered":"2023-11-28T21:34:42.595588+0000","last_clean":"2023-11-28T21:34:42.595588+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:34:42.595588+0000","last_undegraded":"2023-11-28T21:34:42.595588+0000","last_fullsized":"2023-11-28T21:34:42.595588+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343566+0000","last_change":"2023-11-28T21:34:37.345104+0000","last_active":"2023-11-28T21:34:41.343566+0000","last_peered":"2023-11-28T21:34:41.343566+0000","last_clean":"2023-11-28T21:34:41.343566+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:34:41.343566+0000","last_undegraded":"2023-11-28T21:34:41.343566+0000","last_fullsized":"2023-11-28T21:34:41.343566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:25:16.125179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593312+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:34:42.593312+0000","last_peered":"2023-11-28T21:34:42.593312+0000","last_clean":"2023-11-28T21:34:42.593312+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:34:42.593312+0000","last_undegraded":"2023-11-28T21:34:42.593312+0000","last_fullsized":"2023-11-28T21:34:42.593312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:36.968220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359721+0000","last_change":"2023-11-28T21:34:41.475704+0000","last_active":"2023-11-28T21:34:42.359721+0000","last_peered":"2023-11-28T21:34:42.359721+0000","last_clean":"2023-11-28T21:34:42.359721+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:34:42.359721+0000","last_undegraded":"2023-11-28T21:34:42.359721+0000","last_fullsized":"2023-11-28T21:34:42.359721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:01:34.494307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341552+0000","last_change":"2023-11-28T21:34:37.330831+0000","last_active":"2023-11-28T21:34:41.341552+0000","last_peered":"2023-11-28T21:34:41.341552+0000","last_clean":"2023-11-28T21:34:41.341552+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:34:41.341552+0000","last_undegraded":"2023-11-28T21:34:41.341552+0000","last_fullsized":"2023-11-28T21:34:41.341552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:50:38.612360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.598662+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:34:42.598662+0000","last_peered":"2023-11-28T21:34:42.598662+0000","last_clean":"2023-11-28T21:34:42.598662+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:34:42.598662+0000","last_undegraded":"2023-11-28T21:34:42.598662+0000","last_fullsized":"2023-11-28T21:34:42.598662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592137+0000","last_change":"2023-11-28T21:34:41.475481+0000","last_active":"2023-11-28T21:34:42.592137+0000","last_peered":"2023-11-28T21:34:42.592137+0000","last_clean":"2023-11-28T21:34:42.592137+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:34:42.592137+0000","last_undegraded":"2023-11-28T21:34:42.592137+0000","last_fullsized":"2023-11-28T21:34:42.592137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:22:18.635807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.347169+0000","last_change":"2023-11-28T21:34:37.337846+0000","last_active":"2023-11-28T21:34:41.347169+0000","last_peered":"2023-11-28T21:34:41.347169+0000","last_clean":"2023-11-28T21:34:41.347169+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:34:41.347169+0000","last_undegraded":"2023-11-28T21:34:41.347169+0000","last_fullsized":"2023-11-28T21:34:41.347169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:46:21.358603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342465+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:34:41.342465+0000","last_peered":"2023-11-28T21:34:41.342465+0000","last_clean":"2023-11-28T21:34:41.342465+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:34:41.342465+0000","last_undegraded":"2023-11-28T21:34:41.342465+0000","last_fullsized":"2023-11-28T21:34:41.342465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:08:15.270352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591609+0000","last_change":"2023-11-28T21:34:41.370410+0000","last_active":"2023-11-28T21:34:42.591609+0000","last_peered":"2023-11-28T21:34:42.591609+0000","last_clean":"2023-11-28T21:34:42.591609+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:34:42.591609+0000","last_undegraded":"2023-11-28T21:34:42.591609+0000","last_fullsized":"2023-11-28T21:34:42.591609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342823+0000","last_change":"2023-11-28T21:34:37.324608+0000","last_active":"2023-11-28T21:34:41.342823+0000","last_peered":"2023-11-28T21:34:41.342823+0000","last_clean":"2023-11-28T21:34:41.342823+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:34:41.342823+0000","last_undegraded":"2023-11-28T21:34:41.342823+0000","last_fullsized":"2023-11-28T21:34:41.342823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:13:53.438473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341590+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:34:41.341590+0000","last_peered":"2023-11-28T21:34:41.341590+0000","last_clean":"2023-11-28T21:34:41.341590+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:34:41.341590+0000","last_undegraded":"2023-11-28T21:34:41.341590+0000","last_fullsized":"2023-11-28T21:34:41.341590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592949+0000","last_change":"2023-11-28T21:34:41.380599+0000","last_active":"2023-11-28T21:34:42.592949+0000","last_peered":"2023-11-28T21:34:42.592949+0000","last_clean":"2023-11-28T21:34:42.592949+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:34:42.592949+0000","last_undegraded":"2023-11-28T21:34:42.592949+0000","last_fullsized":"2023-11-28T21:34:42.592949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342555+0000","last_change":"2023-11-28T21:34:37.330869+0000","last_active":"2023-11-28T21:34:41.342555+0000","last_peered":"2023-11-28T21:34:41.342555+0000","last_clean":"2023-11-28T21:34:41.342555+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:34:41.342555+0000","last_undegraded":"2023-11-28T21:34:41.342555+0000","last_fullsized":"2023-11-28T21:34:41.342555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:58:55.987235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360360+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:34:42.360360+0000","last_peered":"2023-11-28T21:34:42.360360+0000","last_clean":"2023-11-28T21:34:42.360360+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:34:42.360360+0000","last_undegraded":"2023-11-28T21:34:42.360360+0000","last_fullsized":"2023-11-28T21:34:42.360360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593284+0000","last_change":"2023-11-28T21:34:41.371897+0000","last_active":"2023-11-28T21:34:42.593284+0000","last_peered":"2023-11-28T21:34:42.593284+0000","last_clean":"2023-11-28T21:34:42.593284+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:34:42.593284+0000","last_undegraded":"2023-11-28T21:34:42.593284+0000","last_fullsized":"2023-11-28T21:34:42.593284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343750+0000","last_change":"2023-11-28T21:34:37.319833+0000","last_active":"2023-11-28T21:34:41.343750+0000","last_peered":"2023-11-28T21:34:41.343750+0000","last_clean":"2023-11-28T21:34:41.343750+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:34:41.343750+0000","last_undegraded":"2023-11-28T21:34:41.343750+0000","last_fullsized":"2023-11-28T21:34:41.343750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:02:17.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595914+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:34:42.595914+0000","last_peered":"2023-11-28T21:34:42.595914+0000","last_clean":"2023-11-28T21:34:42.595914+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:34:42.595914+0000","last_undegraded":"2023-11-28T21:34:42.595914+0000","last_fullsized":"2023-11-28T21:34:42.595914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592392+0000","last_change":"2023-11-28T21:34:41.475053+0000","last_active":"2023-11-28T21:34:42.592392+0000","last_peered":"2023-11-28T21:34:42.592392+0000","last_clean":"2023-11-28T21:34:42.592392+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:34:42.592392+0000","last_undegraded":"2023-11-28T21:34:42.592392+0000","last_fullsized":"2023-11-28T21:34:42.592392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:32.658001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340067+0000","last_change":"2023-11-28T21:34:37.358890+0000","last_active":"2023-11-28T21:34:41.340067+0000","last_peered":"2023-11-28T21:34:41.340067+0000","last_clean":"2023-11-28T21:34:41.340067+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:34:41.340067+0000","last_undegraded":"2023-11-28T21:34:41.340067+0000","last_fullsized":"2023-11-28T21:34:41.340067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:54.878141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"24'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.332378+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:35:08.332378+0000","last_peered":"2023-11-28T21:35:08.332378+0000","last_clean":"2023-11-28T21:35:08.332378+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:35:08.332378+0000","last_undegraded":"2023-11-28T21:35:08.332378+0000","last_fullsized":"2023-11-28T21:35:08.332378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.469889+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:34:41.469889+0000","last_peered":"2023-11-28T21:34:41.469889+0000","last_clean":"2023-11-28T21:34:41.469889+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:34:41.469889+0000","last_undegraded":"2023-11-28T21:34:41.469889+0000","last_fullsized":"2023-11-28T21:34:41.469889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594727+0000","last_change":"2023-11-28T21:34:41.477522+0000","last_active":"2023-11-28T21:34:42.594727+0000","last_peered":"2023-11-28T21:34:42.594727+0000","last_clean":"2023-11-28T21:34:42.594727+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:34:42.594727+0000","last_undegraded":"2023-11-28T21:34:42.594727+0000","last_fullsized":"2023-11-28T21:34:42.594727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:36:36.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471469+0000","last_change":"2023-11-28T21:34:37.350441+0000","last_active":"2023-11-28T21:34:41.471469+0000","last_peered":"2023-11-28T21:34:41.471469+0000","last_clean":"2023-11-28T21:34:41.471469+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:34:41.471469+0000","last_undegraded":"2023-11-28T21:34:41.471469+0000","last_fullsized":"2023-11-28T21:34:41.471469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:54.775495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.355701+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:34:41.355701+0000","last_peered":"2023-11-28T21:34:41.355701+0000","last_clean":"2023-11-28T21:34:41.355701+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:34:41.355701+0000","last_undegraded":"2023-11-28T21:34:41.355701+0000","last_fullsized":"2023-11-28T21:34:41.355701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593850+0000","last_change":"2023-11-28T21:34:41.485154+0000","last_active":"2023-11-28T21:34:42.593850+0000","last_peered":"2023-11-28T21:34:42.593850+0000","last_clean":"2023-11-28T21:34:42.593850+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:34:42.593850+0000","last_undegraded":"2023-11-28T21:34:42.593850+0000","last_fullsized":"2023-11-28T21:34:42.593850+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:35:21.006172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343008+0000","last_change":"2023-11-28T21:34:37.345652+0000","last_active":"2023-11-28T21:34:41.343008+0000","last_peered":"2023-11-28T21:34:41.343008+0000","last_clean":"2023-11-28T21:34:41.343008+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:34:41.343008+0000","last_undegraded":"2023-11-28T21:34:41.343008+0000","last_fullsized":"2023-11-28T21:34:41.343008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:21:17.474200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343244+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:34:42.343244+0000","last_peered":"2023-11-28T21:34:42.343244+0000","last_clean":"2023-11-28T21:34:42.343244+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:34:42.343244+0000","last_undegraded":"2023-11-28T21:34:42.343244+0000","last_fullsized":"2023-11-28T21:34:42.343244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:48.073446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592358+0000","last_change":"2023-11-28T21:34:41.370764+0000","last_active":"2023-11-28T21:34:42.592358+0000","last_peered":"2023-11-28T21:34:42.592358+0000","last_clean":"2023-11-28T21:34:42.592358+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:34:42.592358+0000","last_undegraded":"2023-11-28T21:34:42.592358+0000","last_fullsized":"2023-11-28T21:34:42.592358+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:06:42.297098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471917+0000","last_change":"2023-11-28T21:34:37.359589+0000","last_active":"2023-11-28T21:34:41.471917+0000","last_peered":"2023-11-28T21:34:41.471917+0000","last_clean":"2023-11-28T21:34:41.471917+0000","last_became_active":"2023-11-28T21:34:37.359042+0000","last_became_peered":"2023-11-28T21:34:37.359042+0000","last_unstale":"2023-11-28T21:34:41.471917+0000","last_undegraded":"2023-11-28T21:34:41.471917+0000","last_fullsized":"2023-11-28T21:34:41.471917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:56:27.020614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.339535+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:34:42.339535+0000","last_peered":"2023-11-28T21:34:42.339535+0000","last_clean":"2023-11-28T21:34:42.339535+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:34:42.339535+0000","last_undegraded":"2023-11-28T21:34:42.339535+0000","last_fullsized":"2023-11-28T21:34:42.339535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.362523+0000","last_change":"2023-11-28T21:34:41.477127+0000","last_active":"2023-11-28T21:34:42.362523+0000","last_peered":"2023-11-28T21:34:42.362523+0000","last_clean":"2023-11-28T21:34:42.362523+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:34:42.362523+0000","last_undegraded":"2023-11-28T21:34:42.362523+0000","last_fullsized":"2023-11-28T21:34:42.362523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:32.798817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471715+0000","last_change":"2023-11-28T21:34:37.355693+0000","last_active":"2023-11-28T21:34:41.471715+0000","last_peered":"2023-11-28T21:34:41.471715+0000","last_clean":"2023-11-28T21:34:41.471715+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:34:41.471715+0000","last_undegraded":"2023-11-28T21:34:41.471715+0000","last_fullsized":"2023-11-28T21:34:41.471715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:12.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.338222+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:34:42.338222+0000","last_peered":"2023-11-28T21:34:42.338222+0000","last_clean":"2023-11-28T21:34:42.338222+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:34:42.338222+0000","last_undegraded":"2023-11-28T21:34:42.338222+0000","last_fullsized":"2023-11-28T21:34:42.338222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594073+0000","last_change":"2023-11-28T21:34:41.382468+0000","last_active":"2023-11-28T21:34:42.594073+0000","last_peered":"2023-11-28T21:34:42.594073+0000","last_clean":"2023-11-28T21:34:42.594073+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:34:42.594073+0000","last_undegraded":"2023-11-28T21:34:42.594073+0000","last_fullsized":"2023-11-28T21:34:42.594073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:17.410943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470942+0000","last_change":"2023-11-28T21:34:37.353232+0000","last_active":"2023-11-28T21:34:41.470942+0000","last_peered":"2023-11-28T21:34:41.470942+0000","last_clean":"2023-11-28T21:34:41.470942+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:34:41.470942+0000","last_undegraded":"2023-11-28T21:34:41.470942+0000","last_fullsized":"2023-11-28T21:34:41.470942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:11.285378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'10","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.317627+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:35:08.317627+0000","last_peered":"2023-11-28T21:35:08.317627+0000","last_clean":"2023-11-28T21:35:08.317627+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:35:08.317627+0000","last_undegraded":"2023-11-28T21:35:08.317627+0000","last_fullsized":"2023-11-28T21:35:08.317627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592317+0000","last_change":"2023-11-28T21:34:41.385400+0000","last_active":"2023-11-28T21:34:42.592317+0000","last_peered":"2023-11-28T21:34:42.592317+0000","last_clean":"2023-11-28T21:34:42.592317+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:34:42.592317+0000","last_undegraded":"2023-11-28T21:34:42.592317+0000","last_fullsized":"2023-11-28T21:34:42.592317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:36.433439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365468+0000","last_change":"2023-11-28T21:34:37.333461+0000","last_active":"2023-11-28T21:34:41.365468+0000","last_peered":"2023-11-28T21:34:41.365468+0000","last_clean":"2023-11-28T21:34:41.365468+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:34:41.365468+0000","last_undegraded":"2023-11-28T21:34:41.365468+0000","last_fullsized":"2023-11-28T21:34:41.365468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:25.513524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.366026+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:34:41.366026+0000","last_peered":"2023-11-28T21:34:41.366026+0000","last_clean":"2023-11-28T21:34:41.366026+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:34:41.366026+0000","last_undegraded":"2023-11-28T21:34:41.366026+0000","last_fullsized":"2023-11-28T21:34:41.366026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361391+0000","last_change":"2023-11-28T21:34:41.479919+0000","last_active":"2023-11-28T21:34:42.361391+0000","last_peered":"2023-11-28T21:34:42.361391+0000","last_clean":"2023-11-28T21:34:42.361391+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:34:42.361391+0000","last_undegraded":"2023-11-28T21:34:42.361391+0000","last_fullsized":"2023-11-28T21:34:42.361391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:11.183117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471078+0000","last_change":"2023-11-28T21:34:37.355729+0000","last_active":"2023-11-28T21:34:41.471078+0000","last_peered":"2023-11-28T21:34:41.471078+0000","last_clean":"2023-11-28T21:34:41.471078+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:34:41.471078+0000","last_undegraded":"2023-11-28T21:34:41.471078+0000","last_fullsized":"2023-11-28T21:34:41.471078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:39.178378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"24'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.335897+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:35:08.335897+0000","last_peered":"2023-11-28T21:35:08.335897+0000","last_clean":"2023-11-28T21:35:08.335897+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:35:08.335897+0000","last_undegraded":"2023-11-28T21:35:08.335897+0000","last_fullsized":"2023-11-28T21:35:08.335897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361306+0000","last_change":"2023-11-28T21:34:41.478483+0000","last_active":"2023-11-28T21:34:42.361306+0000","last_peered":"2023-11-28T21:34:42.361306+0000","last_clean":"2023-11-28T21:34:42.361306+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:34:42.361306+0000","last_undegraded":"2023-11-28T21:34:42.361306+0000","last_fullsized":"2023-11-28T21:34:42.361306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:25:54.342692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345020+0000","last_change":"2023-11-28T21:34:37.345476+0000","last_active":"2023-11-28T21:34:41.345020+0000","last_peered":"2023-11-28T21:34:41.345020+0000","last_clean":"2023-11-28T21:34:41.345020+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:34:41.345020+0000","last_undegraded":"2023-11-28T21:34:41.345020+0000","last_fullsized":"2023-11-28T21:34:41.345020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.660412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.337504+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:35:08.337504+0000","last_peered":"2023-11-28T21:35:08.337504+0000","last_clean":"2023-11-28T21:35:08.337504+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:35:08.337504+0000","last_undegraded":"2023-11-28T21:35:08.337504+0000","last_fullsized":"2023-11-28T21:35:08.337504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:28.694184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363244+0000","last_change":"2023-11-28T21:34:41.477926+0000","last_active":"2023-11-28T21:34:42.363244+0000","last_peered":"2023-11-28T21:34:42.363244+0000","last_clean":"2023-11-28T21:34:42.363244+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:34:42.363244+0000","last_undegraded":"2023-11-28T21:34:42.363244+0000","last_fullsized":"2023-11-28T21:34:42.363244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:26.403506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471448+0000","last_change":"2023-11-28T21:34:37.343885+0000","last_active":"2023-11-28T21:34:41.471448+0000","last_peered":"2023-11-28T21:34:41.471448+0000","last_clean":"2023-11-28T21:34:41.471448+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:34:41.471448+0000","last_undegraded":"2023-11-28T21:34:41.471448+0000","last_fullsized":"2023-11-28T21:34:41.471448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:36.255834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'11","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.336181+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:35:08.336181+0000","last_peered":"2023-11-28T21:35:08.336181+0000","last_clean":"2023-11-28T21:35:08.336181+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:35:08.336181+0000","last_undegraded":"2023-11-28T21:35:08.336181+0000","last_fullsized":"2023-11-28T21:35:08.336181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591595+0000","last_change":"2023-11-28T21:34:41.380565+0000","last_active":"2023-11-28T21:34:42.591595+0000","last_peered":"2023-11-28T21:34:42.591595+0000","last_clean":"2023-11-28T21:34:42.591595+0000","last_became_active":"2023-11-28T21:34:41.380080+0000","last_became_peered":"2023-11-28T21:34:41.380080+0000","last_unstale":"2023-11-28T21:34:42.591595+0000","last_undegraded":"2023-11-28T21:34:42.591595+0000","last_fullsized":"2023-11-28T21:34:42.591595+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343524+0000","last_change":"2023-11-28T21:34:37.345635+0000","last_active":"2023-11-28T21:34:41.343524+0000","last_peered":"2023-11-28T21:34:41.343524+0000","last_clean":"2023-11-28T21:34:41.343524+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:34:41.343524+0000","last_undegraded":"2023-11-28T21:34:41.343524+0000","last_fullsized":"2023-11-28T21:34:41.343524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:52:12.144267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593063+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:34:42.593063+0000","last_peered":"2023-11-28T21:34:42.593063+0000","last_clean":"2023-11-28T21:34:42.593063+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:34:42.593063+0000","last_undegraded":"2023-11-28T21:34:42.593063+0000","last_fullsized":"2023-11-28T21:34:42.593063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:35.266650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597618+0000","last_change":"2023-11-28T21:34:41.365758+0000","last_active":"2023-11-28T21:34:42.597618+0000","last_peered":"2023-11-28T21:34:42.597618+0000","last_clean":"2023-11-28T21:34:42.597618+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:34:42.597618+0000","last_undegraded":"2023-11-28T21:34:42.597618+0000","last_fullsized":"2023-11-28T21:34:42.597618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344832+0000","last_change":"2023-11-28T21:34:37.333607+0000","last_active":"2023-11-28T21:34:41.344832+0000","last_peered":"2023-11-28T21:34:41.344832+0000","last_clean":"2023-11-28T21:34:41.344832+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:34:41.344832+0000","last_undegraded":"2023-11-28T21:34:41.344832+0000","last_fullsized":"2023-11-28T21:34:41.344832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:13:01.639908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"24'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.334144+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:35:08.334144+0000","last_peered":"2023-11-28T21:35:08.334144+0000","last_clean":"2023-11-28T21:35:08.334144+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:35:08.334144+0000","last_undegraded":"2023-11-28T21:35:08.334144+0000","last_fullsized":"2023-11-28T21:35:08.334144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359893+0000","last_change":"2023-11-28T21:34:41.380413+0000","last_active":"2023-11-28T21:34:42.359893+0000","last_peered":"2023-11-28T21:34:42.359893+0000","last_clean":"2023-11-28T21:34:42.359893+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:34:42.359893+0000","last_undegraded":"2023-11-28T21:34:42.359893+0000","last_fullsized":"2023-11-28T21:34:42.359893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:20:28.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342001+0000","last_change":"2023-11-28T21:34:37.351009+0000","last_active":"2023-11-28T21:34:41.342001+0000","last_peered":"2023-11-28T21:34:41.342001+0000","last_clean":"2023-11-28T21:34:41.342001+0000","last_became_active":"2023-11-28T21:34:37.350321+0000","last_became_peered":"2023-11-28T21:34:37.350321+0000","last_unstale":"2023-11-28T21:34:41.342001+0000","last_undegraded":"2023-11-28T21:34:41.342001+0000","last_fullsized":"2023-11-28T21:34:41.342001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:27:42.022466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"24'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.335647+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:35:08.335647+0000","last_peered":"2023-11-28T21:35:08.335647+0000","last_clean":"2023-11-28T21:35:08.335647+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:35:08.335647+0000","last_undegraded":"2023-11-28T21:35:08.335647+0000","last_fullsized":"2023-11-28T21:35:08.335647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359889+0000","last_change":"2023-11-28T21:34:41.475185+0000","last_active":"2023-11-28T21:34:42.359889+0000","last_peered":"2023-11-28T21:34:42.359889+0000","last_clean":"2023-11-28T21:34:42.359889+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:34:42.359889+0000","last_undegraded":"2023-11-28T21:34:42.359889+0000","last_fullsized":"2023-11-28T21:34:42.359889+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:09.389881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592347+0000","last_change":"2023-11-28T21:34:41.479956+0000","last_active":"2023-11-28T21:34:42.592347+0000","last_peered":"2023-11-28T21:34:42.592347+0000","last_clean":"2023-11-28T21:34:42.592347+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:34:42.592347+0000","last_undegraded":"2023-11-28T21:34:42.592347+0000","last_fullsized":"2023-11-28T21:34:42.592347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:46.004852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343050+0000","last_change":"2023-11-28T21:34:37.317513+0000","last_active":"2023-11-28T21:34:41.343050+0000","last_peered":"2023-11-28T21:34:41.343050+0000","last_clean":"2023-11-28T21:34:41.343050+0000","last_became_active":"2023-11-28T21:34:37.317082+0000","last_became_peered":"2023-11-28T21:34:37.317082+0000","last_unstale":"2023-11-28T21:34:41.343050+0000","last_undegraded":"2023-11-28T21:34:41.343050+0000","last_fullsized":"2023-11-28T21:34:41.343050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:05:37.846078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"24'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.362815+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:34:42.362815+0000","last_peered":"2023-11-28T21:34:42.362815+0000","last_clean":"2023-11-28T21:34:42.362815+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:34:42.362815+0000","last_undegraded":"2023-11-28T21:34:42.362815+0000","last_fullsized":"2023-11-28T21:34:42.362815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595858+0000","last_change":"2023-11-28T21:34:41.385969+0000","last_active":"2023-11-28T21:34:42.595858+0000","last_peered":"2023-11-28T21:34:42.595858+0000","last_clean":"2023-11-28T21:34:42.595858+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:34:42.595858+0000","last_undegraded":"2023-11-28T21:34:42.595858+0000","last_fullsized":"2023-11-28T21:34:42.595858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:01.692549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339578+0000","last_change":"2023-11-28T21:34:37.314087+0000","last_active":"2023-11-28T21:34:41.339578+0000","last_peered":"2023-11-28T21:34:41.339578+0000","last_clean":"2023-11-28T21:34:41.339578+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:34:41.339578+0000","last_undegraded":"2023-11-28T21:34:41.339578+0000","last_fullsized":"2023-11-28T21:34:41.339578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:25:11.912313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340387+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:34:41.340387+0000","last_peered":"2023-11-28T21:34:41.340387+0000","last_clean":"2023-11-28T21:34:41.340387+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:34:41.340387+0000","last_undegraded":"2023-11-28T21:34:41.340387+0000","last_fullsized":"2023-11-28T21:34:41.340387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.343264+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:34:42.343264+0000","last_peered":"2023-11-28T21:34:42.343264+0000","last_clean":"2023-11-28T21:34:42.343264+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:34:42.343264+0000","last_undegraded":"2023-11-28T21:34:42.343264+0000","last_fullsized":"2023-11-28T21:34:42.343264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345191+0000","last_change":"2023-11-28T21:34:37.319801+0000","last_active":"2023-11-28T21:34:41.345191+0000","last_peered":"2023-11-28T21:34:41.345191+0000","last_clean":"2023-11-28T21:34:41.345191+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:34:41.345191+0000","last_undegraded":"2023-11-28T21:34:41.345191+0000","last_fullsized":"2023-11-28T21:34:41.345191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:40:13.251522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360747+0000","last_change":"2023-11-28T21:34:41.370523+0000","last_active":"2023-11-28T21:34:42.360747+0000","last_peered":"2023-11-28T21:34:42.360747+0000","last_clean":"2023-11-28T21:34:42.360747+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:34:42.360747+0000","last_undegraded":"2023-11-28T21:34:42.360747+0000","last_fullsized":"2023-11-28T21:34:42.360747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:17:36.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'5","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.334661+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:35:08.334661+0000","last_peered":"2023-11-28T21:35:08.334661+0000","last_clean":"2023-11-28T21:35:08.334661+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:35:08.334661+0000","last_undegraded":"2023-11-28T21:35:08.334661+0000","last_fullsized":"2023-11-28T21:35:08.334661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342331+0000","last_change":"2023-11-28T21:34:37.328809+0000","last_active":"2023-11-28T21:34:41.342331+0000","last_peered":"2023-11-28T21:34:41.342331+0000","last_clean":"2023-11-28T21:34:41.342331+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:34:41.342331+0000","last_undegraded":"2023-11-28T21:34:41.342331+0000","last_fullsized":"2023-11-28T21:34:41.342331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:27.300090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361127+0000","last_change":"2023-11-28T21:34:41.363087+0000","last_active":"2023-11-28T21:34:42.361127+0000","last_peered":"2023-11-28T21:34:42.361127+0000","last_clean":"2023-11-28T21:34:42.361127+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:34:42.361127+0000","last_undegraded":"2023-11-28T21:34:42.361127+0000","last_fullsized":"2023-11-28T21:34:42.361127+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:35.046132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365193+0000","last_change":"2023-11-28T21:34:37.333670+0000","last_active":"2023-11-28T21:34:41.365193+0000","last_peered":"2023-11-28T21:34:41.365193+0000","last_clean":"2023-11-28T21:34:41.365193+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:34:41.365193+0000","last_undegraded":"2023-11-28T21:34:41.365193+0000","last_fullsized":"2023-11-28T21:34:41.365193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:32.598014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.355458+0000","last_change":"2023-11-28T21:34:37.347101+0000","last_active":"2023-11-28T21:34:41.355458+0000","last_peered":"2023-11-28T21:34:41.355458+0000","last_clean":"2023-11-28T21:34:41.355458+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:34:41.355458+0000","last_undegraded":"2023-11-28T21:34:41.355458+0000","last_fullsized":"2023-11-28T21:34:41.355458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:39:43.186110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341187+0000","last_change":"2023-11-28T21:34:37.338576+0000","last_active":"2023-11-28T21:34:41.341187+0000","last_peered":"2023-11-28T21:34:41.341187+0000","last_clean":"2023-11-28T21:34:41.341187+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:34:41.341187+0000","last_undegraded":"2023-11-28T21:34:41.341187+0000","last_fullsized":"2023-11-28T21:34:41.341187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:53:02.356097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345097+0000","last_change":"2023-11-28T21:34:37.316270+0000","last_active":"2023-11-28T21:34:41.345097+0000","last_peered":"2023-11-28T21:34:41.345097+0000","last_clean":"2023-11-28T21:34:41.345097+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:34:41.345097+0000","last_undegraded":"2023-11-28T21:34:41.345097+0000","last_fullsized":"2023-11-28T21:34:41.345097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:43.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340024+0000","last_change":"2023-11-28T21:34:37.334973+0000","last_active":"2023-11-28T21:34:41.340024+0000","last_peered":"2023-11-28T21:34:41.340024+0000","last_clean":"2023-11-28T21:34:41.340024+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:34:41.340024+0000","last_undegraded":"2023-11-28T21:34:41.340024+0000","last_fullsized":"2023-11-28T21:34:41.340024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:55.050945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345807+0000","last_change":"2023-11-28T21:34:37.318155+0000","last_active":"2023-11-28T21:34:41.345807+0000","last_peered":"2023-11-28T21:34:41.345807+0000","last_clean":"2023-11-28T21:34:41.345807+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:34:41.345807+0000","last_undegraded":"2023-11-28T21:34:41.345807+0000","last_fullsized":"2023-11-28T21:34:41.345807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:29.840255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367255+0000","last_change":"2023-11-28T21:34:37.345758+0000","last_active":"2023-11-28T21:34:41.367255+0000","last_peered":"2023-11-28T21:34:41.367255+0000","last_clean":"2023-11-28T21:34:41.367255+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:34:41.367255+0000","last_undegraded":"2023-11-28T21:34:41.367255+0000","last_fullsized":"2023-11-28T21:34:41.367255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:26.444654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343483+0000","last_change":"2023-11-28T21:34:37.334989+0000","last_active":"2023-11-28T21:34:41.343483+0000","last_peered":"2023-11-28T21:34:41.343483+0000","last_clean":"2023-11-28T21:34:41.343483+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:34:41.343483+0000","last_undegraded":"2023-11-28T21:34:41.343483+0000","last_fullsized":"2023-11-28T21:34:41.343483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:36:57.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346163+0000","last_change":"2023-11-28T21:34:37.333075+0000","last_active":"2023-11-28T21:34:41.346163+0000","last_peered":"2023-11-28T21:34:41.346163+0000","last_clean":"2023-11-28T21:34:41.346163+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:34:41.346163+0000","last_undegraded":"2023-11-28T21:34:41.346163+0000","last_fullsized":"2023-11-28T21:34:41.346163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:50:43.748349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339854+0000","last_change":"2023-11-28T21:34:37.337230+0000","last_active":"2023-11-28T21:34:41.339854+0000","last_peered":"2023-11-28T21:34:41.339854+0000","last_clean":"2023-11-28T21:34:41.339854+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:34:41.339854+0000","last_undegraded":"2023-11-28T21:34:41.339854+0000","last_fullsized":"2023-11-28T21:34:41.339854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:48:45.994559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340945+0000","last_change":"2023-11-28T21:34:37.349894+0000","last_active":"2023-11-28T21:34:41.340945+0000","last_peered":"2023-11-28T21:34:41.340945+0000","last_clean":"2023-11-28T21:34:41.340945+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:34:41.340945+0000","last_undegraded":"2023-11-28T21:34:41.340945+0000","last_fullsized":"2023-11-28T21:34:41.340945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:10.293911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345003+0000","last_change":"2023-11-28T21:34:37.344269+0000","last_active":"2023-11-28T21:34:41.345003+0000","last_peered":"2023-11-28T21:34:41.345003+0000","last_clean":"2023-11-28T21:34:41.345003+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:34:41.345003+0000","last_undegraded":"2023-11-28T21:34:41.345003+0000","last_fullsized":"2023-11-28T21:34:41.345003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:03.204740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.365122+0000","last_change":"2023-11-28T21:34:37.353059+0000","last_active":"2023-11-28T21:34:41.365122+0000","last_peered":"2023-11-28T21:34:41.365122+0000","last_clean":"2023-11-28T21:34:41.365122+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:34:41.365122+0000","last_undegraded":"2023-11-28T21:34:41.365122+0000","last_fullsized":"2023-11-28T21:34:41.365122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:45:12.640797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471801+0000","last_change":"2023-11-28T21:34:37.352965+0000","last_active":"2023-11-28T21:34:41.471801+0000","last_peered":"2023-11-28T21:34:41.471801+0000","last_clean":"2023-11-28T21:34:41.471801+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:34:41.471801+0000","last_undegraded":"2023-11-28T21:34:41.471801+0000","last_fullsized":"2023-11-28T21:34:41.471801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:30:42.087715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.356122+0000","last_change":"2023-11-28T21:34:37.336830+0000","last_active":"2023-11-28T21:34:41.356122+0000","last_peered":"2023-11-28T21:34:41.356122+0000","last_clean":"2023-11-28T21:34:41.356122+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:34:41.356122+0000","last_undegraded":"2023-11-28T21:34:41.356122+0000","last_fullsized":"2023-11-28T21:34:41.356122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:57.318311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.341567+0000","last_change":"2023-11-28T21:34:37.349181+0000","last_active":"2023-11-28T21:34:41.341567+0000","last_peered":"2023-11-28T21:34:41.341567+0000","last_clean":"2023-11-28T21:34:41.341567+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:34:41.341567+0000","last_undegraded":"2023-11-28T21:34:41.341567+0000","last_fullsized":"2023-11-28T21:34:41.341567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:27:59.989299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.345968+0000","last_change":"2023-11-28T21:34:37.336668+0000","last_active":"2023-11-28T21:34:41.345968+0000","last_peered":"2023-11-28T21:34:41.345968+0000","last_clean":"2023-11-28T21:34:41.345968+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:34:41.345968+0000","last_undegraded":"2023-11-28T21:34:41.345968+0000","last_fullsized":"2023-11-28T21:34:41.345968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:06.416717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343147+0000","last_change":"2023-11-28T21:34:37.332790+0000","last_active":"2023-11-28T21:34:41.343147+0000","last_peered":"2023-11-28T21:34:41.343147+0000","last_clean":"2023-11-28T21:34:41.343147+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:34:41.343147+0000","last_undegraded":"2023-11-28T21:34:41.343147+0000","last_fullsized":"2023-11-28T21:34:41.343147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:32:45.920387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344930+0000","last_change":"2023-11-28T21:34:37.320794+0000","last_active":"2023-11-28T21:34:41.344930+0000","last_peered":"2023-11-28T21:34:41.344930+0000","last_clean":"2023-11-28T21:34:41.344930+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:34:41.344930+0000","last_undegraded":"2023-11-28T21:34:41.344930+0000","last_fullsized":"2023-11-28T21:34:41.344930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:16:46.347683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339818+0000","last_change":"2023-11-28T21:34:37.337726+0000","last_active":"2023-11-28T21:34:41.339818+0000","last_peered":"2023-11-28T21:34:41.339818+0000","last_clean":"2023-11-28T21:34:41.339818+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:34:41.339818+0000","last_undegraded":"2023-11-28T21:34:41.339818+0000","last_fullsized":"2023-11-28T21:34:41.339818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:38:43.884668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.343259+0000","last_change":"2023-11-28T21:34:37.334677+0000","last_active":"2023-11-28T21:34:41.343259+0000","last_peered":"2023-11-28T21:34:41.343259+0000","last_clean":"2023-11-28T21:34:41.343259+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:34:41.343259+0000","last_undegraded":"2023-11-28T21:34:41.343259+0000","last_fullsized":"2023-11-28T21:34:41.343259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:05:54.981899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.366695+0000","last_change":"2023-11-28T21:34:37.354861+0000","last_active":"2023-11-28T21:34:41.366695+0000","last_peered":"2023-11-28T21:34:41.366695+0000","last_clean":"2023-11-28T21:34:41.366695+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:34:41.366695+0000","last_undegraded":"2023-11-28T21:34:41.366695+0000","last_fullsized":"2023-11-28T21:34:41.366695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:30.602284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.367055+0000","last_change":"2023-11-28T21:34:37.347675+0000","last_active":"2023-11-28T21:34:41.367055+0000","last_peered":"2023-11-28T21:34:41.367055+0000","last_clean":"2023-11-28T21:34:41.367055+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:34:41.367055+0000","last_undegraded":"2023-11-28T21:34:41.367055+0000","last_fullsized":"2023-11-28T21:34:41.367055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:51:38.803896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471861+0000","last_change":"2023-11-28T21:34:37.357735+0000","last_active":"2023-11-28T21:34:41.471861+0000","last_peered":"2023-11-28T21:34:41.471861+0000","last_clean":"2023-11-28T21:34:41.471861+0000","last_became_active":"2023-11-28T21:34:37.357173+0000","last_became_peered":"2023-11-28T21:34:37.357173+0000","last_unstale":"2023-11-28T21:34:41.471861+0000","last_undegraded":"2023-11-28T21:34:41.471861+0000","last_fullsized":"2023-11-28T21:34:41.471861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:38.660713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.339546+0000","last_change":"2023-11-28T21:34:37.358851+0000","last_active":"2023-11-28T21:34:41.339546+0000","last_peered":"2023-11-28T21:34:41.339546+0000","last_clean":"2023-11-28T21:34:41.339546+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:34:41.339546+0000","last_undegraded":"2023-11-28T21:34:41.339546+0000","last_fullsized":"2023-11-28T21:34:41.339546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.378519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470714+0000","last_change":"2023-11-28T21:34:37.343972+0000","last_active":"2023-11-28T21:34:41.470714+0000","last_peered":"2023-11-28T21:34:41.470714+0000","last_clean":"2023-11-28T21:34:41.470714+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:34:41.470714+0000","last_undegraded":"2023-11-28T21:34:41.470714+0000","last_fullsized":"2023-11-28T21:34:41.470714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:56:00.591888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.340667+0000","last_change":"2023-11-28T21:34:37.346299+0000","last_active":"2023-11-28T21:34:41.340667+0000","last_peered":"2023-11-28T21:34:41.340667+0000","last_clean":"2023-11-28T21:34:41.340667+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:34:41.340667+0000","last_undegraded":"2023-11-28T21:34:41.340667+0000","last_fullsized":"2023-11-28T21:34:41.340667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:06.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.470739+0000","last_change":"2023-11-28T21:34:37.343410+0000","last_active":"2023-11-28T21:34:41.470739+0000","last_peered":"2023-11-28T21:34:41.470739+0000","last_clean":"2023-11-28T21:34:41.470739+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:34:41.470739+0000","last_undegraded":"2023-11-28T21:34:41.470739+0000","last_fullsized":"2023-11-28T21:34:41.470739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:29:45.070478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.342784+0000","last_change":"2023-11-28T21:34:37.348270+0000","last_active":"2023-11-28T21:34:41.342784+0000","last_peered":"2023-11-28T21:34:41.342784+0000","last_clean":"2023-11-28T21:34:41.342784+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:34:41.342784+0000","last_undegraded":"2023-11-28T21:34:41.342784+0000","last_fullsized":"2023-11-28T21:34:41.342784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:36:10.938478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.344314+0000","last_change":"2023-11-28T21:34:37.330786+0000","last_active":"2023-11-28T21:34:41.344314+0000","last_peered":"2023-11-28T21:34:41.344314+0000","last_clean":"2023-11-28T21:34:41.344314+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:34:41.344314+0000","last_undegraded":"2023-11-28T21:34:41.344314+0000","last_fullsized":"2023-11-28T21:34:41.344314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:52:36.738867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.346251+0000","last_change":"2023-11-28T21:34:37.318725+0000","last_active":"2023-11-28T21:34:41.346251+0000","last_peered":"2023-11-28T21:34:41.346251+0000","last_clean":"2023-11-28T21:34:41.346251+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:34:41.346251+0000","last_undegraded":"2023-11-28T21:34:41.346251+0000","last_fullsized":"2023-11-28T21:34:41.346251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:49:21.775066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-11-28T21:34:41.471764+0000","last_change":"2023-11-28T21:34:37.349829+0000","last_active":"2023-11-28T21:34:41.471764+0000","last_peered":"2023-11-28T21:34:41.471764+0000","last_clean":"2023-11-28T21:34:41.471764+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:34:41.471764+0000","last_undegraded":"2023-11-28T21:34:41.471764+0000","last_fullsized":"2023-11-28T21:34:41.471764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:07.752862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6045,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":12090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":690742,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":100624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":689779,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:35:11.692 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T21:35:11.693 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T21:35:11.693 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T21:35:11.693 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T21:35:11.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f9179b76640 1 Processor -- start 2023-11-28T21:35:11.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f9179b76640 1 -- start start 2023-11-28T21:35:11.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f9179b76640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f917415a980 0x7f91740b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f9179b76640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91740b44a0 con 0x7f91740b4010 2023-11-28T21:35:11.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f9179b76640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917415d390 con 0x7f9174153fb0 2023-11-28T21:35:11.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f9179b76640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91740b2b60 con 0x7f917415a980 2023-11-28T21:35:11.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f9179375640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f91740b4010 0x7f91740cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36516/0 (socket says 172.21.15.12:36516) 2023-11-28T21:35:11.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f9179375640 1 -- 172.21.15.12:0/2152794665 learned_addr learned my addr 172.21.15.12:0/2152794665 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:11.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f9173fff640 1 --2- 172.21.15.12:0/2152794665 >> v2:172.21.15.144:3300/0 conn(0x7f917415a980 0x7f91740b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131935870 0 0) 0x7f91740b44a0 con 0x7f91740b4010 2023-11-28T21:35:11.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9164003530 con 0x7f91740b4010 2023-11-28T21:35:11.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3238153968 0 0) 0x7f9164003530 con 0x7f91740b4010 2023-11-28T21:35:11.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 >> v2:172.21.15.144:3300/0 conn(0x7f917415a980 msgr2=0x7f91740b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:11.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 --2- 172.21.15.12:0/2152794665 >> v2:172.21.15.144:3300/0 conn(0x7f917415a980 0x7f91740b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 >> v1:172.21.15.144:6789/0 conn(0x7f9174153fb0 legacy=0x7f91740ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917415d630 con 0x7f91740b4010 2023-11-28T21:35:11.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f916c001a00 con 0x7f91740b4010 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 >> v1:172.21.15.12:6789/0 conn(0x7f91740b4010 legacy=0x7f91740cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9164003980 0x7f9164003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91640047d0 0x7f9164004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9164005190 0x7f916400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91740b2b60 con 0x7f9164005190 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916400c040 con 0x7f9164003980 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.872+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916400c200 con 0x7f91640047d0 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f9179375640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9164005190 0x7f916400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f9173fff640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91640047d0 0x7f9164004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f9178b74640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9164003980 0x7f9164003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f9179375640 1 -- 172.21.15.12:0/2152794665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91640047d0 msgr2=0x7f9164004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f9179375640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91640047d0 0x7f9164004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f9179375640 1 -- 172.21.15.12:0/2152794665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9164003980 msgr2=0x7f9164003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f9179375640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9164003980 0x7f9164003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f9179375640 1 -- 172.21.15.12:0/2152794665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9164003530 con 0x7f9164005190 2023-11-28T21:35:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f9179375640 1 -- 172.21.15.12:0/2152794665 reap_dead start 2023-11-28T21:35:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f9179375640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9164005190 0x7f916400b990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f916c00a820 tx=0x7f916c01d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11d3692856276e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:11.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0072d0 con 0x7f9164005190 2023-11-28T21:35:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c0a8070 con 0x7f9164005190 2023-11-28T21:35:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.876+0000 7f91737fe640 1 -- 172.21.15.12:0/2152794665 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c0076b0 con 0x7f9164005190 2023-11-28T21:35:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 -- 172.21.15.12:0/2152794665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9164005190 msgr2=0x7f916400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9164005190 0x7f916400b990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f916c00a820 tx=0x7f916c01d320 comp rx=0 tx=0).stop 2023-11-28T21:35:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 -- 172.21.15.12:0/2152794665 shutdown_connections 2023-11-28T21:35:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 --2- 172.21.15.12:0/2152794665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9164005190 0x7f916400b990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 -- 172.21.15.12:0/2152794665 >> 172.21.15.12:0/2152794665 conn(0x7f9174071ce0 msgr2=0x7f91740720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 -- 172.21.15.12:0/2152794665 shutdown_connections 2023-11-28T21:35:11.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 -- 172.21.15.12:0/2152794665 wait complete. 2023-11-28T21:35:11.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 Processor -- start 2023-11-28T21:35:11.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 -- start start 2023-11-28T21:35:11.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9164005190 0x7f9174158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91740cd950 0x7f91741590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91741e5030 0x7f91741e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917415e1b0 con 0x7f91740cd950 2023-11-28T21:35:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91740c5f50 con 0x7f91741e5030 2023-11-28T21:35:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179b76640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91740ce0c0 con 0x7f9164005190 2023-11-28T21:35:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9173fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91740cd950 0x7f91741590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9173fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91740cd950 0x7f91741590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38470/0 (socket says 172.21.15.12:38470) 2023-11-28T21:35:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9173fff640 1 -- 172.21.15.12:0/4055735720 learned_addr learned my addr 172.21.15.12:0/4055735720 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:11.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9179375640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91741e5030 0x7f91741e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9178b74640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9164005190 0x7f9174158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9173fff640 1 -- 172.21.15.12:0/4055735720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9164005190 msgr2=0x7f9174158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9173fff640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9164005190 0x7f9174158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9173fff640 1 -- 172.21.15.12:0/4055735720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91741e5030 msgr2=0x7f91741e54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:11.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9173fff640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91741e5030 0x7f91741e54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:11.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.880+0000 7f9173fff640 1 -- 172.21.15.12:0/4055735720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91741e59f0 con 0x7f91740cd950 2023-11-28T21:35:11.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f9173fff640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91740cd950 0x7f91741590b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9160011960 tx=0x7f9160011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:11.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f91717fa640 1 -- 172.21.15.12:0/4055735720 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160004240 con 0x7f91740cd950 2023-11-28T21:35:11.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f91717fa640 1 -- 172.21.15.12:0/4055735720 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91600043a0 con 0x7f91740cd950 2023-11-28T21:35:11.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f91717fa640 1 -- 172.21.15.12:0/4055735720 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916000fa40 con 0x7f91740cd950 2023-11-28T21:35:11.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f9179b76640 1 -- 172.21.15.12:0/4055735720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91740c6180 con 0x7f91740cd950 2023-11-28T21:35:11.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f9179b76640 1 -- 172.21.15.12:0/4055735720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91740c5a60 con 0x7f91740cd950 2023-11-28T21:35:11.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f9179b76640 1 -- 172.21.15.12:0/4055735720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91740c6600 con 0x7f91740cd950 2023-11-28T21:35:11.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f91717fa640 1 -- 172.21.15.12:0/4055735720 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f916000fba0 con 0x7f91740cd950 2023-11-28T21:35:11.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f91717fa640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9158066470 0x7f9158068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:11.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f91717fa640 1 -- 172.21.15.12:0/4055735720 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9160084910 con 0x7f91740cd950 2023-11-28T21:35:11.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.884+0000 7f9178b74640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9158066470 0x7f9158068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:11.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.888+0000 7f9178b74640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9158066470 0x7f9158068930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9168000c10 tx=0x7f9168009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:11.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:11.888+0000 7f91717fa640 1 -- 172.21.15.12:0/4055735720 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9160051970 con 0x7f91740cd950 2023-11-28T21:35:12.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 -- 172.21.15.12:0/4055735720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9174074f60 con 0x7f91740cd950 2023-11-28T21:35:12.082 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f91717fa640 1 -- 172.21.15.12:0/4055735720 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9160023040 con 0x7f91740cd950 2023-11-28T21:35:12.082 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:12.082 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 -- 172.21.15.12:0/4055735720 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9158066470 msgr2=0x7f9158068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9158066470 0x7f9158068930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9168000c10 tx=0x7f9168009040 comp rx=0 tx=0).stop 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 -- 172.21.15.12:0/4055735720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91740cd950 msgr2=0x7f91741590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91740cd950 0x7f91741590b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9160011960 tx=0x7f9160011e20 comp rx=0 tx=0).stop 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 -- 172.21.15.12:0/4055735720 shutdown_connections 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9158066470 0x7f9158068930 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91741e5030 0x7f91741e54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f91740cd950 0x7f91741590b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 --2- 172.21.15.12:0/4055735720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9164005190 0x7f9174158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 -- 172.21.15.12:0/4055735720 >> 172.21.15.12:0/4055735720 conn(0x7f9174071ce0 msgr2=0x7f9174152810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:12.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 -- 172.21.15.12:0/4055735720 shutdown_connections 2023-11-28T21:35:12.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.080+0000 7f9179b76640 1 -- 172.21.15.12:0/4055735720 wait complete. 2023-11-28T21:35:12.104 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T21:35:12.105 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:12.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.244+0000 7f0eafa32640 1 Processor -- start 2023-11-28T21:35:12.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.244+0000 7f0eafa32640 1 -- start start 2023-11-28T21:35:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eafa32640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0ea8070bd0 0x7f0ea8071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eafa32640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea80c6350 con 0x7f0ea806a5b0 2023-11-28T21:35:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eafa32640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea80c65d0 con 0x7f0ea8153ee0 2023-11-28T21:35:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eafa32640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea80c6b90 con 0x7f0ea8070bd0 2023-11-28T21:35:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eadfcf640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0ea8070bd0 0x7f0ea8071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eacfcd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0ea806a5b0 0x7f0ea80c7a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36518/0 (socket says 172.21.15.12:36518) 2023-11-28T21:35:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eadfcf640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0ea8070bd0 0x7f0ea8071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38474/0 (socket says 172.21.15.12:38474) 2023-11-28T21:35:12.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eacfcd640 1 -- 172.21.15.12:0/503880621 learned_addr learned my addr 172.21.15.12:0/503880621 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eadfcf640 1 -- 172.21.15.12:0/503880621 >> v1:172.21.15.144:6789/0 conn(0x7f0ea8153ee0 legacy=0x7f0ea815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eadfcf640 1 -- 172.21.15.12:0/503880621 >> v1:172.21.15.12:6789/0 conn(0x7f0ea806a5b0 legacy=0x7f0ea80c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.248+0000 7f0eadfcf640 1 -- 172.21.15.12:0/503880621 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80c68b0 con 0x7f0ea8070bd0 2023-11-28T21:35:12.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eadfcf640 1 --2- 172.21.15.12:0/503880621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 0x7f0ea8071030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009d90 tx=0x7f0ea001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b5b73d014c7fd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:12.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0e9ffff640 1 -- 172.21.15.12:0/503880621 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00af600 con 0x7f0ea8070bd0 2023-11-28T21:35:12.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0e9ffff640 1 -- 172.21.15.12:0/503880621 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea00ab050 con 0x7f0ea8070bd0 2023-11-28T21:35:12.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0e9ffff640 1 -- 172.21.15.12:0/503880621 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00af980 con 0x7f0ea8070bd0 2023-11-28T21:35:12.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 -- 172.21.15.12:0/503880621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 msgr2=0x7f0ea8071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 --2- 172.21.15.12:0/503880621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 0x7f0ea8071030 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009d90 tx=0x7f0ea001f130 comp rx=0 tx=0).stop 2023-11-28T21:35:12.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 -- 172.21.15.12:0/503880621 shutdown_connections 2023-11-28T21:35:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 --2- 172.21.15.12:0/503880621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 0x7f0ea8071030 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 -- 172.21.15.12:0/503880621 >> 172.21.15.12:0/503880621 conn(0x7f0ea80c8450 msgr2=0x7f0ea80c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 -- 172.21.15.12:0/503880621 shutdown_connections 2023-11-28T21:35:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 -- 172.21.15.12:0/503880621 wait complete. 2023-11-28T21:35:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 Processor -- start 2023-11-28T21:35:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 -- start start 2023-11-28T21:35:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea806a5b0 0x7f0ea814b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 0x7f0ea81578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea8153ee0 0x7f0ea8154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81eff90 con 0x7f0ea8070bd0 2023-11-28T21:35:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81f0270 con 0x7f0ea8153ee0 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.252+0000 7f0eafa32640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81f0550 con 0x7f0ea806a5b0 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eacfcd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 0x7f0ea81578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eacfcd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 0x7f0ea81578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38488/0 (socket says 172.21.15.12:38488) 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eacfcd640 1 -- 172.21.15.12:0/231983824 learned_addr learned my addr 172.21.15.12:0/231983824 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eadfcf640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea8153ee0 0x7f0ea8154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0ead7ce640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea806a5b0 0x7f0ea814b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eacfcd640 1 -- 172.21.15.12:0/231983824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea806a5b0 msgr2=0x7f0ea814b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eacfcd640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea806a5b0 0x7f0ea814b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eacfcd640 1 -- 172.21.15.12:0/231983824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea8153ee0 msgr2=0x7f0ea8154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eacfcd640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea8153ee0 0x7f0ea8154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eacfcd640 1 -- 172.21.15.12:0/231983824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea0009a70 con 0x7f0ea8070bd0 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0ead7ce640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea806a5b0 0x7f0ea814b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eadfcf640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea8153ee0 0x7f0ea8154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eacfcd640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 0x7f0ea81578e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800c4d0 tx=0x7f0e9800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:12.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0e9dffb640 1 -- 172.21.15.12:0/231983824 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98007480 con 0x7f0ea8070bd0 2023-11-28T21:35:12.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0e9dffb640 1 -- 172.21.15.12:0/231983824 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e980075e0 con 0x7f0ea8070bd0 2023-11-28T21:35:12.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eafa32640 1 -- 172.21.15.12:0/231983824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea81f0830 con 0x7f0ea8070bd0 2023-11-28T21:35:12.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0e9dffb640 1 -- 172.21.15.12:0/231983824 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e980059b0 con 0x7f0ea8070bd0 2023-11-28T21:35:12.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eafa32640 1 -- 172.21.15.12:0/231983824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea80c2940 con 0x7f0ea8070bd0 2023-11-28T21:35:12.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0eafa32640 1 -- 172.21.15.12:0/231983824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea8074f60 con 0x7f0ea8070bd0 2023-11-28T21:35:12.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.256+0000 7f0e9dffb640 1 -- 172.21.15.12:0/231983824 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0e98005b10 con 0x7f0ea8070bd0 2023-11-28T21:35:12.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.260+0000 7f0e9dffb640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e8c0664a0 0x7f0e8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.260+0000 7f0e9dffb640 1 -- 172.21.15.12:0/231983824 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0e980835e0 con 0x7f0ea8070bd0 2023-11-28T21:35:12.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.260+0000 7f0ead7ce640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e8c0664a0 0x7f0e8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.260+0000 7f0ead7ce640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e8c0664a0 0x7f0e8c068960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000f70 tx=0x7f0ea4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:12.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.264+0000 7f0e9dffb640 1 -- 172.21.15.12:0/231983824 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0e98050810 con 0x7f0ea8070bd0 2023-11-28T21:35:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.424+0000 7f0eafa32640 1 -- 172.21.15.12:0/231983824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ea806e930 con 0x7f0ea8070bd0 2023-11-28T21:35:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.424+0000 7f0e9dffb640 1 -- 172.21.15.12:0/231983824 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7f0e98017090 con 0x7f0ea8070bd0 2023-11-28T21:35:12.427 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:12.428 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 -- 172.21.15.12:0/231983824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e8c0664a0 msgr2=0x7f0e8c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e8c0664a0 0x7f0e8c068960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000f70 tx=0x7f0ea4009040 comp rx=0 tx=0).stop 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 -- 172.21.15.12:0/231983824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 msgr2=0x7f0ea81578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 0x7f0ea81578e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800c4d0 tx=0x7f0e9800c990 comp rx=0 tx=0).stop 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 -- 172.21.15.12:0/231983824 shutdown_connections 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e8c0664a0 0x7f0e8c068960 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea8153ee0 0x7f0ea8154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ea8070bd0 0x7f0ea81578e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 --2- 172.21.15.12:0/231983824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea806a5b0 0x7f0ea814b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 -- 172.21.15.12:0/231983824 >> 172.21.15.12:0/231983824 conn(0x7f0ea80c8450 msgr2=0x7f0ea8070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 -- 172.21.15.12:0/231983824 shutdown_connections 2023-11-28T21:35:12.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.428+0000 7f0eafa32640 1 -- 172.21.15.12:0/231983824 wait complete. 2023-11-28T21:35:12.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T21:34:29.878190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T21:34:34.039032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T21:34:35.369379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T21:34:39.373079+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T21:35:12.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T21:35:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c6499d640 1 Processor -- start 2023-11-28T21:35:12.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c6499d640 1 -- start start 2023-11-28T21:35:12.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c6499d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c60150c60 0x7f1c60072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c6499d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1c6015d2c0 0x7f1c6015a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c6499d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1c6006b530 0x7f1c6006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c6499d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c600726d0 con 0x7f1c6015d2c0 2023-11-28T21:35:12.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c6499d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6015acd0 con 0x7f1c6006b530 2023-11-28T21:35:12.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c6499d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c600c7d50 con 0x7f1c60150c60 2023-11-28T21:35:12.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c57fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1c6015d2c0 0x7f1c6015a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c57fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1c6015d2c0 0x7f1c6015a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38492/0 (socket says 172.21.15.12:38492) 2023-11-28T21:35:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c57fff640 1 -- 172.21.15.12:0/534089487 learned_addr learned my addr 172.21.15.12:0/534089487 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c5ed9d640 1 --2- 172.21.15.12:0/534089487 >> v2:172.21.15.144:3300/0 conn(0x7f1c6006b530 0x7f1c6006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c5e59c640 1 --2- 172.21.15.12:0/534089487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c60150c60 0x7f1c60072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c57fff640 1 -- 172.21.15.12:0/534089487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c60150c60 msgr2=0x7f1c60072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c57fff640 1 --2- 172.21.15.12:0/534089487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c60150c60 0x7f1c60072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c57fff640 1 -- 172.21.15.12:0/534089487 >> v2:172.21.15.144:3300/0 conn(0x7f1c6006b530 msgr2=0x7f1c6006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c57fff640 1 --2- 172.21.15.12:0/534089487 >> v2:172.21.15.144:3300/0 conn(0x7f1c6006b530 0x7f1c6006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.596+0000 7f1c57fff640 1 -- 172.21.15.12:0/534089487 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c60159cd0 con 0x7f1c6015d2c0 2023-11-28T21:35:12.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c57fff640 1 --2- 172.21.15.12:0/534089487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6015d2c0 0x7f1c6015a790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1c60071360 tx=0x7f1c4801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66832cb9c1cf8b84 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:12.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c5dd9b640 1 -- 172.21.15.12:0/534089487 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c480a7070 con 0x7f1c6015d2c0 2023-11-28T21:35:12.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c5dd9b640 1 -- 172.21.15.12:0/534089487 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c480ab430 con 0x7f1c6015d2c0 2023-11-28T21:35:12.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c5dd9b640 1 -- 172.21.15.12:0/534089487 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c480ab7c0 con 0x7f1c6015d2c0 2023-11-28T21:35:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 -- 172.21.15.12:0/534089487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6015d2c0 msgr2=0x7f1c6015a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 --2- 172.21.15.12:0/534089487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6015d2c0 0x7f1c6015a790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1c60071360 tx=0x7f1c4801c470 comp rx=0 tx=0).stop 2023-11-28T21:35:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 -- 172.21.15.12:0/534089487 shutdown_connections 2023-11-28T21:35:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 --2- 172.21.15.12:0/534089487 >> v2:172.21.15.144:3300/0 conn(0x7f1c6006b530 0x7f1c6006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 --2- 172.21.15.12:0/534089487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6015d2c0 0x7f1c6015a790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 --2- 172.21.15.12:0/534089487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c60150c60 0x7f1c60072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 -- 172.21.15.12:0/534089487 >> 172.21.15.12:0/534089487 conn(0x7f1c6006d8d0 msgr2=0x7f1c6006dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:12.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 -- 172.21.15.12:0/534089487 shutdown_connections 2023-11-28T21:35:12.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 -- 172.21.15.12:0/534089487 wait complete. 2023-11-28T21:35:12.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 Processor -- start 2023-11-28T21:35:12.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.600+0000 7f1c6499d640 1 -- start start 2023-11-28T21:35:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c6499d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6006b530 0x7f1c60157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c6499d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c60150c60 0x7f1c601f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c6499d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c6015d2c0 0x7f1c6014fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c6499d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60155f20 con 0x7f1c6006b530 2023-11-28T21:35:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c6499d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60155610 con 0x7f1c60150c60 2023-11-28T21:35:12.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c6499d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60163fd0 con 0x7f1c6015d2c0 2023-11-28T21:35:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6006b530 0x7f1c60157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6006b530 0x7f1c60157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38506/0 (socket says 172.21.15.12:38506) 2023-11-28T21:35:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5e59c640 1 -- 172.21.15.12:0/2325178961 learned_addr learned my addr 172.21.15.12:0/2325178961 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c57fff640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c60150c60 0x7f1c601f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5ed9d640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c6015d2c0 0x7f1c6014fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5e59c640 1 -- 172.21.15.12:0/2325178961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c6015d2c0 msgr2=0x7f1c6014fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5e59c640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c6015d2c0 0x7f1c6014fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5e59c640 1 -- 172.21.15.12:0/2325178961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c60150c60 msgr2=0x7f1c601f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5e59c640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c60150c60 0x7f1c601f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5e59c640 1 -- 172.21.15.12:0/2325178961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c480094d0 con 0x7f1c6006b530 2023-11-28T21:35:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5ed9d640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c6015d2c0 0x7f1c6014fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c5e59c640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6006b530 0x7f1c60157290 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000d5f0 tx=0x7f1c5000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c56ffd640 1 -- 172.21.15.12:0/2325178961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c500125f0 con 0x7f1c6006b530 2023-11-28T21:35:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c56ffd640 1 -- 172.21.15.12:0/2325178961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c50012750 con 0x7f1c6006b530 2023-11-28T21:35:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c56ffd640 1 -- 172.21.15.12:0/2325178961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50012ae0 con 0x7f1c6006b530 2023-11-28T21:35:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c6499d640 1 -- 172.21.15.12:0/2325178961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c601642b0 con 0x7f1c6006b530 2023-11-28T21:35:12.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c6499d640 1 -- 172.21.15.12:0/2325178961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c60155b40 con 0x7f1c6006b530 2023-11-28T21:35:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.604+0000 7f1c6499d640 1 -- 172.21.15.12:0/2325178961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c60156190 con 0x7f1c6006b530 2023-11-28T21:35:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.608+0000 7f1c56ffd640 1 -- 172.21.15.12:0/2325178961 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1c50012c40 con 0x7f1c6006b530 2023-11-28T21:35:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.608+0000 7f1c56ffd640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c40066470 0x7f1c40068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.608+0000 7f1c56ffd640 1 -- 172.21.15.12:0/2325178961 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1c500845b0 con 0x7f1c6006b530 2023-11-28T21:35:12.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.608+0000 7f1c57fff640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c40066470 0x7f1c40068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.608+0000 7f1c57fff640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c40066470 0x7f1c40068930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1c480af000 tx=0x7f1c48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:12.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.612+0000 7f1c56ffd640 1 -- 172.21.15.12:0/2325178961 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1c50051730 con 0x7f1c6006b530 2023-11-28T21:35:12.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.768+0000 7f1c6499d640 1 -- 172.21.15.12:0/2325178961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c60072960 con 0x7f1c6006b530 2023-11-28T21:35:12.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.768+0000 7f1c56ffd640 1 -- 172.21.15.12:0/2325178961 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c50054d50 con 0x7f1c6006b530 2023-11-28T21:35:12.773 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-11-28T21:35:12.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 -- 172.21.15.12:0/2325178961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c40066470 msgr2=0x7f1c40068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c40066470 0x7f1c40068930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1c480af000 tx=0x7f1c48008040 comp rx=0 tx=0).stop 2023-11-28T21:35:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 -- 172.21.15.12:0/2325178961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6006b530 msgr2=0x7f1c60157290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6006b530 0x7f1c60157290 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000d5f0 tx=0x7f1c5000dab0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 -- 172.21.15.12:0/2325178961 shutdown_connections 2023-11-28T21:35:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c40066470 0x7f1c40068930 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c6015d2c0 0x7f1c6014fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c60150c60 0x7f1c601f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 --2- 172.21.15.12:0/2325178961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c6006b530 0x7f1c60157290 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 -- 172.21.15.12:0/2325178961 >> 172.21.15.12:0/2325178961 conn(0x7f1c6006d8d0 msgr2=0x7f1c600c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 -- 172.21.15.12:0/2325178961 shutdown_connections 2023-11-28T21:35:12.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.772+0000 7f1c6499d640 1 -- 172.21.15.12:0/2325178961 wait complete. 2023-11-28T21:35:12.794 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T21:35:12.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.936+0000 7fb07e7b8640 1 Processor -- start 2023-11-28T21:35:12.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb07e7b8640 1 -- start start 2023-11-28T21:35:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb07e7b8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb078153e40 0x7fb0780ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb07e7b8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07815acc0 con 0x7fb0780cb180 2023-11-28T21:35:12.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb07e7b8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb078159ea0 con 0x7fb07815dc80 2023-11-28T21:35:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb07e7b8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb078159cb0 con 0x7fb078153e40 2023-11-28T21:35:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb07cd55640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb0780cb180 0x7fb0780cb580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36524/0 (socket says 172.21.15.12:36524) 2023-11-28T21:35:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb07cd55640 1 -- 172.21.15.12:0/2909536048 learned_addr learned my addr 172.21.15.12:0/2909536048 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb077fff640 1 --2- 172.21.15.12:0/2909536048 >> v2:172.21.15.12:3300/0 conn(0x7fb078153e40 0x7fb0780ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb0777fe640 1 -- 172.21.15.12:0/2909536048 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837119560 0 0) 0x7fb07815acc0 con 0x7fb0780cb180 2023-11-28T21:35:12.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb0777fe640 1 -- 172.21.15.12:0/2909536048 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb064003530 con 0x7fb0780cb180 2023-11-28T21:35:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb077fff640 1 -- 172.21.15.12:0/2909536048 >> v1:172.21.15.144:6789/0 conn(0x7fb07815dc80 legacy=0x7fb0780cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb077fff640 1 -- 172.21.15.12:0/2909536048 >> v1:172.21.15.12:6789/0 conn(0x7fb0780cb180 legacy=0x7fb0780cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.940+0000 7fb077fff640 1 -- 172.21.15.12:0/2909536048 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07815a140 con 0x7fb078153e40 2023-11-28T21:35:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb077fff640 1 --2- 172.21.15.12:0/2909536048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb078153e40 0x7fb0780ca110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb0680097c0 tx=0x7fb06801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddf92c2eabc9270 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb0777fe640 1 -- 172.21.15.12:0/2909536048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0680ac620 con 0x7fb078153e40 2023-11-28T21:35:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb0777fe640 1 -- 172.21.15.12:0/2909536048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0680ac780 con 0x7fb078153e40 2023-11-28T21:35:12.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb0777fe640 1 -- 172.21.15.12:0/2909536048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0680acaf0 con 0x7fb078153e40 2023-11-28T21:35:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb07e7b8640 1 -- 172.21.15.12:0/2909536048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb078153e40 msgr2=0x7fb0780ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb07e7b8640 1 --2- 172.21.15.12:0/2909536048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb078153e40 0x7fb0780ca110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb0680097c0 tx=0x7fb06801c3c0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb07e7b8640 1 -- 172.21.15.12:0/2909536048 shutdown_connections 2023-11-28T21:35:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb07e7b8640 1 --2- 172.21.15.12:0/2909536048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb078153e40 0x7fb0780ca110 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb07e7b8640 1 -- 172.21.15.12:0/2909536048 >> 172.21.15.12:0/2909536048 conn(0x7fb07806b7b0 msgr2=0x7fb078066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb07e7b8640 1 -- 172.21.15.12:0/2909536048 shutdown_connections 2023-11-28T21:35:12.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb07e7b8640 1 -- 172.21.15.12:0/2909536048 wait complete. 2023-11-28T21:35:12.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb07e7b8640 1 Processor -- start 2023-11-28T21:35:12.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.944+0000 7fb07e7b8640 1 -- start start 2023-11-28T21:35:12.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb07e7b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0780cb180 0x7fb0780b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb07e7b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb078153e40 0x7fb078151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb07e7b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb07815dc80 0x7fb0781f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb07e7b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07815d3a0 con 0x7fb0780cb180 2023-11-28T21:35:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb07e7b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0781f2e30 con 0x7fb078153e40 2023-11-28T21:35:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb07e7b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07815d0c0 con 0x7fb07815dc80 2023-11-28T21:35:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb077fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0780cb180 0x7fb0780b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb077fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0780cb180 0x7fb0780b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38528/0 (socket says 172.21.15.12:38528) 2023-11-28T21:35:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb077fff640 1 -- 172.21.15.12:0/877524420 learned_addr learned my addr 172.21.15.12:0/877524420 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:12.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb06ffff640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb078153e40 0x7fb078151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb07cd55640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb07815dc80 0x7fb0781f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb077fff640 1 -- 172.21.15.12:0/877524420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb07815dc80 msgr2=0x7fb0781f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb077fff640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb07815dc80 0x7fb0781f5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb077fff640 1 -- 172.21.15.12:0/877524420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb078153e40 msgr2=0x7fb078151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb077fff640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb078153e40 0x7fb078151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:12.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb077fff640 1 -- 172.21.15.12:0/877524420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb068009420 con 0x7fb0780cb180 2023-11-28T21:35:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb06ffff640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb078153e40 0x7fb078151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb077fff640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0780cb180 0x7fb0780b3fb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb068019ff0 tx=0x7fb068019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb0757fa640 1 -- 172.21.15.12:0/877524420 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0680ac620 con 0x7fb0780cb180 2023-11-28T21:35:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb0757fa640 1 -- 172.21.15.12:0/877524420 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb068004030 con 0x7fb0780cb180 2023-11-28T21:35:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb0757fa640 1 -- 172.21.15.12:0/877524420 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0680043a0 con 0x7fb0780cb180 2023-11-28T21:35:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb07e7b8640 1 -- 172.21.15.12:0/877524420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0781f2070 con 0x7fb0780cb180 2023-11-28T21:35:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb07e7b8640 1 -- 172.21.15.12:0/877524420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0781f2630 con 0x7fb0780cb180 2023-11-28T21:35:12.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.948+0000 7fb07e7b8640 1 -- 172.21.15.12:0/877524420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb078074f60 con 0x7fb0780cb180 2023-11-28T21:35:12.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.952+0000 7fb0757fa640 1 -- 172.21.15.12:0/877524420 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb0680b3040 con 0x7fb0780cb180 2023-11-28T21:35:12.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.952+0000 7fb0757fa640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb058066470 0x7fb058068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:12.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.952+0000 7fb0757fa640 1 -- 172.21.15.12:0/877524420 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb06811d4a0 con 0x7fb0780cb180 2023-11-28T21:35:12.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.952+0000 7fb06ffff640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb058066470 0x7fb058068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:12.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.952+0000 7fb06ffff640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb058066470 0x7fb058068930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb07806ca60 tx=0x7fb060009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:12.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:12.956+0000 7fb0757fa640 1 -- 172.21.15.12:0/877524420 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb0680ea5a0 con 0x7fb0780cb180 2023-11-28T21:35:13.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 -- 172.21.15.12:0/877524420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb078159200 con 0x7fb0780cb180 2023-11-28T21:35:13.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb0757fa640 1 -- 172.21.15.12:0/877524420 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb0680edbc0 con 0x7fb0780cb180 2023-11-28T21:35:13.119 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 -- 172.21.15.12:0/877524420 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb058066470 msgr2=0x7fb058068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb058066470 0x7fb058068930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb07806ca60 tx=0x7fb060009040 comp rx=0 tx=0).stop 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 -- 172.21.15.12:0/877524420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0780cb180 msgr2=0x7fb0780b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0780cb180 0x7fb0780b3fb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb068019ff0 tx=0x7fb068019160 comp rx=0 tx=0).stop 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 -- 172.21.15.12:0/877524420 shutdown_connections 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb058066470 0x7fb058068930 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb07815dc80 0x7fb0781f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb078153e40 0x7fb078151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 --2- 172.21.15.12:0/877524420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0780cb180 0x7fb0780b3fb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 -- 172.21.15.12:0/877524420 >> 172.21.15.12:0/877524420 conn(0x7fb07806b7b0 msgr2=0x7fb078158af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:13.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 -- 172.21.15.12:0/877524420 shutdown_connections 2023-11-28T21:35:13.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.116+0000 7fb07e7b8640 1 -- 172.21.15.12:0/877524420 wait complete. 2023-11-28T21:35:13.140 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T21:35:13.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.284+0000 7f7033217640 1 Processor -- start 2023-11-28T21:35:18.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.284+0000 7f7033217640 1 -- start start 2023-11-28T21:35:18.611 INFO:tasks.ceph.mds.f.smithi144.stderr:2023-11-28T21:35:15.595+0000 7fcaf2464640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-11-28T21:35:18.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7033217640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f702c156af0 0x7f702c161a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7033217640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702c0c6ce0 con 0x7f702c1500d0 2023-11-28T21:35:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7033217640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702c0c5df0 con 0x7f702c0c27c0 2023-11-28T21:35:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7033217640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f702c0c7170 con 0x7f702c156af0 2023-11-28T21:35:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7030fb3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f702c1500d0 0x7f702c0c2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36534/0 (socket says 172.21.15.12:36534) 2023-11-28T21:35:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7030fb3640 1 -- 172.21.15.12:0/3530450835 learned_addr learned my addr 172.21.15.12:0/3530450835 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f70317b4640 1 --2- 172.21.15.12:0/3530450835 >> v2:172.21.15.144:3300/0 conn(0x7f702c156af0 0x7f702c161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663807949 0 0) 0x7f702c0c6ce0 con 0x7f702c1500d0 2023-11-28T21:35:18.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f701c003450 con 0x7f702c1500d0 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 349887771 0 0) 0x7f701c003450 con 0x7f702c1500d0 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 >> v2:172.21.15.144:3300/0 conn(0x7f702c156af0 msgr2=0x7f702c161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 --2- 172.21.15.12:0/3530450835 >> v2:172.21.15.144:3300/0 conn(0x7f702c156af0 0x7f702c161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 >> v1:172.21.15.144:6789/0 conn(0x7f702c0c27c0 legacy=0x7f702c06d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c0c60d0 con 0x7f702c1500d0 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f7020001680 con 0x7f702c1500d0 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 >> v1:172.21.15.12:6789/0 conn(0x7f702c1500d0 legacy=0x7f702c0c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f701c003ae0 0x7f701c003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f701c0088b0 0x7f701c00ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f701c00b500 0x7f701c00d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c0c7170 con 0x7f701c003ae0 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c00e2a0 con 0x7f701c0088b0 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c00e460 con 0x7f701c00b500 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7030fb3640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f701c003ae0 0x7f701c003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f702bfff640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f701c0088b0 0x7f701c00ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f70317b4640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f701c00b500 0x7f701c00d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7030fb3640 1 -- 172.21.15.12:0/3530450835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f701c00b500 msgr2=0x7f701c00d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7030fb3640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f701c00b500 0x7f701c00d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7030fb3640 1 -- 172.21.15.12:0/3530450835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f701c0088b0 msgr2=0x7f701c00ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7030fb3640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f701c0088b0 0x7f701c00ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f7030fb3640 1 -- 172.21.15.12:0/3530450835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c003450 con 0x7f701c003ae0 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.288+0000 7f70317b4640 1 -- 172.21.15.12:0/3530450835 reap_dead start 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f7030fb3640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f701c003ae0 0x7f701c003f00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f702000c240 tx=0x7f702001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c9fbbd2a36cd55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020005ae0 con 0x7f701c003ae0 2023-11-28T21:35:18.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7020005c40 con 0x7f701c003ae0 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f702b7fe640 1 -- 172.21.15.12:0/3530450835 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70200a5790 con 0x7f701c003ae0 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f7033217640 1 -- 172.21.15.12:0/3530450835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f701c003ae0 msgr2=0x7f701c003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f7033217640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f701c003ae0 0x7f701c003f00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f702000c240 tx=0x7f702001cbc0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f7033217640 1 -- 172.21.15.12:0/3530450835 shutdown_connections 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f7033217640 1 --2- 172.21.15.12:0/3530450835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f701c003ae0 0x7f701c003f00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f7033217640 1 -- 172.21.15.12:0/3530450835 >> 172.21.15.12:0/3530450835 conn(0x7f702c066360 msgr2=0x7f702c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f7033217640 1 -- 172.21.15.12:0/3530450835 shutdown_connections 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f7033217640 1 -- 172.21.15.12:0/3530450835 wait complete. 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f7033217640 1 Processor -- start 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.292+0000 7f7033217640 1 -- start start 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f7033217640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f701c003ae0 0x7f702c1f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f7033217640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f702c0c2010 0x7f702c14aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f7033217640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702c071370 0x7f702c071820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f7033217640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c1f2f60 con 0x7f702c0c2010 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f7033217640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702c070ee0 con 0x7f702c071370 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f7033217640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f702c071d60 con 0x7f701c003ae0 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f702bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f702c0c2010 0x7f702c14aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f702bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f702c0c2010 0x7f702c14aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38536/0 (socket says 172.21.15.12:38536) 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f702bfff640 1 -- 172.21.15.12:0/2352406380 learned_addr learned my addr 172.21.15.12:0/2352406380 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f7030fb3640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f701c003ae0 0x7f702c1f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f70317b4640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702c071370 0x7f702c071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f702bfff640 1 -- 172.21.15.12:0/2352406380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f701c003ae0 msgr2=0x7f702c1f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f702bfff640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f701c003ae0 0x7f702c1f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f702bfff640 1 -- 172.21.15.12:0/2352406380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702c071370 msgr2=0x7f702c071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f702bfff640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702c071370 0x7f702c071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f702bfff640 1 -- 172.21.15.12:0/2352406380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c14b0e0 con 0x7f702c0c2010 2023-11-28T21:35:18.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f702bfff640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f702c0c2010 0x7f702c14aba0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f701800e480 tx=0x7f701800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f70297fa640 1 -- 172.21.15.12:0/2352406380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70180052e0 con 0x7f702c0c2010 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f70297fa640 1 -- 172.21.15.12:0/2352406380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7018005440 con 0x7f702c0c2010 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f7033217640 1 -- 172.21.15.12:0/2352406380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702c0c24e0 con 0x7f702c0c2010 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f70297fa640 1 -- 172.21.15.12:0/2352406380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018003840 con 0x7f702c0c2010 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f7033217640 1 -- 172.21.15.12:0/2352406380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702c0705b0 con 0x7f702c0c2010 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.296+0000 7f7033217640 1 -- 172.21.15.12:0/2352406380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702c071050 con 0x7f702c0c2010 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.300+0000 7f70297fa640 1 -- 172.21.15.12:0/2352406380 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f70180039a0 con 0x7f702c0c2010 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.300+0000 7f70297fa640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7010066470 0x7f7010068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.300+0000 7f70297fa640 1 -- 172.21.15.12:0/2352406380 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7018083e30 con 0x7f702c0c2010 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.300+0000 7f7030fb3640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7010066470 0x7f7010068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.300+0000 7f7030fb3640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7010066470 0x7f7010068930 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f70200ab000 tx=0x7f70200a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.304+0000 7f70297fa640 1 -- 172.21.15.12:0/2352406380 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7018051060 con 0x7f702c0c2010 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.464+0000 7f7033217640 1 -- 172.21.15.12:0/2352406380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f702c074f60 con 0x7f702c0c2010 2023-11-28T21:35:18.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.464+0000 7f70297fa640 1 -- 172.21.15.12:0/2352406380 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7018054680 con 0x7f702c0c2010 2023-11-28T21:35:18.625 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-11-28T21:35:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 -- 172.21.15.12:0/2352406380 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7010066470 msgr2=0x7f7010068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7010066470 0x7f7010068930 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f70200ab000 tx=0x7f70200a7070 comp rx=0 tx=0).stop 2023-11-28T21:35:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 -- 172.21.15.12:0/2352406380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f702c0c2010 msgr2=0x7f702c14aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f702c0c2010 0x7f702c14aba0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f701800e480 tx=0x7f701800e940 comp rx=0 tx=0).stop 2023-11-28T21:35:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 -- 172.21.15.12:0/2352406380 shutdown_connections 2023-11-28T21:35:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7010066470 0x7f7010068930 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702c071370 0x7f702c071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f702c0c2010 0x7f702c14aba0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 --2- 172.21.15.12:0/2352406380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f701c003ae0 0x7f702c1f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 -- 172.21.15.12:0/2352406380 >> 172.21.15.12:0/2352406380 conn(0x7f702c066360 msgr2=0x7f702c14ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 -- 172.21.15.12:0/2352406380 shutdown_connections 2023-11-28T21:35:18.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:13.468+0000 7f7033217640 1 -- 172.21.15.12:0/2352406380 wait complete. 2023-11-28T21:35:18.627 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T21:35:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2ed765640 1 Processor -- start 2023-11-28T21:35:18.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2ed765640 1 -- start start 2023-11-28T21:35:18.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2ed765640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff2e806e300 0x7ff2e80c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2ed765640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e8157b70 con 0x7ff2e8151ed0 2023-11-28T21:35:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2ed765640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e8157e50 con 0x7ff2e8151af0 2023-11-28T21:35:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2ed765640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e80707f0 con 0x7ff2e806e300 2023-11-28T21:35:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2e77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff2e806e300 0x7ff2e80c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2de7fc640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff2e8151ed0 0x7ff2e8072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41902/0 (socket says 172.21.15.12:41902) 2023-11-28T21:35:18.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2de7fc640 1 -- 172.21.15.12:0/261848657 learned_addr learned my addr 172.21.15.12:0/261848657 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2e77fe640 1 -- 172.21.15.12:0/261848657 >> v1:172.21.15.144:6789/0 conn(0x7ff2e8151af0 legacy=0x7ff2e80c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2e77fe640 1 -- 172.21.15.12:0/261848657 >> v1:172.21.15.12:6789/0 conn(0x7ff2e8151ed0 legacy=0x7ff2e8072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2e77fe640 1 -- 172.21.15.12:0/261848657 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e8158130 con 0x7ff2e806e300 2023-11-28T21:35:18.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.784+0000 7ff2e67fc640 1 -- 172.21.15.12:0/261848657 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886542559 0 0) 0x7ff2e8157b70 con 0x7ff2e8151ed0 2023-11-28T21:35:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2e77fe640 1 --2- 172.21.15.12:0/261848657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e806e300 0x7ff2e80c7650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009d90 tx=0x7ff2e001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fee6c42bc5403f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2e67fc640 1 -- 172.21.15.12:0/261848657 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e00a5810 con 0x7ff2e806e300 2023-11-28T21:35:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2e67fc640 1 -- 172.21.15.12:0/261848657 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e00a5970 con 0x7ff2e806e300 2023-11-28T21:35:18.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2e67fc640 1 -- 172.21.15.12:0/261848657 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e00a5d00 con 0x7ff2e806e300 2023-11-28T21:35:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2ed765640 1 -- 172.21.15.12:0/261848657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e806e300 msgr2=0x7ff2e80c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2ed765640 1 --2- 172.21.15.12:0/261848657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e806e300 0x7ff2e80c7650 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009d90 tx=0x7ff2e001c950 comp rx=0 tx=0).stop 2023-11-28T21:35:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2ed765640 1 -- 172.21.15.12:0/261848657 shutdown_connections 2023-11-28T21:35:18.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2ed765640 1 --2- 172.21.15.12:0/261848657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e806e300 0x7ff2e80c7650 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2ed765640 1 -- 172.21.15.12:0/261848657 >> 172.21.15.12:0/261848657 conn(0x7ff2e8066360 msgr2=0x7ff2e8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2ed765640 1 -- 172.21.15.12:0/261848657 shutdown_connections 2023-11-28T21:35:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2ed765640 1 -- 172.21.15.12:0/261848657 wait complete. 2023-11-28T21:35:18.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2ed765640 1 Processor -- start 2023-11-28T21:35:18.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.788+0000 7ff2ed765640 1 -- start start 2023-11-28T21:35:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2ed765640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2e806e300 0x7ff2e80ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2ed765640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e8151af0 0x7ff2e80ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2ed765640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e814c2c0 0x7ff2e813fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2ed765640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e8157250 con 0x7ff2e814c2c0 2023-11-28T21:35:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2ed765640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e814c950 con 0x7ff2e8151af0 2023-11-28T21:35:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2ed765640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e813f890 con 0x7ff2e806e300 2023-11-28T21:35:18.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e814c2c0 0x7ff2e813fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e814c2c0 0x7ff2e813fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39648/0 (socket says 172.21.15.12:39648) 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2e77fe640 1 -- 172.21.15.12:0/1032412182 learned_addr learned my addr 172.21.15.12:0/1032412182 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2e6ffd640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2e806e300 0x7ff2e80ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2de7fc640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e8151af0 0x7ff2e80ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2e77fe640 1 -- 172.21.15.12:0/1032412182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2e806e300 msgr2=0x7ff2e80ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2e77fe640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2e806e300 0x7ff2e80ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2e77fe640 1 -- 172.21.15.12:0/1032412182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e8151af0 msgr2=0x7ff2e80ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2e77fe640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e8151af0 0x7ff2e80ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2e77fe640 1 -- 172.21.15.12:0/1032412182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e0009a70 con 0x7ff2e814c2c0 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2e77fe640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e814c2c0 0x7ff2e813fdd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00b0040 tx=0x7ff2e00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2dffff640 1 -- 172.21.15.12:0/1032412182 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e00b9860 con 0x7ff2e814c2c0 2023-11-28T21:35:18.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2dffff640 1 -- 172.21.15.12:0/1032412182 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e00b99c0 con 0x7ff2e814c2c0 2023-11-28T21:35:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2dffff640 1 -- 172.21.15.12:0/1032412182 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e00b9d70 con 0x7ff2e814c2c0 2023-11-28T21:35:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2ed765640 1 -- 172.21.15.12:0/1032412182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e80c90c0 con 0x7ff2e814c2c0 2023-11-28T21:35:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2ed765640 1 -- 172.21.15.12:0/1032412182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e814cb60 con 0x7ff2e814c2c0 2023-11-28T21:35:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.792+0000 7ff2ed765640 1 -- 172.21.15.12:0/1032412182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e8074f60 con 0x7ff2e814c2c0 2023-11-28T21:35:18.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.796+0000 7ff2dffff640 1 -- 172.21.15.12:0/1032412182 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff2e00ad440 con 0x7ff2e814c2c0 2023-11-28T21:35:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.796+0000 7ff2dffff640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2c80664f0 0x7ff2c80689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.796+0000 7ff2dffff640 1 -- 172.21.15.12:0/1032412182 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff2e011f3b0 con 0x7ff2e814c2c0 2023-11-28T21:35:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.796+0000 7ff2e6ffd640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2c80664f0 0x7ff2c80689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:18.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.796+0000 7ff2e6ffd640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2c80664f0 0x7ff2c80689b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8000f70 tx=0x7ff2d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:18.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.800+0000 7ff2dffff640 1 -- 172.21.15.12:0/1032412182 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff2e00ec770 con 0x7ff2e814c2c0 2023-11-28T21:35:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 -- 172.21.15.12:0/1032412182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff2e806b3b0 con 0x7ff2e814c2c0 2023-11-28T21:35:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2dffff640 1 -- 172.21.15.12:0/1032412182 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff2e00efd90 con 0x7ff2e814c2c0 2023-11-28T21:35:18.959 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:35:18.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 -- 172.21.15.12:0/1032412182 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2c80664f0 msgr2=0x7ff2c80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2c80664f0 0x7ff2c80689b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8000f70 tx=0x7ff2d8009040 comp rx=0 tx=0).stop 2023-11-28T21:35:18.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 -- 172.21.15.12:0/1032412182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e814c2c0 msgr2=0x7ff2e813fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:18.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e814c2c0 0x7ff2e813fdd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00b0040 tx=0x7ff2e00196f0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 -- 172.21.15.12:0/1032412182 shutdown_connections 2023-11-28T21:35:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2c80664f0 0x7ff2c80689b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2e814c2c0 0x7ff2e813fdd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2e8151af0 0x7ff2e80ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 --2- 172.21.15.12:0/1032412182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2e806e300 0x7ff2e80ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 -- 172.21.15.12:0/1032412182 >> 172.21.15.12:0/1032412182 conn(0x7ff2e8066360 msgr2=0x7ff2e814fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 -- 172.21.15.12:0/1032412182 shutdown_connections 2023-11-28T21:35:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:18.956+0000 7ff2ed765640 1 -- 172.21.15.12:0/1032412182 wait complete. 2023-11-28T21:35:18.980 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:35:19.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.120+0000 7f9c501b8640 1 Processor -- start 2023-11-28T21:35:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.120+0000 7f9c501b8640 1 -- start start 2023-11-28T21:35:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.120+0000 7f9c501b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48158b20 0x7f9c48156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.120+0000 7f9c501b8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c48067410 con 0x7f9c4806e300 2023-11-28T21:35:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.120+0000 7f9c501b8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4806e090 con 0x7f9c48150c20 2023-11-28T21:35:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.120+0000 7f9c501b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48157f30 con 0x7f9c48158b20 2023-11-28T21:35:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.120+0000 7f9c4d753640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9c4806e300 0x7f9c48158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41914/0 (socket says 172.21.15.12:41914) 2023-11-28T21:35:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.120+0000 7f9c4d753640 1 -- 172.21.15.12:0/3279458093 learned_addr learned my addr 172.21.15.12:0/3279458093 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:19.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.120+0000 7f9c4e755640 1 --2- 172.21.15.12:0/3279458093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48158b20 0x7f9c48156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4cf52640 1 -- 172.21.15.12:0/3279458093 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506652976 0 0) 0x7f9c48067410 con 0x7f9c4806e300 2023-11-28T21:35:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4cf52640 1 -- 172.21.15.12:0/3279458093 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c34003530 con 0x7f9c4806e300 2023-11-28T21:35:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4cf52640 1 -- 172.21.15.12:0/3279458093 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307346649 0 0) 0x7f9c4806e090 con 0x7f9c48150c20 2023-11-28T21:35:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4cf52640 1 -- 172.21.15.12:0/3279458093 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c48067410 con 0x7f9c48150c20 2023-11-28T21:35:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4cf52640 1 -- 172.21.15.12:0/3279458093 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1006659183 0 0) 0x7f9c34003530 con 0x7f9c4806e300 2023-11-28T21:35:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4e755640 1 -- 172.21.15.12:0/3279458093 >> v1:172.21.15.144:6789/0 conn(0x7f9c48150c20 legacy=0x7f9c480c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4e755640 1 -- 172.21.15.12:0/3279458093 >> v1:172.21.15.12:6789/0 conn(0x7f9c4806e300 legacy=0x7f9c48158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4e755640 1 -- 172.21.15.12:0/3279458093 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c48151000 con 0x7f9c48158b20 2023-11-28T21:35:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4e755640 1 --2- 172.21.15.12:0/3279458093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48158b20 0x7f9c48156910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009dc0 tx=0x7f9c4001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d6b439a44d5266 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4cf52640 1 -- 172.21.15.12:0/3279458093 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38001680 con 0x7f9c48158b20 2023-11-28T21:35:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4cf52640 1 -- 172.21.15.12:0/3279458093 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c400a55e0 con 0x7f9c48158b20 2023-11-28T21:35:19.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.124+0000 7f9c4cf52640 1 -- 172.21.15.12:0/3279458093 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c400a5970 con 0x7f9c48158b20 2023-11-28T21:35:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 -- 172.21.15.12:0/3279458093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48158b20 msgr2=0x7f9c48156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 --2- 172.21.15.12:0/3279458093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48158b20 0x7f9c48156910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009dc0 tx=0x7f9c4001c890 comp rx=0 tx=0).stop 2023-11-28T21:35:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 -- 172.21.15.12:0/3279458093 shutdown_connections 2023-11-28T21:35:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 --2- 172.21.15.12:0/3279458093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48158b20 0x7f9c48156910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 -- 172.21.15.12:0/3279458093 >> 172.21.15.12:0/3279458093 conn(0x7f9c48066360 msgr2=0x7f9c48066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 -- 172.21.15.12:0/3279458093 shutdown_connections 2023-11-28T21:35:19.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 -- 172.21.15.12:0/3279458093 wait complete. 2023-11-28T21:35:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 Processor -- start 2023-11-28T21:35:19.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 -- start start 2023-11-28T21:35:19.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c4806e300 0x7f9c481f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48150c20 0x7f9c48070e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c48158b20 0x7f9c48071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c481f6a00 con 0x7f9c48158b20 2023-11-28T21:35:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4814b7d0 con 0x7f9c4806e300 2023-11-28T21:35:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c501b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c481f6c50 con 0x7f9c48150c20 2023-11-28T21:35:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c4e755640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c48158b20 0x7f9c48071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c4e755640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c48158b20 0x7f9c48071370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39662/0 (socket says 172.21.15.12:39662) 2023-11-28T21:35:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c4e755640 1 -- 172.21.15.12:0/190950718 learned_addr learned my addr 172.21.15.12:0/190950718 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:19.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c4d753640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48150c20 0x7f9c48070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c4df54640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c4806e300 0x7f9c481f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c4e755640 1 -- 172.21.15.12:0/190950718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48150c20 msgr2=0x7f9c48070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c4e755640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48150c20 0x7f9c48070e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c4e755640 1 -- 172.21.15.12:0/190950718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c4806e300 msgr2=0x7f9c481f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c4e755640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c4806e300 0x7f9c481f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.128+0000 7f9c4e755640 1 -- 172.21.15.12:0/190950718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c40009a70 con 0x7f9c48158b20 2023-11-28T21:35:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c4e755640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c48158b20 0x7f9c48071370 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9c40098380 tx=0x7f9c40019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c3e7fc640 1 -- 172.21.15.12:0/190950718 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c400b9650 con 0x7f9c48158b20 2023-11-28T21:35:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c3e7fc640 1 -- 172.21.15.12:0/190950718 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c400b97b0 con 0x7f9c48158b20 2023-11-28T21:35:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c3e7fc640 1 -- 172.21.15.12:0/190950718 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c400b9b40 con 0x7f9c48158b20 2023-11-28T21:35:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c501b8640 1 -- 172.21.15.12:0/190950718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4814ba50 con 0x7f9c48158b20 2023-11-28T21:35:19.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c501b8640 1 -- 172.21.15.12:0/190950718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4814c010 con 0x7f9c48158b20 2023-11-28T21:35:19.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c501b8640 1 -- 172.21.15.12:0/190950718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4814c200 con 0x7f9c48158b20 2023-11-28T21:35:19.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c3e7fc640 1 -- 172.21.15.12:0/190950718 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9c400b9ca0 con 0x7f9c48158b20 2023-11-28T21:35:19.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c3e7fc640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9c2c0664a0 0x7f9c2c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c3e7fc640 1 -- 172.21.15.12:0/190950718 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9c4011f280 con 0x7f9c48158b20 2023-11-28T21:35:19.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.132+0000 7f9c4df54640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9c2c0664a0 0x7f9c2c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.136+0000 7f9c4df54640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9c2c0664a0 0x7f9c2c068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9c440011f0 tx=0x7f9c44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:19.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.136+0000 7f9c3e7fc640 1 -- 172.21.15.12:0/190950718 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9c400ec370 con 0x7f9c48158b20 2023-11-28T21:35:19.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 -- 172.21.15.12:0/190950718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c48074f60 con 0x7f9c48158b20 2023-11-28T21:35:19.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c3e7fc640 1 -- 172.21.15.12:0/190950718 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6529 (secure 0 0 0) 0x7f9c400ef990 con 0x7f9c48158b20 2023-11-28T21:35:19.331 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:19.331 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:35:14.337396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:35:19.332 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2023-11-28T21:35:19.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 -- 172.21.15.12:0/190950718 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9c2c0664a0 msgr2=0x7f9c2c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9c2c0664a0 0x7f9c2c068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9c440011f0 tx=0x7f9c44009040 comp rx=0 tx=0).stop 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 -- 172.21.15.12:0/190950718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c48158b20 msgr2=0x7f9c48071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c48158b20 0x7f9c48071370 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9c40098380 tx=0x7f9c40019630 comp rx=0 tx=0).stop 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 -- 172.21.15.12:0/190950718 shutdown_connections 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9c2c0664a0 0x7f9c2c068960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9c48158b20 0x7f9c48071370 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c48150c20 0x7f9c48070e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 --2- 172.21.15.12:0/190950718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c4806e300 0x7f9c481f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 -- 172.21.15.12:0/190950718 >> 172.21.15.12:0/190950718 conn(0x7f9c48066360 msgr2=0x7f9c481560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 -- 172.21.15.12:0/190950718 shutdown_connections 2023-11-28T21:35:19.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.328+0000 7f9c501b8640 1 -- 172.21.15.12:0/190950718 wait complete. 2023-11-28T21:35:19.352 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:19.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b26fd640 1 Processor -- start 2023-11-28T21:35:19.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b26fd640 1 -- start start 2023-11-28T21:35:19.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b26fd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f03ac150c20 0x7f03ac156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b26fd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f03ac06cf40 0x7f03ac158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b26fd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac067410 con 0x7f03ac06e300 2023-11-28T21:35:19.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b26fd640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac06e090 con 0x7f03ac06cf40 2023-11-28T21:35:19.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b26fd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac157f30 con 0x7f03ac150c20 2023-11-28T21:35:19.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03ab7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f03ac06e300 0x7f03ac0c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41924/0 (socket says 172.21.15.12:41924) 2023-11-28T21:35:19.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03ab7fe640 1 -- 172.21.15.12:0/2251035175 learned_addr learned my addr 172.21.15.12:0/2251035175 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:19.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b0c9a640 1 --2- 172.21.15.12:0/2251035175 >> v2:172.21.15.12:3300/0 conn(0x7f03ac06cf40 0x7f03ac158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03abfff640 1 --2- 172.21.15.12:0/2251035175 >> v2:172.21.15.144:3300/0 conn(0x7f03ac150c20 0x7f03ac156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b0c9a640 1 -- 172.21.15.12:0/2251035175 >> v2:172.21.15.144:3300/0 conn(0x7f03ac150c20 msgr2=0x7f03ac156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:19.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b0c9a640 1 --2- 172.21.15.12:0/2251035175 >> v2:172.21.15.144:3300/0 conn(0x7f03ac150c20 0x7f03ac156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b0c9a640 1 -- 172.21.15.12:0/2251035175 >> v1:172.21.15.12:6789/0 conn(0x7f03ac06e300 legacy=0x7f03ac0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03b0c9a640 1 -- 172.21.15.12:0/2251035175 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac151000 con 0x7f03ac06cf40 2023-11-28T21:35:19.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.492+0000 7f03aaffd640 1 -- 172.21.15.12:0/2251035175 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122664145 0 0) 0x7f03ac067410 con 0x7f03ac06e300 2023-11-28T21:35:19.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03b0c9a640 1 --2- 172.21.15.12:0/2251035175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac06cf40 0x7f03ac158410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f03ac06c6a0 tx=0x7f03a401eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220f9e5d568e041e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:19.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03aaffd640 1 -- 172.21.15.12:0/2251035175 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a40a7c80 con 0x7f03ac06cf40 2023-11-28T21:35:19.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03aaffd640 1 -- 172.21.15.12:0/2251035175 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a40a7de0 con 0x7f03ac06cf40 2023-11-28T21:35:19.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03aaffd640 1 -- 172.21.15.12:0/2251035175 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a40af660 con 0x7f03ac06cf40 2023-11-28T21:35:19.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03b26fd640 1 -- 172.21.15.12:0/2251035175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac06cf40 msgr2=0x7f03ac158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03b26fd640 1 --2- 172.21.15.12:0/2251035175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac06cf40 0x7f03ac158410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f03ac06c6a0 tx=0x7f03a401eda0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03b26fd640 1 -- 172.21.15.12:0/2251035175 shutdown_connections 2023-11-28T21:35:19.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03b26fd640 1 --2- 172.21.15.12:0/2251035175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac06cf40 0x7f03ac158410 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03b26fd640 1 --2- 172.21.15.12:0/2251035175 >> v2:172.21.15.144:3300/0 conn(0x7f03ac150c20 0x7f03ac156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03b26fd640 1 -- 172.21.15.12:0/2251035175 >> 172.21.15.12:0/2251035175 conn(0x7f03ac066360 msgr2=0x7f03ac066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:19.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03b26fd640 1 -- 172.21.15.12:0/2251035175 shutdown_connections 2023-11-28T21:35:19.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.496+0000 7f03b26fd640 1 -- 172.21.15.12:0/2251035175 wait complete. 2023-11-28T21:35:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b26fd640 1 Processor -- start 2023-11-28T21:35:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b26fd640 1 -- start start 2023-11-28T21:35:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b26fd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03ac06cf40 0x7f03ac151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b26fd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03ac06e300 0x7f03ac14c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b26fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac150c20 0x7f03ac1f0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b26fd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac14a700 con 0x7f03ac150c20 2023-11-28T21:35:19.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b26fd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac148d80 con 0x7f03ac06cf40 2023-11-28T21:35:19.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b26fd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac149af0 con 0x7f03ac06e300 2023-11-28T21:35:19.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b0c9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac150c20 0x7f03ac1f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b0c9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac150c20 0x7f03ac1f0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39678/0 (socket says 172.21.15.12:39678) 2023-11-28T21:35:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b0c9a640 1 -- 172.21.15.12:0/3030984287 learned_addr learned my addr 172.21.15.12:0/3030984287 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03ab7fe640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03ac06e300 0x7f03ac14c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03abfff640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03ac06cf40 0x7f03ac151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b0c9a640 1 -- 172.21.15.12:0/3030984287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03ac06e300 msgr2=0x7f03ac14c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b0c9a640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03ac06e300 0x7f03ac14c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b0c9a640 1 -- 172.21.15.12:0/3030984287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03ac06cf40 msgr2=0x7f03ac151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b0c9a640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03ac06cf40 0x7f03ac151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b0c9a640 1 -- 172.21.15.12:0/3030984287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a4009a70 con 0x7f03ac150c20 2023-11-28T21:35:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03abfff640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03ac06cf40 0x7f03ac151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:19.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b0c9a640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac150c20 0x7f03ac1f0130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f03a40b2040 tx=0x7f03a401bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03a8ff9640 1 -- 172.21.15.12:0/3030984287 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a40afcc0 con 0x7f03ac150c20 2023-11-28T21:35:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03a8ff9640 1 -- 172.21.15.12:0/3030984287 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a40afe20 con 0x7f03ac150c20 2023-11-28T21:35:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b26fd640 1 -- 172.21.15.12:0/3030984287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac149dd0 con 0x7f03ac150c20 2023-11-28T21:35:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03a8ff9640 1 -- 172.21.15.12:0/3030984287 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a40bb650 con 0x7f03ac150c20 2023-11-28T21:35:19.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.500+0000 7f03b26fd640 1 -- 172.21.15.12:0/3030984287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac14a390 con 0x7f03ac150c20 2023-11-28T21:35:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.504+0000 7f03b26fd640 1 -- 172.21.15.12:0/3030984287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac06c8c0 con 0x7f03ac150c20 2023-11-28T21:35:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.504+0000 7f03a8ff9640 1 -- 172.21.15.12:0/3030984287 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f03a40bb7b0 con 0x7f03ac150c20 2023-11-28T21:35:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.504+0000 7f03a8ff9640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0398066470 0x7f0398068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.504+0000 7f03a8ff9640 1 -- 172.21.15.12:0/3030984287 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f03a4121850 con 0x7f03ac150c20 2023-11-28T21:35:19.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.504+0000 7f03abfff640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0398066470 0x7f0398068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.504+0000 7f03abfff640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0398066470 0x7f0398068930 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f039c009b10 tx=0x7f039c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:19.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.508+0000 7f03a8ff9640 1 -- 172.21.15.12:0/3030984287 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03a40eea70 con 0x7f03ac150c20 2023-11-28T21:35:19.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.668+0000 7f03b26fd640 1 -- 172.21.15.12:0/3030984287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03ac06c1d0 con 0x7f03ac150c20 2023-11-28T21:35:19.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.668+0000 7f03a8ff9640 1 -- 172.21.15.12:0/3030984287 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7f03a40f2090 con 0x7f03ac150c20 2023-11-28T21:35:19.672 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:19.672 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 -- 172.21.15.12:0/3030984287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0398066470 msgr2=0x7f0398068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0398066470 0x7f0398068930 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f039c009b10 tx=0x7f039c008040 comp rx=0 tx=0).stop 2023-11-28T21:35:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 -- 172.21.15.12:0/3030984287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac150c20 msgr2=0x7f03ac1f0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac150c20 0x7f03ac1f0130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f03a40b2040 tx=0x7f03a401bb40 comp rx=0 tx=0).stop 2023-11-28T21:35:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 -- 172.21.15.12:0/3030984287 shutdown_connections 2023-11-28T21:35:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0398066470 0x7f0398068930 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03ac150c20 0x7f03ac1f0130 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03ac06e300 0x7f03ac14c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 --2- 172.21.15.12:0/3030984287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03ac06cf40 0x7f03ac151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 -- 172.21.15.12:0/3030984287 >> 172.21.15.12:0/3030984287 conn(0x7f03ac066360 msgr2=0x7f03ac071460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:19.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 -- 172.21.15.12:0/3030984287 shutdown_connections 2023-11-28T21:35:19.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.672+0000 7f03b26fd640 1 -- 172.21.15.12:0/3030984287 wait complete. 2023-11-28T21:35:19.694 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T21:34:29.878190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T21:34:34.039032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T21:34:35.369379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T21:34:39.373079+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T21:35:19.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T21:35:19.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f180bf2a640 1 Processor -- start 2023-11-28T21:35:19.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f180bf2a640 1 -- start start 2023-11-28T21:35:19.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f180bf2a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f18041502f0 0x7f18040c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f180bf2a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1804071120 con 0x7f18041596f0 2023-11-28T21:35:19.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f180bf2a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1804071400 con 0x7f1804071890 2023-11-28T21:35:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f180bf2a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18040c7810 con 0x7f18041502f0 2023-11-28T21:35:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f180a4c7640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f18041596f0 0x7f1804159b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41930/0 (socket says 172.21.15.12:41930) 2023-11-28T21:35:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f1809cc6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f18041502f0 0x7f18040c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f180a4c7640 1 -- 172.21.15.12:0/2281775580 learned_addr learned my addr 172.21.15.12:0/2281775580 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f1808cc4640 1 -- 172.21.15.12:0/2281775580 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360048475 0 0) 0x7f1804071120 con 0x7f18041596f0 2023-11-28T21:35:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f1809cc6640 1 -- 172.21.15.12:0/2281775580 >> v1:172.21.15.144:6789/0 conn(0x7f1804071890 legacy=0x7f1804071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f1809cc6640 1 -- 172.21.15.12:0/2281775580 >> v1:172.21.15.12:6789/0 conn(0x7f18041596f0 legacy=0x7f1804159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.840+0000 7f1809cc6640 1 -- 172.21.15.12:0/2281775580 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18040c6aa0 con 0x7f18041502f0 2023-11-28T21:35:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f1809cc6640 1 --2- 172.21.15.12:0/2281775580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18041502f0 0x7f18040c6ca0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f18000097c0 tx=0x7f180001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d53c87b99d130a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f1808cc4640 1 -- 172.21.15.12:0/2281775580 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18000a8970 con 0x7f18041502f0 2023-11-28T21:35:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f1808cc4640 1 -- 172.21.15.12:0/2281775580 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18000a8ad0 con 0x7f18041502f0 2023-11-28T21:35:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f1808cc4640 1 -- 172.21.15.12:0/2281775580 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18000a8e60 con 0x7f18041502f0 2023-11-28T21:35:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f180bf2a640 1 -- 172.21.15.12:0/2281775580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18041502f0 msgr2=0x7f18040c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f180bf2a640 1 --2- 172.21.15.12:0/2281775580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18041502f0 0x7f18040c6ca0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f18000097c0 tx=0x7f180001ec70 comp rx=0 tx=0).stop 2023-11-28T21:35:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f180bf2a640 1 -- 172.21.15.12:0/2281775580 shutdown_connections 2023-11-28T21:35:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f180bf2a640 1 --2- 172.21.15.12:0/2281775580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18041502f0 0x7f18040c6ca0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f180bf2a640 1 -- 172.21.15.12:0/2281775580 >> 172.21.15.12:0/2281775580 conn(0x7f18040c8450 msgr2=0x7f18040c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f180bf2a640 1 -- 172.21.15.12:0/2281775580 shutdown_connections 2023-11-28T21:35:19.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.844+0000 7f180bf2a640 1 -- 172.21.15.12:0/2281775580 wait complete. 2023-11-28T21:35:19.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180bf2a640 1 Processor -- start 2023-11-28T21:35:19.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180bf2a640 1 -- start start 2023-11-28T21:35:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180bf2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1804071890 0x7f180414c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180bf2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18041502f0 0x7f1804144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180bf2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18041596f0 0x7f18041e3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180bf2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041ed7d0 con 0x7f1804071890 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180bf2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041eda00 con 0x7f18041596f0 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180bf2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18041edca0 con 0x7f18041502f0 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f1809cc6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1804071890 0x7f180414c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f1809cc6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1804071890 0x7f180414c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39692/0 (socket says 172.21.15.12:39692) 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f1809cc6640 1 -- 172.21.15.12:0/2176306238 learned_addr learned my addr 172.21.15.12:0/2176306238 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180a4c7640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18041596f0 0x7f18041e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f18094c5640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18041502f0 0x7f1804144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f1809cc6640 1 -- 172.21.15.12:0/2176306238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18041502f0 msgr2=0x7f1804144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f1809cc6640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18041502f0 0x7f1804144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f1809cc6640 1 -- 172.21.15.12:0/2176306238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18041596f0 msgr2=0x7f18041e3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f1809cc6640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18041596f0 0x7f18041e3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f1809cc6640 1 -- 172.21.15.12:0/2176306238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1800009420 con 0x7f1804071890 2023-11-28T21:35:19.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f18094c5640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18041502f0 0x7f1804144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:35:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f1809cc6640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1804071890 0x7f180414c070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f180001b880 tx=0x7f180001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f17fa7fc640 1 -- 172.21.15.12:0/2176306238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18000a8970 con 0x7f1804071890 2023-11-28T21:35:19.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f17fa7fc640 1 -- 172.21.15.12:0/2176306238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18000b9410 con 0x7f1804071890 2023-11-28T21:35:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180bf2a640 1 -- 172.21.15.12:0/2176306238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18041edf80 con 0x7f1804071890 2023-11-28T21:35:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f17fa7fc640 1 -- 172.21.15.12:0/2176306238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18000b97a0 con 0x7f1804071890 2023-11-28T21:35:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.848+0000 7f180bf2a640 1 -- 172.21.15.12:0/2176306238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18041ee540 con 0x7f1804071890 2023-11-28T21:35:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.852+0000 7f180bf2a640 1 -- 172.21.15.12:0/2176306238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18041ee730 con 0x7f1804071890 2023-11-28T21:35:19.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.852+0000 7f17fa7fc640 1 -- 172.21.15.12:0/2176306238 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f18000b7040 con 0x7f1804071890 2023-11-28T21:35:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.852+0000 7f17fa7fc640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f17e8066470 0x7f17e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.852+0000 7f17fa7fc640 1 -- 172.21.15.12:0/2176306238 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1800121e10 con 0x7f1804071890 2023-11-28T21:35:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.852+0000 7f18094c5640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f17e8066470 0x7f17e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:19.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.852+0000 7f18094c5640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f17e8066470 0x7f17e8068930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f18040c3a10 tx=0x7f17f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:19.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:19.856+0000 7f17fa7fc640 1 -- 172.21.15.12:0/2176306238 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f18000ef010 con 0x7f1804071890 2023-11-28T21:35:20.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 -- 172.21.15.12:0/2176306238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1804074f60 con 0x7f1804071890 2023-11-28T21:35:20.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f17fa7fc640 1 -- 172.21.15.12:0/2176306238 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18000b5020 con 0x7f1804071890 2023-11-28T21:35:20.022 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-11-28T21:35:20.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 -- 172.21.15.12:0/2176306238 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f17e8066470 msgr2=0x7f17e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f17e8066470 0x7f17e8068930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f18040c3a10 tx=0x7f17f4009040 comp rx=0 tx=0).stop 2023-11-28T21:35:20.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 -- 172.21.15.12:0/2176306238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1804071890 msgr2=0x7f180414c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1804071890 0x7f180414c070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f180001b880 tx=0x7f180001ba10 comp rx=0 tx=0).stop 2023-11-28T21:35:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 -- 172.21.15.12:0/2176306238 shutdown_connections 2023-11-28T21:35:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f17e8066470 0x7f17e8068930 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18041596f0 0x7f18041e3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18041502f0 0x7f1804144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 --2- 172.21.15.12:0/2176306238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1804071890 0x7f180414c070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 -- 172.21.15.12:0/2176306238 >> 172.21.15.12:0/2176306238 conn(0x7f18040c8450 msgr2=0x7f180406b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 -- 172.21.15.12:0/2176306238 shutdown_connections 2023-11-28T21:35:20.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.020+0000 7f180bf2a640 1 -- 172.21.15.12:0/2176306238 wait complete. 2023-11-28T21:35:20.043 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T21:35:20.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f293e0fd640 1 Processor -- start 2023-11-28T21:35:20.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f293e0fd640 1 -- start start 2023-11-28T21:35:20.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f293e0fd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2938153ee0 0x7f293815a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f293e0fd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938070bd0 0x7f2938071080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f293e0fd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29380c6350 con 0x7f293806a5b0 2023-11-28T21:35:20.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f293e0fd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29380c6b90 con 0x7f2938153ee0 2023-11-28T21:35:20.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f293e0fd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2938072410 con 0x7f2938070bd0 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2936ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f293806a5b0 0x7f29380c7a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41946/0 (socket says 172.21.15.12:41946) 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2936ffd640 1 -- 172.21.15.12:0/47542370 learned_addr learned my addr 172.21.15.12:0/47542370 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29377fe640 1 --2- 172.21.15.12:0/47542370 >> v2:172.21.15.144:3300/0 conn(0x7f2938153ee0 0x7f293815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2937fff640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938070bd0 0x7f2938071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115127966 0 0) 0x7f29380c6350 con 0x7f293806a5b0 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2920003530 con 0x7f293806a5b0 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4011002319 0 0) 0x7f2920003530 con 0x7f293806a5b0 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938070bd0 msgr2=0x7f2938071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938070bd0 0x7f2938071080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 >> v2:172.21.15.144:3300/0 conn(0x7f2938153ee0 msgr2=0x7f293815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:20.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 --2- 172.21.15.12:0/47542370 >> v2:172.21.15.144:3300/0 conn(0x7f2938153ee0 0x7f293815a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29380c65d0 con 0x7f293806a5b0 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f292c001820 con 0x7f293806a5b0 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 >> v1:172.21.15.12:6789/0 conn(0x7f293806a5b0 legacy=0x7f29380c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938070bd0 0x7f29200039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920003fc0 0x7f29200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920004d10 0x7f2920005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380c6b90 con 0x7f2920004d10 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292000ba20 con 0x7f2920003fc0 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292000bbe0 con 0x7f2938070bd0 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2937fff640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920004d10 0x7f2920005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2936ffd640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920003fc0 0x7f29200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f29377fe640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938070bd0 0x7f29200039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2937fff640 1 -- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938070bd0 msgr2=0x7f29200039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:20.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2937fff640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938070bd0 0x7f29200039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2937fff640 1 -- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920003fc0 msgr2=0x7f29200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:20.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2937fff640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920003fc0 0x7f29200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2937fff640 1 -- 172.21.15.12:0/47542370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920003530 con 0x7f2920004d10 2023-11-28T21:35:20.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.188+0000 7f2936ffd640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920003fc0 0x7f29200047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f2937fff640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920004d10 0x7f2920005190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2930009fc0 tx=0x7f293001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efcc47b2d9d142c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29300a5c70 con 0x7f2920004d10 2023-11-28T21:35:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293000f040 con 0x7f2920004d10 2023-11-28T21:35:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f29367fc640 1 -- 172.21.15.12:0/47542370 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29300ad440 con 0x7f2920004d10 2023-11-28T21:35:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f293e0fd640 1 -- 172.21.15.12:0/47542370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920004d10 msgr2=0x7f2920005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f293e0fd640 1 --2- 172.21.15.12:0/47542370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2920004d10 0x7f2920005190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2930009fc0 tx=0x7f293001cc60 comp rx=0 tx=0).stop 2023-11-28T21:35:20.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f2937fff640 1 -- 172.21.15.12:0/47542370 reap_dead start 2023-11-28T21:35:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f293e0fd640 1 -- 172.21.15.12:0/47542370 shutdown_connections 2023-11-28T21:35:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f293e0fd640 1 -- 172.21.15.12:0/47542370 >> 172.21.15.12:0/47542370 conn(0x7f29380c8450 msgr2=0x7f29380c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f293e0fd640 1 -- 172.21.15.12:0/47542370 shutdown_connections 2023-11-28T21:35:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.192+0000 7f293e0fd640 1 -- 172.21.15.12:0/47542370 wait complete. 2023-11-28T21:35:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f293e0fd640 1 Processor -- start 2023-11-28T21:35:20.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f293e0fd640 1 -- start start 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f293e0fd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920004d10 0x7f29381f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f293e0fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f293814e110 0x7f293814b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f293e0fd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29381464d0 0x7f2938146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f293e0fd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293814eb20 con 0x7f293814e110 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f293e0fd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2938146e90 con 0x7f2920004d10 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f293e0fd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29380c36b0 con 0x7f29381464d0 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f2936ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f293814e110 0x7f293814b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f2936ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f293814e110 0x7f293814b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39700/0 (socket says 172.21.15.12:39700) 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f2936ffd640 1 -- 172.21.15.12:0/3419137477 learned_addr learned my addr 172.21.15.12:0/3419137477 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f29377fe640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920004d10 0x7f29381f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f2937fff640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29381464d0 0x7f2938146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f2936ffd640 1 -- 172.21.15.12:0/3419137477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29381464d0 msgr2=0x7f2938146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f2936ffd640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29381464d0 0x7f2938146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f2936ffd640 1 -- 172.21.15.12:0/3419137477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920004d10 msgr2=0x7f29381f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f2936ffd640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920004d10 0x7f29381f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f2936ffd640 1 -- 172.21.15.12:0/3419137477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29380c30f0 con 0x7f293814e110 2023-11-28T21:35:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f2936ffd640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f293814e110 0x7f293814b5a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f292c00e730 tx=0x7f292c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f291ffff640 1 -- 172.21.15.12:0/3419137477 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c0070d0 con 0x7f293814e110 2023-11-28T21:35:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f291ffff640 1 -- 172.21.15.12:0/3419137477 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c007230 con 0x7f293814e110 2023-11-28T21:35:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f291ffff640 1 -- 172.21.15.12:0/3419137477 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c0075c0 con 0x7f293814e110 2023-11-28T21:35:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f293e0fd640 1 -- 172.21.15.12:0/3419137477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29380c33d0 con 0x7f293814e110 2023-11-28T21:35:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.196+0000 7f293e0fd640 1 -- 172.21.15.12:0/3419137477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29380c3e90 con 0x7f293814e110 2023-11-28T21:35:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.200+0000 7f293e0fd640 1 -- 172.21.15.12:0/3419137477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2938074f60 con 0x7f293814e110 2023-11-28T21:35:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.200+0000 7f291ffff640 1 -- 172.21.15.12:0/3419137477 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f292c0057b0 con 0x7f293814e110 2023-11-28T21:35:20.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.200+0000 7f291ffff640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2924066470 0x7f2924068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.200+0000 7f291ffff640 1 -- 172.21.15.12:0/3419137477 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f292c0841e0 con 0x7f293814e110 2023-11-28T21:35:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.200+0000 7f29377fe640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2924066470 0x7f2924068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.200+0000 7f29377fe640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2924066470 0x7f2924068930 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2928009ad0 tx=0x7f2928008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.204+0000 7f291ffff640 1 -- 172.21.15.12:0/3419137477 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f292c0512d0 con 0x7f293814e110 2023-11-28T21:35:20.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 -- 172.21.15.12:0/3419137477 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29380c5e00 con 0x7f293814e110 2023-11-28T21:35:20.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f291ffff640 1 -- 172.21.15.12:0/3419137477 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f292c0548f0 con 0x7f293814e110 2023-11-28T21:35:20.368 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:35:20.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 -- 172.21.15.12:0/3419137477 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2924066470 msgr2=0x7f2924068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2924066470 0x7f2924068930 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2928009ad0 tx=0x7f2928008040 comp rx=0 tx=0).stop 2023-11-28T21:35:20.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 -- 172.21.15.12:0/3419137477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f293814e110 msgr2=0x7f293814b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f293814e110 0x7f293814b5a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f292c00e730 tx=0x7f292c00ebf0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 -- 172.21.15.12:0/3419137477 shutdown_connections 2023-11-28T21:35:20.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2924066470 0x7f2924068930 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29381464d0 0x7f2938146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f293814e110 0x7f293814b5a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 --2- 172.21.15.12:0/3419137477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2920004d10 0x7f29381f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 -- 172.21.15.12:0/3419137477 >> 172.21.15.12:0/3419137477 conn(0x7f29380c8450 msgr2=0x7f2938073b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:20.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.364+0000 7f293e0fd640 1 -- 172.21.15.12:0/3419137477 shutdown_connections 2023-11-28T21:35:20.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.368+0000 7f293e0fd640 1 -- 172.21.15.12:0/3419137477 wait complete. 2023-11-28T21:35:20.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T21:35:20.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb9654fd640 1 Processor -- start 2023-11-28T21:35:20.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb9654fd640 1 -- start start 2023-11-28T21:35:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb9654fd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb9600cba30 0x7fb96006ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb9654fd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96015ab40 con 0x7fb960153e00 2023-11-28T21:35:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb9654fd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb960159fe0 con 0x7fb96006b1b0 2023-11-28T21:35:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb9654fd640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96006a850 con 0x7fb9600cba30 2023-11-28T21:35:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb95e7fc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb9600cba30 0x7fb96006ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb95effd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb960153e00 0x7fb9600ca3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41962/0 (socket says 172.21.15.12:41962) 2023-11-28T21:35:20.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb95effd640 1 -- 172.21.15.12:0/2063216485 learned_addr learned my addr 172.21.15.12:0/2063216485 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb95e7fc640 1 -- 172.21.15.12:0/2063216485 >> v1:172.21.15.144:6789/0 conn(0x7fb96006b1b0 legacy=0x7fb9600caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb95e7fc640 1 -- 172.21.15.12:0/2063216485 >> v1:172.21.15.12:6789/0 conn(0x7fb960153e00 legacy=0x7fb9600ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb95e7fc640 1 -- 172.21.15.12:0/2063216485 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96015a2a0 con 0x7fb9600cba30 2023-11-28T21:35:20.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.532+0000 7fb95dffb640 1 -- 172.21.15.12:0/2063216485 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667383436 0 0) 0x7fb96015ab40 con 0x7fb960153e00 2023-11-28T21:35:20.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.536+0000 7fb95e7fc640 1 --2- 172.21.15.12:0/2063216485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600cba30 0x7fb96006ac70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9540097c0 tx=0x7fb95401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3591de4f211985a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:20.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb95dffb640 1 -- 172.21.15.12:0/2063216485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9540a8970 con 0x7fb9600cba30 2023-11-28T21:35:20.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb95dffb640 1 -- 172.21.15.12:0/2063216485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9540a8ad0 con 0x7fb9600cba30 2023-11-28T21:35:20.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb95dffb640 1 -- 172.21.15.12:0/2063216485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9540a8e60 con 0x7fb9600cba30 2023-11-28T21:35:20.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 -- 172.21.15.12:0/2063216485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600cba30 msgr2=0x7fb96006ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 --2- 172.21.15.12:0/2063216485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600cba30 0x7fb96006ac70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb9540097c0 tx=0x7fb95401ec70 comp rx=0 tx=0).stop 2023-11-28T21:35:20.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 -- 172.21.15.12:0/2063216485 shutdown_connections 2023-11-28T21:35:20.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 --2- 172.21.15.12:0/2063216485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb9600cba30 0x7fb96006ac70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 -- 172.21.15.12:0/2063216485 >> 172.21.15.12:0/2063216485 conn(0x7fb9600cf4e0 msgr2=0x7fb9600cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:20.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 -- 172.21.15.12:0/2063216485 shutdown_connections 2023-11-28T21:35:20.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 -- 172.21.15.12:0/2063216485 wait complete. 2023-11-28T21:35:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 Processor -- start 2023-11-28T21:35:20.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 -- start start 2023-11-28T21:35:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb96006b1b0 0x7fb960150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600cba30 0x7fb9601f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb960153e00 0x7fb9600b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600b2670 con 0x7fb96006b1b0 2023-11-28T21:35:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600c9250 con 0x7fb960153e00 2023-11-28T21:35:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.540+0000 7fb9654fd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb960151280 con 0x7fb9600cba30 2023-11-28T21:35:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb96006b1b0 0x7fb960150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb96006b1b0 0x7fb960150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39714/0 (socket says 172.21.15.12:39714) 2023-11-28T21:35:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95effd640 1 -- 172.21.15.12:0/2948273877 learned_addr learned my addr 172.21.15.12:0/2948273877 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95e7fc640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600cba30 0x7fb9601f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95f7fe640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb960153e00 0x7fb9600b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95effd640 1 -- 172.21.15.12:0/2948273877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600cba30 msgr2=0x7fb9601f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95effd640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600cba30 0x7fb9601f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95effd640 1 -- 172.21.15.12:0/2948273877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb960153e00 msgr2=0x7fb9600b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95effd640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb960153e00 0x7fb9600b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95effd640 1 -- 172.21.15.12:0/2948273877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb954009420 con 0x7fb96006b1b0 2023-11-28T21:35:20.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95e7fc640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600cba30 0x7fb9601f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:20.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb95effd640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb96006b1b0 0x7fb960150d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb95000c4a0 tx=0x7fb95000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:20.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb9477fe640 1 -- 172.21.15.12:0/2948273877 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9500074e0 con 0x7fb96006b1b0 2023-11-28T21:35:20.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb9477fe640 1 -- 172.21.15.12:0/2948273877 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb950007640 con 0x7fb96006b1b0 2023-11-28T21:35:20.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb9654fd640 1 -- 172.21.15.12:0/2948273877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9600c6eb0 con 0x7fb96006b1b0 2023-11-28T21:35:20.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb9477fe640 1 -- 172.21.15.12:0/2948273877 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950004a30 con 0x7fb96006b1b0 2023-11-28T21:35:20.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb9654fd640 1 -- 172.21.15.12:0/2948273877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb960144e00 con 0x7fb96006b1b0 2023-11-28T21:35:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb9654fd640 1 -- 172.21.15.12:0/2948273877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb960074f60 con 0x7fb96006b1b0 2023-11-28T21:35:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb9477fe640 1 -- 172.21.15.12:0/2948273877 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb950004b90 con 0x7fb96006b1b0 2023-11-28T21:35:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.544+0000 7fb9477fe640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb94c0664a0 0x7fb94c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.548+0000 7fb9477fe640 1 -- 172.21.15.12:0/2948273877 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb950083630 con 0x7fb96006b1b0 2023-11-28T21:35:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.548+0000 7fb95e7fc640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb94c0664a0 0x7fb94c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.548+0000 7fb95e7fc640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb94c0664a0 0x7fb94c068960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb9540b2000 tx=0x7fb954008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:20.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.552+0000 7fb9477fe640 1 -- 172.21.15.12:0/2948273877 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb9500507c0 con 0x7fb96006b1b0 2023-11-28T21:35:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.708+0000 7fb9654fd640 1 -- 172.21.15.12:0/2948273877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9600caca0 con 0x7fb96006b1b0 2023-11-28T21:35:20.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.708+0000 7fb9477fe640 1 -- 172.21.15.12:0/2948273877 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb950018020 con 0x7fb96006b1b0 2023-11-28T21:35:20.711 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-11-28T21:35:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.708+0000 7fb9654fd640 1 -- 172.21.15.12:0/2948273877 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb94c0664a0 msgr2=0x7fb94c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.708+0000 7fb9654fd640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb94c0664a0 0x7fb94c068960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb9540b2000 tx=0x7fb954008040 comp rx=0 tx=0).stop 2023-11-28T21:35:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.708+0000 7fb9654fd640 1 -- 172.21.15.12:0/2948273877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb96006b1b0 msgr2=0x7fb960150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.708+0000 7fb9654fd640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb96006b1b0 0x7fb960150d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb95000c4a0 tx=0x7fb95000c960 comp rx=0 tx=0).stop 2023-11-28T21:35:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.712+0000 7fb9654fd640 1 -- 172.21.15.12:0/2948273877 shutdown_connections 2023-11-28T21:35:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.712+0000 7fb9654fd640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb94c0664a0 0x7fb94c068960 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.712+0000 7fb9654fd640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb960153e00 0x7fb9600b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.712+0000 7fb9654fd640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9600cba30 0x7fb9601f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.712+0000 7fb9654fd640 1 --2- 172.21.15.12:0/2948273877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb96006b1b0 0x7fb960150d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.712+0000 7fb9654fd640 1 -- 172.21.15.12:0/2948273877 >> 172.21.15.12:0/2948273877 conn(0x7fb9600cf4e0 msgr2=0x7fb9600caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:20.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.712+0000 7fb9654fd640 1 -- 172.21.15.12:0/2948273877 shutdown_connections 2023-11-28T21:35:20.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.712+0000 7fb9654fd640 1 -- 172.21.15.12:0/2948273877 wait complete. 2023-11-28T21:35:20.733 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T21:35:20.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19c291d640 1 Processor -- start 2023-11-28T21:35:20.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19c291d640 1 -- start start 2023-11-28T21:35:20.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19c291d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19bc1540f0 0x7f19bc0c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19c291d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19bc15b180 0x7f19bc15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19c291d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f19bc15d210 0x7f19bc15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19c291d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19bc06e200 con 0x7f19bc15b180 2023-11-28T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19c291d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19bc06d2a0 con 0x7f19bc15d210 2023-11-28T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19c291d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc06d580 con 0x7f19bc1540f0 2023-11-28T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19bb7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19bc15b180 0x7f19bc15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19bb7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19bc15b180 0x7f19bc15b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39716/0 (socket says 172.21.15.12:39716) 2023-11-28T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19bb7fe640 1 -- 172.21.15.12:0/1746995414 learned_addr learned my addr 172.21.15.12:0/1746995414 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19bbfff640 1 --2- 172.21.15.12:0/1746995414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19bc1540f0 0x7f19bc0c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19c0eba640 1 --2- 172.21.15.12:0/1746995414 >> v2:172.21.15.144:3300/0 conn(0x7f19bc15d210 0x7f19bc15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19bb7fe640 1 -- 172.21.15.12:0/1746995414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19bc1540f0 msgr2=0x7f19bc0c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:20.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19bb7fe640 1 --2- 172.21.15.12:0/1746995414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19bc1540f0 0x7f19bc0c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19bb7fe640 1 -- 172.21.15.12:0/1746995414 >> v2:172.21.15.144:3300/0 conn(0x7f19bc15d210 msgr2=0x7f19bc15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:20.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19bb7fe640 1 --2- 172.21.15.12:0/1746995414 >> v2:172.21.15.144:3300/0 conn(0x7f19bc15d210 0x7f19bc15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19bb7fe640 1 -- 172.21.15.12:0/1746995414 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc06d860 con 0x7f19bc15b180 2023-11-28T21:35:20.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.876+0000 7f19c0eba640 1 --2- 172.21.15.12:0/1746995414 >> v2:172.21.15.144:3300/0 conn(0x7f19bc15d210 0x7f19bc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19bb7fe640 1 --2- 172.21.15.12:0/1746995414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15b180 0x7f19bc15b5a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f19bc15e310 tx=0x7f19b001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99785186ee820efd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19baffd640 1 -- 172.21.15.12:0/1746995414 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b00ac600 con 0x7f19bc15b180 2023-11-28T21:35:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19baffd640 1 -- 172.21.15.12:0/1746995414 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b00a8050 con 0x7f19bc15b180 2023-11-28T21:35:20.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19baffd640 1 -- 172.21.15.12:0/1746995414 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b00ac980 con 0x7f19bc15b180 2023-11-28T21:35:20.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19c291d640 1 -- 172.21.15.12:0/1746995414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15b180 msgr2=0x7f19bc15b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19c291d640 1 --2- 172.21.15.12:0/1746995414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15b180 0x7f19bc15b5a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f19bc15e310 tx=0x7f19b001c4d0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19c291d640 1 -- 172.21.15.12:0/1746995414 shutdown_connections 2023-11-28T21:35:20.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19c291d640 1 --2- 172.21.15.12:0/1746995414 >> v2:172.21.15.144:3300/0 conn(0x7f19bc15d210 0x7f19bc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19c291d640 1 --2- 172.21.15.12:0/1746995414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15b180 0x7f19bc15b5a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19c291d640 1 --2- 172.21.15.12:0/1746995414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19bc1540f0 0x7f19bc0c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19c291d640 1 -- 172.21.15.12:0/1746995414 >> 172.21.15.12:0/1746995414 conn(0x7f19bc066360 msgr2=0x7f19bc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:20.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19c291d640 1 -- 172.21.15.12:0/1746995414 shutdown_connections 2023-11-28T21:35:20.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.880+0000 7f19c291d640 1 -- 172.21.15.12:0/1746995414 wait complete. 2023-11-28T21:35:20.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c291d640 1 Processor -- start 2023-11-28T21:35:20.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c291d640 1 -- start start 2023-11-28T21:35:20.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c291d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19bc1540f0 0x7f19bc1f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c291d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19bc15b180 0x7f19bc15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c291d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15d210 0x7f19bc15c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c291d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc144970 con 0x7f19bc15d210 2023-11-28T21:35:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c291d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc1f5c20 con 0x7f19bc15b180 2023-11-28T21:35:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c291d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc150f70 con 0x7f19bc1540f0 2023-11-28T21:35:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c0eba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15d210 0x7f19bc15c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c0eba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15d210 0x7f19bc15c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39720/0 (socket says 172.21.15.12:39720) 2023-11-28T21:35:20.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c0eba640 1 -- 172.21.15.12:0/4059188943 learned_addr learned my addr 172.21.15.12:0/4059188943 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19bb7fe640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19bc15b180 0x7f19bc15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19bbfff640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19bc1540f0 0x7f19bc1f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c0eba640 1 -- 172.21.15.12:0/4059188943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19bc1540f0 msgr2=0x7f19bc1f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c0eba640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19bc1540f0 0x7f19bc1f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c0eba640 1 -- 172.21.15.12:0/4059188943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19bc15b180 msgr2=0x7f19bc15c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c0eba640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19bc15b180 0x7f19bc15c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c0eba640 1 -- 172.21.15.12:0/4059188943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b00094d0 con 0x7f19bc15d210 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19bb7fe640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19bc15b180 0x7f19bc15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c0eba640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15d210 0x7f19bc15c940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f19b4007df0 tx=0x7f19b400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19b8ff9640 1 -- 172.21.15.12:0/4059188943 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b40039a0 con 0x7f19bc15d210 2023-11-28T21:35:20.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19b8ff9640 1 -- 172.21.15.12:0/4059188943 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b400f040 con 0x7f19bc15d210 2023-11-28T21:35:20.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c291d640 1 -- 172.21.15.12:0/4059188943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc151250 con 0x7f19bc15d210 2023-11-28T21:35:20.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19b8ff9640 1 -- 172.21.15.12:0/4059188943 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4003d40 con 0x7f19bc15d210 2023-11-28T21:35:20.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.884+0000 7f19c291d640 1 -- 172.21.15.12:0/4059188943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19bc1f6150 con 0x7f19bc15d210 2023-11-28T21:35:20.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.888+0000 7f19c291d640 1 -- 172.21.15.12:0/4059188943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19bc074f60 con 0x7f19bc15d210 2023-11-28T21:35:20.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.888+0000 7f19b8ff9640 1 -- 172.21.15.12:0/4059188943 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f19b4004030 con 0x7f19bc15d210 2023-11-28T21:35:20.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.888+0000 7f19b8ff9640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19a8066470 0x7f19a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:20.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.888+0000 7f19b8ff9640 1 -- 172.21.15.12:0/4059188943 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f19b40844f0 con 0x7f19bc15d210 2023-11-28T21:35:20.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.888+0000 7f19bbfff640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19a8066470 0x7f19a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:20.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.888+0000 7f19bbfff640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19a8066470 0x7f19a8068930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009a70 tx=0x7f19ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:20.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:20.892+0000 7f19b8ff9640 1 -- 172.21.15.12:0/4059188943 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f19b40515e0 con 0x7f19bc15d210 2023-11-28T21:35:21.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 -- 172.21.15.12:0/4059188943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f19bc0c5610 con 0x7f19bc15d210 2023-11-28T21:35:21.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19b8ff9640 1 -- 172.21.15.12:0/4059188943 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f19b4054c00 con 0x7f19bc15d210 2023-11-28T21:35:21.051 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:35:21.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 -- 172.21.15.12:0/4059188943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19a8066470 msgr2=0x7f19a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19a8066470 0x7f19a8068930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f19ac009a70 tx=0x7f19ac008040 comp rx=0 tx=0).stop 2023-11-28T21:35:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 -- 172.21.15.12:0/4059188943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15d210 msgr2=0x7f19bc15c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15d210 0x7f19bc15c940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f19b4007df0 tx=0x7f19b400d4a0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 -- 172.21.15.12:0/4059188943 shutdown_connections 2023-11-28T21:35:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19a8066470 0x7f19a8068930 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19bc15d210 0x7f19bc15c940 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19bc15b180 0x7f19bc15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 --2- 172.21.15.12:0/4059188943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19bc1540f0 0x7f19bc1f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 -- 172.21.15.12:0/4059188943 >> 172.21.15.12:0/4059188943 conn(0x7f19bc066360 msgr2=0x7f19bc158840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 -- 172.21.15.12:0/4059188943 shutdown_connections 2023-11-28T21:35:21.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.048+0000 7f19c291d640 1 -- 172.21.15.12:0/4059188943 wait complete. 2023-11-28T21:35:21.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:35:21.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.212+0000 7f8a3629c640 1 Processor -- start 2023-11-28T21:35:21.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.212+0000 7f8a3629c640 1 -- start start 2023-11-28T21:35:21.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a3629c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8a30151ed0 0x7f8a300c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a3629c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8a3006e300 0x7f8a30072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a3629c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a30157b70 con 0x7f8a30151af0 2023-11-28T21:35:21.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a3629c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a300707f0 con 0x7f8a30151ed0 2023-11-28T21:35:21.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a3629c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a300c7b90 con 0x7f8a3006e300 2023-11-28T21:35:21.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a2ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8a30151af0 0x7f8a300c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41978/0 (socket says 172.21.15.12:41978) 2023-11-28T21:35:21.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a2ffff640 1 -- 172.21.15.12:0/476666322 learned_addr learned my addr 172.21.15.12:0/476666322 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:21.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a2f7fe640 1 --2- 172.21.15.12:0/476666322 >> v2:172.21.15.12:3300/0 conn(0x7f8a30151ed0 0x7f8a300c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a34839640 1 --2- 172.21.15.12:0/476666322 >> v2:172.21.15.144:3300/0 conn(0x7f8a3006e300 0x7f8a30072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a2f7fe640 1 -- 172.21.15.12:0/476666322 >> v2:172.21.15.144:3300/0 conn(0x7f8a3006e300 msgr2=0x7f8a30072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:21.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a2f7fe640 1 --2- 172.21.15.12:0/476666322 >> v2:172.21.15.144:3300/0 conn(0x7f8a3006e300 0x7f8a30072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a2f7fe640 1 -- 172.21.15.12:0/476666322 >> v1:172.21.15.12:6789/0 conn(0x7f8a30151af0 legacy=0x7f8a300c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a2f7fe640 1 -- 172.21.15.12:0/476666322 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a30157e50 con 0x7f8a30151ed0 2023-11-28T21:35:21.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.216+0000 7f8a2effd640 1 -- 172.21.15.12:0/476666322 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147711781 0 0) 0x7f8a30157b70 con 0x7f8a30151af0 2023-11-28T21:35:21.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a2f7fe640 1 --2- 172.21.15.12:0/476666322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a30151ed0 0x7f8a300c7650 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8a300716f0 tx=0x7f8a2401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57122dcc78b24286 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:21.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a2effd640 1 -- 172.21.15.12:0/476666322 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a240ab600 con 0x7f8a30151ed0 2023-11-28T21:35:21.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a2effd640 1 -- 172.21.15.12:0/476666322 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a240a7050 con 0x7f8a30151ed0 2023-11-28T21:35:21.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a2effd640 1 -- 172.21.15.12:0/476666322 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a240ab980 con 0x7f8a30151ed0 2023-11-28T21:35:21.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 -- 172.21.15.12:0/476666322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a30151ed0 msgr2=0x7f8a300c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 --2- 172.21.15.12:0/476666322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a30151ed0 0x7f8a300c7650 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8a300716f0 tx=0x7f8a2401c460 comp rx=0 tx=0).stop 2023-11-28T21:35:21.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 -- 172.21.15.12:0/476666322 shutdown_connections 2023-11-28T21:35:21.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 --2- 172.21.15.12:0/476666322 >> v2:172.21.15.144:3300/0 conn(0x7f8a3006e300 0x7f8a30072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 --2- 172.21.15.12:0/476666322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a30151ed0 0x7f8a300c7650 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 -- 172.21.15.12:0/476666322 >> 172.21.15.12:0/476666322 conn(0x7f8a30066360 msgr2=0x7f8a30066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:21.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 -- 172.21.15.12:0/476666322 shutdown_connections 2023-11-28T21:35:21.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 -- 172.21.15.12:0/476666322 wait complete. 2023-11-28T21:35:21.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 Processor -- start 2023-11-28T21:35:21.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 -- start start 2023-11-28T21:35:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.220+0000 7f8a3629c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3006e300 0x7f8a3014aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a3629c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a30151af0 0x7f8a30141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a3629c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a30141770 0x7f8a3014d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a3629c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a30141c20 con 0x7f8a3006e300 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a3629c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a30149600 con 0x7f8a30151af0 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a3629c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3014afd0 con 0x7f8a30141770 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3006e300 0x7f8a3014aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3006e300 0x7f8a3014aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39744/0 (socket says 172.21.15.12:39744) 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2ffff640 1 -- 172.21.15.12:0/1079481870 learned_addr learned my addr 172.21.15.12:0/1079481870 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2f7fe640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a30151af0 0x7f8a30141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a34839640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a30141770 0x7f8a3014d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2ffff640 1 -- 172.21.15.12:0/1079481870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a30141770 msgr2=0x7f8a3014d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2ffff640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a30141770 0x7f8a3014d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2ffff640 1 -- 172.21.15.12:0/1079481870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a30151af0 msgr2=0x7f8a30141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2ffff640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a30151af0 0x7f8a30141230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2ffff640 1 -- 172.21.15.12:0/1079481870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a24009420 con 0x7f8a3006e300 2023-11-28T21:35:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2ffff640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3006e300 0x7f8a3014aa90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8a200011f0 tx=0x7f8a2000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2cff9640 1 -- 172.21.15.12:0/1079481870 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200074e0 con 0x7f8a3006e300 2023-11-28T21:35:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2cff9640 1 -- 172.21.15.12:0/1079481870 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a20007640 con 0x7f8a3006e300 2023-11-28T21:35:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a3629c640 1 -- 172.21.15.12:0/1079481870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3014b530 con 0x7f8a3006e300 2023-11-28T21:35:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2cff9640 1 -- 172.21.15.12:0/1079481870 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a20004a10 con 0x7f8a3006e300 2023-11-28T21:35:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a3629c640 1 -- 172.21.15.12:0/1079481870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a30149890 con 0x7f8a3006e300 2023-11-28T21:35:21.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a3629c640 1 -- 172.21.15.12:0/1079481870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a30074f60 con 0x7f8a3006e300 2023-11-28T21:35:21.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.224+0000 7f8a2cff9640 1 -- 172.21.15.12:0/1079481870 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8a20004b70 con 0x7f8a3006e300 2023-11-28T21:35:21.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.228+0000 7f8a2cff9640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a1c066470 0x7f8a1c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.228+0000 7f8a2cff9640 1 -- 172.21.15.12:0/1079481870 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8a20083760 con 0x7f8a3006e300 2023-11-28T21:35:21.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.228+0000 7f8a2f7fe640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a1c066470 0x7f8a1c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.228+0000 7f8a2f7fe640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a1c066470 0x7f8a1c068930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a240ad000 tx=0x7f8a24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:21.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.228+0000 7f8a2cff9640 1 -- 172.21.15.12:0/1079481870 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a200508e0 con 0x7f8a3006e300 2023-11-28T21:35:21.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 -- 172.21.15.12:0/1079481870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a30162130 con 0x7f8a3006e300 2023-11-28T21:35:21.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a2cff9640 1 -- 172.21.15.12:0/1079481870 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6529 (secure 0 0 0) 0x7f8a20053f00 con 0x7f8a3006e300 2023-11-28T21:35:21.427 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:21.427 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:35:14.337396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:35:21.428 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2023-11-28T21:35:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 -- 172.21.15.12:0/1079481870 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a1c066470 msgr2=0x7f8a1c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a1c066470 0x7f8a1c068930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a240ad000 tx=0x7f8a24008040 comp rx=0 tx=0).stop 2023-11-28T21:35:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 -- 172.21.15.12:0/1079481870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3006e300 msgr2=0x7f8a3014aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3006e300 0x7f8a3014aa90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8a200011f0 tx=0x7f8a2000c9b0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 -- 172.21.15.12:0/1079481870 shutdown_connections 2023-11-28T21:35:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a1c066470 0x7f8a1c068930 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a30141770 0x7f8a3014d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a30151af0 0x7f8a30141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 --2- 172.21.15.12:0/1079481870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a3006e300 0x7f8a3014aa90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.424+0000 7f8a3629c640 1 -- 172.21.15.12:0/1079481870 >> 172.21.15.12:0/1079481870 conn(0x7f8a30066360 msgr2=0x7f8a300c6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.428+0000 7f8a3629c640 1 -- 172.21.15.12:0/1079481870 shutdown_connections 2023-11-28T21:35:21.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.428+0000 7f8a3629c640 1 -- 172.21.15.12:0/1079481870 wait complete. 2023-11-28T21:35:21.449 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:21.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7089fc4640 1 Processor -- start 2023-11-28T21:35:21.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7089fc4640 1 -- start start 2023-11-28T21:35:21.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7089fc4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70840ca110 0x7f70840cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7089fc4640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708415acc0 con 0x7f70840cfba0 2023-11-28T21:35:21.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7089fc4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7084159ea0 con 0x7f7084153e40 2023-11-28T21:35:21.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7089fc4640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70840ca520 con 0x7f70840ca110 2023-11-28T21:35:21.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7082ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f70840ca110 0x7f70840cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7083fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f70840cfba0 0x7f70840cb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41986/0 (socket says 172.21.15.12:41986) 2023-11-28T21:35:21.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7083fff640 1 -- 172.21.15.12:0/1553118046 learned_addr learned my addr 172.21.15.12:0/1553118046 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:21.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7082ffd640 1 -- 172.21.15.12:0/1553118046 >> v1:172.21.15.144:6789/0 conn(0x7f7084153e40 legacy=0x7f708415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:21.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7082ffd640 1 -- 172.21.15.12:0/1553118046 >> v1:172.21.15.12:6789/0 conn(0x7f70840cfba0 legacy=0x7f70840cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f70827fc640 1 -- 172.21.15.12:0/1553118046 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884015346 0 0) 0x7f708415acc0 con 0x7f70840cfba0 2023-11-28T21:35:21.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.592+0000 7f7082ffd640 1 -- 172.21.15.12:0/1553118046 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708415a140 con 0x7f70840ca110 2023-11-28T21:35:21.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f7082ffd640 1 --2- 172.21.15.12:0/1553118046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840ca110 0x7f70840cf660 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70780097c0 tx=0x7f707801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b577318c768798 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f70827fc640 1 -- 172.21.15.12:0/1553118046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70780ab600 con 0x7f70840ca110 2023-11-28T21:35:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f70827fc640 1 -- 172.21.15.12:0/1553118046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70780a7050 con 0x7f70840ca110 2023-11-28T21:35:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f70827fc640 1 -- 172.21.15.12:0/1553118046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70780ab980 con 0x7f70840ca110 2023-11-28T21:35:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f7089fc4640 1 -- 172.21.15.12:0/1553118046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840ca110 msgr2=0x7f70840cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f7089fc4640 1 --2- 172.21.15.12:0/1553118046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840ca110 0x7f70840cf660 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70780097c0 tx=0x7f707801c440 comp rx=0 tx=0).stop 2023-11-28T21:35:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f7089fc4640 1 -- 172.21.15.12:0/1553118046 shutdown_connections 2023-11-28T21:35:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f7089fc4640 1 --2- 172.21.15.12:0/1553118046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840ca110 0x7f70840cf660 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f7089fc4640 1 -- 172.21.15.12:0/1553118046 >> 172.21.15.12:0/1553118046 conn(0x7f708406b7b0 msgr2=0x7f7084066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f7089fc4640 1 -- 172.21.15.12:0/1553118046 shutdown_connections 2023-11-28T21:35:21.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.596+0000 7f7089fc4640 1 -- 172.21.15.12:0/1553118046 wait complete. 2023-11-28T21:35:21.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7089fc4640 1 Processor -- start 2023-11-28T21:35:21.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7089fc4640 1 -- start start 2023-11-28T21:35:21.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7089fc4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70840ca110 0x7f7084147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7089fc4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840cfba0 0x7f708414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7089fc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7084153e40 0x7f708414e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7089fc4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708414edd0 con 0x7f70840cfba0 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7089fc4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708414ffb0 con 0x7f7084153e40 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7089fc4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708414b090 con 0x7f70840ca110 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7082ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840cfba0 0x7f708414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7082ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840cfba0 0x7f708414e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39756/0 (socket says 172.21.15.12:39756) 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7082ffd640 1 -- 172.21.15.12:0/172980444 learned_addr learned my addr 172.21.15.12:0/172980444 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f70837fe640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70840ca110 0x7f7084147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7083fff640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7084153e40 0x7f708414e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7082ffd640 1 -- 172.21.15.12:0/172980444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70840ca110 msgr2=0x7f7084147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7082ffd640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70840ca110 0x7f7084147f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7082ffd640 1 -- 172.21.15.12:0/172980444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7084153e40 msgr2=0x7f708414e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7082ffd640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7084153e40 0x7f708414e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7082ffd640 1 -- 172.21.15.12:0/172980444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078009420 con 0x7f70840cfba0 2023-11-28T21:35:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f70837fe640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70840ca110 0x7f7084147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7082ffd640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840cfba0 0x7f708414e350 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f707801a070 tx=0x7f707801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f706bfff640 1 -- 172.21.15.12:0/172980444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078004220 con 0x7f70840cfba0 2023-11-28T21:35:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f706bfff640 1 -- 172.21.15.12:0/172980444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7078004380 con 0x7f70840cfba0 2023-11-28T21:35:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7089fc4640 1 -- 172.21.15.12:0/172980444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708414adb0 con 0x7f70840cfba0 2023-11-28T21:35:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f706bfff640 1 -- 172.21.15.12:0/172980444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078003820 con 0x7f70840cfba0 2023-11-28T21:35:21.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.600+0000 7f7089fc4640 1 -- 172.21.15.12:0/172980444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70840b3960 con 0x7f70840cfba0 2023-11-28T21:35:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.604+0000 7f7089fc4640 1 -- 172.21.15.12:0/172980444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7084074f60 con 0x7f70840cfba0 2023-11-28T21:35:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.604+0000 7f706bfff640 1 -- 172.21.15.12:0/172980444 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7078003980 con 0x7f70840cfba0 2023-11-28T21:35:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.604+0000 7f706bfff640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7070066470 0x7f7070068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.604+0000 7f706bfff640 1 -- 172.21.15.12:0/172980444 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f707811e750 con 0x7f70840cfba0 2023-11-28T21:35:21.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.604+0000 7f70837fe640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7070066470 0x7f7070068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.604+0000 7f70837fe640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7070066470 0x7f7070068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7074003ec0 tx=0x7f7074009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:21.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.608+0000 7f706bfff640 1 -- 172.21.15.12:0/172980444 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f70780eb840 con 0x7f70840cfba0 2023-11-28T21:35:21.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 -- 172.21.15.12:0/172980444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7084158300 con 0x7f70840cfba0 2023-11-28T21:35:21.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f706bfff640 1 -- 172.21.15.12:0/172980444 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18358 (secure 0 0 0) 0x7f70780c3090 con 0x7f70840cfba0 2023-11-28T21:35:21.771 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:21.771 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:34:41.754419+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:21.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 -- 172.21.15.12:0/172980444 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7070066470 msgr2=0x7f7070068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7070066470 0x7f7070068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7074003ec0 tx=0x7f7074009040 comp rx=0 tx=0).stop 2023-11-28T21:35:21.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 -- 172.21.15.12:0/172980444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840cfba0 msgr2=0x7f708414e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840cfba0 0x7f708414e350 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f707801a070 tx=0x7f707801ac80 comp rx=0 tx=0).stop 2023-11-28T21:35:21.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 -- 172.21.15.12:0/172980444 shutdown_connections 2023-11-28T21:35:21.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7070066470 0x7f7070068930 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7084153e40 0x7f708414e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70840cfba0 0x7f708414e350 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 --2- 172.21.15.12:0/172980444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70840ca110 0x7f7084147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 -- 172.21.15.12:0/172980444 >> 172.21.15.12:0/172980444 conn(0x7f708406b7b0 msgr2=0x7f7084150cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 -- 172.21.15.12:0/172980444 shutdown_connections 2023-11-28T21:35:21.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.768+0000 7f7089fc4640 1 -- 172.21.15.12:0/172980444 wait complete. 2023-11-28T21:35:21.796 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:35:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.936+0000 7f9e5e679640 1 Processor -- start 2023-11-28T21:35:21.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.936+0000 7f9e5e679640 1 -- start start 2023-11-28T21:35:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e5e679640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58153ee0 0x7f9e5815a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e5e679640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9e5806a5b0 0x7f9e580c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e5e679640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e580c6350 con 0x7f9e58070bd0 2023-11-28T21:35:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e5e679640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e580c6b90 con 0x7f9e5806a5b0 2023-11-28T21:35:21.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e5e679640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58072410 con 0x7f9e58153ee0 2023-11-28T21:35:21.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e577fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9e5806a5b0 0x7f9e580c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58153ee0 0x7f9e5815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58153ee0 0x7f9e5815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:42612/0 (socket says 172.21.15.12:42612) 2023-11-28T21:35:21.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e57fff640 1 -- 172.21.15.12:0/4112969183 learned_addr learned my addr 172.21.15.12:0/4112969183 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:21.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e5cc16640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9e58070bd0 0x7f9e58071060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:40952/0 (socket says 172.21.15.12:40952) 2023-11-28T21:35:21.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e57fff640 1 -- 172.21.15.12:0/4112969183 >> v2:172.21.15.144:3300/0 conn(0x7f9e5806a5b0 msgr2=0x7f9e580c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:21.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e57fff640 1 --2- 172.21.15.12:0/4112969183 >> v2:172.21.15.144:3300/0 conn(0x7f9e5806a5b0 0x7f9e580c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e57fff640 1 -- 172.21.15.12:0/4112969183 >> v1:172.21.15.144:6789/0 conn(0x7f9e58070bd0 legacy=0x7f9e58071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.940+0000 7f9e57fff640 1 -- 172.21.15.12:0/4112969183 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580c65d0 con 0x7f9e58153ee0 2023-11-28T21:35:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e57fff640 1 --2- 172.21.15.12:0/4112969183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58153ee0 0x7f9e5815a680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009870 tx=0x7f9e4801ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ab2b4d3aa332b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e56ffd640 1 -- 172.21.15.12:0/4112969183 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e480a8a30 con 0x7f9e58153ee0 2023-11-28T21:35:21.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e56ffd640 1 -- 172.21.15.12:0/4112969183 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e480a8b90 con 0x7f9e58153ee0 2023-11-28T21:35:21.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e56ffd640 1 -- 172.21.15.12:0/4112969183 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e480b0440 con 0x7f9e58153ee0 2023-11-28T21:35:21.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 -- 172.21.15.12:0/4112969183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58153ee0 msgr2=0x7f9e5815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 --2- 172.21.15.12:0/4112969183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58153ee0 0x7f9e5815a680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009870 tx=0x7f9e4801ed30 comp rx=0 tx=0).stop 2023-11-28T21:35:21.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 -- 172.21.15.12:0/4112969183 shutdown_connections 2023-11-28T21:35:21.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 --2- 172.21.15.12:0/4112969183 >> v2:172.21.15.144:3300/0 conn(0x7f9e5806a5b0 0x7f9e580c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 --2- 172.21.15.12:0/4112969183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58153ee0 0x7f9e5815a680 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 -- 172.21.15.12:0/4112969183 >> 172.21.15.12:0/4112969183 conn(0x7f9e580c8450 msgr2=0x7f9e580c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:21.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 -- 172.21.15.12:0/4112969183 shutdown_connections 2023-11-28T21:35:21.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 -- 172.21.15.12:0/4112969183 wait complete. 2023-11-28T21:35:21.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 Processor -- start 2023-11-28T21:35:21.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 -- start start 2023-11-28T21:35:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e5806a5b0 0x7f9e580c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.944+0000 7f9e5e679640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58070bd0 0x7f9e581526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e5e679640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e58153ee0 0x7f9e581f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e5e679640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580c42b0 con 0x7f9e5806a5b0 2023-11-28T21:35:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e5e679640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e581f0360 con 0x7f9e58153ee0 2023-11-28T21:35:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e5e679640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580c2cd0 con 0x7f9e58070bd0 2023-11-28T21:35:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e5806a5b0 0x7f9e580c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e5806a5b0 0x7f9e580c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39762/0 (socket says 172.21.15.12:39762) 2023-11-28T21:35:21.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e57fff640 1 -- 172.21.15.12:0/258735428 learned_addr learned my addr 172.21.15.12:0/258735428 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:21.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e577fe640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58070bd0 0x7f9e581526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e5cc16640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e58153ee0 0x7f9e581f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e57fff640 1 -- 172.21.15.12:0/258735428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58070bd0 msgr2=0x7f9e581526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e57fff640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58070bd0 0x7f9e581526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e57fff640 1 -- 172.21.15.12:0/258735428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e58153ee0 msgr2=0x7f9e581f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:21.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e57fff640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e58153ee0 0x7f9e581f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:21.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e57fff640 1 -- 172.21.15.12:0/258735428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e480094d0 con 0x7f9e5806a5b0 2023-11-28T21:35:21.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e57fff640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e5806a5b0 0x7f9e580c3d70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9e480b3040 tx=0x7f9e4801bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:21.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e54ff9640 1 -- 172.21.15.12:0/258735428 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e480b9620 con 0x7f9e5806a5b0 2023-11-28T21:35:21.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e54ff9640 1 -- 172.21.15.12:0/258735428 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e480b97c0 con 0x7f9e5806a5b0 2023-11-28T21:35:21.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e54ff9640 1 -- 172.21.15.12:0/258735428 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e480b9b70 con 0x7f9e5806a5b0 2023-11-28T21:35:21.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e5e679640 1 -- 172.21.15.12:0/258735428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e580c2fb0 con 0x7f9e5806a5b0 2023-11-28T21:35:21.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e5e679640 1 -- 172.21.15.12:0/258735428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e580c3570 con 0x7f9e5806a5b0 2023-11-28T21:35:21.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e5e679640 1 -- 172.21.15.12:0/258735428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e58074f60 con 0x7f9e5806a5b0 2023-11-28T21:35:21.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.948+0000 7f9e54ff9640 1 -- 172.21.15.12:0/258735428 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9e480b7070 con 0x7f9e5806a5b0 2023-11-28T21:35:21.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.952+0000 7f9e54ff9640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9e44066470 0x7f9e44068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:21.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.952+0000 7f9e54ff9640 1 -- 172.21.15.12:0/258735428 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9e481220f0 con 0x7f9e5806a5b0 2023-11-28T21:35:21.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.952+0000 7f9e577fe640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9e44066470 0x7f9e44068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:21.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.952+0000 7f9e577fe640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9e44066470 0x7f9e44068930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e580ca410 tx=0x7f9e4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:21.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:21.956+0000 7f9e54ff9640 1 -- 172.21.15.12:0/258735428 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9e480ef280 con 0x7f9e5806a5b0 2023-11-28T21:35:22.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 -- 172.21.15.12:0/258735428 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e580c5e00 con 0x7f9e5806a5b0 2023-11-28T21:35:22.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e54ff9640 1 -- 172.21.15.12:0/258735428 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6529 (secure 0 0 0) 0x7f9e480f28a0 con 0x7f9e5806a5b0 2023-11-28T21:35:22.149 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:22.150 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:35:14.337396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:35:22.152 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2023-11-28T21:35:22.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 -- 172.21.15.12:0/258735428 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9e44066470 msgr2=0x7f9e44068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9e44066470 0x7f9e44068930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e580ca410 tx=0x7f9e4c008040 comp rx=0 tx=0).stop 2023-11-28T21:35:22.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 -- 172.21.15.12:0/258735428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e5806a5b0 msgr2=0x7f9e580c3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e5806a5b0 0x7f9e580c3d70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9e480b3040 tx=0x7f9e4801bad0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 -- 172.21.15.12:0/258735428 shutdown_connections 2023-11-28T21:35:22.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9e44066470 0x7f9e44068930 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e58153ee0 0x7f9e581f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e58070bd0 0x7f9e581526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 --2- 172.21.15.12:0/258735428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e5806a5b0 0x7f9e580c3d70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 -- 172.21.15.12:0/258735428 >> 172.21.15.12:0/258735428 conn(0x7f9e580c8450 msgr2=0x7f9e580702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:22.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 -- 172.21.15.12:0/258735428 shutdown_connections 2023-11-28T21:35:22.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.144+0000 7f9e5e679640 1 -- 172.21.15.12:0/258735428 wait complete. 2023-11-28T21:35:22.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T21:34:36.378416+0000', 'modified': '2023-11-28T21:35:14.337396+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4261, 'mds_1': 4250, 'mds_2': 4247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4247': {'gid': 4247, 'name': 'a', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.12:6838/949980653', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6834', 'nonce': 949980653}, {'type': 'v1', 'addr': '172.21.15.12:6838', 'nonce': 949980653}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4250': {'gid': 4250, 'name': 'e', 'rank': 1, 'incarnation': 8, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.12:6839/3027655738', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6836', 'nonce': 3027655738}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 3027655738}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4261': {'gid': 4261, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.144:6833/3053317078', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3053317078}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3053317078}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T21:35:22.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-28T21:35:22.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T21:35:22.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.920+0000 7f0a935d8640 1 Processor -- start 2023-11-28T21:35:22.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.920+0000 7f0a935d8640 1 -- start start 2023-11-28T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a935d8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a8c15a910 0x7f0a8c070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a935d8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0a8c0710e0 0x7f0a8c0714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a935d8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a8c06f050 con 0x7f0a8c143070 2023-11-28T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a935d8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a8c071a30 con 0x7f0a8c15a910 2023-11-28T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a935d8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a8c06f580 con 0x7f0a8c0710e0 2023-11-28T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a90b73640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a8c15a910 0x7f0a8c070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a90b73640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0a8c15a910 0x7f0a8c070ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39774/0 (socket says 172.21.15.12:39774) 2023-11-28T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a91374640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0a8c143070 0x7f0a8c15b340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41992/0 (socket says 172.21.15.12:41992) 2023-11-28T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a90b73640 1 -- 172.21.15.12:0/3196183839 learned_addr learned my addr 172.21.15.12:0/3196183839 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:22.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a91b75640 1 --2- 172.21.15.12:0/3196183839 >> v2:172.21.15.144:3300/0 conn(0x7f0a8c0710e0 0x7f0a8c0714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a90b73640 1 -- 172.21.15.12:0/3196183839 >> v2:172.21.15.144:3300/0 conn(0x7f0a8c0710e0 msgr2=0x7f0a8c0714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a90b73640 1 --2- 172.21.15.12:0/3196183839 >> v2:172.21.15.144:3300/0 conn(0x7f0a8c0710e0 0x7f0a8c0714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a90b73640 1 -- 172.21.15.12:0/3196183839 >> v1:172.21.15.12:6789/0 conn(0x7f0a8c143070 legacy=0x7f0a8c15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a90b73640 1 -- 172.21.15.12:0/3196183839 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c15e330 con 0x7f0a8c15a910 2023-11-28T21:35:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a83fff640 1 -- 172.21.15.12:0/3196183839 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498880083 0 0) 0x7f0a8c06f050 con 0x7f0a8c143070 2023-11-28T21:35:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a90b73640 1 --2- 172.21.15.12:0/3196183839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 0x7f0a8c070ba0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c070680 tx=0x7f0a7c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbe343a41b8e0e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a83fff640 1 -- 172.21.15.12:0/3196183839 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0aa430 con 0x7f0a8c15a910 2023-11-28T21:35:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a83fff640 1 -- 172.21.15.12:0/3196183839 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c0aa590 con 0x7f0a8c15a910 2023-11-28T21:35:22.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.924+0000 7f0a83fff640 1 -- 172.21.15.12:0/3196183839 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0aa920 con 0x7f0a8c15a910 2023-11-28T21:35:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 -- 172.21.15.12:0/3196183839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 msgr2=0x7f0a8c070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 --2- 172.21.15.12:0/3196183839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 0x7f0a8c070ba0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c070680 tx=0x7f0a7c01c4d0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 -- 172.21.15.12:0/3196183839 shutdown_connections 2023-11-28T21:35:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 --2- 172.21.15.12:0/3196183839 >> v2:172.21.15.144:3300/0 conn(0x7f0a8c0710e0 0x7f0a8c0714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 --2- 172.21.15.12:0/3196183839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 0x7f0a8c070ba0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 -- 172.21.15.12:0/3196183839 >> 172.21.15.12:0/3196183839 conn(0x7f0a8c06b6c0 msgr2=0x7f0a8c06bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 -- 172.21.15.12:0/3196183839 shutdown_connections 2023-11-28T21:35:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 -- 172.21.15.12:0/3196183839 wait complete. 2023-11-28T21:35:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 Processor -- start 2023-11-28T21:35:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 -- start start 2023-11-28T21:35:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a8c0710e0 0x7f0a8c158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a8c143070 0x7f0a8c13e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 0x7f0a8c13ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c13c060 con 0x7f0a8c15a910 2023-11-28T21:35:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c157f30 con 0x7f0a8c0710e0 2023-11-28T21:35:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a935d8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1583c0 con 0x7f0a8c143070 2023-11-28T21:35:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a91b75640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 0x7f0a8c13ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a91b75640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 0x7f0a8c13ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39782/0 (socket says 172.21.15.12:39782) 2023-11-28T21:35:22.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a91b75640 1 -- 172.21.15.12:0/4189617252 learned_addr learned my addr 172.21.15.12:0/4189617252 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:22.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a90b73640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a8c143070 0x7f0a8c13e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.928+0000 7f0a91374640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a8c0710e0 0x7f0a8c158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a91b75640 1 -- 172.21.15.12:0/4189617252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a8c143070 msgr2=0x7f0a8c13e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a91b75640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a8c143070 0x7f0a8c13e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a91b75640 1 -- 172.21.15.12:0/4189617252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a8c0710e0 msgr2=0x7f0a8c158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a91b75640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a8c0710e0 0x7f0a8c158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a91b75640 1 -- 172.21.15.12:0/4189617252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c0094d0 con 0x7f0a8c15a910 2023-11-28T21:35:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a91b75640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 0x7f0a8c13ebb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007e40 tx=0x7f0a8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a81ffb640 1 -- 172.21.15.12:0/4189617252 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84014620 con 0x7f0a8c15a910 2023-11-28T21:35:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a81ffb640 1 -- 172.21.15.12:0/4189617252 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8400ee90 con 0x7f0a8c15a910 2023-11-28T21:35:22.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a81ffb640 1 -- 172.21.15.12:0/4189617252 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840149e0 con 0x7f0a8c15a910 2023-11-28T21:35:22.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a935d8640 1 -- 172.21.15.12:0/4189617252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8c1f2b90 con 0x7f0a8c15a910 2023-11-28T21:35:22.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a935d8640 1 -- 172.21.15.12:0/4189617252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c1589a0 con 0x7f0a8c15a910 2023-11-28T21:35:22.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a81ffb640 1 -- 172.21.15.12:0/4189617252 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0a84014b40 con 0x7f0a8c15a910 2023-11-28T21:35:22.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.932+0000 7f0a81ffb640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0a700664c0 0x7f0a70068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a81ffb640 1 -- 172.21.15.12:0/4189617252 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0a84084a30 con 0x7f0a8c15a910 2023-11-28T21:35:22.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0a700664c0 0x7f0a70068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 Processor -- start 2023-11-28T21:35:22.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 -- start start 2023-11-28T21:35:22.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0a700664c0 0x7f0a70068980 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007fd0 tx=0x7f0a88007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2a5930 0x559fcc2a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2aa9f0 0x559fcc2aadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc2ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc0d4130 con 0x559fcc2a5930 2023-11-28T21:35:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc0c19a0 con 0x559fcc2ab310 2023-11-28T21:35:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fcc0beed0 con 0x559fcc2aa9f0 2023-11-28T21:35:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2a5930 0x559fcc2a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2a5930 0x559fcc2a5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39790/0 (socket says 172.21.15.12:39790) 2023-11-28T21:35:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 -- 172.21.15.12:0/3149409954 learned_addr learned my addr 172.21.15.12:0/3149409954 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a90b73640 1 --2- 172.21.15.12:0/3149409954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2aa9f0 0x559fcc2aadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91b75640 1 --2- 172.21.15.12:0/3149409954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc2ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 -- 172.21.15.12:0/3149409954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2aa9f0 msgr2=0x559fcc2aadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 --2- 172.21.15.12:0/3149409954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2aa9f0 0x559fcc2aadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 -- 172.21.15.12:0/3149409954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 msgr2=0x559fcc2ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 --2- 172.21.15.12:0/3149409954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc2ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 -- 172.21.15.12:0/3149409954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fcbdb2890 con 0x559fcc2a5930 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a91374640 1 --2- 172.21.15.12:0/3149409954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2a5930 0x559fcc2a5d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0a88004fa0 tx=0x7f0a88013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e445f907cab5470 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a6effd640 1 -- 172.21.15.12:0/3149409954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88013b50 con 0x559fcc2a5930 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a6effd640 1 -- 172.21.15.12:0/3149409954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a88013cb0 con 0x559fcc2a5930 2023-11-28T21:35:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a6effd640 1 -- 172.21.15.12:0/3149409954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88012550 con 0x559fcc2a5930 2023-11-28T21:35:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 -- 172.21.15.12:0/3149409954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2a5930 msgr2=0x559fcc2a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 --2- 172.21.15.12:0/3149409954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2a5930 0x559fcc2a5d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0a88004fa0 tx=0x7f0a88013000 comp rx=0 tx=0).stop 2023-11-28T21:35:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 -- 172.21.15.12:0/3149409954 shutdown_connections 2023-11-28T21:35:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 --2- 172.21.15.12:0/3149409954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc2ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 --2- 172.21.15.12:0/3149409954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2aa9f0 0x559fcc2aadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 --2- 172.21.15.12:0/3149409954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2a5930 0x559fcc2a5d10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 -- 172.21.15.12:0/3149409954 >> 172.21.15.12:0/3149409954 conn(0x559fcc2a67a0 msgr2=0x559fcc2a8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 -- 172.21.15.12:0/3149409954 shutdown_connections 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.936+0000 7f0a9693d000 1 -- 172.21.15.12:0/3149409954 wait complete. 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 -- start start 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 Processor -- start 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc2a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc2a5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc2b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc0ba7c0 con 0x559fcc2aa9f0 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc072bf0 con 0x559fcc2ab310 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fcc078a90 con 0x559fcc2a5930 2023-11-28T21:35:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 10 client.0 _ll_register_callbacks cb 0x559fcc2553c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a90b73640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc2a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a90b73640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc2a5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39802/0 (socket says 172.21.15.12:39802) 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a90b73640 1 -- 172.21.15.12:0/1561601842 learned_addr learned my addr 172.21.15.12:0/1561601842 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a91374640 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc2a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a91b75640 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc2b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a90b73640 1 -- 172.21.15.12:0/1561601842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 msgr2=0x559fcc2a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a90b73640 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc2a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a90b73640 1 -- 172.21.15.12:0/1561601842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 msgr2=0x559fcc2b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a90b73640 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc2b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a90b73640 1 -- 172.21.15.12:0/1561601842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a88004c80 con 0x559fcc2aa9f0 2023-11-28T21:35:22.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a91b75640 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc2b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a90b73640 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc2a5250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c01c510 tx=0x7f0a7c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d9195e16be1ac2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a67fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-11-28T21:35:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a67fff640 1 -- 172.21.15.12:0/1561601842 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0b8040 con 0x559fcc2aa9f0 2023-11-28T21:35:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a67fff640 1 -- 172.21.15.12:0/1561601842 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0a7c0bfda0 con 0x559fcc2aa9f0 2023-11-28T21:35:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 -- 172.21.15.12:0/1561601842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fcc07c470 con 0x559fcc2aa9f0 2023-11-28T21:35:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a67fff640 1 -- 172.21.15.12:0/1561601842 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a7c0059a0 con 0x559fcc2aa9f0 2023-11-28T21:35:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 10 client.4845 fetch_fsmap learned FSMap version 13 2023-11-28T21:35:22.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 -- 172.21.15.12:0/1561601842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x559fcc07b190 con 0x559fcc2aa9f0 2023-11-28T21:35:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a67fff640 1 -- 172.21.15.12:0/1561601842 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559fcc07c470 con 0x559fcc2aa9f0 2023-11-28T21:35:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a67fff640 1 -- 172.21.15.12:0/1561601842 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4476+0+0 (secure 0 0 0) 0x7f0a7c0c5020 con 0x559fcc2aa9f0 2023-11-28T21:35:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 10 client.4845 fetch_fsmap finished waiting for FSMap version 13 2023-11-28T21:35:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 10 client.4845 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T21:35:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 20 client.4845 populate_metadata read hostname 'smithi012' 2023-11-28T21:35:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.12:6834/949980653,v1:172.21.15.12:6838/949980653] conn(0x559fcc2b56d0 0x559fcc2b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 4 client.4845 mds_command: new command op to 4247 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T21:35:22.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.940+0000 7f0a9693d000 1 -- 172.21.15.12:0/1561601842 --> [v2:172.21.15.12:6834/949980653,v1:172.21.15.12:6838/949980653] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fcc07fb50 con 0x559fcc2b56d0 2023-11-28T21:35:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a91374640 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.12:6834/949980653,v1:172.21.15.12:6838/949980653] conn(0x559fcc2b56d0 0x559fcc2b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a91374640 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.12:6834/949980653,v1:172.21.15.12:6838/949980653] conn(0x559fcc2b56d0 0x559fcc2b5ab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53d7db33319213fd server_cookie=22a96b3536d5c32c in_seq=0 out_seq=0 2023-11-28T21:35:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a67fff640 10 client.4845 ms_handle_connect on v2:172.21.15.12:6834/949980653 2023-11-28T21:35:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a67fff640 1 -- 172.21.15.12:0/1561601842 <== mds.2 v2:172.21.15.12:6834/949980653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559fcc07fb50 con 0x559fcc2b56d0 2023-11-28T21:35:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a67fff640 10 client.4845 handle_command_reply: tid=0 2023-11-28T21:35:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 client.4845 shutdown 2023-11-28T21:35:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 -- 172.21.15.12:0/1561601842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 msgr2=0x559fcc2a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc2a5250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c01c510 tx=0x7f0a7c0046d0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 -- 172.21.15.12:0/1561601842 shutdown_connections 2023-11-28T21:35:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.12:6834/949980653,v1:172.21.15.12:6838/949980653] conn(0x559fcc2b56d0 0x559fcc2b5ab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc2b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc2a5250 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1561601842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc2a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a67fff640 0 client.4845 ms_handle_reset on v2:172.21.15.12:6834/949980653 2023-11-28T21:35:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 -- 172.21.15.12:0/1561601842 >> 172.21.15.12:0/1561601842 conn(0x559fcc2a67a0 msgr2=0x559fcc26fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 -- 172.21.15.12:0/1561601842 shutdown_connections 2023-11-28T21:35:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 1 -- 172.21.15.12:0/1561601842 wait complete. 2023-11-28T21:35:22.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.944+0000 7f0a9693d000 20 client.4845 trim_cache size 0 max 16384 2023-11-28T21:35:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a9693d000 1 Processor -- start 2023-11-28T21:35:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a9693d000 1 -- start start 2023-11-28T21:35:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc2b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2aa9f0 0x559fcc2b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2ab310 0x559fcc2b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a9693d000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc2aa340 con 0x559fcc2ab310 2023-11-28T21:35:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a9693d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcbd73940 con 0x559fcc2aa9f0 2023-11-28T21:35:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a9693d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fcc0b8e90 con 0x559fcc2a5930 2023-11-28T21:35:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a91b75640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2ab310 0x559fcc2b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a91b75640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2ab310 0x559fcc2b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39818/0 (socket says 172.21.15.12:39818) 2023-11-28T21:35:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a91b75640 1 -- 172.21.15.12:0/877706952 learned_addr learned my addr 172.21.15.12:0/877706952 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a90b73640 1 --2- 172.21.15.12:0/877706952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2aa9f0 0x559fcc2b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a91374640 1 --2- 172.21.15.12:0/877706952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc2b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a91b75640 1 -- 172.21.15.12:0/877706952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 msgr2=0x559fcc2b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a91b75640 1 --2- 172.21.15.12:0/877706952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc2b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a91b75640 1 -- 172.21.15.12:0/877706952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2aa9f0 msgr2=0x559fcc2b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a91b75640 1 --2- 172.21.15.12:0/877706952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2aa9f0 0x559fcc2b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a91b75640 1 -- 172.21.15.12:0/877706952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fcc061490 con 0x559fcc2ab310 2023-11-28T21:35:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a91b75640 1 --2- 172.21.15.12:0/877706952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2ab310 0x559fcc2b43d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400e9b0 tx=0x7f0a84055910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf3f14c3d00b5a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a6f7fe640 1 -- 172.21.15.12:0/877706952 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84082d60 con 0x559fcc2ab310 2023-11-28T21:35:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a6f7fe640 1 -- 172.21.15.12:0/877706952 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a84053cb0 con 0x559fcc2ab310 2023-11-28T21:35:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.960+0000 7f0a6f7fe640 1 -- 172.21.15.12:0/877706952 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84054e00 con 0x559fcc2ab310 2023-11-28T21:35:22.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- 172.21.15.12:0/877706952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2ab310 msgr2=0x559fcc2b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 --2- 172.21.15.12:0/877706952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2ab310 0x559fcc2b43d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400e9b0 tx=0x7f0a84055910 comp rx=0 tx=0).stop 2023-11-28T21:35:22.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- 172.21.15.12:0/877706952 shutdown_connections 2023-11-28T21:35:22.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 --2- 172.21.15.12:0/877706952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2ab310 0x559fcc2b43d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 --2- 172.21.15.12:0/877706952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2aa9f0 0x559fcc2b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 --2- 172.21.15.12:0/877706952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc2b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- 172.21.15.12:0/877706952 >> 172.21.15.12:0/877706952 conn(0x559fcc2a67a0 msgr2=0x559fcc2b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:22.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- 172.21.15.12:0/877706952 shutdown_connections 2023-11-28T21:35:22.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- 172.21.15.12:0/877706952 wait complete. 2023-11-28T21:35:22.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- start start 2023-11-28T21:35:22.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 Processor -- start 2023-11-28T21:35:22.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc26d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc30ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc30f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc0b6130 con 0x559fcc2aa9f0 2023-11-28T21:35:22.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fcc0fc140 con 0x559fcc2ab310 2023-11-28T21:35:22.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fcc26fdd0 con 0x559fcc2a5930 2023-11-28T21:35:22.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 10 client.0 _ll_register_callbacks cb 0x559fcc235bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T21:35:22.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a90b73640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc30ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a90b73640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc30ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39832/0 (socket says 172.21.15.12:39832) 2023-11-28T21:35:22.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a90b73640 1 -- 172.21.15.12:0/1080560604 learned_addr learned my addr 172.21.15.12:0/1080560604 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:22.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a91374640 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc26d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a91b75640 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc30f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a90b73640 1 -- 172.21.15.12:0/1080560604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 msgr2=0x559fcc26d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:22.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a90b73640 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc26d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a90b73640 1 -- 172.21.15.12:0/1080560604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 msgr2=0x559fcc30f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:22.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a90b73640 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc30f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a90b73640 1 -- 172.21.15.12:0/1080560604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c1589a0 con 0x559fcc2aa9f0 2023-11-28T21:35:22.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a91b75640 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc30f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a90b73640 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc30ec70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0aa560 tx=0x7f0a7c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44afada540a4edbf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a67fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-11-28T21:35:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a67fff640 1 -- 172.21.15.12:0/1080560604 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c0043e0 con 0x559fcc2aa9f0 2023-11-28T21:35:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a67fff640 1 -- 172.21.15.12:0/1080560604 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0a7c0d2db0 con 0x559fcc2aa9f0 2023-11-28T21:35:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- 172.21.15.12:0/1080560604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fcc0b9110 con 0x559fcc2aa9f0 2023-11-28T21:35:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a67fff640 1 -- 172.21.15.12:0/1080560604 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a7c003620 con 0x559fcc2aa9f0 2023-11-28T21:35:22.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 10 client.4851 fetch_fsmap learned FSMap version 13 2023-11-28T21:35:22.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.964+0000 7f0a9693d000 1 -- 172.21.15.12:0/1080560604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x559fcbdb2cc0 con 0x559fcc2aa9f0 2023-11-28T21:35:22.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a67fff640 1 -- 172.21.15.12:0/1080560604 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559fcc0b9110 con 0x559fcc2aa9f0 2023-11-28T21:35:22.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a67fff640 1 -- 172.21.15.12:0/1080560604 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4476+0+0 (secure 0 0 0) 0x7f0a7c0c5040 con 0x559fcc2aa9f0 2023-11-28T21:35:22.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 10 client.4851 fetch_fsmap finished waiting for FSMap version 13 2023-11-28T21:35:22.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 10 client.4851 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T21:35:22.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 20 client.4851 populate_metadata read hostname 'smithi012' 2023-11-28T21:35:22.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.12:6834/949980653,v1:172.21.15.12:6838/949980653] conn(0x559fcc30f6f0 0x559fcc30da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 4 client.4851 mds_command: new command op to 4247 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T21:35:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 -- 172.21.15.12:0/1080560604 --> [v2:172.21.15.12:6834/949980653,v1:172.21.15.12:6838/949980653] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559fcc0e89a0 con 0x559fcc30f6f0 2023-11-28T21:35:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a91374640 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.12:6834/949980653,v1:172.21.15.12:6838/949980653] conn(0x559fcc30f6f0 0x559fcc30da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a91374640 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.12:6834/949980653,v1:172.21.15.12:6838/949980653] conn(0x559fcc30f6f0 0x559fcc30da50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa6ee9672584cb6 server_cookie=336ef633260ffbcc in_seq=0 out_seq=0 2023-11-28T21:35:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a67fff640 10 client.4851 ms_handle_connect on v2:172.21.15.12:6834/949980653 2023-11-28T21:35:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a67fff640 1 -- 172.21.15.12:0/1080560604 <== mds.2 v2:172.21.15.12:6834/949980653 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x559fcc0e89a0 con 0x559fcc30f6f0 2023-11-28T21:35:22.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a67fff640 10 client.4851 handle_command_reply: tid=0 2023-11-28T21:35:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 client.4851 shutdown 2023-11-28T21:35:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 -- 172.21.15.12:0/1080560604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 msgr2=0x559fcc30ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc30ec70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c0aa560 tx=0x7f0a7c01ad10 comp rx=0 tx=0).stop 2023-11-28T21:35:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 -- 172.21.15.12:0/1080560604 shutdown_connections 2023-11-28T21:35:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.12:6834/949980653,v1:172.21.15.12:6838/949980653] conn(0x559fcc30f6f0 0x559fcc30da50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559fcc2ab310 0x559fcc30f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fcc2aa9f0 0x559fcc30ec70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a67fff640 0 client.4851 ms_handle_reset on v2:172.21.15.12:6834/949980653 2023-11-28T21:35:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 --2- 172.21.15.12:0/1080560604 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x559fcc2a5930 0x559fcc26d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 -- 172.21.15.12:0/1080560604 >> 172.21.15.12:0/1080560604 conn(0x559fcc2a67a0 msgr2=0x559fcc2b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 -- 172.21.15.12:0/1080560604 shutdown_connections 2023-11-28T21:35:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 1 -- 172.21.15.12:0/1080560604 wait complete. 2023-11-28T21:35:22.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a9693d000 20 client.4851 trim_cache size 0 max 16384 2023-11-28T21:35:22.980 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:35:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "2975d7d0-8668-45d5-addd-c7b2ef978be7", 2023-11-28T21:35:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2023-11-28T21:35:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4247, 2023-11-28T21:35:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2023-11-28T21:35:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2023-11-28T21:35:22.980 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2023-11-28T21:35:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 13.656203547, 2023-11-28T21:35:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 13, 2023-11-28T21:35:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 24, 2023-11-28T21:35:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 24, 2023-11-28T21:35:22.981 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 53.397623082999999 2023-11-28T21:35:22.981 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:35:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 -- 172.21.15.12:0/4189617252 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0a700664c0 msgr2=0x7f0a70068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0a700664c0 0x7f0a70068980 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a88007fd0 tx=0x7f0a88007d60 comp rx=0 tx=0).stop 2023-11-28T21:35:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 -- 172.21.15.12:0/4189617252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 msgr2=0x7f0a8c13ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 0x7f0a8c13ebb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007e40 tx=0x7f0a8400e4b0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 -- 172.21.15.12:0/4189617252 shutdown_connections 2023-11-28T21:35:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0a700664c0 0x7f0a70068980 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0a8c15a910 0x7f0a8c13ebb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a8c143070 0x7f0a8c13e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 --2- 172.21.15.12:0/4189617252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a8c0710e0 0x7f0a8c158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:22.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 -- 172.21.15.12:0/4189617252 >> 172.21.15.12:0/4189617252 conn(0x7f0a8c06b6c0 msgr2=0x7f0a8c0ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 -- 172.21.15.12:0/4189617252 shutdown_connections 2023-11-28T21:35:22.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:22.968+0000 7f0a935d8640 1 -- 172.21.15.12:0/4189617252 wait complete. 2023-11-28T21:35:22.991 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-11-28T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e43e6640 1 Processor -- start 2023-11-28T21:35:23.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e43e6640 1 -- start start 2023-11-28T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e43e6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79dc150c20 0x7f79dc156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e43e6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79dc06e300 0x7f79dc0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e43e6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79dc067410 con 0x7f79dc06cf40 2023-11-28T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e43e6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc06e090 con 0x7f79dc06e300 2023-11-28T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e43e6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc157f30 con 0x7f79dc150c20 2023-11-28T21:35:23.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e1981640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79dc06e300 0x7f79dc0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e2983640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f79dc06cf40 0x7f79dc158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42006/0 (socket says 172.21.15.12:42006) 2023-11-28T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e2983640 1 -- 172.21.15.12:0/81091101 learned_addr learned my addr 172.21.15.12:0/81091101 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e1981640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79dc06e300 0x7f79dc0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39836/0 (socket says 172.21.15.12:39836) 2023-11-28T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.132+0000 7f79e2182640 1 --2- 172.21.15.12:0/81091101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79dc150c20 0x7f79dc156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.136+0000 7f79e1981640 1 -- 172.21.15.12:0/81091101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79dc150c20 msgr2=0x7f79dc156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.136+0000 7f79e1981640 1 --2- 172.21.15.12:0/81091101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79dc150c20 0x7f79dc156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.136+0000 7f79e1981640 1 -- 172.21.15.12:0/81091101 >> v1:172.21.15.12:6789/0 conn(0x7f79dc06cf40 legacy=0x7f79dc158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:23.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.136+0000 7f79e1981640 1 -- 172.21.15.12:0/81091101 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc151000 con 0x7f79dc06e300 2023-11-28T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.136+0000 7f79e1981640 1 --2- 172.21.15.12:0/81091101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc06e300 0x7f79dc0c6dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0097c0 tx=0x7f79cc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d630735eb523ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:23.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.136+0000 7f79e1180640 1 -- 172.21.15.12:0/81091101 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0ad640 con 0x7f79dc06e300 2023-11-28T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.136+0000 7f79e1180640 1 -- 172.21.15.12:0/81091101 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc0ad7a0 con 0x7f79dc06e300 2023-11-28T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.136+0000 7f79e1180640 1 -- 172.21.15.12:0/81091101 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0adb30 con 0x7f79dc06e300 2023-11-28T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.136+0000 7f79e43e6640 1 -- 172.21.15.12:0/81091101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc06e300 msgr2=0x7f79dc0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:23.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.136+0000 7f79e43e6640 1 --2- 172.21.15.12:0/81091101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc06e300 0x7f79dc0c6dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0097c0 tx=0x7f79cc01c480 comp rx=0 tx=0).stop 2023-11-28T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 -- 172.21.15.12:0/81091101 shutdown_connections 2023-11-28T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 --2- 172.21.15.12:0/81091101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc06e300 0x7f79dc0c6dd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 --2- 172.21.15.12:0/81091101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79dc150c20 0x7f79dc156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 -- 172.21.15.12:0/81091101 >> 172.21.15.12:0/81091101 conn(0x7f79dc066360 msgr2=0x7f79dc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 -- 172.21.15.12:0/81091101 shutdown_connections 2023-11-28T21:35:23.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 -- 172.21.15.12:0/81091101 wait complete. 2023-11-28T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 Processor -- start 2023-11-28T21:35:23.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 -- start start 2023-11-28T21:35:23.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79dc06cf40 0x7f79dc1f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc06e300 0x7f79dc14c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc150c20 0x7f79dc146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc071c20 con 0x7f79dc150c20 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc1ea840 con 0x7f79dc06e300 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e43e6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc1ea2a0 con 0x7f79dc06cf40 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e2983640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc150c20 0x7f79dc146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e2983640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc150c20 0x7f79dc146660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39844/0 (socket says 172.21.15.12:39844) 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e2983640 1 -- 172.21.15.12:0/3694162310 learned_addr learned my addr 172.21.15.12:0/3694162310 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e1981640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc06e300 0x7f79dc14c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e2182640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79dc06cf40 0x7f79dc1f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e2983640 1 -- 172.21.15.12:0/3694162310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79dc06cf40 msgr2=0x7f79dc1f2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e2983640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79dc06cf40 0x7f79dc1f2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e2983640 1 -- 172.21.15.12:0/3694162310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc06e300 msgr2=0x7f79dc14c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e2983640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc06e300 0x7f79dc14c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e2983640 1 -- 172.21.15.12:0/3694162310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc009420 con 0x7f79dc150c20 2023-11-28T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.140+0000 7f79e1981640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc06e300 0x7f79dc14c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:23.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.144+0000 7f79e2983640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc150c20 0x7f79dc146660 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79d400cbc0 tx=0x7f79d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.144+0000 7f79d2ffd640 1 -- 172.21.15.12:0/3694162310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400ad40 con 0x7f79dc150c20 2023-11-28T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.144+0000 7f79d2ffd640 1 -- 172.21.15.12:0/3694162310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d4012400 con 0x7f79dc150c20 2023-11-28T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.144+0000 7f79d2ffd640 1 -- 172.21.15.12:0/3694162310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400aea0 con 0x7f79dc150c20 2023-11-28T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.144+0000 7f79e43e6640 1 -- 172.21.15.12:0/3694162310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc14cdc0 con 0x7f79dc150c20 2023-11-28T21:35:23.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.144+0000 7f79e43e6640 1 -- 172.21.15.12:0/3694162310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc1f6490 con 0x7f79dc150c20 2023-11-28T21:35:23.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.144+0000 7f79d2ffd640 1 -- 172.21.15.12:0/3694162310 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f79d4007070 con 0x7f79dc150c20 2023-11-28T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.144+0000 7f79d2ffd640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79c0066470 0x7f79c0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79d2ffd640 1 -- 172.21.15.12:0/3694162310 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f79d4083fc0 con 0x7f79dc150c20 2023-11-28T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2182640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79c0066470 0x7f79c0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 Processor -- start 2023-11-28T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 -- start start 2023-11-28T21:35:23.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2182640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79c0066470 0x7f79c0068930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f79d80079c0 tx=0x7f79d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f4c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4ce9f0 0x55658f4cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4cf310 0x55658f4cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658f2f8130 con 0x55658f4cf310 2023-11-28T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658f2e59a0 con 0x55658f4ce9f0 2023-11-28T21:35:23.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55658f2e2ed0 con 0x55658f4c9930 2023-11-28T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2983640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4cf310 0x55658f4cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2983640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4cf310 0x55658f4cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39852/0 (socket says 172.21.15.12:39852) 2023-11-28T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2983640 1 -- 172.21.15.12:0/636973782 learned_addr learned my addr 172.21.15.12:0/636973782 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2182640 1 --2- 172.21.15.12:0/636973782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f4c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e1981640 1 --2- 172.21.15.12:0/636973782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4ce9f0 0x55658f4cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2983640 1 -- 172.21.15.12:0/636973782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 msgr2=0x55658f4c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2983640 1 --2- 172.21.15.12:0/636973782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f4c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2983640 1 -- 172.21.15.12:0/636973782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4ce9f0 msgr2=0x55658f4cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2983640 1 --2- 172.21.15.12:0/636973782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4ce9f0 0x55658f4cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2983640 1 -- 172.21.15.12:0/636973782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55658efd6890 con 0x55658f4cf310 2023-11-28T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e2983640 1 --2- 172.21.15.12:0/636973782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4cf310 0x55658f4cf6f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79d40513c0 tx=0x7f79d4054ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4979d83d96e14c3e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:23.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79bffff640 1 -- 172.21.15.12:0/636973782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d405b070 con 0x55658f4cf310 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79bffff640 1 -- 172.21.15.12:0/636973782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d4057070 con 0x55658f4cf310 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79bffff640 1 -- 172.21.15.12:0/636973782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d40515d0 con 0x55658f4cf310 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 -- 172.21.15.12:0/636973782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4cf310 msgr2=0x55658f4cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 --2- 172.21.15.12:0/636973782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4cf310 0x55658f4cf6f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f79d40513c0 tx=0x7f79d4054ec0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 -- 172.21.15.12:0/636973782 shutdown_connections 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 --2- 172.21.15.12:0/636973782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4cf310 0x55658f4cf6f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 --2- 172.21.15.12:0/636973782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4ce9f0 0x55658f4cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 --2- 172.21.15.12:0/636973782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f4c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 -- 172.21.15.12:0/636973782 >> 172.21.15.12:0/636973782 conn(0x55658f4ca7a0 msgr2=0x55658f4ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 -- 172.21.15.12:0/636973782 shutdown_connections 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.148+0000 7f79e7751000 1 -- 172.21.15.12:0/636973782 wait complete. 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 -- start start 2023-11-28T21:35:23.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 Processor -- start 2023-11-28T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f4c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f4c9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4cf310 0x55658f4d8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658f2de7c0 con 0x55658f4ce9f0 2023-11-28T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658f296bf0 con 0x55658f4cf310 2023-11-28T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55658f29ca90 con 0x55658f4c9930 2023-11-28T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 10 client.0 _ll_register_callbacks cb 0x55658f4793c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T21:35:23.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e1981640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f4c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e1981640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f4c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39868/0 (socket says 172.21.15.12:39868) 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e1981640 1 -- 172.21.15.12:0/47875001 learned_addr learned my addr 172.21.15.12:0/47875001 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e2182640 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f4c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e2983640 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4cf310 0x55658f4d8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e1981640 1 -- 172.21.15.12:0/47875001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 msgr2=0x55658f4c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e1981640 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f4c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e1981640 1 -- 172.21.15.12:0/47875001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4cf310 msgr2=0x55658f4d8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e1981640 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4cf310 0x55658f4d8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e1981640 1 -- 172.21.15.12:0/47875001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc1f6490 con 0x55658f4ce9f0 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e1981640 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f4c9250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0097c0 tx=0x7f79cc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0331091b051c2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79bd7fa640 1 -- 172.21.15.12:0/47875001 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0b8620 con 0x55658f4ce9f0 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79bd7fa640 1 -- 172.21.15.12:0/47875001 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f79cc0ad5e0 con 0x55658f4ce9f0 2023-11-28T21:35:23.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 -- 172.21.15.12:0/47875001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55658f2a0470 con 0x55658f4ce9f0 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79bd7fa640 1 -- 172.21.15.12:0/47875001 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79cc0b8780 con 0x55658f4ce9f0 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 10 client.4866 fetch_fsmap learned FSMap version 13 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 -- 172.21.15.12:0/47875001 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55658f29f190 con 0x55658f4ce9f0 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79bd7fa640 1 -- 172.21.15.12:0/47875001 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55658f2a0470 con 0x55658f4ce9f0 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79bd7fa640 1 -- 172.21.15.12:0/47875001 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4476+0+0 (secure 0 0 0) 0x7f79cc0b88e0 con 0x55658f4ce9f0 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 10 client.4866 fetch_fsmap finished waiting for FSMap version 13 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 10 client.4866 resolve_mds: resolved name 'e' to daemon mds.e 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 20 client.4866 populate_metadata read hostname 'smithi012' 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.12:6836/3027655738,v1:172.21.15.12:6839/3027655738] conn(0x55658f4d96d0 0x55658f4d9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 4 client.4866 mds_command: new command op to 4250 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.152+0000 7f79e7751000 1 -- 172.21.15.12:0/47875001 --> [v2:172.21.15.12:6836/3027655738,v1:172.21.15.12:6839/3027655738] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55658f2a3b50 con 0x55658f4d96d0 2023-11-28T21:35:23.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e2182640 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.12:6836/3027655738,v1:172.21.15.12:6839/3027655738] conn(0x55658f4d96d0 0x55658f4d9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e2182640 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.12:6836/3027655738,v1:172.21.15.12:6839/3027655738] conn(0x55658f4d96d0 0x55658f4d9ab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54aeb6e5e219edf2 server_cookie=a6dac9a3306f6a75 in_seq=0 out_seq=0 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79bd7fa640 10 client.4866 ms_handle_connect on v2:172.21.15.12:6836/3027655738 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79bd7fa640 1 -- 172.21.15.12:0/47875001 <== mds.1 v2:172.21.15.12:6836/3027655738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55658f2a3b50 con 0x55658f4d96d0 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79bd7fa640 10 client.4866 handle_command_reply: tid=0 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 client.4866 shutdown 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 -- 172.21.15.12:0/47875001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 msgr2=0x55658f4c9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f4c9250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0097c0 tx=0x7f79cc01acc0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 -- 172.21.15.12:0/47875001 shutdown_connections 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4cf310 0x55658f4d8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f4c9250 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.12:6836/3027655738,v1:172.21.15.12:6839/3027655738] conn(0x55658f4d96d0 0x55658f4d9ab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 --2- 172.21.15.12:0/47875001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f4c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 -- 172.21.15.12:0/47875001 >> 172.21.15.12:0/47875001 conn(0x55658f4ca7a0 msgr2=0x55658f493ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:23.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79bd7fa640 0 client.4866 ms_handle_reset on v2:172.21.15.12:6836/3027655738 2023-11-28T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 -- 172.21.15.12:0/47875001 shutdown_connections 2023-11-28T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 1 -- 172.21.15.12:0/47875001 wait complete. 2023-11-28T21:35:23.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.156+0000 7f79e7751000 20 client.4866 trim_cache size 0 max 16384 2023-11-28T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e7751000 1 Processor -- start 2023-11-28T21:35:23.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e7751000 1 -- start start 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e7751000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4c9930 0x55658f491fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e7751000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4ce9f0 0x55658f492500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e7751000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4cf310 0x55658f4d7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e7751000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658ef97940 con 0x55658f4c9930 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e7751000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658f2b8e30 con 0x55658f4ce9f0 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e7751000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55658f2da5c0 con 0x55658f4cf310 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2182640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4c9930 0x55658f491fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2182640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4c9930 0x55658f491fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39872/0 (socket says 172.21.15.12:39872) 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2182640 1 -- 172.21.15.12:0/3865657871 learned_addr learned my addr 172.21.15.12:0/3865657871 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e1981640 1 --2- 172.21.15.12:0/3865657871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4ce9f0 0x55658f492500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2983640 1 --2- 172.21.15.12:0/3865657871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4cf310 0x55658f4d7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2182640 1 -- 172.21.15.12:0/3865657871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4cf310 msgr2=0x55658f4d7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:23.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2182640 1 --2- 172.21.15.12:0/3865657871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4cf310 0x55658f4d7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2182640 1 -- 172.21.15.12:0/3865657871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4ce9f0 msgr2=0x55658f492500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2182640 1 --2- 172.21.15.12:0/3865657871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4ce9f0 0x55658f492500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2182640 1 -- 172.21.15.12:0/3865657871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55658f4ce2f0 con 0x55658f4c9930 2023-11-28T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2983640 1 --2- 172.21.15.12:0/3865657871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4cf310 0x55658f4d7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79e2182640 1 --2- 172.21.15.12:0/3865657871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4c9930 0x55658f491fc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f79d8010be0 tx=0x7f79d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cc8c3e46e7481c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79d0ff9640 1 -- 172.21.15.12:0/3865657871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8029070 con 0x55658f4c9930 2023-11-28T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79d0ff9640 1 -- 172.21.15.12:0/3865657871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d8010e90 con 0x55658f4c9930 2023-11-28T21:35:23.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.172+0000 7f79d0ff9640 1 -- 172.21.15.12:0/3865657871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d80168d0 con 0x55658f4c9930 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- 172.21.15.12:0/3865657871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4c9930 msgr2=0x55658f491fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 --2- 172.21.15.12:0/3865657871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4c9930 0x55658f491fc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f79d8010be0 tx=0x7f79d8016000 comp rx=0 tx=0).stop 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- 172.21.15.12:0/3865657871 shutdown_connections 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 --2- 172.21.15.12:0/3865657871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4cf310 0x55658f4d7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 --2- 172.21.15.12:0/3865657871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4ce9f0 0x55658f492500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 --2- 172.21.15.12:0/3865657871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4c9930 0x55658f491fc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- 172.21.15.12:0/3865657871 >> 172.21.15.12:0/3865657871 conn(0x55658f4ca7a0 msgr2=0x55658f4daee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- 172.21.15.12:0/3865657871 shutdown_connections 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- 172.21.15.12:0/3865657871 wait complete. 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- start start 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 Processor -- start 2023-11-28T21:35:23.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f491e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f532c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4cf310 0x55658f5331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658f2d8ac0 con 0x55658f4ce9f0 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55658f493dd0 con 0x55658f4cf310 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55658f2798a0 con 0x55658f4c9930 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 10 client.0 _ll_register_callbacks cb 0x55658f459bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e1981640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f532c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e1981640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f532c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39876/0 (socket says 172.21.15.12:39876) 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e1981640 1 -- 172.21.15.12:0/2245102513 learned_addr learned my addr 172.21.15.12:0/2245102513 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e2182640 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f491e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e2983640 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4cf310 0x55658f5331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e1981640 1 -- 172.21.15.12:0/2245102513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 msgr2=0x55658f491e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:23.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e1981640 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f491e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e1981640 1 -- 172.21.15.12:0/2245102513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4cf310 msgr2=0x55658f5331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e1981640 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4cf310 0x55658f5331b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e1981640 1 -- 172.21.15.12:0/2245102513 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d800e040 con 0x55658f4ce9f0 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e1981640 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f532c70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f79cc01ad00 tx=0x7f79cc0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45287362b0be0f82 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79bd7fa640 1 -- 172.21.15.12:0/2245102513 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0b8830 con 0x55658f4ce9f0 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79bd7fa640 1 -- 172.21.15.12:0/2245102513 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f79cc0ad5e0 con 0x55658f4ce9f0 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- 172.21.15.12:0/2245102513 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55658f2dd110 con 0x55658f4ce9f0 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79bd7fa640 1 -- 172.21.15.12:0/2245102513 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79cc006e20 con 0x55658f4ce9f0 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 10 client.4872 fetch_fsmap learned FSMap version 13 2023-11-28T21:35:23.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.176+0000 7f79e7751000 1 -- 172.21.15.12:0/2245102513 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55658f285490 con 0x55658f4ce9f0 2023-11-28T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79bd7fa640 1 -- 172.21.15.12:0/2245102513 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55658f2dd110 con 0x55658f4ce9f0 2023-11-28T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79bd7fa640 1 -- 172.21.15.12:0/2245102513 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4476+0+0 (secure 0 0 0) 0x7f79cc0d5020 con 0x55658f4ce9f0 2023-11-28T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 10 client.4872 fetch_fsmap finished waiting for FSMap version 13 2023-11-28T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 10 client.4872 resolve_mds: resolved name 'e' to daemon mds.e 2023-11-28T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 20 client.4872 populate_metadata read hostname 'smithi012' 2023-11-28T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.12:6836/3027655738,v1:172.21.15.12:6839/3027655738] conn(0x55658f531a50 0x55658f531e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 4 client.4872 mds_command: new command op to 4250 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 -- 172.21.15.12:0/2245102513 --> [v2:172.21.15.12:6836/3027655738,v1:172.21.15.12:6839/3027655738] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55658f30c9a0 con 0x55658f531a50 2023-11-28T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e2182640 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.12:6836/3027655738,v1:172.21.15.12:6839/3027655738] conn(0x55658f531a50 0x55658f531e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:23.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e2182640 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.12:6836/3027655738,v1:172.21.15.12:6839/3027655738] conn(0x55658f531a50 0x55658f531e30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c02ec0377a0a06 server_cookie=6f1cd1e35dbdd07b in_seq=0 out_seq=0 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79bd7fa640 10 client.4872 ms_handle_connect on v2:172.21.15.12:6836/3027655738 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79bd7fa640 1 -- 172.21.15.12:0/2245102513 <== mds.1 v2:172.21.15.12:6836/3027655738 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55658f30c9a0 con 0x55658f531a50 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79bd7fa640 10 client.4872 handle_command_reply: tid=0 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 client.4872 shutdown 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 -- 172.21.15.12:0/2245102513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 msgr2=0x55658f532c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f532c70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f79cc01ad00 tx=0x7f79cc0a7000 comp rx=0 tx=0).stop 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 -- 172.21.15.12:0/2245102513 shutdown_connections 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55658f4cf310 0x55658f5331b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55658f4ce9f0 0x55658f532c70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.12:6836/3027655738,v1:172.21.15.12:6839/3027655738] conn(0x55658f531a50 0x55658f531e30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 --2- 172.21.15.12:0/2245102513 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55658f4c9930 0x55658f491e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 -- 172.21.15.12:0/2245102513 >> 172.21.15.12:0/2245102513 conn(0x55658f4ca7a0 msgr2=0x55658f4d9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:23.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79bd7fa640 0 client.4872 ms_handle_reset on v2:172.21.15.12:6836/3027655738 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 -- 172.21.15.12:0/2245102513 shutdown_connections 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 1 -- 172.21.15.12:0/2245102513 wait complete. 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e7751000 20 client.4872 trim_cache size 0 max 16384 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "2975d7d0-8668-45d5-addd-c7b2ef978be7", 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4250, 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2023-11-28T21:35:23.187 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 19.892285414, 2023-11-28T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 13, 2023-11-28T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 24, 2023-11-28T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 24, 2023-11-28T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 53.608776630000001 2023-11-28T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:35:23.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 -- 172.21.15.12:0/3694162310 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79c0066470 msgr2=0x7f79c0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79c0066470 0x7f79c0068930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f79d80079c0 tx=0x7f79d8008040 comp rx=0 tx=0).stop 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 -- 172.21.15.12:0/3694162310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc150c20 msgr2=0x7f79dc146660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc150c20 0x7f79dc146660 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79d400cbc0 tx=0x7f79d400a430 comp rx=0 tx=0).stop 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 -- 172.21.15.12:0/3694162310 shutdown_connections 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79c0066470 0x7f79c0068930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79dc150c20 0x7f79dc146660 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79dc06e300 0x7f79dc14c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 --2- 172.21.15.12:0/3694162310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79dc06cf40 0x7f79dc1f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 -- 172.21.15.12:0/3694162310 >> 172.21.15.12:0/3694162310 conn(0x7f79dc066360 msgr2=0x7f79dc15a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 -- 172.21.15.12:0/3694162310 shutdown_connections 2023-11-28T21:35:23.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.180+0000 7f79e43e6640 1 -- 172.21.15.12:0/3694162310 wait complete. 2023-11-28T21:35:23.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-11-28T21:35:23.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.344+0000 7fdb774fe640 1 Processor -- start 2023-11-28T21:35:25.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.344+0000 7fdb774fe640 1 -- start start 2023-11-28T21:35:25.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.344+0000 7fdb774fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb70150c20 0x7fdb70156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.344+0000 7fdb774fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006e300 0x7fdb700c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.344+0000 7fdb774fe640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb70067410 con 0x7fdb7006cf40 2023-11-28T21:35:25.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.344+0000 7fdb774fe640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7006e090 con 0x7fdb70150c20 2023-11-28T21:35:25.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.344+0000 7fdb774fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70157f30 con 0x7fdb7006e300 2023-11-28T21:35:25.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.344+0000 7fdb7529a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb70150c20 0x7fdb70156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb7529a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb70150c20 0x7fdb70156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39892/0 (socket says 172.21.15.12:39892) 2023-11-28T21:35:25.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb7529a640 1 -- 172.21.15.12:0/486114904 learned_addr learned my addr 172.21.15.12:0/486114904 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:25.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb74a99640 1 --2- 172.21.15.12:0/486114904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006e300 0x7fdb700c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb7529a640 1 -- 172.21.15.12:0/486114904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006e300 msgr2=0x7fdb700c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb7529a640 1 --2- 172.21.15.12:0/486114904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006e300 0x7fdb700c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb7529a640 1 -- 172.21.15.12:0/486114904 >> v1:172.21.15.144:6789/0 conn(0x7fdb7006cf40 legacy=0x7fdb70158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb7529a640 1 -- 172.21.15.12:0/486114904 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70151000 con 0x7fdb70150c20 2023-11-28T21:35:25.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb7529a640 1 --2- 172.21.15.12:0/486114904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 0x7fdb70156910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009870 tx=0x7fdb6c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa5b4c07168370a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb67fff640 1 -- 172.21.15.12:0/486114904 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0ab650 con 0x7fdb70150c20 2023-11-28T21:35:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb67fff640 1 -- 172.21.15.12:0/486114904 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6c0ab7b0 con 0x7fdb70150c20 2023-11-28T21:35:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.348+0000 7fdb67fff640 1 -- 172.21.15.12:0/486114904 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0abb40 con 0x7fdb70150c20 2023-11-28T21:35:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 -- 172.21.15.12:0/486114904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 msgr2=0x7fdb70156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 --2- 172.21.15.12:0/486114904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 0x7fdb70156910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009870 tx=0x7fdb6c01c540 comp rx=0 tx=0).stop 2023-11-28T21:35:25.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 -- 172.21.15.12:0/486114904 shutdown_connections 2023-11-28T21:35:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 --2- 172.21.15.12:0/486114904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006e300 0x7fdb700c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 --2- 172.21.15.12:0/486114904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 0x7fdb70156910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 -- 172.21.15.12:0/486114904 >> 172.21.15.12:0/486114904 conn(0x7fdb70066360 msgr2=0x7fdb70066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 -- 172.21.15.12:0/486114904 shutdown_connections 2023-11-28T21:35:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 -- 172.21.15.12:0/486114904 wait complete. 2023-11-28T21:35:25.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 Processor -- start 2023-11-28T21:35:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 -- start start 2023-11-28T21:35:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006cf40 0x7fdb7013fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb7006e300 0x7fdb70140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 0x7fdb7015f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70145c50 con 0x7fdb70150c20 2023-11-28T21:35:25.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70146140 con 0x7fdb7006e300 2023-11-28T21:35:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb774fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7014d6e0 con 0x7fdb7006cf40 2023-11-28T21:35:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb75a9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 0x7fdb7015f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb75a9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 0x7fdb7015f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39896/0 (socket says 172.21.15.12:39896) 2023-11-28T21:35:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb75a9b640 1 -- 172.21.15.12:0/3946770995 learned_addr learned my addr 172.21.15.12:0/3946770995 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb7529a640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006cf40 0x7fdb7013fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb74a99640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb7006e300 0x7fdb70140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb75a9b640 1 -- 172.21.15.12:0/3946770995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006cf40 msgr2=0x7fdb7013fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006cf40 0x7fdb7013fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb75a9b640 1 -- 172.21.15.12:0/3946770995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb7006e300 msgr2=0x7fdb70140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb7006e300 0x7fdb70140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb75a9b640 1 -- 172.21.15.12:0/3946770995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6c0094d0 con 0x7fdb70150c20 2023-11-28T21:35:25.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb7529a640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006cf40 0x7fdb7013fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.352+0000 7fdb74a99640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb7006e300 0x7fdb70140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.356+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 0x7fdb7015f650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800c7a0 tx=0x7fdb6800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.356+0000 7fdb65ffb640 1 -- 172.21.15.12:0/3946770995 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68003b80 con 0x7fdb70150c20 2023-11-28T21:35:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.356+0000 7fdb65ffb640 1 -- 172.21.15.12:0/3946770995 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb68003ce0 con 0x7fdb70150c20 2023-11-28T21:35:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.356+0000 7fdb65ffb640 1 -- 172.21.15.12:0/3946770995 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6800aa90 con 0x7fdb70150c20 2023-11-28T21:35:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.356+0000 7fdb774fe640 1 -- 172.21.15.12:0/3946770995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb70145e60 con 0x7fdb70150c20 2023-11-28T21:35:25.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.356+0000 7fdb774fe640 1 -- 172.21.15.12:0/3946770995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7014d370 con 0x7fdb70150c20 2023-11-28T21:35:25.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.356+0000 7fdb65ffb640 1 -- 172.21.15.12:0/3946770995 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdb6800abf0 con 0x7fdb70150c20 2023-11-28T21:35:25.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.356+0000 7fdb65ffb640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb54066470 0x7fdb54068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.356+0000 7fdb65ffb640 1 -- 172.21.15.12:0/3946770995 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdb68083fe0 con 0x7fdb70150c20 2023-11-28T21:35:25.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.356+0000 7fdb7529a640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb54066470 0x7fdb54068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 Processor -- start 2023-11-28T21:35:25.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 -- start start 2023-11-28T21:35:25.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7529a640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb54066470 0x7fdb54068930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0af000 tx=0x7fdb6c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:25.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d79f0 0x5632468d7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d8310 0x5632468d86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563246701130 con 0x5632468d2930 2023-11-28T21:35:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632466ee9a0 con 0x5632468d8310 2023-11-28T21:35:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632466ebed0 con 0x5632468d79f0 2023-11-28T21:35:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7529a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7529a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39904/0 (socket says 172.21.15.12:39904) 2023-11-28T21:35:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7529a640 1 -- 172.21.15.12:0/3794029509 learned_addr learned my addr 172.21.15.12:0/3794029509 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/3794029509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d8310 0x5632468d86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb74a99640 1 --2- 172.21.15.12:0/3794029509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d79f0 0x5632468d7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7529a640 1 -- 172.21.15.12:0/3794029509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d79f0 msgr2=0x5632468d7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7529a640 1 --2- 172.21.15.12:0/3794029509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d79f0 0x5632468d7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7529a640 1 -- 172.21.15.12:0/3794029509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d8310 msgr2=0x5632468d86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:25.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7529a640 1 --2- 172.21.15.12:0/3794029509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d8310 0x5632468d86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7529a640 1 -- 172.21.15.12:0/3794029509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632463df890 con 0x5632468d2930 2023-11-28T21:35:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7529a640 1 --2- 172.21.15.12:0/3794029509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d2d10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c01c580 tx=0x7fdb6c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfea2dcaef390df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb52ffd640 1 -- 172.21.15.12:0/3794029509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0b47f0 con 0x5632468d2930 2023-11-28T21:35:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb52ffd640 1 -- 172.21.15.12:0/3794029509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6c0c4070 con 0x5632468d2930 2023-11-28T21:35:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb52ffd640 1 -- 172.21.15.12:0/3794029509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0b4b50 con 0x5632468d2930 2023-11-28T21:35:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 -- 172.21.15.12:0/3794029509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 msgr2=0x5632468d2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:25.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 --2- 172.21.15.12:0/3794029509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d2d10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c01c580 tx=0x7fdb6c004740 comp rx=0 tx=0).stop 2023-11-28T21:35:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 -- 172.21.15.12:0/3794029509 shutdown_connections 2023-11-28T21:35:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 --2- 172.21.15.12:0/3794029509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d8310 0x5632468d86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 --2- 172.21.15.12:0/3794029509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d79f0 0x5632468d7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 --2- 172.21.15.12:0/3794029509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d2d10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 -- 172.21.15.12:0/3794029509 >> 172.21.15.12:0/3794029509 conn(0x5632468d37a0 msgr2=0x5632468d5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 -- 172.21.15.12:0/3794029509 shutdown_connections 2023-11-28T21:35:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 -- 172.21.15.12:0/3794029509 wait complete. 2023-11-28T21:35:25.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.360+0000 7fdb7a866000 1 -- start start 2023-11-28T21:35:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 Processor -- start 2023-11-28T21:35:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d79f0 0x5632468d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d8310 0x5632468e1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632466e77c0 con 0x5632468d2930 2023-11-28T21:35:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324669fbf0 con 0x5632468d8310 2023-11-28T21:35:25.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632466a5a90 con 0x5632468d79f0 2023-11-28T21:35:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 10 client.0 _ll_register_callbacks cb 0x5632468823c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T21:35:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7529a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7529a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39914/0 (socket says 172.21.15.12:39914) 2023-11-28T21:35:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7529a640 1 -- 172.21.15.12:0/2231471465 learned_addr learned my addr 172.21.15.12:0/2231471465 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d8310 0x5632468e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb74a99640 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d79f0 0x5632468d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7529a640 1 -- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d79f0 msgr2=0x5632468d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7529a640 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d79f0 0x5632468d2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7529a640 1 -- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d8310 msgr2=0x5632468e1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7529a640 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d8310 0x5632468e1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7529a640 1 -- 172.21.15.12:0/2231471465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6c0ba070 con 0x5632468d2930 2023-11-28T21:35:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7529a640 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d1d10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c01ca20 tx=0x7fdb6c01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8737d2b726f8e424 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-11-28T21:35:25.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb4bfff640 1 -- 172.21.15.12:0/2231471465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0b49a0 con 0x5632468d2930 2023-11-28T21:35:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb4bfff640 1 -- 172.21.15.12:0/2231471465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdb6c0d3dc0 con 0x5632468d2930 2023-11-28T21:35:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 -- 172.21.15.12:0/2231471465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632466a9470 con 0x5632468d2930 2023-11-28T21:35:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb4bfff640 1 -- 172.21.15.12:0/2231471465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb6c0b4b00 con 0x5632468d2930 2023-11-28T21:35:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 10 client.4884 fetch_fsmap learned FSMap version 13 2023-11-28T21:35:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 -- 172.21.15.12:0/2231471465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x5632466a8190 con 0x5632468d2930 2023-11-28T21:35:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb4bfff640 1 -- 172.21.15.12:0/2231471465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632466a9470 con 0x5632468d2930 2023-11-28T21:35:25.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb4bfff640 1 -- 172.21.15.12:0/2231471465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4476+0+0 (secure 0 0 0) 0x7fdb6c0b4c60 con 0x5632468d2930 2023-11-28T21:35:25.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 10 client.4884 fetch_fsmap finished waiting for FSMap version 13 2023-11-28T21:35:25.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 10 client.4884 resolve_mds: resolved name 'f' to daemon mds.f 2023-11-28T21:35:25.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 20 client.4884 populate_metadata read hostname 'smithi012' 2023-11-28T21:35:25.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x5632468e2640 0x5632468e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 4 client.4884 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-28T21:35:25.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb7a866000 1 -- 172.21.15.12:0/2231471465 --> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632466acb50 con 0x5632468e2640 2023-11-28T21:35:25.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.364+0000 7fdb74a99640 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x5632468e2640 0x5632468e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb74a99640 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x5632468e2640 0x5632468e2a20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f37f188c6c5a056 server_cookie=8547aab533144b93 in_seq=0 out_seq=0 2023-11-28T21:35:25.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb4bfff640 10 client.4884 ms_handle_connect on v2:172.21.15.144:6832/3053317078 2023-11-28T21:35:25.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb4bfff640 1 -- 172.21.15.12:0/2231471465 <== mds.0 v2:172.21.15.144:6832/3053317078 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632466acb50 con 0x5632468e2640 2023-11-28T21:35:25.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb4bfff640 10 client.4884 handle_command_reply: tid=0 2023-11-28T21:35:25.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 client.4884 shutdown 2023-11-28T21:35:25.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 -- 172.21.15.12:0/2231471465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 msgr2=0x5632468d1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:25.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d1d10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c01ca20 tx=0x7fdb6c01ad80 comp rx=0 tx=0).stop 2023-11-28T21:35:25.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 -- 172.21.15.12:0/2231471465 shutdown_connections 2023-11-28T21:35:25.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x5632468e2640 0x5632468e2a20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d8310 0x5632468e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d79f0 0x5632468d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 --2- 172.21.15.12:0/2231471465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d2930 0x5632468d1d10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 -- 172.21.15.12:0/2231471465 >> 172.21.15.12:0/2231471465 conn(0x5632468d37a0 msgr2=0x56324689cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb4bfff640 0 client.4884 ms_handle_reset on v2:172.21.15.144:6832/3053317078 2023-11-28T21:35:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 -- 172.21.15.12:0/2231471465 shutdown_connections 2023-11-28T21:35:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 1 -- 172.21.15.12:0/2231471465 wait complete. 2023-11-28T21:35:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.368+0000 7fdb7a866000 20 client.4884 trim_cache size 0 max 16384 2023-11-28T21:35:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb7a866000 1 Processor -- start 2023-11-28T21:35:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb7a866000 1 -- start start 2023-11-28T21:35:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d2930 0x5632468e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d79f0 0x5632468e1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x56324689b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb7a866000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632468d6590 con 0x5632468d8310 2023-11-28T21:35:25.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb7a866000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632463a0940 con 0x5632468d79f0 2023-11-28T21:35:25.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb7a866000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632466a6e20 con 0x5632468d2930 2023-11-28T21:35:25.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb75a9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x56324689b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb75a9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x56324689b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39930/0 (socket says 172.21.15.12:39930) 2023-11-28T21:35:25.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb75a9b640 1 -- 172.21.15.12:0/2820493723 learned_addr learned my addr 172.21.15.12:0/2820493723 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:25.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb7529a640 1 --2- 172.21.15.12:0/2820493723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d2930 0x5632468e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb74a99640 1 --2- 172.21.15.12:0/2820493723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d79f0 0x5632468e1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb75a9b640 1 -- 172.21.15.12:0/2820493723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d2930 msgr2=0x5632468e13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:25.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/2820493723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d2930 0x5632468e13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb75a9b640 1 -- 172.21.15.12:0/2820493723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d79f0 msgr2=0x5632468e1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:25.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/2820493723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d79f0 0x5632468e1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb75a9b640 1 -- 172.21.15.12:0/2820493723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56324668e490 con 0x5632468d8310 2023-11-28T21:35:25.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/2820493723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x56324689b230 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb68051340 tx=0x7fdb6805b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9b5edd32772ea9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:25.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.384+0000 7fdb537fe640 1 -- 172.21.15.12:0/2820493723 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68054e40 con 0x5632468d8310 2023-11-28T21:35:25.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb537fe640 1 -- 172.21.15.12:0/2820493723 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb68058f90 con 0x5632468d8310 2023-11-28T21:35:25.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb537fe640 1 -- 172.21.15.12:0/2820493723 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6805d140 con 0x5632468d8310 2023-11-28T21:35:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 -- 172.21.15.12:0/2820493723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 msgr2=0x56324689b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 --2- 172.21.15.12:0/2820493723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x56324689b230 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdb68051340 tx=0x7fdb6805b040 comp rx=0 tx=0).stop 2023-11-28T21:35:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 -- 172.21.15.12:0/2820493723 shutdown_connections 2023-11-28T21:35:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 --2- 172.21.15.12:0/2820493723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x56324689b230 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 --2- 172.21.15.12:0/2820493723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d79f0 0x5632468e1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 --2- 172.21.15.12:0/2820493723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d2930 0x5632468e13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 -- 172.21.15.12:0/2820493723 >> 172.21.15.12:0/2820493723 conn(0x5632468d37a0 msgr2=0x5632468e0730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 -- 172.21.15.12:0/2820493723 shutdown_connections 2023-11-28T21:35:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 -- 172.21.15.12:0/2820493723 wait complete. 2023-11-28T21:35:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 -- start start 2023-11-28T21:35:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 Processor -- start 2023-11-28T21:35:25.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d2930 0x5632468d7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d79f0 0x563246898e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x563246899370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632466a0d60 con 0x5632468d8310 2023-11-28T21:35:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632466e4fd0 con 0x5632468d79f0 2023-11-28T21:35:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632466e6650 con 0x5632468d2930 2023-11-28T21:35:25.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 10 client.0 _ll_register_callbacks cb 0x563246862bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T21:35:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb75a9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x563246899370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb75a9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x563246899370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39944/0 (socket says 172.21.15.12:39944) 2023-11-28T21:35:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb75a9b640 1 -- 172.21.15.12:0/1681161586 learned_addr learned my addr 172.21.15.12:0/1681161586 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7529a640 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d2930 0x5632468d7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb74a99640 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d79f0 0x563246898e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb75a9b640 1 -- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d2930 msgr2=0x5632468d7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d2930 0x5632468d7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb75a9b640 1 -- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d79f0 msgr2=0x563246898e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d79f0 0x563246898e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb75a9b640 1 -- 172.21.15.12:0/1681161586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7014d370 con 0x5632468d8310 2023-11-28T21:35:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb75a9b640 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x563246899370 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdb6801a000 tx=0x7fdb68082100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddd9c2588f6f546 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:25.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2023-11-28T21:35:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb4bfff640 1 -- 172.21.15.12:0/1681161586 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68054e40 con 0x5632468d8310 2023-11-28T21:35:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb4bfff640 1 -- 172.21.15.12:0/1681161586 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdb6805d140 con 0x5632468d8310 2023-11-28T21:35:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.388+0000 7fdb7a866000 1 -- 172.21.15.12:0/1681161586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632466e6110 con 0x5632468d8310 2023-11-28T21:35:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb4bfff640 1 -- 172.21.15.12:0/1681161586 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb68051550 con 0x5632468d8310 2023-11-28T21:35:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 10 client.4890 fetch_fsmap learned FSMap version 13 2023-11-28T21:35:25.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 -- 172.21.15.12:0/1681161586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x5632463dfcc0 con 0x5632468d8310 2023-11-28T21:35:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb4bfff640 1 -- 172.21.15.12:0/1681161586 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb68062070 con 0x5632468d8310 2023-11-28T21:35:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb4bfff640 1 -- 172.21.15.12:0/1681161586 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 13) v1 ==== 4476+0+0 (secure 0 0 0) 0x7fdb68019020 con 0x5632468d8310 2023-11-28T21:35:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 10 client.4890 fetch_fsmap finished waiting for FSMap version 13 2023-11-28T21:35:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 10 client.4890 resolve_mds: resolved name 'f' to daemon mds.f 2023-11-28T21:35:25.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 20 client.4890 populate_metadata read hostname 'smithi012' 2023-11-28T21:35:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x5632469398a0 0x563246939c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 4 client.4890 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-28T21:35:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 -- 172.21.15.12:0/1681161586 --> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5632467159a0 con 0x5632469398a0 2023-11-28T21:35:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7529a640 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x5632469398a0 0x563246939c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7529a640 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x5632469398a0 0x563246939c80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=636b998b4cae3167 server_cookie=3f0c2cde747b97c9 in_seq=0 out_seq=0 2023-11-28T21:35:25.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb4bfff640 10 client.4890 ms_handle_connect on v2:172.21.15.144:6832/3053317078 2023-11-28T21:35:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb4bfff640 1 -- 172.21.15.12:0/1681161586 <== mds.0 v2:172.21.15.144:6832/3053317078 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5632467159a0 con 0x5632469398a0 2023-11-28T21:35:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb4bfff640 10 client.4890 handle_command_reply: tid=0 2023-11-28T21:35:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 client.4890 shutdown 2023-11-28T21:35:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 -- 172.21.15.12:0/1681161586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 msgr2=0x563246899370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x563246899370 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdb6801a000 tx=0x7fdb68082100 comp rx=0 tx=0).stop 2023-11-28T21:35:25.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 -- 172.21.15.12:0/1681161586 shutdown_connections 2023-11-28T21:35:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:6832/3053317078,v1:172.21.15.144:6833/3053317078] conn(0x5632469398a0 0x563246939c80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632468d8310 0x563246899370 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632468d79f0 0x563246898e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb4bfff640 0 client.4890 ms_handle_reset on v2:172.21.15.144:6832/3053317078 2023-11-28T21:35:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 --2- 172.21.15.12:0/1681161586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5632468d2930 0x5632468d7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 -- 172.21.15.12:0/1681161586 >> 172.21.15.12:0/1681161586 conn(0x5632468d37a0 msgr2=0x5632468e0730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:25.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 -- 172.21.15.12:0/1681161586 shutdown_connections 2023-11-28T21:35:25.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 1 -- 172.21.15.12:0/1681161586 wait complete. 2023-11-28T21:35:25.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb7a866000 20 client.4890 trim_cache size 0 max 16384 2023-11-28T21:35:25.481 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:35:25.481 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "2975d7d0-8668-45d5-addd-c7b2ef978be7", 2023-11-28T21:35:25.481 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2023-11-28T21:35:25.481 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 4261, 2023-11-28T21:35:25.481 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2023-11-28T21:35:25.481 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2023-11-28T21:35:25.482 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2023-11-28T21:35:25.482 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 46.069366500999998, 2023-11-28T21:35:25.482 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 13, 2023-11-28T21:35:25.482 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 24, 2023-11-28T21:35:25.482 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 24, 2023-11-28T21:35:25.483 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 53.839718318999999 2023-11-28T21:35:25.483 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:35:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 -- 172.21.15.12:0/3946770995 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb54066470 msgr2=0x7fdb54068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb54066470 0x7fdb54068930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0af000 tx=0x7fdb6c0a7000 comp rx=0 tx=0).stop 2023-11-28T21:35:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 -- 172.21.15.12:0/3946770995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 msgr2=0x7fdb7015f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:25.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 0x7fdb7015f650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800c7a0 tx=0x7fdb6800cc60 comp rx=0 tx=0).stop 2023-11-28T21:35:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 -- 172.21.15.12:0/3946770995 shutdown_connections 2023-11-28T21:35:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb54066470 0x7fdb54068930 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb70150c20 0x7fdb7015f650 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb7006e300 0x7fdb70140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 --2- 172.21.15.12:0/3946770995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb7006cf40 0x7fdb7013fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 -- 172.21.15.12:0/3946770995 >> 172.21.15.12:0/3946770995 conn(0x7fdb70066360 msgr2=0x7fdb70071630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 -- 172.21.15.12:0/3946770995 shutdown_connections 2023-11-28T21:35:25.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:23.392+0000 7fdb774fe640 1 -- 172.21.15.12:0/3946770995 wait complete. 2023-11-28T21:35:25.488 INFO:teuthology.run_tasks:Running task kclient... 2023-11-28T21:35:25.506 INFO:tasks.kclient:Mounting kernel clients... 2023-11-28T21:35:25.507 INFO:tasks.kclient:config is {'client.0': {}} 2023-11-28T21:35:25.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:25.507 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-11-28T21:35:25.517 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-28T21:35:25.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:25.519 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T21:35:25.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:25.805 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2023-11-28T21:35:25.949 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-11-28T21:35:25.949 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T21:35:25.949 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T21:35:25.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T21:35:26.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.092+0000 7f0e1c39d640 1 Processor -- start 2023-11-28T21:35:26.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.092+0000 7f0e1c39d640 1 -- start start 2023-11-28T21:35:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.092+0000 7f0e1c39d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0e1406e300 0x7f0e14156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.092+0000 7f0e1c39d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1406cf40 0x7f0e14158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.092+0000 7f0e1c39d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e14067410 con 0x7f0e14150c20 2023-11-28T21:35:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.092+0000 7f0e1c39d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1406e090 con 0x7f0e1406e300 2023-11-28T21:35:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.092+0000 7f0e1c39d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14157f30 con 0x7f0e1406cf40 2023-11-28T21:35:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19938640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0e1406e300 0x7f0e14156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:26.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19938640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0e1406e300 0x7f0e14156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41550/0 (socket says 172.21.15.12:41550) 2023-11-28T21:35:26.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19938640 1 -- 172.21.15.12:0/493047302 learned_addr learned my addr 172.21.15.12:0/493047302 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:26.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e1a93a640 1 --2- 172.21.15.12:0/493047302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1406cf40 0x7f0e14158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:26.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19938640 1 -- 172.21.15.12:0/493047302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1406cf40 msgr2=0x7f0e14158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:26.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19938640 1 --2- 172.21.15.12:0/493047302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1406cf40 0x7f0e14158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:26.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19938640 1 -- 172.21.15.12:0/493047302 >> v1:172.21.15.144:6789/0 conn(0x7f0e14150c20 legacy=0x7f0e140c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:26.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19938640 1 -- 172.21.15.12:0/493047302 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14151000 con 0x7f0e1406e300 2023-11-28T21:35:26.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19938640 1 --2- 172.21.15.12:0/493047302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 0x7f0e14156910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009870 tx=0x7f0e0401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab4d4ab8119fb97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19137640 1 -- 172.21.15.12:0/493047302 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040ab650 con 0x7f0e1406e300 2023-11-28T21:35:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19137640 1 -- 172.21.15.12:0/493047302 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e040ab7b0 con 0x7f0e1406e300 2023-11-28T21:35:26.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.096+0000 7f0e19137640 1 -- 172.21.15.12:0/493047302 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040abb40 con 0x7f0e1406e300 2023-11-28T21:35:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 -- 172.21.15.12:0/493047302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 msgr2=0x7f0e14156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 --2- 172.21.15.12:0/493047302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 0x7f0e14156910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009870 tx=0x7f0e0401c540 comp rx=0 tx=0).stop 2023-11-28T21:35:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 -- 172.21.15.12:0/493047302 shutdown_connections 2023-11-28T21:35:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 --2- 172.21.15.12:0/493047302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1406cf40 0x7f0e14158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 --2- 172.21.15.12:0/493047302 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 0x7f0e14156910 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 -- 172.21.15.12:0/493047302 >> 172.21.15.12:0/493047302 conn(0x7f0e14066360 msgr2=0x7f0e14066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 -- 172.21.15.12:0/493047302 shutdown_connections 2023-11-28T21:35:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 -- 172.21.15.12:0/493047302 wait complete. 2023-11-28T21:35:26.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 Processor -- start 2023-11-28T21:35:26.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 -- start start 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1406cf40 0x7f0e14160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 0x7f0e1415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e14150c20 0x7f0e1415b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e141f1ac0 con 0x7f0e1406e300 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e141f0bc0 con 0x7f0e14150c20 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1c39d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14150240 con 0x7f0e1406cf40 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e19938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 0x7f0e1415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e19938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 0x7f0e1415af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41554/0 (socket says 172.21.15.12:41554) 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e19938640 1 -- 172.21.15.12:0/1470283173 learned_addr learned my addr 172.21.15.12:0/1470283173 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1a139640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1406cf40 0x7f0e14160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:26.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e1a93a640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e14150c20 0x7f0e1415b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e19938640 1 -- 172.21.15.12:0/1470283173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1406cf40 msgr2=0x7f0e14160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e19938640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1406cf40 0x7f0e14160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e19938640 1 -- 172.21.15.12:0/1470283173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e14150c20 msgr2=0x7f0e1415b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e19938640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e14150c20 0x7f0e1415b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e19938640 1 -- 172.21.15.12:0/1470283173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e040094d0 con 0x7f0e1406e300 2023-11-28T21:35:26.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.100+0000 7f0e19938640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 0x7f0e1415af70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0e040ad040 tx=0x7f0e0401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e0affd640 1 -- 172.21.15.12:0/1470283173 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040b4800 con 0x7f0e1406e300 2023-11-28T21:35:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e0affd640 1 -- 172.21.15.12:0/1470283173 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e040b4980 con 0x7f0e1406e300 2023-11-28T21:35:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e0affd640 1 -- 172.21.15.12:0/1470283173 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040b4d30 con 0x7f0e1406e300 2023-11-28T21:35:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e1c39d640 1 -- 172.21.15.12:0/1470283173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e141f0e40 con 0x7f0e1406e300 2023-11-28T21:35:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e1c39d640 1 -- 172.21.15.12:0/1470283173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e141f1400 con 0x7f0e1406e300 2023-11-28T21:35:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e1c39d640 1 -- 172.21.15.12:0/1470283173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e14074f60 con 0x7f0e1406e300 2023-11-28T21:35:26.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e0affd640 1 -- 172.21.15.12:0/1470283173 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0e040be3f0 con 0x7f0e1406e300 2023-11-28T21:35:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e0affd640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0df8066470 0x7f0df8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e0affd640 1 -- 172.21.15.12:0/1470283173 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0e0411ed40 con 0x7f0e1406e300 2023-11-28T21:35:26.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e1a139640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0df8066470 0x7f0df8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:26.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.104+0000 7f0e1a139640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0df8066470 0x7f0df8068930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0e10000f70 tx=0x7f0e10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:26.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.108+0000 7f0e0affd640 1 -- 172.21.15.12:0/1470283173 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0e040ebea0 con 0x7f0e1406e300 2023-11-28T21:35:26.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 -- 172.21.15.12:0/1470283173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0e140cad00 con 0x7f0e1406e300 2023-11-28T21:35:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e0affd640 1 -- 172.21.15.12:0/1470283173 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0e040c4020 con 0x7f0e1406e300 2023-11-28T21:35:26.302 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T21:35:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 -- 172.21.15.12:0/1470283173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0df8066470 msgr2=0x7f0df8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0df8066470 0x7f0df8068930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0e10000f70 tx=0x7f0e10009040 comp rx=0 tx=0).stop 2023-11-28T21:35:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 -- 172.21.15.12:0/1470283173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 msgr2=0x7f0e1415af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 0x7f0e1415af70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0e040ad040 tx=0x7f0e0401ad80 comp rx=0 tx=0).stop 2023-11-28T21:35:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 -- 172.21.15.12:0/1470283173 shutdown_connections 2023-11-28T21:35:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0df8066470 0x7f0df8068930 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e14150c20 0x7f0e1415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e1406e300 0x7f0e1415af70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:26.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 --2- 172.21.15.12:0/1470283173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1406cf40 0x7f0e14160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 -- 172.21.15.12:0/1470283173 >> 172.21.15.12:0/1470283173 conn(0x7f0e14066360 msgr2=0x7f0e1406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 -- 172.21.15.12:0/1470283173 shutdown_connections 2023-11-28T21:35:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:26.300+0000 7f0e1c39d640 1 -- 172.21.15.12:0/1470283173 wait complete. 2023-11-28T21:35:26.323 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T21:35:26.323 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T21:35:26.323 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2023-11-28T21:35:26.323 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T21:35:26.323 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T21:35:26.323 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T21:35:26.323 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T21:35:26.323 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T21:35:26.323 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T21:35:26.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:26.324 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2023-11-28T21:35:26.333 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T21:35:26.333 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T21:35:26.333 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T21:35:26.333 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-11-28T21:35:26.333 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2023-11-28T21:35:26.333 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-11-28T21:35:26.334 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T21:35:26.334 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2023-11-28T21:35:26.334 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2023-11-28T21:35:26.334 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T21:35:26.334 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2023-11-28T21:35:26.334 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2023-11-28T21:35:26.334 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T21:35:26.334 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T21:35:26.334 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2023-11-28T21:35:26.334 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-28T21:35:26.335 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42349sec preferred_lft 42349sec 2023-11-28T21:35:26.335 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link 2023-11-28T21:35:26.335 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-11-28T21:35:26.335 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-28T21:35:26.335 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T21:35:26.335 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2023-11-28T21:35:26.335 INFO:teuthology.orchestra.run.smithi012.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-28T21:35:26.335 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 02:42:f3:cf:11:01 brd ff:ff:ff:ff:ff:ff 2023-11-28T21:35:26.335 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-28T21:35:26.335 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2023-11-28T21:35:26.336 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T21:35:26.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:26.337 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-11-28T21:35:26.337 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-11-28T21:35:26.337 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2023-11-28T21:35:26.337 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2023-11-28T21:35:26.337 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2023-11-28T21:35:26.337 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T21:35:26.337 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-11-28T21:35:26.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:26.452 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T21:35:26.472 INFO:teuthology.orchestra.run.smithi012.stdout:1 2023-11-28T21:35:26.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:26.474 DEBUG:teuthology.orchestra.run.smithi012:> route 2023-11-28T21:35:26.531 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2023-11-28T21:35:26.531 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T21:35:26.532 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-28T21:35:26.532 INFO:teuthology.orchestra.run.smithi012.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-28T21:35:26.532 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-28T21:35:26.532 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T21:35:26.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:26.532 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-11-28T21:35:26.532 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-11-28T21:35:26.532 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T21:35:26.532 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T21:35:26.533 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T21:35:26.533 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-11-28T21:35:26.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:26.631 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-11-28T21:35:26.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:26.641 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2023-11-28T21:35:26.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:26.690 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-11-28T21:35:26.691 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-11-28T21:35:26.691 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T21:35:26.691 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T21:35:26.691 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-11-28T21:35:26.774 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T21:35:26.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:26.774 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-11-28T21:35:26.774 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-11-28T21:35:26.774 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T21:35:26.774 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T21:35:26.774 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T21:35:26.774 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T21:35:26.775 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T21:35:26.775 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-11-28T21:35:26.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:26.920 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec bash -c ' 2023-11-28T21:35:26.920 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2023-11-28T21:35:26.920 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2023-11-28T21:35:26.920 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T21:35:26.920 DEBUG:teuthology.orchestra.run.smithi012:> ') 2023-11-28T21:35:26.961 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T21:35:26.962 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T21:35:27.018 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T21:35:27.018 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T21:35:27.018 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T21:35:27.069 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-11-28T21:35:27.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:27.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T21:35:27.164 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T21:35:27.165 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T21:35:27.165 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-11-28T21:35:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T21:35:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T21:35:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-11-28T21:35:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-11-28T21:35:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T21:35:27.287 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-11-28T21:35:27.288 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-11-28T21:35:27.288 INFO:teuthology.orchestra.run.smithi012.stdout:invalid new device string format 2023-11-28T21:35:27.288 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-11-28T21:35:27.288 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-28T21:35:27.288 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:6789,172.21.15.144:6789,172.21.15.144:6790" 2023-11-28T21:35:27.288 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.12:6789,172.21.15.144:6789,172.21.15.144:6790:/ 2023-11-28T21:35:27.288 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=2975d7d0-8668-45d5-addd-c7b2ef978be7" will pass to kernel 2023-11-28T21:35:27.292 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T21:35:27.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:27.292 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T21:35:27.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:27.314 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T21:35:27.369 INFO:teuthology.orchestra.run.smithi012.stdout:client4697 2023-11-28T21:35:27.369 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:35:27.369 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T21:35:27.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:27.422 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T21:35:27.476 INFO:teuthology.orchestra.run.smithi012.stdout:client4697 2023-11-28T21:35:27.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:27.477 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-28T21:35:27.528 INFO:teuthology.orchestra.run.smithi012.stdout:client4697 2023-11-28T21:35:27.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:35:27.528 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2975d7d0-8668-45d5-addd-c7b2ef978be7.client4697/status') 2023-11-28T21:35:27.590 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.4697 (3)172.21.15.12:0/1853951501 2023-11-28T21:35:27.590 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2023-11-28T21:35:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2023-11-28T21:35:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2023-11-28T21:35:27.591 INFO:teuthology.orchestra.run.smithi012.stderr:70 bytes copied, 5.8952e-05 s, 1.2 MB/s 2023-11-28T21:35:27.593 INFO:teuthology.run_tasks:Running task mds_thrash... 2023-11-28T21:35:27.618 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f92338e3640 1 Processor -- start 2023-11-28T21:35:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f92338e3640 1 -- start start 2023-11-28T21:35:27.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f92338e3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f922c158420 0x7f922c06d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:27.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f92338e3640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f922c0c38f0 0x7f922c0c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:27.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f92338e3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c0c3460 con 0x7f922c15e010 2023-11-28T21:35:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f92338e3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c158800 con 0x7f922c158420 2023-11-28T21:35:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f92338e3640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c162610 con 0x7f922c0c38f0 2023-11-28T21:35:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f923167f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f922c158420 0x7f922c06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f923167f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f922c158420 0x7f922c06d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41562/0 (socket says 172.21.15.12:41562) 2023-11-28T21:35:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f9231e80640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f922c15e010 0x7f922c15e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45172/0 (socket says 172.21.15.12:45172) 2023-11-28T21:35:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f923167f640 1 -- 172.21.15.12:0/3550227266 learned_addr learned my addr 172.21.15.12:0/3550227266 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f9230e7e640 1 --2- 172.21.15.12:0/3550227266 >> v2:172.21.15.144:3300/0 conn(0x7f922c0c38f0 0x7f922c0c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f923167f640 1 -- 172.21.15.12:0/3550227266 >> v2:172.21.15.144:3300/0 conn(0x7f922c0c38f0 msgr2=0x7f922c0c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f923167f640 1 --2- 172.21.15.12:0/3550227266 >> v2:172.21.15.144:3300/0 conn(0x7f922c0c38f0 0x7f922c0c3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f923167f640 1 -- 172.21.15.12:0/3550227266 >> v1:172.21.15.12:6789/0 conn(0x7f922c15e010 legacy=0x7f922c15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f9223fff640 1 -- 172.21.15.12:0/3550227266 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118375599 0 0) 0x7f922c0c3460 con 0x7f922c15e010 2023-11-28T21:35:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.772+0000 7f923167f640 1 -- 172.21.15.12:0/3550227266 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c0c2b70 con 0x7f922c158420 2023-11-28T21:35:27.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f923167f640 1 --2- 172.21.15.12:0/3550227266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c158420 0x7f922c06d950 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f922c06b930 tx=0x7f922801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9cd3c439b335c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f9223fff640 1 -- 172.21.15.12:0/3550227266 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92280ab600 con 0x7f922c158420 2023-11-28T21:35:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f9223fff640 1 -- 172.21.15.12:0/3550227266 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92280a7050 con 0x7f922c158420 2023-11-28T21:35:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f9223fff640 1 -- 172.21.15.12:0/3550227266 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92280ab980 con 0x7f922c158420 2023-11-28T21:35:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f92338e3640 1 -- 172.21.15.12:0/3550227266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c158420 msgr2=0x7f922c06d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f92338e3640 1 --2- 172.21.15.12:0/3550227266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c158420 0x7f922c06d950 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f922c06b930 tx=0x7f922801c500 comp rx=0 tx=0).stop 2023-11-28T21:35:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f92338e3640 1 -- 172.21.15.12:0/3550227266 shutdown_connections 2023-11-28T21:35:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f92338e3640 1 --2- 172.21.15.12:0/3550227266 >> v2:172.21.15.144:3300/0 conn(0x7f922c0c38f0 0x7f922c0c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f92338e3640 1 --2- 172.21.15.12:0/3550227266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c158420 0x7f922c06d950 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f92338e3640 1 -- 172.21.15.12:0/3550227266 >> 172.21.15.12:0/3550227266 conn(0x7f922c066360 msgr2=0x7f922c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f92338e3640 1 -- 172.21.15.12:0/3550227266 shutdown_connections 2023-11-28T21:35:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.776+0000 7f92338e3640 1 -- 172.21.15.12:0/3550227266 wait complete. 2023-11-28T21:35:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f92338e3640 1 Processor -- start 2023-11-28T21:35:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f92338e3640 1 -- start start 2023-11-28T21:35:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f92338e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c0c38f0 0x7f922c153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f92338e3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f922c158420 0x7f922c153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f92338e3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f922c15e010 0x7f922c159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f92338e3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c159000 con 0x7f922c0c38f0 2023-11-28T21:35:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f92338e3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c14fb10 con 0x7f922c15e010 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f92338e3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c150b10 con 0x7f922c158420 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f923167f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c0c38f0 0x7f922c153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f923167f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c0c38f0 0x7f922c153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41570/0 (socket says 172.21.15.12:41570) 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f923167f640 1 -- 172.21.15.12:0/2366176464 learned_addr learned my addr 172.21.15.12:0/2366176464 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f9230e7e640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f922c158420 0x7f922c153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f9231e80640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f922c15e010 0x7f922c159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f923167f640 1 -- 172.21.15.12:0/2366176464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f922c158420 msgr2=0x7f922c153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f923167f640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f922c158420 0x7f922c153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f923167f640 1 -- 172.21.15.12:0/2366176464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f922c15e010 msgr2=0x7f922c159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f923167f640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f922c15e010 0x7f922c159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:27.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f923167f640 1 -- 172.21.15.12:0/2366176464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280094d0 con 0x7f922c0c38f0 2023-11-28T21:35:27.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f923167f640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c0c38f0 0x7f922c153fe0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f92280ae040 tx=0x7f92280192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:27.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f9221ffb640 1 -- 172.21.15.12:0/2366176464 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92280b4440 con 0x7f922c0c38f0 2023-11-28T21:35:27.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f9221ffb640 1 -- 172.21.15.12:0/2366176464 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92280b45a0 con 0x7f922c0c38f0 2023-11-28T21:35:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f92338e3640 1 -- 172.21.15.12:0/2366176464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c1546b0 con 0x7f922c0c38f0 2023-11-28T21:35:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f9221ffb640 1 -- 172.21.15.12:0/2366176464 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92280b4930 con 0x7f922c0c38f0 2023-11-28T21:35:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.780+0000 7f92338e3640 1 -- 172.21.15.12:0/2366176464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c150030 con 0x7f922c0c38f0 2023-11-28T21:35:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.784+0000 7f92338e3640 1 -- 172.21.15.12:0/2366176464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c074f60 con 0x7f922c0c38f0 2023-11-28T21:35:27.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.784+0000 7f9221ffb640 1 -- 172.21.15.12:0/2366176464 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f92280b2070 con 0x7f922c0c38f0 2023-11-28T21:35:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.784+0000 7f9221ffb640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f92100664a0 0x7f9210068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.784+0000 7f9221ffb640 1 -- 172.21.15.12:0/2366176464 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f922811e120 con 0x7f922c0c38f0 2023-11-28T21:35:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.784+0000 7f9230e7e640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f92100664a0 0x7f9210068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.788+0000 7f9230e7e640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f92100664a0 0x7f9210068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f922c157d00 tx=0x7f921c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.788+0000 7f9221ffb640 1 -- 172.21.15.12:0/2366176464 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f92280eb1c0 con 0x7f922c0c38f0 2023-11-28T21:35:27.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.948+0000 7f92338e3640 1 -- 172.21.15.12:0/2366176464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f922c154a60 con 0x7f922c0c38f0 2023-11-28T21:35:27.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.948+0000 7f9221ffb640 1 -- 172.21.15.12:0/2366176464 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18746 (secure 0 0 0) 0x7f92280ee7e0 con 0x7f922c0c38f0 2023-11-28T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:27.952 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:35:25.566387+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 -- 172.21.15.12:0/2366176464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f92100664a0 msgr2=0x7f9210068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f92100664a0 0x7f9210068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f922c157d00 tx=0x7f921c008040 comp rx=0 tx=0).stop 2023-11-28T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 -- 172.21.15.12:0/2366176464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c0c38f0 msgr2=0x7f922c153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c0c38f0 0x7f922c153fe0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f92280ae040 tx=0x7f92280192a0 comp rx=0 tx=0).stop 2023-11-28T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 -- 172.21.15.12:0/2366176464 shutdown_connections 2023-11-28T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f92100664a0 0x7f9210068960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f922c15e010 0x7f922c159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f922c158420 0x7f922c153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 --2- 172.21.15.12:0/2366176464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f922c0c38f0 0x7f922c153fe0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 -- 172.21.15.12:0/2366176464 >> 172.21.15.12:0/2366176464 conn(0x7f922c066360 msgr2=0x7f922c15e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 -- 172.21.15.12:0/2366176464 shutdown_connections 2023-11-28T21:35:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:27.952+0000 7f92338e3640 1 -- 172.21.15.12:0/2366176464 wait complete. 2023-11-28T21:35:27.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T21:34:29.878190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T21:34:34.039032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T21:34:35.369379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T21:34:39.373079+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T21:35:27.974 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T21:35:28.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.116+0000 7fa7e40c9640 1 Processor -- start 2023-11-28T21:35:28.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.116+0000 7fa7e40c9640 1 -- start start 2023-11-28T21:35:28.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e40c9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa7dc06d950 0x7fa7dc161d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e40c9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0c3460 con 0x7fa7dc158420 2023-11-28T21:35:28.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e40c9640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0c2b70 con 0x7fa7dc0c4000 2023-11-28T21:35:28.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e40c9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7dc158800 con 0x7fa7dc06d950 2023-11-28T21:35:28.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e1e65640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa7dc158420 0x7fa7dc0c38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45184/0 (socket says 172.21.15.12:45184) 2023-11-28T21:35:28.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e1e65640 1 -- 172.21.15.12:0/1086064352 learned_addr learned my addr 172.21.15.12:0/1086064352 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e2666640 1 --2- 172.21.15.12:0/1086064352 >> v2:172.21.15.144:3300/0 conn(0x7fa7dc06d950 0x7fa7dc161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469372003 0 0) 0x7fa7dc0c3460 con 0x7fa7dc158420 2023-11-28T21:35:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c8003530 con 0x7fa7dc158420 2023-11-28T21:35:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434870982 0 0) 0x7fa7c8003530 con 0x7fa7dc158420 2023-11-28T21:35:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 >> v2:172.21.15.144:3300/0 conn(0x7fa7dc06d950 msgr2=0x7fa7dc161d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 --2- 172.21.15.12:0/1086064352 >> v2:172.21.15.144:3300/0 conn(0x7fa7dc06d950 0x7fa7dc161d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 >> v1:172.21.15.144:6789/0 conn(0x7fa7dc0c4000 legacy=0x7fa7dc161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc06e090 con 0x7fa7dc158420 2023-11-28T21:35:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fa7d8001680 con 0x7fa7dc158420 2023-11-28T21:35:28.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 >> v1:172.21.15.12:6789/0 conn(0x7fa7dc158420 legacy=0x7fa7dc0c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c80039f0 0x7fa7c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80047d0 0x7fa7c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c8005190 0x7fa7c800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc158800 con 0x7fa7c80039f0 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c800c010 con 0x7fa7c8005190 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c800c1d0 con 0x7fa7c80047d0 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e1e65640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c80039f0 0x7fa7c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e1664640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80047d0 0x7fa7c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e2666640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c8005190 0x7fa7c800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e1e65640 1 -- 172.21.15.12:0/1086064352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80047d0 msgr2=0x7fa7c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e1e65640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80047d0 0x7fa7c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e1e65640 1 -- 172.21.15.12:0/1086064352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c8005190 msgr2=0x7fa7c800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:28.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e1e65640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c8005190 0x7fa7c800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e1e65640 1 -- 172.21.15.12:0/1086064352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c8003530 con 0x7fa7c80039f0 2023-11-28T21:35:28.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.120+0000 7fa7e2666640 1 -- 172.21.15.12:0/1086064352 reap_dead start 2023-11-28T21:35:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e1e65640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c80039f0 0x7fa7c8003dd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c190 tx=0x7fa7d801cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57f451f772cd726 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80058e0 con 0x7fa7c80039f0 2023-11-28T21:35:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8005a60 con 0x7fa7c80039f0 2023-11-28T21:35:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e0e63640 1 -- 172.21.15.12:0/1086064352 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8005e50 con 0x7fa7c80039f0 2023-11-28T21:35:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e40c9640 1 -- 172.21.15.12:0/1086064352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c80039f0 msgr2=0x7fa7c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e40c9640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c80039f0 0x7fa7c8003dd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c190 tx=0x7fa7d801cba0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e40c9640 1 -- 172.21.15.12:0/1086064352 shutdown_connections 2023-11-28T21:35:28.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e40c9640 1 --2- 172.21.15.12:0/1086064352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7c80039f0 0x7fa7c8003dd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e40c9640 1 -- 172.21.15.12:0/1086064352 >> 172.21.15.12:0/1086064352 conn(0x7fa7dc066360 msgr2=0x7fa7dc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:28.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e40c9640 1 -- 172.21.15.12:0/1086064352 shutdown_connections 2023-11-28T21:35:28.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.124+0000 7fa7e40c9640 1 -- 172.21.15.12:0/1086064352 wait complete. 2023-11-28T21:35:28.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e40c9640 1 Processor -- start 2023-11-28T21:35:28.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e40c9640 1 -- start start 2023-11-28T21:35:28.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e40c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80039f0 0x7fa7dc070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e40c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc0c21f0 0x7fa7dc15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e40c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7dc154760 0x7fa7dc154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e40c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc155120 con 0x7fa7dc154760 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e40c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0b9d90 con 0x7fa7dc0c21f0 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e40c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc15fe70 con 0x7fa7c80039f0 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e2666640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7dc154760 0x7fa7dc154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e1664640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc0c21f0 0x7fa7dc15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e2666640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7dc154760 0x7fa7dc154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41586/0 (socket says 172.21.15.12:41586) 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e2666640 1 -- 172.21.15.12:0/298941217 learned_addr learned my addr 172.21.15.12:0/298941217 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e1e65640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80039f0 0x7fa7dc070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e2666640 1 -- 172.21.15.12:0/298941217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80039f0 msgr2=0x7fa7dc070a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e2666640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80039f0 0x7fa7dc070a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e2666640 1 -- 172.21.15.12:0/298941217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc0c21f0 msgr2=0x7fa7dc15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e2666640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc0c21f0 0x7fa7dc15b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e2666640 1 -- 172.21.15.12:0/298941217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc159ac0 con 0x7fa7dc154760 2023-11-28T21:35:28.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e2666640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7dc154760 0x7fa7dc154be0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0c5980 tx=0x7fa7d40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:28.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7d27fc640 1 -- 172.21.15.12:0/298941217 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400e070 con 0x7fa7dc154760 2023-11-28T21:35:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7d27fc640 1 -- 172.21.15.12:0/298941217 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d4011e10 con 0x7fa7dc154760 2023-11-28T21:35:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7d27fc640 1 -- 172.21.15.12:0/298941217 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40039a0 con 0x7fa7dc154760 2023-11-28T21:35:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e40c9640 1 -- 172.21.15.12:0/298941217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc159da0 con 0x7fa7dc154760 2023-11-28T21:35:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.128+0000 7fa7e40c9640 1 -- 172.21.15.12:0/298941217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc15a360 con 0x7fa7dc154760 2023-11-28T21:35:28.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.132+0000 7fa7e40c9640 1 -- 172.21.15.12:0/298941217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7dc074f60 con 0x7fa7dc154760 2023-11-28T21:35:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.132+0000 7fa7d27fc640 1 -- 172.21.15.12:0/298941217 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa7d4003b00 con 0x7fa7dc154760 2023-11-28T21:35:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.132+0000 7fa7d27fc640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa7c0066470 0x7fa7c0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.132+0000 7fa7d27fc640 1 -- 172.21.15.12:0/298941217 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa7d40847e0 con 0x7fa7dc154760 2023-11-28T21:35:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.132+0000 7fa7e1e65640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa7c0066470 0x7fa7c0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.132+0000 7fa7e1e65640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa7c0066470 0x7fa7c0068930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80aa000 tx=0x7fa7d8001990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:28.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.136+0000 7fa7d27fc640 1 -- 172.21.15.12:0/298941217 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7d4051890 con 0x7fa7dc154760 2023-11-28T21:35:28.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 -- 172.21.15.12:0/298941217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7dc15d9e0 con 0x7fa7dc154760 2023-11-28T21:35:28.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7d27fc640 1 -- 172.21.15.12:0/298941217 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7d4054eb0 con 0x7fa7dc154760 2023-11-28T21:35:28.298 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-11-28T21:35:28.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 -- 172.21.15.12:0/298941217 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa7c0066470 msgr2=0x7fa7c0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa7c0066470 0x7fa7c0068930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80aa000 tx=0x7fa7d8001990 comp rx=0 tx=0).stop 2023-11-28T21:35:28.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 -- 172.21.15.12:0/298941217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7dc154760 msgr2=0x7fa7dc154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7dc154760 0x7fa7dc154be0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0c5980 tx=0x7fa7d40115a0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 -- 172.21.15.12:0/298941217 shutdown_connections 2023-11-28T21:35:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa7c0066470 0x7fa7c0068930 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7dc154760 0x7fa7dc154be0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc0c21f0 0x7fa7dc15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 --2- 172.21.15.12:0/298941217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80039f0 0x7fa7dc070a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 -- 172.21.15.12:0/298941217 >> 172.21.15.12:0/298941217 conn(0x7fa7dc066360 msgr2=0x7fa7dc0719b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 -- 172.21.15.12:0/298941217 shutdown_connections 2023-11-28T21:35:28.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.296+0000 7fa7e40c9640 1 -- 172.21.15.12:0/298941217 wait complete. 2023-11-28T21:35:28.320 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.460+0000 7f2303f1d640 1 Processor -- start 2023-11-28T21:35:28.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.460+0000 7f2303f1d640 1 -- start start 2023-11-28T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f2303f1d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc151af0 0x7f22fc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f2303f1d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f22fc151ed0 0x7f22fc0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f2303f1d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22fc157b70 con 0x7f22fc06e300 2023-11-28T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f2303f1d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22fc0707f0 con 0x7f22fc151ed0 2023-11-28T21:35:28.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f2303f1d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0c7b90 con 0x7f22fc151af0 2023-11-28T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f23014b8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f22fc151ed0 0x7f22fc0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f2301cb9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc151af0 0x7f22fc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f23014b8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f22fc151ed0 0x7f22fc0c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:51584/0 (socket says 172.21.15.12:51584) 2023-11-28T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f23014b8640 1 -- 172.21.15.12:0/3869580262 learned_addr learned my addr 172.21.15.12:0/3869580262 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f23024ba640 1 --1- 172.21.15.12:0/3869580262 >> v1:172.21.15.144:6789/0 conn(0x7f22fc06e300 0x7f22fc072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:46960/0 (socket says 172.21.15.12:46960) 2023-11-28T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f2301cb9640 1 -- 172.21.15.12:0/3869580262 >> v2:172.21.15.144:3300/0 conn(0x7f22fc151ed0 msgr2=0x7f22fc0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f2301cb9640 1 --2- 172.21.15.12:0/3869580262 >> v2:172.21.15.144:3300/0 conn(0x7f22fc151ed0 0x7f22fc0c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f2301cb9640 1 -- 172.21.15.12:0/3869580262 >> v1:172.21.15.144:6789/0 conn(0x7f22fc06e300 legacy=0x7f22fc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.464+0000 7f2301cb9640 1 -- 172.21.15.12:0/3869580262 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc157e50 con 0x7f22fc151af0 2023-11-28T21:35:28.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2301cb9640 1 --2- 172.21.15.12:0/3869580262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc151af0 0x7f22fc0c7650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22f80097c0 tx=0x7f22f801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9ad18b69fb24a15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2300cb7640 1 -- 172.21.15.12:0/3869580262 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f80ab600 con 0x7f22fc151af0 2023-11-28T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2300cb7640 1 -- 172.21.15.12:0/3869580262 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f80a7050 con 0x7f22fc151af0 2023-11-28T21:35:28.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2300cb7640 1 -- 172.21.15.12:0/3869580262 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f80ab980 con 0x7f22fc151af0 2023-11-28T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2303f1d640 1 -- 172.21.15.12:0/3869580262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc151af0 msgr2=0x7f22fc0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2303f1d640 1 --2- 172.21.15.12:0/3869580262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc151af0 0x7f22fc0c7650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f22f80097c0 tx=0x7f22f801c480 comp rx=0 tx=0).stop 2023-11-28T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2303f1d640 1 -- 172.21.15.12:0/3869580262 shutdown_connections 2023-11-28T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2303f1d640 1 --2- 172.21.15.12:0/3869580262 >> v2:172.21.15.144:3300/0 conn(0x7f22fc151ed0 0x7f22fc0c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2303f1d640 1 --2- 172.21.15.12:0/3869580262 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc151af0 0x7f22fc0c7650 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2303f1d640 1 -- 172.21.15.12:0/3869580262 >> 172.21.15.12:0/3869580262 conn(0x7f22fc066360 msgr2=0x7f22fc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:28.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2303f1d640 1 -- 172.21.15.12:0/3869580262 shutdown_connections 2023-11-28T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.468+0000 7f2303f1d640 1 -- 172.21.15.12:0/3869580262 wait complete. 2023-11-28T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 Processor -- start 2023-11-28T21:35:28.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 -- start start 2023-11-28T21:35:28.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc06e300 0x7f22fc142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22fc151af0 0x7f22fc14bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22fc14c2e0 0x7f22fc140d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc1f39b0 con 0x7f22fc14c2e0 2023-11-28T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc1f2e10 con 0x7f22fc151af0 2023-11-28T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc1f3080 con 0x7f22fc06e300 2023-11-28T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f23024ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22fc14c2e0 0x7f22fc140d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f23024ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22fc14c2e0 0x7f22fc140d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41588/0 (socket says 172.21.15.12:41588) 2023-11-28T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f23024ba640 1 -- 172.21.15.12:0/1204281400 learned_addr learned my addr 172.21.15.12:0/1204281400 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f23014b8640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22fc151af0 0x7f22fc14bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2301cb9640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc06e300 0x7f22fc142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f23024ba640 1 -- 172.21.15.12:0/1204281400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc06e300 msgr2=0x7f22fc142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f23024ba640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc06e300 0x7f22fc142460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f23024ba640 1 -- 172.21.15.12:0/1204281400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22fc151af0 msgr2=0x7f22fc14bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f23024ba640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22fc151af0 0x7f22fc14bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f23024ba640 1 -- 172.21.15.12:0/1204281400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8009420 con 0x7f22fc14c2e0 2023-11-28T21:35:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f23024ba640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22fc14c2e0 0x7f22fc140d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c6c0 tx=0x7f22f400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f22f27fc640 1 -- 172.21.15.12:0/1204281400 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f4004d70 con 0x7f22fc14c2e0 2023-11-28T21:35:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f22f27fc640 1 -- 172.21.15.12:0/1204281400 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f400a640 con 0x7f22fc14c2e0 2023-11-28T21:35:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f22f27fc640 1 -- 172.21.15.12:0/1204281400 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f400a9b0 con 0x7f22fc14c2e0 2023-11-28T21:35:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 -- 172.21.15.12:0/1204281400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc1f3360 con 0x7f22fc14c2e0 2023-11-28T21:35:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 -- 172.21.15.12:0/1204281400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc1429a0 con 0x7f22fc14c2e0 2023-11-28T21:35:28.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.472+0000 7f2303f1d640 1 -- 172.21.15.12:0/1204281400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc1f3b60 con 0x7f22fc14c2e0 2023-11-28T21:35:28.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.476+0000 7f22f27fc640 1 -- 172.21.15.12:0/1204281400 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f22f400ab10 con 0x7f22fc14c2e0 2023-11-28T21:35:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.476+0000 7f22f27fc640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f22e00664a0 0x7f22e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.476+0000 7f22f27fc640 1 -- 172.21.15.12:0/1204281400 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f22f4083590 con 0x7f22fc14c2e0 2023-11-28T21:35:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.476+0000 7f2301cb9640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f22e00664a0 0x7f22e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.476+0000 7f2301cb9640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f22e00664a0 0x7f22e0068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f22f80ad000 tx=0x7f22f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:28.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.480+0000 7f22f27fc640 1 -- 172.21.15.12:0/1204281400 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f22fc1f3b60 con 0x7f22fc14c2e0 2023-11-28T21:35:28.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 -- 172.21.15.12:0/1204281400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22fc1f3810 con 0x7f22fc14c2e0 2023-11-28T21:35:28.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f22f27fc640 1 -- 172.21.15.12:0/1204281400 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f22fc1f3810 con 0x7f22fc14c2e0 2023-11-28T21:35:28.642 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:35:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 -- 172.21.15.12:0/1204281400 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f22e00664a0 msgr2=0x7f22e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f22e00664a0 0x7f22e0068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f22f80ad000 tx=0x7f22f8008040 comp rx=0 tx=0).stop 2023-11-28T21:35:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 -- 172.21.15.12:0/1204281400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22fc14c2e0 msgr2=0x7f22fc140d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22fc14c2e0 0x7f22fc140d10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f22f400c6c0 tx=0x7f22f400cb80 comp rx=0 tx=0).stop 2023-11-28T21:35:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 -- 172.21.15.12:0/1204281400 shutdown_connections 2023-11-28T21:35:28.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f22e00664a0 0x7f22e0068960 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22fc14c2e0 0x7f22fc140d10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22fc151af0 0x7f22fc14bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 --2- 172.21.15.12:0/1204281400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22fc06e300 0x7f22fc142460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 -- 172.21.15.12:0/1204281400 >> 172.21.15.12:0/1204281400 conn(0x7f22fc066360 msgr2=0x7f22fc06cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 -- 172.21.15.12:0/1204281400 shutdown_connections 2023-11-28T21:35:28.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.640+0000 7f2303f1d640 1 -- 172.21.15.12:0/1204281400 wait complete. 2023-11-28T21:35:28.663 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T21:35:28.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f677e8c9640 1 Processor -- start 2023-11-28T21:35:28.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f677e8c9640 1 -- start start 2023-11-28T21:35:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f677e8c9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f677806e300 0x7f6778156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f677e8c9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6778067410 con 0x7f6778150c20 2023-11-28T21:35:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f677e8c9640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677806e090 con 0x7f677806cf40 2023-11-28T21:35:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f677e8c9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6778157f30 con 0x7f677806e300 2023-11-28T21:35:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6777fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6778150c20 0x7f67780c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45192/0 (socket says 172.21.15.12:45192) 2023-11-28T21:35:28.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6777fff640 1 -- 172.21.15.12:0/2420937894 learned_addr learned my addr 172.21.15.12:0/2420937894 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:28.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f67777fe640 1 --2- 172.21.15.12:0/2420937894 >> v2:172.21.15.144:3300/0 conn(0x7f677806e300 0x7f6778156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242494336 0 0) 0x7f6778067410 con 0x7f6778150c20 2023-11-28T21:35:28.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6760003530 con 0x7f6778150c20 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952712031 0 0) 0x7f677806e090 con 0x7f677806cf40 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6778067410 con 0x7f677806cf40 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3185365102 0 0) 0x7f6760003530 con 0x7f6778150c20 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 >> v2:172.21.15.144:3300/0 conn(0x7f677806e300 msgr2=0x7f6778156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 --2- 172.21.15.12:0/2420937894 >> v2:172.21.15.144:3300/0 conn(0x7f677806e300 0x7f6778156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 >> v1:172.21.15.144:6789/0 conn(0x7f677806cf40 legacy=0x7f6778158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778151000 con 0x7f6778150c20 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f6768001680 con 0x7f6778150c20 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 >> v1:172.21.15.12:6789/0 conn(0x7f6778150c20 legacy=0x7f67780c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6760003b80 0x7f6760003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6760004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6760005190 0x7f676000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6778157f30 con 0x7f67600047d0 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676000c090 con 0x7f6760005190 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676000c250 con 0x7f6760003b80 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f67777fe640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6760004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f6777fff640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6760003b80 0x7f6760003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f677ce66640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6760005190 0x7f676000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f67777fe640 1 -- 172.21.15.12:0/2420937894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6760003b80 msgr2=0x7f6760003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f67777fe640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6760003b80 0x7f6760003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f67777fe640 1 -- 172.21.15.12:0/2420937894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6760005190 msgr2=0x7f676000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f67777fe640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6760005190 0x7f676000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f67777fe640 1 -- 172.21.15.12:0/2420937894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6760003530 con 0x7f67600047d0 2023-11-28T21:35:28.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.808+0000 7f677ce66640 1 -- 172.21.15.12:0/2420937894 reap_dead start 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f67777fe640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6760004c50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f676c009ab0 tx=0x7f676c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f280387935fa08db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c0a5620 con 0x7f67600047d0 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c0a5780 con 0x7f67600047d0 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f6776ffd640 1 -- 172.21.15.12:0/2420937894 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c0a5b30 con 0x7f67600047d0 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f677e8c9640 1 -- 172.21.15.12:0/2420937894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 msgr2=0x7f6760004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f677e8c9640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6760004c50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f676c009ab0 tx=0x7f676c01ca30 comp rx=0 tx=0).stop 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f677e8c9640 1 -- 172.21.15.12:0/2420937894 shutdown_connections 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f677e8c9640 1 --2- 172.21.15.12:0/2420937894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6760004c50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f677e8c9640 1 -- 172.21.15.12:0/2420937894 >> 172.21.15.12:0/2420937894 conn(0x7f6778066360 msgr2=0x7f6778066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f677e8c9640 1 -- 172.21.15.12:0/2420937894 shutdown_connections 2023-11-28T21:35:28.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.812+0000 7f677e8c9640 1 -- 172.21.15.12:0/2420937894 wait complete. 2023-11-28T21:35:28.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677e8c9640 1 Processor -- start 2023-11-28T21:35:28.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677e8c9640 1 -- start start 2023-11-28T21:35:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677e8c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6778152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677e8c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6778151240 0x7f67781563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677e8c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6778153210 0x7f6778153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677e8c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6778151ad0 con 0x7f67600047d0 2023-11-28T21:35:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677e8c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67781ef720 con 0x7f6778151240 2023-11-28T21:35:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677e8c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67781529a0 con 0x7f6778153210 2023-11-28T21:35:28.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6777fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6778152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6777fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6778152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41598/0 (socket says 172.21.15.12:41598) 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6777fff640 1 -- 172.21.15.12:0/2358745910 learned_addr learned my addr 172.21.15.12:0/2358745910 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677ce66640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6778153210 0x7f6778153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f67777fe640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6778151240 0x7f67781563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6777fff640 1 -- 172.21.15.12:0/2358745910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6778153210 msgr2=0x7f6778153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6777fff640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6778153210 0x7f6778153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6777fff640 1 -- 172.21.15.12:0/2358745910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6778151240 msgr2=0x7f67781563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6777fff640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6778151240 0x7f67781563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6777fff640 1 -- 172.21.15.12:0/2358745910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778153bb0 con 0x7f67600047d0 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6777fff640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6778152460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6768001830 tx=0x7f676800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6774ff9640 1 -- 172.21.15.12:0/2358745910 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676800ee60 con 0x7f67600047d0 2023-11-28T21:35:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6774ff9640 1 -- 172.21.15.12:0/2358745910 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676800c8d0 con 0x7f67600047d0 2023-11-28T21:35:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677e8c9640 1 -- 172.21.15.12:0/2358745910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67781ef930 con 0x7f67600047d0 2023-11-28T21:35:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f6774ff9640 1 -- 172.21.15.12:0/2358745910 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676800cc60 con 0x7f67600047d0 2023-11-28T21:35:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.816+0000 7f677e8c9640 1 -- 172.21.15.12:0/2358745910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67781efef0 con 0x7f67600047d0 2023-11-28T21:35:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.820+0000 7f677e8c9640 1 -- 172.21.15.12:0/2358745910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6778074f60 con 0x7f67600047d0 2023-11-28T21:35:28.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.820+0000 7f6774ff9640 1 -- 172.21.15.12:0/2358745910 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6768013420 con 0x7f67600047d0 2023-11-28T21:35:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.820+0000 7f6774ff9640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f67640664a0 0x7f6764068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.820+0000 7f6774ff9640 1 -- 172.21.15.12:0/2358745910 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6768083ed0 con 0x7f67600047d0 2023-11-28T21:35:28.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.820+0000 7f67777fe640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f67640664a0 0x7f6764068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:28.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.820+0000 7f67777fe640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f67640664a0 0x7f6764068960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f676c0af000 tx=0x7f676c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:28.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.824+0000 7f6774ff9640 1 -- 172.21.15.12:0/2358745910 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6768051020 con 0x7f67600047d0 2023-11-28T21:35:28.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 -- 172.21.15.12:0/2358745910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67780c2c30 con 0x7f67600047d0 2023-11-28T21:35:28.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f6774ff9640 1 -- 172.21.15.12:0/2358745910 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6768054640 con 0x7f67600047d0 2023-11-28T21:35:28.988 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-11-28T21:35:28.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 -- 172.21.15.12:0/2358745910 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f67640664a0 msgr2=0x7f6764068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f67640664a0 0x7f6764068960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f676c0af000 tx=0x7f676c008040 comp rx=0 tx=0).stop 2023-11-28T21:35:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 -- 172.21.15.12:0/2358745910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 msgr2=0x7f6778152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6778152460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6768001830 tx=0x7f676800ec10 comp rx=0 tx=0).stop 2023-11-28T21:35:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 -- 172.21.15.12:0/2358745910 shutdown_connections 2023-11-28T21:35:28.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f67640664a0 0x7f6764068960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6778153210 0x7f6778153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6778151240 0x7f67781563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 --2- 172.21.15.12:0/2358745910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f67600047d0 0x7f6778152460 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.984+0000 7f677e8c9640 1 -- 172.21.15.12:0/2358745910 >> 172.21.15.12:0/2358745910 conn(0x7f6778066360 msgr2=0x7f6778155230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:28.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.988+0000 7f677e8c9640 1 -- 172.21.15.12:0/2358745910 shutdown_connections 2023-11-28T21:35:28.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:28.988+0000 7f677e8c9640 1 -- 172.21.15.12:0/2358745910 wait complete. 2023-11-28T21:35:29.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T21:35:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.152+0000 7fdaa84e3640 1 Processor -- start 2023-11-28T21:35:29.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.152+0000 7fdaa84e3640 1 -- start start 2023-11-28T21:35:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.152+0000 7fdaa84e3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa006e300 0x7fdaa0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.152+0000 7fdaa84e3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa0067410 con 0x7fdaa0150c20 2023-11-28T21:35:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.152+0000 7fdaa84e3640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa006e090 con 0x7fdaa006cf40 2023-11-28T21:35:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.152+0000 7fdaa84e3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa0157f30 con 0x7fdaa006e300 2023-11-28T21:35:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa627f640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdaa0150c20 0x7fdaa00c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45194/0 (socket says 172.21.15.12:45194) 2023-11-28T21:35:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa627f640 1 -- 172.21.15.12:0/1821574568 learned_addr learned my addr 172.21.15.12:0/1821574568 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa5a7e640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa006e300 0x7fdaa0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737804902 0 0) 0x7fdaa0067410 con 0x7fdaa0150c20 2023-11-28T21:35:29.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda8c003530 con 0x7fdaa0150c20 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695537718 0 0) 0x7fdaa006e090 con 0x7fdaa006cf40 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa0067410 con 0x7fdaa006cf40 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2433241654 0 0) 0x7fda8c003530 con 0x7fdaa0150c20 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa006e300 msgr2=0x7fdaa0156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa006e300 0x7fdaa0156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 >> v1:172.21.15.144:6789/0 conn(0x7fdaa006cf40 legacy=0x7fdaa0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa0151000 con 0x7fdaa0150c20 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fda9c001680 con 0x7fdaa0150c20 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 >> v1:172.21.15.12:6789/0 conn(0x7fdaa0150c20 legacy=0x7fdaa00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8c003bb0 0x7fda8c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8c0047d0 0x7fda8c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8c005190 0x7fda8c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa0157f30 con 0x7fda8c005190 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c00e270 con 0x7fda8c0047d0 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c00e430 con 0x7fda8c003bb0 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa5a7e640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa006e300 0x7fdaa0156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa6a80640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8c005190 0x7fda8c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa627f640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8c003bb0 0x7fda8c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa5a7e640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8c0047d0 0x7fda8c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa6a80640 1 -- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8c003bb0 msgr2=0x7fda8c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa6a80640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda8c003bb0 0x7fda8c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa6a80640 1 -- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8c0047d0 msgr2=0x7fda8c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa6a80640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8c0047d0 0x7fda8c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.156+0000 7fdaa6a80640 1 -- 172.21.15.12:0/1821574568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c003530 con 0x7fda8c005190 2023-11-28T21:35:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa6a80640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8c005190 0x7fda8c00b990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda98001fc0 tx=0x7fda9801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894fcdcf876504df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98003b80 con 0x7fda8c005190 2023-11-28T21:35:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda98003ce0 con 0x7fda8c005190 2023-11-28T21:35:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa527d640 1 -- 172.21.15.12:0/1821574568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda980052f0 con 0x7fda8c005190 2023-11-28T21:35:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa84e3640 1 -- 172.21.15.12:0/1821574568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8c005190 msgr2=0x7fda8c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa84e3640 1 --2- 172.21.15.12:0/1821574568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda8c005190 0x7fda8c00b990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda98001fc0 tx=0x7fda9801c8e0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa6a80640 1 -- 172.21.15.12:0/1821574568 reap_dead start 2023-11-28T21:35:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa84e3640 1 -- 172.21.15.12:0/1821574568 shutdown_connections 2023-11-28T21:35:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa84e3640 1 -- 172.21.15.12:0/1821574568 >> 172.21.15.12:0/1821574568 conn(0x7fdaa0066360 msgr2=0x7fdaa0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa84e3640 1 -- 172.21.15.12:0/1821574568 shutdown_connections 2023-11-28T21:35:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa84e3640 1 -- 172.21.15.12:0/1821574568 wait complete. 2023-11-28T21:35:29.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa84e3640 1 Processor -- start 2023-11-28T21:35:29.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.160+0000 7fdaa84e3640 1 -- start start 2023-11-28T21:35:29.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa84e3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8c005190 0x7fdaa00c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa84e3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa0150000 0x7fdaa006a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa84e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa01f2a50 0x7fdaa01f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa84e3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01f25a0 con 0x7fdaa01f2a50 2023-11-28T21:35:29.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa84e3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01f3410 con 0x7fda8c005190 2023-11-28T21:35:29.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa84e3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa014f000 con 0x7fdaa0150000 2023-11-28T21:35:29.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa6a80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa01f2a50 0x7fdaa01f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa6a80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa01f2a50 0x7fdaa01f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41624/0 (socket says 172.21.15.12:41624) 2023-11-28T21:35:29.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa6a80640 1 -- 172.21.15.12:0/2242531108 learned_addr learned my addr 172.21.15.12:0/2242531108 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:29.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa627f640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8c005190 0x7fdaa00c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa5a7e640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa0150000 0x7fdaa006a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa6a80640 1 -- 172.21.15.12:0/2242531108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa0150000 msgr2=0x7fdaa006a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa6a80640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa0150000 0x7fdaa006a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa6a80640 1 -- 172.21.15.12:0/2242531108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8c005190 msgr2=0x7fdaa00c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa6a80640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8c005190 0x7fdaa00c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa6a80640 1 -- 172.21.15.12:0/2242531108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa0070ae0 con 0x7fdaa01f2a50 2023-11-28T21:35:29.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa6a80640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa01f2a50 0x7fdaa01f2ed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fda98005040 tx=0x7fda8c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fda96ffd640 1 -- 172.21.15.12:0/2242531108 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98006d60 con 0x7fdaa01f2a50 2023-11-28T21:35:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fda96ffd640 1 -- 172.21.15.12:0/2242531108 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda980050d0 con 0x7fdaa01f2a50 2023-11-28T21:35:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa84e3640 1 -- 172.21.15.12:0/2242531108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa014f2a0 con 0x7fdaa01f2a50 2023-11-28T21:35:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fda96ffd640 1 -- 172.21.15.12:0/2242531108 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98005440 con 0x7fdaa01f2a50 2023-11-28T21:35:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa84e3640 1 -- 172.21.15.12:0/2242531108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa014f860 con 0x7fdaa01f2a50 2023-11-28T21:35:29.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fdaa84e3640 1 -- 172.21.15.12:0/2242531108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa01f2500 con 0x7fdaa01f2a50 2023-11-28T21:35:29.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.164+0000 7fda96ffd640 1 -- 172.21.15.12:0/2242531108 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fda980055a0 con 0x7fdaa01f2a50 2023-11-28T21:35:29.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.168+0000 7fda96ffd640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda840664a0 0x7fda84068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.168+0000 7fda96ffd640 1 -- 172.21.15.12:0/2242531108 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fda980aa070 con 0x7fdaa01f2a50 2023-11-28T21:35:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.168+0000 7fdaa627f640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda840664a0 0x7fda84068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.168+0000 7fdaa627f640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda840664a0 0x7fda84068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda9c000c10 tx=0x7fda9c000ee0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:29.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.172+0000 7fda96ffd640 1 -- 172.21.15.12:0/2242531108 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fda980eabb0 con 0x7fdaa01f2a50 2023-11-28T21:35:29.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 -- 172.21.15.12:0/2242531108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdaa0074f60 con 0x7fdaa01f2a50 2023-11-28T21:35:29.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fda96ffd640 1 -- 172.21.15.12:0/2242531108 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fda980bc0a0 con 0x7fdaa01f2a50 2023-11-28T21:35:29.330 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:35:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 -- 172.21.15.12:0/2242531108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda840664a0 msgr2=0x7fda84068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda840664a0 0x7fda84068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda9c000c10 tx=0x7fda9c000ee0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 -- 172.21.15.12:0/2242531108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa01f2a50 msgr2=0x7fdaa01f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa01f2a50 0x7fdaa01f2ed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fda98005040 tx=0x7fda8c005800 comp rx=0 tx=0).stop 2023-11-28T21:35:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 -- 172.21.15.12:0/2242531108 shutdown_connections 2023-11-28T21:35:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda840664a0 0x7fda84068960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa01f2a50 0x7fdaa01f2ed0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa0150000 0x7fdaa006a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 --2- 172.21.15.12:0/2242531108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda8c005190 0x7fdaa00c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 -- 172.21.15.12:0/2242531108 >> 172.21.15.12:0/2242531108 conn(0x7fdaa0066360 msgr2=0x7fdaa00c6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 -- 172.21.15.12:0/2242531108 shutdown_connections 2023-11-28T21:35:29.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.328+0000 7fdaa84e3640 1 -- 172.21.15.12:0/2242531108 wait complete. 2023-11-28T21:35:29.352 INFO:tasks.mds_thrash:mds thrasher using random seed: 1701207329 2023-11-28T21:35:29.352 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:29.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc725580640 1 Processor -- start 2023-11-28T21:35:29.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc725580640 1 -- start start 2023-11-28T21:35:29.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc725580640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc720154100 0x7fc72015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc725580640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7200ca3f0 0x7fc7200c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc725580640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72015e270 con 0x7fc7200c9fd0 2023-11-28T21:35:29.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc725580640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7200c4e60 con 0x7fc720154100 2023-11-28T21:35:29.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc725580640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7201544e0 con 0x7fc7200ca3f0 2023-11-28T21:35:29.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc717fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc7200c9fd0 0x7fc72006a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45198/0 (socket says 172.21.15.12:45198) 2023-11-28T21:35:29.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc717fff640 1 -- 172.21.15.12:0/1520586800 learned_addr learned my addr 172.21.15.12:0/1520586800 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:29.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71effd640 1 --2- 172.21.15.12:0/1520586800 >> v2:172.21.15.144:3300/0 conn(0x7fc720154100 0x7fc72015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71f7fe640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7200ca3f0 0x7fc7200c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932345261 0 0) 0x7fc72015e270 con 0x7fc7200c9fd0 2023-11-28T21:35:29.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc70c003530 con 0x7fc7200c9fd0 2023-11-28T21:35:29.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2597305815 0 0) 0x7fc70c003530 con 0x7fc7200c9fd0 2023-11-28T21:35:29.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7200ca3f0 msgr2=0x7fc7200c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:29.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7200ca3f0 0x7fc7200c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 >> v2:172.21.15.144:3300/0 conn(0x7fc720154100 msgr2=0x7fc72015d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:29.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 --2- 172.21.15.12:0/1520586800 >> v2:172.21.15.144:3300/0 conn(0x7fc720154100 0x7fc72015d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72015d9d0 con 0x7fc7200c9fd0 2023-11-28T21:35:29.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fc708001680 con 0x7fc7200c9fd0 2023-11-28T21:35:29.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71effd640 1 --2- 172.21.15.12:0/1520586800 >> v2:172.21.15.144:3300/0 conn(0x7fc720154100 0x7fc72015d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:35:29.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 >> v1:172.21.15.12:6789/0 conn(0x7fc7200c9fd0 legacy=0x7fc72006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 0x7fc70c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc70c0047d0 0x7fc70c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7200ca3f0 0x7fc70c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7200c4e60 con 0x7fc70c0039f0 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7201544e0 con 0x7fc70c0047d0 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c00b990 con 0x7fc7200ca3f0 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc71effd640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 0x7fc70c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.496+0000 7fc717fff640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc70c0047d0 0x7fc70c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71f7fe640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7200ca3f0 0x7fc70c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71effd640 1 -- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7200ca3f0 msgr2=0x7fc70c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71effd640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7200ca3f0 0x7fc70c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71effd640 1 -- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc70c0047d0 msgr2=0x7fc70c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71effd640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc70c0047d0 0x7fc70c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71effd640 1 -- 172.21.15.12:0/1520586800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c003530 con 0x7fc70c0039f0 2023-11-28T21:35:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71effd640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 0x7fc70c003e10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc710009ce0 tx=0x7fc71001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369db966ff780818 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:29.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7100a57e0 con 0x7fc70c0039f0 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7100a5940 con 0x7fc70c0039f0 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71e7fc640 1 -- 172.21.15.12:0/1520586800 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7100a5cf0 con 0x7fc70c0039f0 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc725580640 1 -- 172.21.15.12:0/1520586800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 msgr2=0x7fc70c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc725580640 1 --2- 172.21.15.12:0/1520586800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 0x7fc70c003e10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc710009ce0 tx=0x7fc71001cc20 comp rx=0 tx=0).stop 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc71f7fe640 1 -- 172.21.15.12:0/1520586800 reap_dead start 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc725580640 1 -- 172.21.15.12:0/1520586800 shutdown_connections 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc725580640 1 -- 172.21.15.12:0/1520586800 >> 172.21.15.12:0/1520586800 conn(0x7fc72006c690 msgr2=0x7fc720066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.500+0000 7fc725580640 1 -- 172.21.15.12:0/1520586800 shutdown_connections 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc725580640 1 -- 172.21.15.12:0/1520586800 wait complete. 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc725580640 1 Processor -- start 2023-11-28T21:35:29.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc725580640 1 -- start start 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc725580640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 0x7fc72014f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc725580640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc72014e6e0 0x7fc7201f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc725580640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7200c94b0 0x7fc7201f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc725580640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc720150a50 con 0x7fc70c0039f0 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc725580640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7200c9a80 con 0x7fc7200c94b0 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc725580640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc720150e60 con 0x7fc72014e6e0 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc71effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 0x7fc72014f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc71effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 0x7fc72014f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41640/0 (socket says 172.21.15.12:41640) 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc71effd640 1 -- 172.21.15.12:0/1952755835 learned_addr learned my addr 172.21.15.12:0/1952755835 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc71f7fe640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7200c94b0 0x7fc7201f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc717fff640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc72014e6e0 0x7fc7201f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc71effd640 1 -- 172.21.15.12:0/1952755835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc72014e6e0 msgr2=0x7fc7201f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc71effd640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc72014e6e0 0x7fc7201f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc71effd640 1 -- 172.21.15.12:0/1952755835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7200c94b0 msgr2=0x7fc7201f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc71effd640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7200c94b0 0x7fc7201f44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc71effd640 1 -- 172.21.15.12:0/1952755835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc720151140 con 0x7fc70c0039f0 2023-11-28T21:35:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc71effd640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 0x7fc72014f020 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc710009ce0 tx=0x7fc70c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc7177fe640 1 -- 172.21.15.12:0/1952755835 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7100a5cb0 con 0x7fc70c0039f0 2023-11-28T21:35:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc7177fe640 1 -- 172.21.15.12:0/1952755835 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7100a5e30 con 0x7fc70c0039f0 2023-11-28T21:35:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc725580640 1 -- 172.21.15.12:0/1952755835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc720151420 con 0x7fc70c0039f0 2023-11-28T21:35:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.504+0000 7fc7177fe640 1 -- 172.21.15.12:0/1952755835 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710004260 con 0x7fc70c0039f0 2023-11-28T21:35:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.508+0000 7fc725580640 1 -- 172.21.15.12:0/1952755835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72015d1a0 con 0x7fc70c0039f0 2023-11-28T21:35:29.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.508+0000 7fc725580640 1 -- 172.21.15.12:0/1952755835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7201f3e00 con 0x7fc70c0039f0 2023-11-28T21:35:29.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.508+0000 7fc7177fe640 1 -- 172.21.15.12:0/1952755835 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc7100b4040 con 0x7fc70c0039f0 2023-11-28T21:35:29.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.508+0000 7fc7177fe640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc700066470 0x7fc700068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.508+0000 7fc7177fe640 1 -- 172.21.15.12:0/1952755835 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc71011e100 con 0x7fc70c0039f0 2023-11-28T21:35:29.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.508+0000 7fc717fff640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc700066470 0x7fc700068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.508+0000 7fc717fff640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc700066470 0x7fc700068930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc72014df30 tx=0x7fc708009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:29.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.512+0000 7fc7177fe640 1 -- 172.21.15.12:0/1952755835 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc7100eb2f0 con 0x7fc70c0039f0 2023-11-28T21:35:29.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.676+0000 7fc725580640 1 -- 172.21.15.12:0/1952755835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc720067c40 con 0x7fc70c0039f0 2023-11-28T21:35:29.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.676+0000 7fc7177fe640 1 -- 172.21.15.12:0/1952755835 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18746 (secure 0 0 0) 0x7fc7100ee910 con 0x7fc70c0039f0 2023-11-28T21:35:29.682 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:29.682 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:35:25.566387+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:29.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 -- 172.21.15.12:0/1952755835 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc700066470 msgr2=0x7fc700068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc700066470 0x7fc700068930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc72014df30 tx=0x7fc708009040 comp rx=0 tx=0).stop 2023-11-28T21:35:29.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 -- 172.21.15.12:0/1952755835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 msgr2=0x7fc72014f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 0x7fc72014f020 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc710009ce0 tx=0x7fc70c005800 comp rx=0 tx=0).stop 2023-11-28T21:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 -- 172.21.15.12:0/1952755835 shutdown_connections 2023-11-28T21:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc700066470 0x7fc700068930 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7200c94b0 0x7fc7201f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc72014e6e0 0x7fc7201f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 --2- 172.21.15.12:0/1952755835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc70c0039f0 0x7fc72014f020 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 -- 172.21.15.12:0/1952755835 >> 172.21.15.12:0/1952755835 conn(0x7fc72006c690 msgr2=0x7fc720151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 -- 172.21.15.12:0/1952755835 shutdown_connections 2023-11-28T21:35:29.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.680+0000 7fc725580640 1 -- 172.21.15.12:0/1952755835 wait complete. 2023-11-28T21:35:29.703 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T21:34:29.878190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T21:34:34.039032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T21:34:35.369379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T21:34:39.373079+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T21:35:29.703 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T21:35:29.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f997238d640 1 Processor -- start 2023-11-28T21:35:29.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f997238d640 1 -- start start 2023-11-28T21:35:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f997238d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 0x7f996c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f997238d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f996c151ed0 0x7f996c0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f997238d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f996c06e300 0x7f996c0c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f997238d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996c0707f0 con 0x7f996c151ed0 2023-11-28T21:35:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f997238d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996c0c7b90 con 0x7f996c06e300 2023-11-28T21:35:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f997238d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c071c20 con 0x7f996c151af0 2023-11-28T21:35:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f996b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f996c151ed0 0x7f996c0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f996b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f996c151ed0 0x7f996c0c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41642/0 (socket says 172.21.15.12:41642) 2023-11-28T21:35:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.848+0000 7f996b7fe640 1 -- 172.21.15.12:0/3289679732 learned_addr learned my addr 172.21.15.12:0/3289679732 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f997092a640 1 --2- 172.21.15.12:0/3289679732 >> v2:172.21.15.144:3300/0 conn(0x7f996c06e300 0x7f996c0c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996bfff640 1 --2- 172.21.15.12:0/3289679732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 0x7f996c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996b7fe640 1 -- 172.21.15.12:0/3289679732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 msgr2=0x7f996c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996b7fe640 1 --2- 172.21.15.12:0/3289679732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 0x7f996c0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996b7fe640 1 -- 172.21.15.12:0/3289679732 >> v2:172.21.15.144:3300/0 conn(0x7f996c06e300 msgr2=0x7f996c0c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:29.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996b7fe640 1 --2- 172.21.15.12:0/3289679732 >> v2:172.21.15.144:3300/0 conn(0x7f996c06e300 0x7f996c0c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996b7fe640 1 -- 172.21.15.12:0/3289679732 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c157b70 con 0x7f996c151ed0 2023-11-28T21:35:29.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f997092a640 1 --2- 172.21.15.12:0/3289679732 >> v2:172.21.15.144:3300/0 conn(0x7f996c06e300 0x7f996c0c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:29.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996bfff640 1 --2- 172.21.15.12:0/3289679732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 0x7f996c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:29.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996b7fe640 1 --2- 172.21.15.12:0/3289679732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c151ed0 0x7f996c0c49d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f996c06b940 tx=0x7f996001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b59ecf48c404bf1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:29.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996affd640 1 -- 172.21.15.12:0/3289679732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99600ac410 con 0x7f996c151ed0 2023-11-28T21:35:29.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996affd640 1 -- 172.21.15.12:0/3289679732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99600ac570 con 0x7f996c151ed0 2023-11-28T21:35:29.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.852+0000 7f996affd640 1 -- 172.21.15.12:0/3289679732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99600ac900 con 0x7f996c151ed0 2023-11-28T21:35:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 -- 172.21.15.12:0/3289679732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c151ed0 msgr2=0x7f996c0c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 --2- 172.21.15.12:0/3289679732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c151ed0 0x7f996c0c49d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f996c06b940 tx=0x7f996001c500 comp rx=0 tx=0).stop 2023-11-28T21:35:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 -- 172.21.15.12:0/3289679732 shutdown_connections 2023-11-28T21:35:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 --2- 172.21.15.12:0/3289679732 >> v2:172.21.15.144:3300/0 conn(0x7f996c06e300 0x7f996c0c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 --2- 172.21.15.12:0/3289679732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c151ed0 0x7f996c0c49d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 --2- 172.21.15.12:0/3289679732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 0x7f996c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 -- 172.21.15.12:0/3289679732 >> 172.21.15.12:0/3289679732 conn(0x7f996c066360 msgr2=0x7f996c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:29.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 -- 172.21.15.12:0/3289679732 shutdown_connections 2023-11-28T21:35:29.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 -- 172.21.15.12:0/3289679732 wait complete. 2023-11-28T21:35:29.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 Processor -- start 2023-11-28T21:35:29.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 -- start start 2023-11-28T21:35:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c06e300 0x7f996c0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 0x7f996c1f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c158b90 0x7f996c1431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1f3e40 con 0x7f996c158b90 2023-11-28T21:35:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c143e20 con 0x7f996c06e300 2023-11-28T21:35:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997238d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1f2e40 con 0x7f996c151af0 2023-11-28T21:35:29.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997092a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c158b90 0x7f996c1431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997092a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c158b90 0x7f996c1431d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41646/0 (socket says 172.21.15.12:41646) 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f997092a640 1 -- 172.21.15.12:0/3804241481 learned_addr learned my addr 172.21.15.12:0/3804241481 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.856+0000 7f996b7fe640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 0x7f996c1f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f996bfff640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c06e300 0x7f996c0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f997092a640 1 -- 172.21.15.12:0/3804241481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 msgr2=0x7f996c1f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f997092a640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 0x7f996c1f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f997092a640 1 -- 172.21.15.12:0/3804241481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c06e300 msgr2=0x7f996c0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f997092a640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c06e300 0x7f996c0cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f997092a640 1 -- 172.21.15.12:0/3804241481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99600094d0 con 0x7f996c158b90 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f997092a640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c158b90 0x7f996c1431d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9964007de0 tx=0x7f996400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f9968ff9640 1 -- 172.21.15.12:0/3804241481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9964003ba0 con 0x7f996c158b90 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f9968ff9640 1 -- 172.21.15.12:0/3804241481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996400f040 con 0x7f996c158b90 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f997238d640 1 -- 172.21.15.12:0/3804241481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c15a5d0 con 0x7f996c158b90 2023-11-28T21:35:29.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f9968ff9640 1 -- 172.21.15.12:0/3804241481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996400de20 con 0x7f996c158b90 2023-11-28T21:35:29.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f997238d640 1 -- 172.21.15.12:0/3804241481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c15ab90 con 0x7f996c158b90 2023-11-28T21:35:29.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f997238d640 1 -- 172.21.15.12:0/3804241481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c074f60 con 0x7f996c158b90 2023-11-28T21:35:29.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f9968ff9640 1 -- 172.21.15.12:0/3804241481 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9964004030 con 0x7f996c158b90 2023-11-28T21:35:29.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f9968ff9640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9958066470 0x7f9958068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:29.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.860+0000 7f9968ff9640 1 -- 172.21.15.12:0/3804241481 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f99640845e0 con 0x7f996c158b90 2023-11-28T21:35:29.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.864+0000 7f996bfff640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9958066470 0x7f9958068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:29.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.864+0000 7f996bfff640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9958066470 0x7f9958068930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c900 tx=0x7f995c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:29.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:29.864+0000 7f9968ff9640 1 -- 172.21.15.12:0/3804241481 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f99640516b0 con 0x7f996c158b90 2023-11-28T21:35:30.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.024+0000 7f997238d640 1 -- 172.21.15.12:0/3804241481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f996c14fe70 con 0x7f996c158b90 2023-11-28T21:35:30.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f9968ff9640 1 -- 172.21.15.12:0/3804241481 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9964054cd0 con 0x7f996c158b90 2023-11-28T21:35:30.030 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-11-28T21:35:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 -- 172.21.15.12:0/3804241481 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9958066470 msgr2=0x7f9958068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9958066470 0x7f9958068930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c900 tx=0x7f995c008040 comp rx=0 tx=0).stop 2023-11-28T21:35:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 -- 172.21.15.12:0/3804241481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c158b90 msgr2=0x7f996c1431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c158b90 0x7f996c1431d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9964007de0 tx=0x7f996400d4a0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 -- 172.21.15.12:0/3804241481 shutdown_connections 2023-11-28T21:35:30.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9958066470 0x7f9958068930 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f996c158b90 0x7f996c1431d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c151af0 0x7f996c1f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 --2- 172.21.15.12:0/3804241481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c06e300 0x7f996c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 -- 172.21.15.12:0/3804241481 >> 172.21.15.12:0/3804241481 conn(0x7f996c066360 msgr2=0x7f996c144040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 -- 172.21.15.12:0/3804241481 shutdown_connections 2023-11-28T21:35:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.028+0000 7f997238d640 1 -- 172.21.15.12:0/3804241481 wait complete. 2023-11-28T21:35:30.051 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T21:35:30.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.192+0000 7fcefe2d6640 1 Processor -- start 2023-11-28T21:35:30.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.192+0000 7fcefe2d6640 1 -- start start 2023-11-28T21:35:30.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcefe2d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcef81500d0 0x7fcef80c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcefe2d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcef806d7d0 0x7fcef806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcefe2d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef80c6ce0 con 0x7fcef8156af0 2023-11-28T21:35:30.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcefe2d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef80c7170 con 0x7fcef81500d0 2023-11-28T21:35:30.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcefe2d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8156f60 con 0x7fcef806d7d0 2023-11-28T21:35:30.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcef7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcef81500d0 0x7fcef80c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcef7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcef81500d0 0x7fcef80c27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41660/0 (socket says 172.21.15.12:41660) 2023-11-28T21:35:30.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcef7fff640 1 -- 172.21.15.12:0/3767323686 learned_addr learned my addr 172.21.15.12:0/3767323686 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:30.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcef77fe640 1 --1- 172.21.15.12:0/3767323686 >> v1:172.21.15.12:6789/0 conn(0x7fcef8156af0 0x7fcef80c2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45212/0 (socket says 172.21.15.12:45212) 2023-11-28T21:35:30.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcefc873640 1 --2- 172.21.15.12:0/3767323686 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcef806d7d0 0x7fcef806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcef7fff640 1 -- 172.21.15.12:0/3767323686 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcef806d7d0 msgr2=0x7fcef806dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:30.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcef7fff640 1 --2- 172.21.15.12:0/3767323686 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcef806d7d0 0x7fcef806dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcef7fff640 1 -- 172.21.15.12:0/3767323686 >> v1:172.21.15.12:6789/0 conn(0x7fcef8156af0 legacy=0x7fcef80c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.196+0000 7fcef7fff640 1 -- 172.21.15.12:0/3767323686 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef80c5df0 con 0x7fcef81500d0 2023-11-28T21:35:30.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcef7fff640 1 --2- 172.21.15.12:0/3767323686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef81500d0 0x7fcef80c27c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009870 tx=0x7fcee801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812a31116d5031f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:30.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcef6ffd640 1 -- 172.21.15.12:0/3767323686 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee80aa620 con 0x7fcef81500d0 2023-11-28T21:35:30.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcef6ffd640 1 -- 172.21.15.12:0/3767323686 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee80aa780 con 0x7fcef81500d0 2023-11-28T21:35:30.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcef6ffd640 1 -- 172.21.15.12:0/3767323686 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee80aab10 con 0x7fcef81500d0 2023-11-28T21:35:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 -- 172.21.15.12:0/3767323686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef81500d0 msgr2=0x7fcef80c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 --2- 172.21.15.12:0/3767323686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef81500d0 0x7fcef80c27c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009870 tx=0x7fcee801c540 comp rx=0 tx=0).stop 2023-11-28T21:35:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 -- 172.21.15.12:0/3767323686 shutdown_connections 2023-11-28T21:35:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 --2- 172.21.15.12:0/3767323686 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcef806d7d0 0x7fcef806dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 --2- 172.21.15.12:0/3767323686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef81500d0 0x7fcef80c27c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 -- 172.21.15.12:0/3767323686 >> 172.21.15.12:0/3767323686 conn(0x7fcef8066360 msgr2=0x7fcef8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 -- 172.21.15.12:0/3767323686 shutdown_connections 2023-11-28T21:35:30.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 -- 172.21.15.12:0/3767323686 wait complete. 2023-11-28T21:35:30.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 Processor -- start 2023-11-28T21:35:30.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 -- start start 2023-11-28T21:35:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef806d7d0 0x7fcef81584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcef81500d0 0x7fcef814a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcef8156af0 0x7fcef814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8149fb0 con 0x7fcef806d7d0 2023-11-28T21:35:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef81f5b70 con 0x7fcef81500d0 2023-11-28T21:35:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.200+0000 7fcefe2d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef815a1d0 con 0x7fcef8156af0 2023-11-28T21:35:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef806d7d0 0x7fcef81584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef806d7d0 0x7fcef81584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41662/0 (socket says 172.21.15.12:41662) 2023-11-28T21:35:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef7fff640 1 -- 172.21.15.12:0/2660582680 learned_addr learned my addr 172.21.15.12:0/2660582680 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:30.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef77fe640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcef81500d0 0x7fcef814a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcefc873640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcef8156af0 0x7fcef814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef7fff640 1 -- 172.21.15.12:0/2660582680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcef8156af0 msgr2=0x7fcef814f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef7fff640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcef8156af0 0x7fcef814f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef7fff640 1 -- 172.21.15.12:0/2660582680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcef81500d0 msgr2=0x7fcef814a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef7fff640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcef81500d0 0x7fcef814a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef7fff640 1 -- 172.21.15.12:0/2660582680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee80094d0 con 0x7fcef806d7d0 2023-11-28T21:35:30.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef7fff640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef806d7d0 0x7fcef81584d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcee801abf0 tx=0x7fcee801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:30.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef4ff9640 1 -- 172.21.15.12:0/2660582680 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee80bd3f0 con 0x7fcef806d7d0 2023-11-28T21:35:30.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef4ff9640 1 -- 172.21.15.12:0/2660582680 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee80bd570 con 0x7fcef806d7d0 2023-11-28T21:35:30.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcefe2d6640 1 -- 172.21.15.12:0/2660582680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef8153220 con 0x7fcef806d7d0 2023-11-28T21:35:30.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef4ff9640 1 -- 172.21.15.12:0/2660582680 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee80bd920 con 0x7fcef806d7d0 2023-11-28T21:35:30.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcefe2d6640 1 -- 172.21.15.12:0/2660582680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef81f5de0 con 0x7fcef806d7d0 2023-11-28T21:35:30.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcefe2d6640 1 -- 172.21.15.12:0/2660582680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef8074f60 con 0x7fcef806d7d0 2023-11-28T21:35:30.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.204+0000 7fcef4ff9640 1 -- 172.21.15.12:0/2660582680 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fcee80b33f0 con 0x7fcef806d7d0 2023-11-28T21:35:30.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.208+0000 7fcef4ff9640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcee4066470 0x7fcee4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.208+0000 7fcef4ff9640 1 -- 172.21.15.12:0/2660582680 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcee811e7e0 con 0x7fcef806d7d0 2023-11-28T21:35:30.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.208+0000 7fcef77fe640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcee4066470 0x7fcee4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.208+0000 7fcef77fe640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcee4066470 0x7fcee4068930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcef815a440 tx=0x7fceec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:30.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.212+0000 7fcef4ff9640 1 -- 172.21.15.12:0/2660582680 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcee80eb930 con 0x7fcef806d7d0 2023-11-28T21:35:30.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.368+0000 7fcefe2d6640 1 -- 172.21.15.12:0/2660582680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcef81f5fd0 con 0x7fcef806d7d0 2023-11-28T21:35:30.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.368+0000 7fcef4ff9640 1 -- 172.21.15.12:0/2660582680 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcee80eef50 con 0x7fcef806d7d0 2023-11-28T21:35:30.373 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:35:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 -- 172.21.15.12:0/2660582680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcee4066470 msgr2=0x7fcee4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcee4066470 0x7fcee4068930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcef815a440 tx=0x7fceec009040 comp rx=0 tx=0).stop 2023-11-28T21:35:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 -- 172.21.15.12:0/2660582680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef806d7d0 msgr2=0x7fcef81584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef806d7d0 0x7fcef81584d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcee801abf0 tx=0x7fcee801ad80 comp rx=0 tx=0).stop 2023-11-28T21:35:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 -- 172.21.15.12:0/2660582680 shutdown_connections 2023-11-28T21:35:30.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcee4066470 0x7fcee4068930 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcef8156af0 0x7fcef814f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcef81500d0 0x7fcef814a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 --2- 172.21.15.12:0/2660582680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcef806d7d0 0x7fcef81584d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 -- 172.21.15.12:0/2660582680 >> 172.21.15.12:0/2660582680 conn(0x7fcef8066360 msgr2=0x7fcef80c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 -- 172.21.15.12:0/2660582680 shutdown_connections 2023-11-28T21:35:30.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.372+0000 7fcefe2d6640 1 -- 172.21.15.12:0/2660582680 wait complete. 2023-11-28T21:35:30.394 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T21:35:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80e6079640 1 Processor -- start 2023-11-28T21:35:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80e6079640 1 -- start start 2023-11-28T21:35:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80e6079640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f80e00c27c0 0x7f80e0156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80e6079640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e00c6ce0 con 0x7f80e01500d0 2023-11-28T21:35:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80e6079640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e00c5df0 con 0x7f80e006d7d0 2023-11-28T21:35:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80e6079640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e0157030 con 0x7f80e00c27c0 2023-11-28T21:35:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80df7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f80e01500d0 0x7f80e00c2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45226/0 (socket says 172.21.15.12:45226) 2023-11-28T21:35:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80df7fe640 1 -- 172.21.15.12:0/2558708389 learned_addr learned my addr 172.21.15.12:0/2558708389 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80deffd640 1 --2- 172.21.15.12:0/2558708389 >> v2:172.21.15.12:3300/0 conn(0x7f80e00c27c0 0x7f80e0156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80deffd640 1 -- 172.21.15.12:0/2558708389 >> v1:172.21.15.144:6789/0 conn(0x7f80e006d7d0 legacy=0x7f80e01609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80de7fc640 1 -- 172.21.15.12:0/2558708389 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360425974 0 0) 0x7f80e00c6ce0 con 0x7f80e01500d0 2023-11-28T21:35:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80deffd640 1 -- 172.21.15.12:0/2558708389 >> v1:172.21.15.12:6789/0 conn(0x7f80e01500d0 legacy=0x7f80e00c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.540+0000 7f80deffd640 1 -- 172.21.15.12:0/2558708389 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e00c60d0 con 0x7f80e00c27c0 2023-11-28T21:35:30.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80deffd640 1 --2- 172.21.15.12:0/2558708389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 0x7f80e0156af0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f80d40097c0 tx=0x7f80d401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7561ec0b0a1ae52 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:30.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80de7fc640 1 -- 172.21.15.12:0/2558708389 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d40ab600 con 0x7f80e00c27c0 2023-11-28T21:35:30.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80de7fc640 1 -- 172.21.15.12:0/2558708389 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d40a7050 con 0x7f80e00c27c0 2023-11-28T21:35:30.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80de7fc640 1 -- 172.21.15.12:0/2558708389 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d40ab980 con 0x7f80e00c27c0 2023-11-28T21:35:30.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80e6079640 1 -- 172.21.15.12:0/2558708389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 msgr2=0x7f80e0156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80e6079640 1 --2- 172.21.15.12:0/2558708389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 0x7f80e0156af0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f80d40097c0 tx=0x7f80d401c480 comp rx=0 tx=0).stop 2023-11-28T21:35:30.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80e6079640 1 -- 172.21.15.12:0/2558708389 shutdown_connections 2023-11-28T21:35:30.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80e6079640 1 --2- 172.21.15.12:0/2558708389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 0x7f80e0156af0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80e6079640 1 -- 172.21.15.12:0/2558708389 >> 172.21.15.12:0/2558708389 conn(0x7f80e0066360 msgr2=0x7f80e0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:30.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80e6079640 1 -- 172.21.15.12:0/2558708389 shutdown_connections 2023-11-28T21:35:30.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.544+0000 7f80e6079640 1 -- 172.21.15.12:0/2558708389 wait complete. 2023-11-28T21:35:30.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80e6079640 1 Processor -- start 2023-11-28T21:35:30.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80e6079640 1 -- start start 2023-11-28T21:35:30.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80e6079640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80e006d7d0 0x7f80e0155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80e6079640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 0x7f80e01f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80e6079640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80e01500d0 0x7f80e00ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80e6079640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e0155200 con 0x7f80e00c27c0 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80e6079640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e01f6600 con 0x7f80e006d7d0 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80e6079640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e01f6fd0 con 0x7f80e01500d0 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80deffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 0x7f80e01f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80deffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 0x7f80e01f6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41678/0 (socket says 172.21.15.12:41678) 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80deffd640 1 -- 172.21.15.12:0/4059672732 learned_addr learned my addr 172.21.15.12:0/4059672732 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80df7fe640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80e006d7d0 0x7f80e0155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80dffff640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80e01500d0 0x7f80e00ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80deffd640 1 -- 172.21.15.12:0/4059672732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80e01500d0 msgr2=0x7f80e00ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80deffd640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80e01500d0 0x7f80e00ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80deffd640 1 -- 172.21.15.12:0/4059672732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80e006d7d0 msgr2=0x7f80e0155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80deffd640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80e006d7d0 0x7f80e0155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80deffd640 1 -- 172.21.15.12:0/4059672732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d4009420 con 0x7f80e00c27c0 2023-11-28T21:35:30.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80deffd640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 0x7f80e01f6a90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f80d401a0b0 tx=0x7f80d4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80c7fff640 1 -- 172.21.15.12:0/4059672732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4004230 con 0x7f80e00c27c0 2023-11-28T21:35:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80c7fff640 1 -- 172.21.15.12:0/4059672732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d40a4e90 con 0x7f80e00c27c0 2023-11-28T21:35:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80c7fff640 1 -- 172.21.15.12:0/4059672732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d40abe10 con 0x7f80e00c27c0 2023-11-28T21:35:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80e6079640 1 -- 172.21.15.12:0/4059672732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e0158900 con 0x7f80e00c27c0 2023-11-28T21:35:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.548+0000 7f80e6079640 1 -- 172.21.15.12:0/4059672732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e0154f20 con 0x7f80e00c27c0 2023-11-28T21:35:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.552+0000 7f80e6079640 1 -- 172.21.15.12:0/4059672732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e0074f60 con 0x7f80e00c27c0 2023-11-28T21:35:30.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.552+0000 7f80c7fff640 1 -- 172.21.15.12:0/4059672732 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f80d40b2040 con 0x7f80e00c27c0 2023-11-28T21:35:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.552+0000 7f80c7fff640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f80cc066470 0x7f80cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.552+0000 7f80c7fff640 1 -- 172.21.15.12:0/4059672732 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f80d411d4b0 con 0x7f80e00c27c0 2023-11-28T21:35:30.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.552+0000 7f80df7fe640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f80cc066470 0x7f80cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.552+0000 7f80df7fe640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f80cc066470 0x7f80cc068930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80d0000f70 tx=0x7f80d0007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:30.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.556+0000 7f80c7fff640 1 -- 172.21.15.12:0/4059672732 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f80d40ea4f0 con 0x7f80e00c27c0 2023-11-28T21:35:30.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.716+0000 7f80e6079640 1 -- 172.21.15.12:0/4059672732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80e0161ef0 con 0x7f80e00c27c0 2023-11-28T21:35:30.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.716+0000 7f80c7fff640 1 -- 172.21.15.12:0/4059672732 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80d40bc0a0 con 0x7f80e00c27c0 2023-11-28T21:35:30.721 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-11-28T21:35:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 -- 172.21.15.12:0/4059672732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f80cc066470 msgr2=0x7f80cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f80cc066470 0x7f80cc068930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80d0000f70 tx=0x7f80d0007240 comp rx=0 tx=0).stop 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 -- 172.21.15.12:0/4059672732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 msgr2=0x7f80e01f6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 0x7f80e01f6a90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f80d401a0b0 tx=0x7f80d4019220 comp rx=0 tx=0).stop 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 -- 172.21.15.12:0/4059672732 shutdown_connections 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f80cc066470 0x7f80cc068930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80e01500d0 0x7f80e00ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80e00c27c0 0x7f80e01f6a90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 --2- 172.21.15.12:0/4059672732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80e006d7d0 0x7f80e0155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 -- 172.21.15.12:0/4059672732 >> 172.21.15.12:0/4059672732 conn(0x7f80e0066360 msgr2=0x7f80e014bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 -- 172.21.15.12:0/4059672732 shutdown_connections 2023-11-28T21:35:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.720+0000 7f80e6079640 1 -- 172.21.15.12:0/4059672732 wait complete. 2023-11-28T21:35:30.742 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T21:35:30.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.884+0000 7f5a54be6640 1 Processor -- start 2023-11-28T21:35:30.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.884+0000 7f5a54be6640 1 -- start start 2023-11-28T21:35:30.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a54be6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50150c20 0x7f5a50156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a54be6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5a5006e300 0x7f5a500c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a54be6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a50067410 con 0x7f5a5006cf40 2023-11-28T21:35:30.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a54be6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5006e090 con 0x7f5a5006e300 2023-11-28T21:35:30.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a54be6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50157f30 con 0x7f5a50150c20 2023-11-28T21:35:30.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a4e59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50150c20 0x7f5a50156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a4dd9b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5a5006e300 0x7f5a500c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a4e59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50150c20 0x7f5a50156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:42238/0 (socket says 172.21.15.12:42238) 2023-11-28T21:35:30.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a4e59c640 1 -- 172.21.15.12:0/3556593745 learned_addr learned my addr 172.21.15.12:0/3556593745 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:30.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a4dd9b640 1 --2- 172.21.15.12:0/3556593745 >> v2:172.21.15.144:3300/0 conn(0x7f5a5006e300 0x7f5a500c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:51712/0 (socket says 172.21.15.12:51712) 2023-11-28T21:35:30.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a4ed9d640 1 --1- 172.21.15.12:0/3556593745 >> v1:172.21.15.144:6789/0 conn(0x7f5a5006cf40 0x7f5a50158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:47000/0 (socket says 172.21.15.12:47000) 2023-11-28T21:35:30.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a4dd9b640 1 -- 172.21.15.12:0/3556593745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50150c20 msgr2=0x7f5a50156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:30.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a4dd9b640 1 --2- 172.21.15.12:0/3556593745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50150c20 0x7f5a50156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a4dd9b640 1 -- 172.21.15.12:0/3556593745 >> v1:172.21.15.144:6789/0 conn(0x7f5a5006cf40 legacy=0x7f5a50158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.888+0000 7f5a4dd9b640 1 -- 172.21.15.12:0/3556593745 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a50151000 con 0x7f5a5006e300 2023-11-28T21:35:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a4dd9b640 1 --2- 172.21.15.12:0/3556593745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5006e300 0x7f5a500c6dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a440097c0 tx=0x7f5a4401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d37b28c92900d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:30.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a4d59a640 1 -- 172.21.15.12:0/3556593745 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440ad640 con 0x7f5a5006e300 2023-11-28T21:35:30.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a4d59a640 1 -- 172.21.15.12:0/3556593745 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a440ad7a0 con 0x7f5a5006e300 2023-11-28T21:35:30.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a4d59a640 1 -- 172.21.15.12:0/3556593745 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440adb30 con 0x7f5a5006e300 2023-11-28T21:35:30.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a54be6640 1 -- 172.21.15.12:0/3556593745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5006e300 msgr2=0x7f5a500c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a54be6640 1 --2- 172.21.15.12:0/3556593745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5006e300 0x7f5a500c6dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a440097c0 tx=0x7f5a4401c480 comp rx=0 tx=0).stop 2023-11-28T21:35:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a54be6640 1 -- 172.21.15.12:0/3556593745 shutdown_connections 2023-11-28T21:35:30.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a54be6640 1 --2- 172.21.15.12:0/3556593745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5006e300 0x7f5a500c6dd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a54be6640 1 --2- 172.21.15.12:0/3556593745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50150c20 0x7f5a50156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a54be6640 1 -- 172.21.15.12:0/3556593745 >> 172.21.15.12:0/3556593745 conn(0x7f5a50066360 msgr2=0x7f5a50066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:30.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a54be6640 1 -- 172.21.15.12:0/3556593745 shutdown_connections 2023-11-28T21:35:30.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.892+0000 7f5a54be6640 1 -- 172.21.15.12:0/3556593745 wait complete. 2023-11-28T21:35:30.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a54be6640 1 Processor -- start 2023-11-28T21:35:30.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a54be6640 1 -- start start 2023-11-28T21:35:30.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a54be6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a5006cf40 0x7f5a50149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a54be6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006e300 0x7f5a501f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a54be6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50150c20 0x7f5a501f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a54be6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50148fc0 con 0x7f5a5006cf40 2023-11-28T21:35:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a54be6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50072b70 con 0x7f5a50150c20 2023-11-28T21:35:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a54be6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50070dc0 con 0x7f5a5006e300 2023-11-28T21:35:30.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a5006cf40 0x7f5a50149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a5006cf40 0x7f5a50149730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41684/0 (socket says 172.21.15.12:41684) 2023-11-28T21:35:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4e59c640 1 -- 172.21.15.12:0/683603298 learned_addr learned my addr 172.21.15.12:0/683603298 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4ed9d640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50150c20 0x7f5a501f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4dd9b640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006e300 0x7f5a501f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4e59c640 1 -- 172.21.15.12:0/683603298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006e300 msgr2=0x7f5a501f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4e59c640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006e300 0x7f5a501f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4e59c640 1 -- 172.21.15.12:0/683603298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50150c20 msgr2=0x7f5a501f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:30.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4e59c640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50150c20 0x7f5a501f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:30.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4e59c640 1 -- 172.21.15.12:0/683603298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a44009420 con 0x7f5a5006cf40 2023-11-28T21:35:30.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a4e59c640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a5006cf40 0x7f5a50149730 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000e870 tx=0x7f5a4000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:30.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a3effd640 1 -- 172.21.15.12:0/683603298 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400094e0 con 0x7f5a5006cf40 2023-11-28T21:35:30.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a3effd640 1 -- 172.21.15.12:0/683603298 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4000c940 con 0x7f5a5006cf40 2023-11-28T21:35:30.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a54be6640 1 -- 172.21.15.12:0/683603298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a500710a0 con 0x7f5a5006cf40 2023-11-28T21:35:30.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a3effd640 1 -- 172.21.15.12:0/683603298 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000cd10 con 0x7f5a5006cf40 2023-11-28T21:35:30.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.896+0000 7f5a54be6640 1 -- 172.21.15.12:0/683603298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a50072080 con 0x7f5a5006cf40 2023-11-28T21:35:30.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.900+0000 7f5a54be6640 1 -- 172.21.15.12:0/683603298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a50074f60 con 0x7f5a5006cf40 2023-11-28T21:35:30.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.900+0000 7f5a3effd640 1 -- 172.21.15.12:0/683603298 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5a40013470 con 0x7f5a5006cf40 2023-11-28T21:35:30.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.900+0000 7f5a3effd640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a30066470 0x7f5a30068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:30.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.900+0000 7f5a3effd640 1 -- 172.21.15.12:0/683603298 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5a400842a0 con 0x7f5a5006cf40 2023-11-28T21:35:30.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.900+0000 7f5a4dd9b640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a30066470 0x7f5a30068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:30.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.900+0000 7f5a4dd9b640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a30066470 0x7f5a30068930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a440af000 tx=0x7f5a440a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:30.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:30.904+0000 7f5a3effd640 1 -- 172.21.15.12:0/683603298 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5a40051400 con 0x7f5a5006cf40 2023-11-28T21:35:31.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.064+0000 7f5a54be6640 1 -- 172.21.15.12:0/683603298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5a500c2c30 con 0x7f5a5006cf40 2023-11-28T21:35:31.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.064+0000 7f5a3effd640 1 -- 172.21.15.12:0/683603298 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5a40054a20 con 0x7f5a5006cf40 2023-11-28T21:35:31.069 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:35:31.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 -- 172.21.15.12:0/683603298 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a30066470 msgr2=0x7f5a30068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a30066470 0x7f5a30068930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a440af000 tx=0x7f5a440a7070 comp rx=0 tx=0).stop 2023-11-28T21:35:31.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 -- 172.21.15.12:0/683603298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a5006cf40 msgr2=0x7f5a50149730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a5006cf40 0x7f5a50149730 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000e870 tx=0x7f5a4000ed30 comp rx=0 tx=0).stop 2023-11-28T21:35:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 -- 172.21.15.12:0/683603298 shutdown_connections 2023-11-28T21:35:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5a30066470 0x7f5a30068930 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a50150c20 0x7f5a501f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006e300 0x7f5a501f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 --2- 172.21.15.12:0/683603298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a5006cf40 0x7f5a50149730 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 -- 172.21.15.12:0/683603298 >> 172.21.15.12:0/683603298 conn(0x7f5a50066360 msgr2=0x7f5a500c9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 -- 172.21.15.12:0/683603298 shutdown_connections 2023-11-28T21:35:31.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.068+0000 7f5a54be6640 1 -- 172.21.15.12:0/683603298 wait complete. 2023-11-28T21:35:31.090 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2023-11-28T21:35:31.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:35:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b2ad96640 1 Processor -- start 2023-11-28T21:35:31.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b2ad96640 1 -- start start 2023-11-28T21:35:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b2ad96640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240cb180 0x7f9b240cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b2ad96640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2415acc0 con 0x7f9b240ca110 2023-11-28T21:35:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b2ad96640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b24159ea0 con 0x7f9b24153e40 2023-11-28T21:35:31.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b2ad96640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b240ca520 con 0x7f9b240cb180 2023-11-28T21:35:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b23fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9b240ca110 0x7f9b240cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45230/0 (socket says 172.21.15.12:45230) 2023-11-28T21:35:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b23fff640 1 -- 172.21.15.12:0/492079279 learned_addr learned my addr 172.21.15.12:0/492079279 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b29333640 1 --2- 172.21.15.12:0/492079279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240cb180 0x7f9b240cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b237fe640 1 -- 172.21.15.12:0/492079279 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296426837 0 0) 0x7f9b2415acc0 con 0x7f9b240ca110 2023-11-28T21:35:31.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b237fe640 1 -- 172.21.15.12:0/492079279 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b14003530 con 0x7f9b240ca110 2023-11-28T21:35:31.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b29333640 1 -- 172.21.15.12:0/492079279 >> v1:172.21.15.144:6789/0 conn(0x7f9b24153e40 legacy=0x7f9b2415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b237fe640 1 -- 172.21.15.12:0/492079279 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254910725 0 0) 0x7f9b24159ea0 con 0x7f9b24153e40 2023-11-28T21:35:31.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b29333640 1 -- 172.21.15.12:0/492079279 >> v1:172.21.15.12:6789/0 conn(0x7f9b240ca110 legacy=0x7f9b240cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.232+0000 7f9b29333640 1 -- 172.21.15.12:0/492079279 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b240cbb10 con 0x7f9b240cb180 2023-11-28T21:35:31.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.236+0000 7f9b29333640 1 --2- 172.21.15.12:0/492079279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240cb180 0x7f9b240cb5d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009dc0 tx=0x7f9b1c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2fc9efd426516b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:31.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.236+0000 7f9b237fe640 1 -- 172.21.15.12:0/492079279 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0a57c0 con 0x7f9b240cb180 2023-11-28T21:35:31.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.236+0000 7f9b237fe640 1 -- 172.21.15.12:0/492079279 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c0a5920 con 0x7f9b240cb180 2023-11-28T21:35:31.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.236+0000 7f9b237fe640 1 -- 172.21.15.12:0/492079279 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0a5c90 con 0x7f9b240cb180 2023-11-28T21:35:31.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.236+0000 7f9b2ad96640 1 -- 172.21.15.12:0/492079279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240cb180 msgr2=0x7f9b240cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.236+0000 7f9b2ad96640 1 --2- 172.21.15.12:0/492079279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240cb180 0x7f9b240cb5d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009dc0 tx=0x7f9b1c01c890 comp rx=0 tx=0).stop 2023-11-28T21:35:31.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.236+0000 7f9b2ad96640 1 -- 172.21.15.12:0/492079279 shutdown_connections 2023-11-28T21:35:31.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.236+0000 7f9b2ad96640 1 --2- 172.21.15.12:0/492079279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240cb180 0x7f9b240cb5d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 -- 172.21.15.12:0/492079279 >> 172.21.15.12:0/492079279 conn(0x7f9b2406b7b0 msgr2=0x7f9b24066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:31.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 -- 172.21.15.12:0/492079279 shutdown_connections 2023-11-28T21:35:31.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 -- 172.21.15.12:0/492079279 wait complete. 2023-11-28T21:35:31.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 Processor -- start 2023-11-28T21:35:31.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 -- start start 2023-11-28T21:35:31.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240ca110 0x7f9b2415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b240cb180 0x7f9b24152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b24153e40 0x7f9b24152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b240b3910 con 0x7f9b24153e40 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24151570 con 0x7f9b240cb180 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b2ad96640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b240b3bb0 con 0x7f9b240ca110 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b29333640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b24153e40 0x7f9b24152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b29333640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b24153e40 0x7f9b24152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41688/0 (socket says 172.21.15.12:41688) 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b29333640 1 -- 172.21.15.12:0/808029470 learned_addr learned my addr 172.21.15.12:0/808029470 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b23fff640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b240cb180 0x7f9b24152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b28b32640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240ca110 0x7f9b2415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b29333640 1 -- 172.21.15.12:0/808029470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240ca110 msgr2=0x7f9b2415ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b29333640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240ca110 0x7f9b2415ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b29333640 1 -- 172.21.15.12:0/808029470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b240cb180 msgr2=0x7f9b24152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b29333640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b240cb180 0x7f9b24152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b29333640 1 -- 172.21.15.12:0/808029470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c009a70 con 0x7f9b24153e40 2023-11-28T21:35:31.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b28b32640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240ca110 0x7f9b2415ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:31.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b23fff640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b240cb180 0x7f9b24152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.240+0000 7f9b29333640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b24153e40 0x7f9b24152550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c01a4c0 tx=0x7f9b1c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b217fa640 1 -- 172.21.15.12:0/808029470 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c003ba0 con 0x7f9b24153e40 2023-11-28T21:35:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b217fa640 1 -- 172.21.15.12:0/808029470 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c003d00 con 0x7f9b24153e40 2023-11-28T21:35:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b217fa640 1 -- 172.21.15.12:0/808029470 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c004240 con 0x7f9b24153e40 2023-11-28T21:35:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b2ad96640 1 -- 172.21.15.12:0/808029470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b240b3e90 con 0x7f9b24153e40 2023-11-28T21:35:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b2ad96640 1 -- 172.21.15.12:0/808029470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b24151af0 con 0x7f9b24153e40 2023-11-28T21:35:31.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b2ad96640 1 -- 172.21.15.12:0/808029470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b24151ce0 con 0x7f9b24153e40 2023-11-28T21:35:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b217fa640 1 -- 172.21.15.12:0/808029470 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9b1c0b4070 con 0x7f9b24153e40 2023-11-28T21:35:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b217fa640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b080664a0 0x7f9b08068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b217fa640 1 -- 172.21.15.12:0/808029470 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9b1c11ef30 con 0x7f9b24153e40 2023-11-28T21:35:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b28b32640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b080664a0 0x7f9b08068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.244+0000 7f9b28b32640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b080664a0 0x7f9b08068960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000f70 tx=0x7f9b18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:31.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.248+0000 7f9b217fa640 1 -- 172.21.15.12:0/808029470 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9b1c0ec100 con 0x7f9b24153e40 2023-11-28T21:35:31.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.440+0000 7f9b2ad96640 1 -- 172.21.15.12:0/808029470 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b240b2210 con 0x7f9b24153e40 2023-11-28T21:35:31.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b217fa640 1 -- 172.21.15.12:0/808029470 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6529 (secure 0 0 0) 0x7f9b1c0ef720 con 0x7f9b24153e40 2023-11-28T21:35:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:31.446 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:35:14.337396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:35:31.446 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2023-11-28T21:35:31.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 -- 172.21.15.12:0/808029470 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b080664a0 msgr2=0x7f9b08068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b080664a0 0x7f9b08068960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9b18000f70 tx=0x7f9b18009040 comp rx=0 tx=0).stop 2023-11-28T21:35:31.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 -- 172.21.15.12:0/808029470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b24153e40 msgr2=0x7f9b24152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b24153e40 0x7f9b24152550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c01a4c0 tx=0x7f9b1c019630 comp rx=0 tx=0).stop 2023-11-28T21:35:31.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 -- 172.21.15.12:0/808029470 shutdown_connections 2023-11-28T21:35:31.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b080664a0 0x7f9b08068960 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b24153e40 0x7f9b24152550 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b240cb180 0x7f9b24152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 --2- 172.21.15.12:0/808029470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b240ca110 0x7f9b2415ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 -- 172.21.15.12:0/808029470 >> 172.21.15.12:0/808029470 conn(0x7f9b2406b7b0 msgr2=0x7f9b24066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:31.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 -- 172.21.15.12:0/808029470 shutdown_connections 2023-11-28T21:35:31.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.444+0000 7f9b2ad96640 1 -- 172.21.15.12:0/808029470 wait complete. 2023-11-28T21:35:31.467 INFO:tasks.mds_thrash:Ready to start thrashing 2023-11-28T21:35:31.467 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2023-11-28T21:35:31.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:35:31.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.608+0000 7efc8a71d640 1 Processor -- start 2023-11-28T21:35:31.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.608+0000 7efc8a71d640 1 -- start start 2023-11-28T21:35:31.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.608+0000 7efc8a71d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc84150c20 0x7efc84156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.608+0000 7efc8a71d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efc8406cf40 0x7efc84158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.608+0000 7efc8a71d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc84067410 con 0x7efc8406e300 2023-11-28T21:35:31.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.608+0000 7efc8a71d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8406e090 con 0x7efc8406cf40 2023-11-28T21:35:31.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.608+0000 7efc8a71d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc84157f30 con 0x7efc84150c20 2023-11-28T21:35:31.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.608+0000 7efc88cba640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efc8406cf40 0x7efc84158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.608+0000 7efc83fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc84150c20 0x7efc84156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.612+0000 7efc83fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc84150c20 0x7efc84156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:42268/0 (socket says 172.21.15.12:42268) 2023-11-28T21:35:31.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.612+0000 7efc83fff640 1 -- 172.21.15.12:0/1271652598 learned_addr learned my addr 172.21.15.12:0/1271652598 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:31.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.612+0000 7efc88cba640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efc8406cf40 0x7efc84158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:51736/0 (socket says 172.21.15.12:51736) 2023-11-28T21:35:31.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.612+0000 7efc83fff640 1 -- 172.21.15.12:0/1271652598 >> v2:172.21.15.144:3300/0 conn(0x7efc8406cf40 msgr2=0x7efc84158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:31.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.612+0000 7efc83fff640 1 --2- 172.21.15.12:0/1271652598 >> v2:172.21.15.144:3300/0 conn(0x7efc8406cf40 0x7efc84158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.612+0000 7efc83fff640 1 -- 172.21.15.12:0/1271652598 >> v1:172.21.15.144:6789/0 conn(0x7efc8406e300 legacy=0x7efc840c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.612+0000 7efc83fff640 1 -- 172.21.15.12:0/1271652598 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc84151000 con 0x7efc84150c20 2023-11-28T21:35:31.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.612+0000 7efc83fff640 1 --2- 172.21.15.12:0/1271652598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc84150c20 0x7efc84156910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efc7400bb10 tx=0x7efc7401ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53ef7966ce5a2978 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:31.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc82ffd640 1 -- 172.21.15.12:0/1271652598 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc740a7af0 con 0x7efc84150c20 2023-11-28T21:35:31.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc82ffd640 1 -- 172.21.15.12:0/1271652598 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc740a7c50 con 0x7efc84150c20 2023-11-28T21:35:31.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc82ffd640 1 -- 172.21.15.12:0/1271652598 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc740af440 con 0x7efc84150c20 2023-11-28T21:35:31.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc8a71d640 1 -- 172.21.15.12:0/1271652598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc84150c20 msgr2=0x7efc84156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc8a71d640 1 --2- 172.21.15.12:0/1271652598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc84150c20 0x7efc84156910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efc7400bb10 tx=0x7efc7401ec10 comp rx=0 tx=0).stop 2023-11-28T21:35:31.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc8a71d640 1 -- 172.21.15.12:0/1271652598 shutdown_connections 2023-11-28T21:35:31.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc8a71d640 1 --2- 172.21.15.12:0/1271652598 >> v2:172.21.15.144:3300/0 conn(0x7efc8406cf40 0x7efc84158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc8a71d640 1 --2- 172.21.15.12:0/1271652598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc84150c20 0x7efc84156910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc8a71d640 1 -- 172.21.15.12:0/1271652598 >> 172.21.15.12:0/1271652598 conn(0x7efc84066360 msgr2=0x7efc84066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:31.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc8a71d640 1 -- 172.21.15.12:0/1271652598 shutdown_connections 2023-11-28T21:35:31.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc8a71d640 1 -- 172.21.15.12:0/1271652598 wait complete. 2023-11-28T21:35:31.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc8a71d640 1 Processor -- start 2023-11-28T21:35:31.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.620+0000 7efc8a71d640 1 -- start start 2023-11-28T21:35:31.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc8a71d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8406cf40 0x7efc840c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc8a71d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc8406e300 0x7efc841f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc8a71d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84150c20 0x7efc8414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc8a71d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc841ef270 con 0x7efc84150c20 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc8a71d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc841f2d20 con 0x7efc8406cf40 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc8a71d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c4c80 con 0x7efc8406e300 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc88cba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84150c20 0x7efc8414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc88cba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84150c20 0x7efc8414ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41704/0 (socket says 172.21.15.12:41704) 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc88cba640 1 -- 172.21.15.12:0/448978445 learned_addr learned my addr 172.21.15.12:0/448978445 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc83fff640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8406cf40 0x7efc840c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc837fe640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc8406e300 0x7efc841f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc88cba640 1 -- 172.21.15.12:0/448978445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc8406e300 msgr2=0x7efc841f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc88cba640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc8406e300 0x7efc841f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc88cba640 1 -- 172.21.15.12:0/448978445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8406cf40 msgr2=0x7efc840c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc88cba640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8406cf40 0x7efc840c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc88cba640 1 -- 172.21.15.12:0/448978445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7400b770 con 0x7efc84150c20 2023-11-28T21:35:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc88cba640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84150c20 0x7efc8414ea70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00efc0 tx=0x7efc7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc80ff9640 1 -- 172.21.15.12:0/448978445 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c00ced0 con 0x7efc84150c20 2023-11-28T21:35:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc80ff9640 1 -- 172.21.15.12:0/448978445 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c00f040 con 0x7efc84150c20 2023-11-28T21:35:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc80ff9640 1 -- 172.21.15.12:0/448978445 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c0146c0 con 0x7efc84150c20 2023-11-28T21:35:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc8a71d640 1 -- 172.21.15.12:0/448978445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc840c4f60 con 0x7efc84150c20 2023-11-28T21:35:31.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc8a71d640 1 -- 172.21.15.12:0/448978445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc841eee90 con 0x7efc84150c20 2023-11-28T21:35:31.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.624+0000 7efc8a71d640 1 -- 172.21.15.12:0/448978445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc841ef3b0 con 0x7efc84150c20 2023-11-28T21:35:31.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.628+0000 7efc80ff9640 1 -- 172.21.15.12:0/448978445 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7efc7c014820 con 0x7efc84150c20 2023-11-28T21:35:31.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.628+0000 7efc80ff9640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efc700664c0 0x7efc70068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:31.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.628+0000 7efc80ff9640 1 -- 172.21.15.12:0/448978445 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efc7c084fb0 con 0x7efc84150c20 2023-11-28T21:35:31.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.628+0000 7efc83fff640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efc700664c0 0x7efc70068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:31.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.628+0000 7efc83fff640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efc700664c0 0x7efc70068980 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc740b1000 tx=0x7efc740a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:31.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.632+0000 7efc80ff9640 1 -- 172.21.15.12:0/448978445 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efc7c0b6280 con 0x7efc84150c20 2023-11-28T21:35:31.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.788+0000 7efc8a71d640 1 -- 172.21.15.12:0/448978445 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc8406b7d0 con 0x7efc700664c0 2023-11-28T21:35:31.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.796+0000 7efc80ff9640 1 -- 172.21.15.12:0/448978445 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345114 (secure 0 0 0) 0x7efc8406b7d0 con 0x7efc700664c0 2023-11-28T21:35:31.803 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:31.805 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:35:31.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 -- 172.21.15.12:0/448978445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efc700664c0 msgr2=0x7efc70068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efc700664c0 0x7efc70068980 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc740b1000 tx=0x7efc740a9070 comp rx=0 tx=0).stop 2023-11-28T21:35:31.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 -- 172.21.15.12:0/448978445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84150c20 msgr2=0x7efc8414ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:31.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84150c20 0x7efc8414ea70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00efc0 tx=0x7efc7c00c490 comp rx=0 tx=0).stop 2023-11-28T21:35:31.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 -- 172.21.15.12:0/448978445 shutdown_connections 2023-11-28T21:35:31.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efc700664c0 0x7efc70068980 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc84150c20 0x7efc8414ea70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc8406e300 0x7efc841f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 --2- 172.21.15.12:0/448978445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc8406cf40 0x7efc840c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:31.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 -- 172.21.15.12:0/448978445 >> 172.21.15.12:0/448978445 conn(0x7efc84066360 msgr2=0x7efc8406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:31.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 -- 172.21.15.12:0/448978445 shutdown_connections 2023-11-28T21:35:31.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:31.800+0000 7efc8a71d640 1 -- 172.21.15.12:0/448978445 wait complete. 2023-11-28T21:35:31.824 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2023-11-28T21:35:29.828105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597819,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":287,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218460,"kb_used_data":2316,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756260,"statfs":{"total":773094113280,"available":772870410240,"internally_reserved":0,"allocated":2371584,"data_stored":2021219,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001700"},"pg_stats":[{"pgid":"2.1f","version":"24'9","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.332958+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:35:14.332958+0000","last_peered":"2023-11-28T21:35:14.332958+0000","last_clean":"2023-11-28T21:35:14.332958+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:35:14.332958+0000","last_undegraded":"2023-11-28T21:35:14.332958+0000","last_fullsized":"2023-11-28T21:35:14.332958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:46.167511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361318+0000","last_change":"2023-11-28T21:34:37.335656+0000","last_active":"2023-11-28T21:34:42.361318+0000","last_peered":"2023-11-28T21:34:42.361318+0000","last_clean":"2023-11-28T21:34:42.361318+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:34:42.361318+0000","last_undegraded":"2023-11-28T21:34:42.361318+0000","last_fullsized":"2023-11-28T21:34:42.361318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:25:49.707378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.849214+0000","last_change":"2023-11-28T21:34:41.483856+0000","last_active":"2023-11-28T21:34:44.849214+0000","last_peered":"2023-11-28T21:34:44.849214+0000","last_clean":"2023-11-28T21:34:44.849214+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:34:44.849214+0000","last_undegraded":"2023-11-28T21:34:44.849214+0000","last_fullsized":"2023-11-28T21:34:44.849214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:50:56.005578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.326653+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:35:14.326653+0000","last_peered":"2023-11-28T21:35:14.326653+0000","last_clean":"2023-11-28T21:35:14.326653+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:35:14.326653+0000","last_undegraded":"2023-11-28T21:35:14.326653+0000","last_fullsized":"2023-11-28T21:35:14.326653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592574+0000","last_change":"2023-11-28T21:34:37.333649+0000","last_active":"2023-11-28T21:34:42.592574+0000","last_peered":"2023-11-28T21:34:42.592574+0000","last_clean":"2023-11-28T21:34:42.592574+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:34:42.592574+0000","last_undegraded":"2023-11-28T21:34:42.592574+0000","last_fullsized":"2023-11-28T21:34:42.592574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:21:42.041590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.848638+0000","last_change":"2023-11-28T21:34:41.388232+0000","last_active":"2023-11-28T21:34:44.848638+0000","last_peered":"2023-11-28T21:34:44.848638+0000","last_clean":"2023-11-28T21:34:44.848638+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:34:44.848638+0000","last_undegraded":"2023-11-28T21:34:44.848638+0000","last_fullsized":"2023-11-28T21:34:44.848638+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:22.960012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'4","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.334545+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:35:14.334545+0000","last_peered":"2023-11-28T21:35:14.334545+0000","last_clean":"2023-11-28T21:35:14.334545+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:35:14.334545+0000","last_undegraded":"2023-11-28T21:35:14.334545+0000","last_fullsized":"2023-11-28T21:35:14.334545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363671+0000","last_change":"2023-11-28T21:34:37.348126+0000","last_active":"2023-11-28T21:34:42.363671+0000","last_peered":"2023-11-28T21:34:42.363671+0000","last_clean":"2023-11-28T21:34:42.363671+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:34:42.363671+0000","last_undegraded":"2023-11-28T21:34:42.363671+0000","last_fullsized":"2023-11-28T21:34:42.363671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:47.982143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.848808+0000","last_change":"2023-11-28T21:34:41.388539+0000","last_active":"2023-11-28T21:34:44.848808+0000","last_peered":"2023-11-28T21:34:44.848808+0000","last_clean":"2023-11-28T21:34:44.848808+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:34:44.848808+0000","last_undegraded":"2023-11-28T21:34:44.848808+0000","last_fullsized":"2023-11-28T21:34:44.848808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.598151+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:34:42.598151+0000","last_peered":"2023-11-28T21:34:42.598151+0000","last_clean":"2023-11-28T21:34:42.598151+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:34:42.598151+0000","last_undegraded":"2023-11-28T21:34:42.598151+0000","last_fullsized":"2023-11-28T21:34:42.598151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363311+0000","last_change":"2023-11-28T21:34:37.337266+0000","last_active":"2023-11-28T21:34:42.363311+0000","last_peered":"2023-11-28T21:34:42.363311+0000","last_clean":"2023-11-28T21:34:42.363311+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:34:42.363311+0000","last_undegraded":"2023-11-28T21:34:42.363311+0000","last_fullsized":"2023-11-28T21:34:42.363311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:32:48.848989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.848772+0000","last_change":"2023-11-28T21:34:41.372957+0000","last_active":"2023-11-28T21:34:44.848772+0000","last_peered":"2023-11-28T21:34:44.848772+0000","last_clean":"2023-11-28T21:34:44.848772+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:34:44.848772+0000","last_undegraded":"2023-11-28T21:34:44.848772+0000","last_fullsized":"2023-11-28T21:34:44.848772+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593775+0000","last_change":"2023-11-28T21:34:37.353174+0000","last_active":"2023-11-28T21:34:42.593775+0000","last_peered":"2023-11-28T21:34:42.593775+0000","last_clean":"2023-11-28T21:34:42.593775+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:34:42.593775+0000","last_undegraded":"2023-11-28T21:34:42.593775+0000","last_fullsized":"2023-11-28T21:34:42.593775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:32.966564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'4","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.307879+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:35:08.307879+0000","last_peered":"2023-11-28T21:35:08.307879+0000","last_clean":"2023-11-28T21:35:08.307879+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:35:08.307879+0000","last_undegraded":"2023-11-28T21:35:08.307879+0000","last_fullsized":"2023-11-28T21:35:08.307879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"peering","last_fresh":"2023-11-28T21:34:44.848931+0000","last_change":"2023-11-28T21:35:24.582154+0000","last_active":"2023-11-28T21:35:24.582154+0000","last_peered":"2023-11-28T21:34:44.848931+0000","last_clean":"2023-11-28T21:34:44.848931+0000","last_became_active":"2023-11-28T21:34:41.362862+0000","last_became_peered":"2023-11-28T21:34:41.362862+0000","last_unstale":"2023-11-28T21:34:44.848931+0000","last_undegraded":"2023-11-28T21:34:44.848931+0000","last_fullsized":"2023-11-28T21:34:44.848931+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,2,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360788+0000","last_change":"2023-11-28T21:34:37.313495+0000","last_active":"2023-11-28T21:34:42.360788+0000","last_peered":"2023-11-28T21:34:42.360788+0000","last_clean":"2023-11-28T21:34:42.360788+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:34:42.360788+0000","last_undegraded":"2023-11-28T21:34:42.360788+0000","last_fullsized":"2023-11-28T21:34:42.360788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:32:50.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"24'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.309744+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:35:14.309744+0000","last_peered":"2023-11-28T21:35:14.309744+0000","last_clean":"2023-11-28T21:35:14.309744+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:35:14.309744+0000","last_undegraded":"2023-11-28T21:35:14.309744+0000","last_fullsized":"2023-11-28T21:35:14.309744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:53.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"peering","last_fresh":"2023-11-28T21:34:44.848883+0000","last_change":"2023-11-28T21:35:24.575106+0000","last_active":"2023-11-28T21:35:24.575106+0000","last_peered":"2023-11-28T21:34:44.848883+0000","last_clean":"2023-11-28T21:34:44.848883+0000","last_became_active":"2023-11-28T21:34:41.366442+0000","last_became_peered":"2023-11-28T21:34:41.366442+0000","last_unstale":"2023-11-28T21:34:44.848883+0000","last_undegraded":"2023-11-28T21:34:44.848883+0000","last_fullsized":"2023-11-28T21:34:44.848883+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,3,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359974+0000","last_change":"2023-11-28T21:34:37.326569+0000","last_active":"2023-11-28T21:34:42.359974+0000","last_peered":"2023-11-28T21:34:42.359974+0000","last_clean":"2023-11-28T21:34:42.359974+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:34:42.359974+0000","last_undegraded":"2023-11-28T21:34:42.359974+0000","last_fullsized":"2023-11-28T21:34:42.359974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:39:14.264355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"24'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.338972+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:35:08.338972+0000","last_peered":"2023-11-28T21:35:08.338972+0000","last_clean":"2023-11-28T21:35:08.338972+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:35:08.338972+0000","last_undegraded":"2023-11-28T21:35:08.338972+0000","last_fullsized":"2023-11-28T21:35:08.338972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.846737+0000","last_change":"2023-11-28T21:34:41.483661+0000","last_active":"2023-11-28T21:34:44.846737+0000","last_peered":"2023-11-28T21:34:44.846737+0000","last_clean":"2023-11-28T21:34:44.846737+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:34:44.846737+0000","last_undegraded":"2023-11-28T21:34:44.846737+0000","last_fullsized":"2023-11-28T21:34:44.846737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.596370+0000","last_change":"2023-11-28T21:34:37.336922+0000","last_active":"2023-11-28T21:34:42.596370+0000","last_peered":"2023-11-28T21:34:42.596370+0000","last_clean":"2023-11-28T21:34:42.596370+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:34:42.596370+0000","last_undegraded":"2023-11-28T21:34:42.596370+0000","last_fullsized":"2023-11-28T21:34:42.596370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:45.663475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"24'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.309939+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:35:08.309939+0000","last_peered":"2023-11-28T21:35:08.309939+0000","last_clean":"2023-11-28T21:35:08.309939+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:35:08.309939+0000","last_undegraded":"2023-11-28T21:35:08.309939+0000","last_fullsized":"2023-11-28T21:35:08.309939+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.848958+0000","last_change":"2023-11-28T21:34:41.382756+0000","last_active":"2023-11-28T21:34:44.848958+0000","last_peered":"2023-11-28T21:34:44.848958+0000","last_clean":"2023-11-28T21:34:44.848958+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:34:44.848958+0000","last_undegraded":"2023-11-28T21:34:44.848958+0000","last_fullsized":"2023-11-28T21:34:44.848958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:07.402513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592438+0000","last_change":"2023-11-28T21:34:37.320446+0000","last_active":"2023-11-28T21:34:42.592438+0000","last_peered":"2023-11-28T21:34:42.592438+0000","last_clean":"2023-11-28T21:34:42.592438+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:34:42.592438+0000","last_undegraded":"2023-11-28T21:34:42.592438+0000","last_fullsized":"2023-11-28T21:34:42.592438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:33.404441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.332259+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:35:08.332259+0000","last_peered":"2023-11-28T21:35:08.332259+0000","last_clean":"2023-11-28T21:35:08.332259+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:35:08.332259+0000","last_undegraded":"2023-11-28T21:35:08.332259+0000","last_fullsized":"2023-11-28T21:35:08.332259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.848731+0000","last_change":"2023-11-28T21:34:41.485107+0000","last_active":"2023-11-28T21:34:44.848731+0000","last_peered":"2023-11-28T21:34:44.848731+0000","last_clean":"2023-11-28T21:34:44.848731+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:34:44.848731+0000","last_undegraded":"2023-11-28T21:34:44.848731+0000","last_fullsized":"2023-11-28T21:34:44.848731+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361521+0000","last_change":"2023-11-28T21:34:37.345104+0000","last_active":"2023-11-28T21:34:42.361521+0000","last_peered":"2023-11-28T21:34:42.361521+0000","last_clean":"2023-11-28T21:34:42.361521+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:34:42.361521+0000","last_undegraded":"2023-11-28T21:34:42.361521+0000","last_fullsized":"2023-11-28T21:34:42.361521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:25:16.125179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.333512+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:35:14.333512+0000","last_peered":"2023-11-28T21:35:14.333512+0000","last_clean":"2023-11-28T21:35:14.333512+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:35:14.333512+0000","last_undegraded":"2023-11-28T21:35:14.333512+0000","last_fullsized":"2023-11-28T21:35:14.333512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:36.968220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.849152+0000","last_change":"2023-11-28T21:34:41.475704+0000","last_active":"2023-11-28T21:34:44.849152+0000","last_peered":"2023-11-28T21:34:44.849152+0000","last_clean":"2023-11-28T21:34:44.849152+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:34:44.849152+0000","last_undegraded":"2023-11-28T21:34:44.849152+0000","last_fullsized":"2023-11-28T21:34:44.849152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:01:34.494307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592942+0000","last_change":"2023-11-28T21:34:37.330831+0000","last_active":"2023-11-28T21:34:42.592942+0000","last_peered":"2023-11-28T21:34:42.592942+0000","last_clean":"2023-11-28T21:34:42.592942+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:34:42.592942+0000","last_undegraded":"2023-11-28T21:34:42.592942+0000","last_fullsized":"2023-11-28T21:34:42.592942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:50:38.612360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"24'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.309678+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:35:08.309678+0000","last_peered":"2023-11-28T21:35:08.309678+0000","last_clean":"2023-11-28T21:35:08.309678+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:35:08.309678+0000","last_undegraded":"2023-11-28T21:35:08.309678+0000","last_fullsized":"2023-11-28T21:35:08.309678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.845814+0000","last_change":"2023-11-28T21:34:41.475481+0000","last_active":"2023-11-28T21:34:44.845814+0000","last_peered":"2023-11-28T21:34:44.845814+0000","last_clean":"2023-11-28T21:34:44.845814+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:34:44.845814+0000","last_undegraded":"2023-11-28T21:34:44.845814+0000","last_fullsized":"2023-11-28T21:34:44.845814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:22:18.635807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363449+0000","last_change":"2023-11-28T21:34:37.337846+0000","last_active":"2023-11-28T21:34:42.363449+0000","last_peered":"2023-11-28T21:34:42.363449+0000","last_clean":"2023-11-28T21:34:42.363449+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:34:42.363449+0000","last_undegraded":"2023-11-28T21:34:42.363449+0000","last_fullsized":"2023-11-28T21:34:42.363449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:46:21.358603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593037+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:34:42.593037+0000","last_peered":"2023-11-28T21:34:42.593037+0000","last_clean":"2023-11-28T21:34:42.593037+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:34:42.593037+0000","last_undegraded":"2023-11-28T21:34:42.593037+0000","last_fullsized":"2023-11-28T21:34:42.593037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:08:15.270352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.846612+0000","last_change":"2023-11-28T21:34:41.370410+0000","last_active":"2023-11-28T21:34:44.846612+0000","last_peered":"2023-11-28T21:34:44.846612+0000","last_clean":"2023-11-28T21:34:44.846612+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:34:44.846612+0000","last_undegraded":"2023-11-28T21:34:44.846612+0000","last_fullsized":"2023-11-28T21:34:44.846612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597968+0000","last_change":"2023-11-28T21:34:37.324608+0000","last_active":"2023-11-28T21:34:42.597968+0000","last_peered":"2023-11-28T21:34:42.597968+0000","last_clean":"2023-11-28T21:34:42.597968+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:34:42.597968+0000","last_undegraded":"2023-11-28T21:34:42.597968+0000","last_fullsized":"2023-11-28T21:34:42.597968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:13:53.438473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"24'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.328552+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:35:14.328552+0000","last_peered":"2023-11-28T21:35:14.328552+0000","last_clean":"2023-11-28T21:35:14.328552+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:35:14.328552+0000","last_undegraded":"2023-11-28T21:35:14.328552+0000","last_fullsized":"2023-11-28T21:35:14.328552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.846644+0000","last_change":"2023-11-28T21:34:41.380599+0000","last_active":"2023-11-28T21:34:44.846644+0000","last_peered":"2023-11-28T21:34:44.846644+0000","last_clean":"2023-11-28T21:34:44.846644+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:34:44.846644+0000","last_undegraded":"2023-11-28T21:34:44.846644+0000","last_fullsized":"2023-11-28T21:34:44.846644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591703+0000","last_change":"2023-11-28T21:34:37.330869+0000","last_active":"2023-11-28T21:34:42.591703+0000","last_peered":"2023-11-28T21:34:42.591703+0000","last_clean":"2023-11-28T21:34:42.591703+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:34:42.591703+0000","last_undegraded":"2023-11-28T21:34:42.591703+0000","last_fullsized":"2023-11-28T21:34:42.591703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:58:55.987235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"24'4","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.329013+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:35:14.329013+0000","last_peered":"2023-11-28T21:35:14.329013+0000","last_clean":"2023-11-28T21:35:14.329013+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:35:14.329013+0000","last_undegraded":"2023-11-28T21:35:14.329013+0000","last_fullsized":"2023-11-28T21:35:14.329013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.846756+0000","last_change":"2023-11-28T21:34:41.371897+0000","last_active":"2023-11-28T21:34:44.846756+0000","last_peered":"2023-11-28T21:34:44.846756+0000","last_clean":"2023-11-28T21:34:44.846756+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:34:44.846756+0000","last_undegraded":"2023-11-28T21:34:44.846756+0000","last_fullsized":"2023-11-28T21:34:44.846756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591919+0000","last_change":"2023-11-28T21:34:37.319833+0000","last_active":"2023-11-28T21:34:42.591919+0000","last_peered":"2023-11-28T21:34:42.591919+0000","last_clean":"2023-11-28T21:34:42.591919+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:34:42.591919+0000","last_undegraded":"2023-11-28T21:34:42.591919+0000","last_fullsized":"2023-11-28T21:34:42.591919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:02:17.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"24'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.328364+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:35:14.328364+0000","last_peered":"2023-11-28T21:35:14.328364+0000","last_clean":"2023-11-28T21:35:14.328364+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:35:14.328364+0000","last_undegraded":"2023-11-28T21:35:14.328364+0000","last_fullsized":"2023-11-28T21:35:14.328364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.845942+0000","last_change":"2023-11-28T21:34:41.475053+0000","last_active":"2023-11-28T21:34:44.845942+0000","last_peered":"2023-11-28T21:34:44.845942+0000","last_clean":"2023-11-28T21:34:44.845942+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:34:44.845942+0000","last_undegraded":"2023-11-28T21:34:44.845942+0000","last_fullsized":"2023-11-28T21:34:44.845942+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:32.658001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360397+0000","last_change":"2023-11-28T21:34:37.358890+0000","last_active":"2023-11-28T21:34:42.360397+0000","last_peered":"2023-11-28T21:34:42.360397+0000","last_clean":"2023-11-28T21:34:42.360397+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:34:42.360397+0000","last_undegraded":"2023-11-28T21:34:42.360397+0000","last_fullsized":"2023-11-28T21:34:42.360397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:54.878141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"24'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.330985+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:35:14.330985+0000","last_peered":"2023-11-28T21:35:14.330985+0000","last_clean":"2023-11-28T21:35:14.330985+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:35:14.330985+0000","last_undegraded":"2023-11-28T21:35:14.330985+0000","last_fullsized":"2023-11-28T21:35:14.330985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.596154+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:34:42.596154+0000","last_peered":"2023-11-28T21:34:42.596154+0000","last_clean":"2023-11-28T21:34:42.596154+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:34:42.596154+0000","last_undegraded":"2023-11-28T21:34:42.596154+0000","last_fullsized":"2023-11-28T21:34:42.596154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.868832+0000","last_change":"2023-11-28T21:34:41.477522+0000","last_active":"2023-11-28T21:34:44.868832+0000","last_peered":"2023-11-28T21:34:44.868832+0000","last_clean":"2023-11-28T21:34:44.868832+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:34:44.868832+0000","last_undegraded":"2023-11-28T21:34:44.868832+0000","last_fullsized":"2023-11-28T21:34:44.868832+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:36:36.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.598266+0000","last_change":"2023-11-28T21:34:37.350441+0000","last_active":"2023-11-28T21:34:42.598266+0000","last_peered":"2023-11-28T21:34:42.598266+0000","last_clean":"2023-11-28T21:34:42.598266+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:34:42.598266+0000","last_undegraded":"2023-11-28T21:34:42.598266+0000","last_fullsized":"2023-11-28T21:34:42.598266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:54.775495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"24'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.330737+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:35:14.330737+0000","last_peered":"2023-11-28T21:35:14.330737+0000","last_clean":"2023-11-28T21:35:14.330737+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:35:14.330737+0000","last_undegraded":"2023-11-28T21:35:14.330737+0000","last_fullsized":"2023-11-28T21:35:14.330737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.868723+0000","last_change":"2023-11-28T21:34:41.485154+0000","last_active":"2023-11-28T21:34:44.868723+0000","last_peered":"2023-11-28T21:34:44.868723+0000","last_clean":"2023-11-28T21:34:44.868723+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:34:44.868723+0000","last_undegraded":"2023-11-28T21:34:44.868723+0000","last_fullsized":"2023-11-28T21:34:44.868723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:35:21.006172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363765+0000","last_change":"2023-11-28T21:34:37.345652+0000","last_active":"2023-11-28T21:34:42.363765+0000","last_peered":"2023-11-28T21:34:42.363765+0000","last_clean":"2023-11-28T21:34:42.363765+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:34:42.363765+0000","last_undegraded":"2023-11-28T21:34:42.363765+0000","last_fullsized":"2023-11-28T21:34:42.363765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:21:17.474200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593118+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:34:42.593118+0000","last_peered":"2023-11-28T21:34:42.593118+0000","last_clean":"2023-11-28T21:34:42.593118+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:34:42.593118+0000","last_undegraded":"2023-11-28T21:34:42.593118+0000","last_fullsized":"2023-11-28T21:34:42.593118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:48.073446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.845834+0000","last_change":"2023-11-28T21:34:41.370764+0000","last_active":"2023-11-28T21:34:44.845834+0000","last_peered":"2023-11-28T21:34:44.845834+0000","last_clean":"2023-11-28T21:34:44.845834+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:34:44.845834+0000","last_undegraded":"2023-11-28T21:34:44.845834+0000","last_fullsized":"2023-11-28T21:34:44.845834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:06:42.297098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":2,"reported_epoch":25,"state":"peering","last_fresh":"2023-11-28T21:34:37.340685+0000","last_change":"2023-11-28T21:35:24.853154+0000","last_active":"2023-11-28T21:34:35.964297+0000","last_peered":"2023-11-28T21:34:35.964297+0000","last_clean":"2023-11-28T21:34:35.964297+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T21:34:37.340685+0000","last_undegraded":"2023-11-28T21:34:37.340685+0000","last_fullsized":"2023-11-28T21:34:37.340685+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360721+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:34:42.360721+0000","last_peered":"2023-11-28T21:34:42.360721+0000","last_clean":"2023-11-28T21:34:42.360721+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:34:42.360721+0000","last_undegraded":"2023-11-28T21:34:42.360721+0000","last_fullsized":"2023-11-28T21:34:42.360721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.847218+0000","last_change":"2023-11-28T21:34:41.477127+0000","last_active":"2023-11-28T21:34:44.847218+0000","last_peered":"2023-11-28T21:34:44.847218+0000","last_clean":"2023-11-28T21:34:44.847218+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:34:44.847218+0000","last_undegraded":"2023-11-28T21:34:44.847218+0000","last_fullsized":"2023-11-28T21:34:44.847218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:32.798817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.598763+0000","last_change":"2023-11-28T21:34:37.355693+0000","last_active":"2023-11-28T21:34:42.598763+0000","last_peered":"2023-11-28T21:34:42.598763+0000","last_clean":"2023-11-28T21:34:42.598763+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:34:42.598763+0000","last_undegraded":"2023-11-28T21:34:42.598763+0000","last_fullsized":"2023-11-28T21:34:42.598763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:12.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:19.726417+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:35:19.726417+0000","last_peered":"2023-11-28T21:35:19.726417+0000","last_clean":"2023-11-28T21:35:19.726417+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:35:19.726417+0000","last_undegraded":"2023-11-28T21:35:19.726417+0000","last_fullsized":"2023-11-28T21:35:19.726417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.848702+0000","last_change":"2023-11-28T21:34:41.382468+0000","last_active":"2023-11-28T21:34:44.848702+0000","last_peered":"2023-11-28T21:34:44.848702+0000","last_clean":"2023-11-28T21:34:44.848702+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:34:44.848702+0000","last_undegraded":"2023-11-28T21:34:44.848702+0000","last_fullsized":"2023-11-28T21:34:44.848702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:17.410943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597019+0000","last_change":"2023-11-28T21:34:37.353232+0000","last_active":"2023-11-28T21:34:42.597019+0000","last_peered":"2023-11-28T21:34:42.597019+0000","last_clean":"2023-11-28T21:34:42.597019+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:34:42.597019+0000","last_undegraded":"2023-11-28T21:34:42.597019+0000","last_fullsized":"2023-11-28T21:34:42.597019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:11.285378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'15","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.329784+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:35:14.329784+0000","last_peered":"2023-11-28T21:35:14.329784+0000","last_clean":"2023-11-28T21:35:14.329784+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:35:14.329784+0000","last_undegraded":"2023-11-28T21:35:14.329784+0000","last_fullsized":"2023-11-28T21:35:14.329784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.845922+0000","last_change":"2023-11-28T21:34:41.385400+0000","last_active":"2023-11-28T21:34:44.845922+0000","last_peered":"2023-11-28T21:34:44.845922+0000","last_clean":"2023-11-28T21:34:44.845922+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:34:44.845922+0000","last_undegraded":"2023-11-28T21:34:44.845922+0000","last_fullsized":"2023-11-28T21:34:44.845922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:36.433439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593571+0000","last_change":"2023-11-28T21:34:37.333461+0000","last_active":"2023-11-28T21:34:42.593571+0000","last_peered":"2023-11-28T21:34:42.593571+0000","last_clean":"2023-11-28T21:34:42.593571+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:34:42.593571+0000","last_undegraded":"2023-11-28T21:34:42.593571+0000","last_fullsized":"2023-11-28T21:34:42.593571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:25.513524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"24'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.337266+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:35:14.337266+0000","last_peered":"2023-11-28T21:35:14.337266+0000","last_clean":"2023-11-28T21:35:14.337266+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:35:14.337266+0000","last_undegraded":"2023-11-28T21:35:14.337266+0000","last_fullsized":"2023-11-28T21:35:14.337266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.849432+0000","last_change":"2023-11-28T21:34:41.479919+0000","last_active":"2023-11-28T21:34:44.849432+0000","last_peered":"2023-11-28T21:34:44.849432+0000","last_clean":"2023-11-28T21:34:44.849432+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:34:44.849432+0000","last_undegraded":"2023-11-28T21:34:44.849432+0000","last_fullsized":"2023-11-28T21:34:44.849432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:11.183117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597806+0000","last_change":"2023-11-28T21:34:37.355729+0000","last_active":"2023-11-28T21:34:42.597806+0000","last_peered":"2023-11-28T21:34:42.597806+0000","last_clean":"2023-11-28T21:34:42.597806+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:34:42.597806+0000","last_undegraded":"2023-11-28T21:34:42.597806+0000","last_fullsized":"2023-11-28T21:34:42.597806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:39.178378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"24'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.336108+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:35:08.336108+0000","last_peered":"2023-11-28T21:35:08.336108+0000","last_clean":"2023-11-28T21:35:08.336108+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:35:08.336108+0000","last_undegraded":"2023-11-28T21:35:08.336108+0000","last_fullsized":"2023-11-28T21:35:08.336108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.849333+0000","last_change":"2023-11-28T21:34:41.478483+0000","last_active":"2023-11-28T21:34:44.849333+0000","last_peered":"2023-11-28T21:34:44.849333+0000","last_clean":"2023-11-28T21:34:44.849333+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:34:44.849333+0000","last_undegraded":"2023-11-28T21:34:44.849333+0000","last_fullsized":"2023-11-28T21:34:44.849333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:25:54.342692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592059+0000","last_change":"2023-11-28T21:34:37.345476+0000","last_active":"2023-11-28T21:34:42.592059+0000","last_peered":"2023-11-28T21:34:42.592059+0000","last_clean":"2023-11-28T21:34:42.592059+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:34:42.592059+0000","last_undegraded":"2023-11-28T21:34:42.592059+0000","last_fullsized":"2023-11-28T21:34:42.592059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.660412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'6","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.328663+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:35:14.328663+0000","last_peered":"2023-11-28T21:35:14.328663+0000","last_clean":"2023-11-28T21:35:14.328663+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:35:14.328663+0000","last_undegraded":"2023-11-28T21:35:14.328663+0000","last_fullsized":"2023-11-28T21:35:14.328663+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:28.694184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.847611+0000","last_change":"2023-11-28T21:34:41.477926+0000","last_active":"2023-11-28T21:34:44.847611+0000","last_peered":"2023-11-28T21:34:44.847611+0000","last_clean":"2023-11-28T21:34:44.847611+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:34:44.847611+0000","last_undegraded":"2023-11-28T21:34:44.847611+0000","last_fullsized":"2023-11-28T21:34:44.847611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:26.403506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.598054+0000","last_change":"2023-11-28T21:34:37.343885+0000","last_active":"2023-11-28T21:34:42.598054+0000","last_peered":"2023-11-28T21:34:42.598054+0000","last_clean":"2023-11-28T21:34:42.598054+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:34:42.598054+0000","last_undegraded":"2023-11-28T21:34:42.598054+0000","last_fullsized":"2023-11-28T21:34:42.598054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:36.255834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'11","reported_seq":27,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.336398+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:35:08.336398+0000","last_peered":"2023-11-28T21:35:08.336398+0000","last_clean":"2023-11-28T21:35:08.336398+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:35:08.336398+0000","last_undegraded":"2023-11-28T21:35:08.336398+0000","last_fullsized":"2023-11-28T21:35:08.336398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":2,"reported_epoch":25,"state":"peering","last_fresh":"2023-11-28T21:34:41.342475+0000","last_change":"2023-11-28T21:35:24.855720+0000","last_active":"2023-11-28T21:34:40.321594+0000","last_peered":"2023-11-28T21:34:40.321594+0000","last_clean":"2023-11-28T21:34:40.321594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T21:34:41.342475+0000","last_undegraded":"2023-11-28T21:34:41.342475+0000","last_fullsized":"2023-11-28T21:34:41.342475+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.598158+0000","last_change":"2023-11-28T21:34:37.345635+0000","last_active":"2023-11-28T21:34:42.598158+0000","last_peered":"2023-11-28T21:34:42.598158+0000","last_clean":"2023-11-28T21:34:42.598158+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:34:42.598158+0000","last_undegraded":"2023-11-28T21:34:42.598158+0000","last_fullsized":"2023-11-28T21:34:42.598158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:52:12.144267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'4","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.338817+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:35:08.338817+0000","last_peered":"2023-11-28T21:35:08.338817+0000","last_clean":"2023-11-28T21:35:08.338817+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:35:08.338817+0000","last_undegraded":"2023-11-28T21:35:08.338817+0000","last_fullsized":"2023-11-28T21:35:08.338817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:35.266650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.848834+0000","last_change":"2023-11-28T21:34:41.365758+0000","last_active":"2023-11-28T21:34:44.848834+0000","last_peered":"2023-11-28T21:34:44.848834+0000","last_clean":"2023-11-28T21:34:44.848834+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:34:44.848834+0000","last_undegraded":"2023-11-28T21:34:44.848834+0000","last_fullsized":"2023-11-28T21:34:44.848834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592652+0000","last_change":"2023-11-28T21:34:37.333607+0000","last_active":"2023-11-28T21:34:42.592652+0000","last_peered":"2023-11-28T21:34:42.592652+0000","last_clean":"2023-11-28T21:34:42.592652+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:34:42.592652+0000","last_undegraded":"2023-11-28T21:34:42.592652+0000","last_fullsized":"2023-11-28T21:34:42.592652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:13:01.639908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"24'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.334532+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:35:08.334532+0000","last_peered":"2023-11-28T21:35:08.334532+0000","last_clean":"2023-11-28T21:35:08.334532+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:35:08.334532+0000","last_undegraded":"2023-11-28T21:35:08.334532+0000","last_fullsized":"2023-11-28T21:35:08.334532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.849180+0000","last_change":"2023-11-28T21:34:41.380413+0000","last_active":"2023-11-28T21:34:44.849180+0000","last_peered":"2023-11-28T21:34:44.849180+0000","last_clean":"2023-11-28T21:34:44.849180+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:34:44.849180+0000","last_undegraded":"2023-11-28T21:34:44.849180+0000","last_fullsized":"2023-11-28T21:34:44.849180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:20:28.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":2,"reported_epoch":25,"state":"peering","last_fresh":"2023-11-28T21:34:42.363343+0000","last_change":"2023-11-28T21:35:24.853086+0000","last_active":"2023-11-28T21:35:24.853086+0000","last_peered":"2023-11-28T21:34:42.363343+0000","last_clean":"2023-11-28T21:34:42.363343+0000","last_became_active":"2023-11-28T21:34:37.350321+0000","last_became_peered":"2023-11-28T21:34:37.350321+0000","last_unstale":"2023-11-28T21:34:42.363343+0000","last_undegraded":"2023-11-28T21:34:42.363343+0000","last_fullsized":"2023-11-28T21:34:42.363343+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"24'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.335962+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:35:08.335962+0000","last_peered":"2023-11-28T21:35:08.335962+0000","last_clean":"2023-11-28T21:35:08.335962+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:35:08.335962+0000","last_undegraded":"2023-11-28T21:35:08.335962+0000","last_fullsized":"2023-11-28T21:35:08.335962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.849230+0000","last_change":"2023-11-28T21:34:41.475185+0000","last_active":"2023-11-28T21:34:44.849230+0000","last_peered":"2023-11-28T21:34:44.849230+0000","last_clean":"2023-11-28T21:34:44.849230+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:34:44.849230+0000","last_undegraded":"2023-11-28T21:34:44.849230+0000","last_fullsized":"2023-11-28T21:34:44.849230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:09.389881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.845905+0000","last_change":"2023-11-28T21:34:41.479956+0000","last_active":"2023-11-28T21:34:44.845905+0000","last_peered":"2023-11-28T21:34:44.845905+0000","last_clean":"2023-11-28T21:34:44.845905+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:34:44.845905+0000","last_undegraded":"2023-11-28T21:34:44.845905+0000","last_fullsized":"2023-11-28T21:34:44.845905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:46.004852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"peering","last_fresh":"2023-11-28T21:34:42.591338+0000","last_change":"2023-11-28T21:35:24.573347+0000","last_active":"2023-11-28T21:35:24.573347+0000","last_peered":"2023-11-28T21:34:42.591338+0000","last_clean":"2023-11-28T21:34:42.591338+0000","last_became_active":"2023-11-28T21:34:37.317082+0000","last_became_peered":"2023-11-28T21:34:37.317082+0000","last_unstale":"2023-11-28T21:34:42.591338+0000","last_undegraded":"2023-11-28T21:34:42.591338+0000","last_fullsized":"2023-11-28T21:34:42.591338+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"24'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.331572+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:35:08.331572+0000","last_peered":"2023-11-28T21:35:08.331572+0000","last_clean":"2023-11-28T21:35:08.331572+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:35:08.331572+0000","last_undegraded":"2023-11-28T21:35:08.331572+0000","last_fullsized":"2023-11-28T21:35:08.331572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.868893+0000","last_change":"2023-11-28T21:34:41.385969+0000","last_active":"2023-11-28T21:34:44.868893+0000","last_peered":"2023-11-28T21:34:44.868893+0000","last_clean":"2023-11-28T21:34:44.868893+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:34:44.868893+0000","last_undegraded":"2023-11-28T21:34:44.868893+0000","last_fullsized":"2023-11-28T21:34:44.868893+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:01.692549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359928+0000","last_change":"2023-11-28T21:34:37.314087+0000","last_active":"2023-11-28T21:34:42.359928+0000","last_peered":"2023-11-28T21:34:42.359928+0000","last_clean":"2023-11-28T21:34:42.359928+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:34:42.359928+0000","last_undegraded":"2023-11-28T21:34:42.359928+0000","last_fullsized":"2023-11-28T21:34:42.359928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:25:11.912313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"24'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.333569+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:35:14.333569+0000","last_peered":"2023-11-28T21:35:14.333569+0000","last_clean":"2023-11-28T21:35:14.333569+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:35:14.333569+0000","last_undegraded":"2023-11-28T21:35:14.333569+0000","last_fullsized":"2023-11-28T21:35:14.333569+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:14.334068+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:35:14.334068+0000","last_peered":"2023-11-28T21:35:14.334068+0000","last_clean":"2023-11-28T21:35:14.334068+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:35:14.334068+0000","last_undegraded":"2023-11-28T21:35:14.334068+0000","last_fullsized":"2023-11-28T21:35:14.334068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592449+0000","last_change":"2023-11-28T21:34:37.319801+0000","last_active":"2023-11-28T21:34:42.592449+0000","last_peered":"2023-11-28T21:34:42.592449+0000","last_clean":"2023-11-28T21:34:42.592449+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:34:42.592449+0000","last_undegraded":"2023-11-28T21:34:42.592449+0000","last_fullsized":"2023-11-28T21:34:42.592449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:40:13.251522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.849245+0000","last_change":"2023-11-28T21:34:41.370523+0000","last_active":"2023-11-28T21:34:44.849245+0000","last_peered":"2023-11-28T21:34:44.849245+0000","last_clean":"2023-11-28T21:34:44.849245+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:34:44.849245+0000","last_undegraded":"2023-11-28T21:34:44.849245+0000","last_fullsized":"2023-11-28T21:34:44.849245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:17:36.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'5","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:35:08.338623+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:35:08.338623+0000","last_peered":"2023-11-28T21:35:08.338623+0000","last_clean":"2023-11-28T21:35:08.338623+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:35:08.338623+0000","last_undegraded":"2023-11-28T21:35:08.338623+0000","last_fullsized":"2023-11-28T21:35:08.338623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595577+0000","last_change":"2023-11-28T21:34:37.328809+0000","last_active":"2023-11-28T21:34:42.595577+0000","last_peered":"2023-11-28T21:34:42.595577+0000","last_clean":"2023-11-28T21:34:42.595577+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:34:42.595577+0000","last_undegraded":"2023-11-28T21:34:42.595577+0000","last_fullsized":"2023-11-28T21:34:42.595577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:27.300090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:44.849368+0000","last_change":"2023-11-28T21:34:41.363087+0000","last_active":"2023-11-28T21:34:44.849368+0000","last_peered":"2023-11-28T21:34:44.849368+0000","last_clean":"2023-11-28T21:34:44.849368+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:34:44.849368+0000","last_undegraded":"2023-11-28T21:34:44.849368+0000","last_fullsized":"2023-11-28T21:34:44.849368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:35.046132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593310+0000","last_change":"2023-11-28T21:34:37.333670+0000","last_active":"2023-11-28T21:34:42.593310+0000","last_peered":"2023-11-28T21:34:42.593310+0000","last_clean":"2023-11-28T21:34:42.593310+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:34:42.593310+0000","last_undegraded":"2023-11-28T21:34:42.593310+0000","last_fullsized":"2023-11-28T21:34:42.593310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:32.598014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360328+0000","last_change":"2023-11-28T21:34:37.347101+0000","last_active":"2023-11-28T21:34:42.360328+0000","last_peered":"2023-11-28T21:34:42.360328+0000","last_clean":"2023-11-28T21:34:42.360328+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:34:42.360328+0000","last_undegraded":"2023-11-28T21:34:42.360328+0000","last_fullsized":"2023-11-28T21:34:42.360328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:39:43.186110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363133+0000","last_change":"2023-11-28T21:34:37.338576+0000","last_active":"2023-11-28T21:34:42.363133+0000","last_peered":"2023-11-28T21:34:42.363133+0000","last_clean":"2023-11-28T21:34:42.363133+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:34:42.363133+0000","last_undegraded":"2023-11-28T21:34:42.363133+0000","last_fullsized":"2023-11-28T21:34:42.363133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:53:02.356097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592408+0000","last_change":"2023-11-28T21:34:37.316270+0000","last_active":"2023-11-28T21:34:42.592408+0000","last_peered":"2023-11-28T21:34:42.592408+0000","last_clean":"2023-11-28T21:34:42.592408+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:34:42.592408+0000","last_undegraded":"2023-11-28T21:34:42.592408+0000","last_fullsized":"2023-11-28T21:34:42.592408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:43.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361466+0000","last_change":"2023-11-28T21:34:37.334973+0000","last_active":"2023-11-28T21:34:42.361466+0000","last_peered":"2023-11-28T21:34:42.361466+0000","last_clean":"2023-11-28T21:34:42.361466+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:34:42.361466+0000","last_undegraded":"2023-11-28T21:34:42.361466+0000","last_fullsized":"2023-11-28T21:34:42.361466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:55.050945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592949+0000","last_change":"2023-11-28T21:34:37.318155+0000","last_active":"2023-11-28T21:34:42.592949+0000","last_peered":"2023-11-28T21:34:42.592949+0000","last_clean":"2023-11-28T21:34:42.592949+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:34:42.592949+0000","last_undegraded":"2023-11-28T21:34:42.592949+0000","last_fullsized":"2023-11-28T21:34:42.592949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:29.840255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595670+0000","last_change":"2023-11-28T21:34:37.345758+0000","last_active":"2023-11-28T21:34:42.595670+0000","last_peered":"2023-11-28T21:34:42.595670+0000","last_clean":"2023-11-28T21:34:42.595670+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:34:42.595670+0000","last_undegraded":"2023-11-28T21:34:42.595670+0000","last_fullsized":"2023-11-28T21:34:42.595670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:26.444654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.361507+0000","last_change":"2023-11-28T21:34:37.334989+0000","last_active":"2023-11-28T21:34:42.361507+0000","last_peered":"2023-11-28T21:34:42.361507+0000","last_clean":"2023-11-28T21:34:42.361507+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:34:42.361507+0000","last_undegraded":"2023-11-28T21:34:42.361507+0000","last_fullsized":"2023-11-28T21:34:42.361507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:36:57.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.362699+0000","last_change":"2023-11-28T21:34:37.333075+0000","last_active":"2023-11-28T21:34:42.362699+0000","last_peered":"2023-11-28T21:34:42.362699+0000","last_clean":"2023-11-28T21:34:42.362699+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:34:42.362699+0000","last_undegraded":"2023-11-28T21:34:42.362699+0000","last_fullsized":"2023-11-28T21:34:42.362699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:50:43.748349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360000+0000","last_change":"2023-11-28T21:34:37.337230+0000","last_active":"2023-11-28T21:34:42.360000+0000","last_peered":"2023-11-28T21:34:42.360000+0000","last_clean":"2023-11-28T21:34:42.360000+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:34:42.360000+0000","last_undegraded":"2023-11-28T21:34:42.360000+0000","last_fullsized":"2023-11-28T21:34:42.360000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:48:45.994559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363063+0000","last_change":"2023-11-28T21:34:37.349894+0000","last_active":"2023-11-28T21:34:42.363063+0000","last_peered":"2023-11-28T21:34:42.363063+0000","last_clean":"2023-11-28T21:34:42.363063+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:34:42.363063+0000","last_undegraded":"2023-11-28T21:34:42.363063+0000","last_fullsized":"2023-11-28T21:34:42.363063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:10.293911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592225+0000","last_change":"2023-11-28T21:34:37.344269+0000","last_active":"2023-11-28T21:34:42.592225+0000","last_peered":"2023-11-28T21:34:42.592225+0000","last_clean":"2023-11-28T21:34:42.592225+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:34:42.592225+0000","last_undegraded":"2023-11-28T21:34:42.592225+0000","last_fullsized":"2023-11-28T21:34:42.592225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:03.204740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595046+0000","last_change":"2023-11-28T21:34:37.353059+0000","last_active":"2023-11-28T21:34:42.595046+0000","last_peered":"2023-11-28T21:34:42.595046+0000","last_clean":"2023-11-28T21:34:42.595046+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:34:42.595046+0000","last_undegraded":"2023-11-28T21:34:42.595046+0000","last_fullsized":"2023-11-28T21:34:42.595046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:45:12.640797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.598547+0000","last_change":"2023-11-28T21:34:37.352965+0000","last_active":"2023-11-28T21:34:42.598547+0000","last_peered":"2023-11-28T21:34:42.598547+0000","last_clean":"2023-11-28T21:34:42.598547+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:34:42.598547+0000","last_undegraded":"2023-11-28T21:34:42.598547+0000","last_fullsized":"2023-11-28T21:34:42.598547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:30:42.087715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360636+0000","last_change":"2023-11-28T21:34:37.336830+0000","last_active":"2023-11-28T21:34:42.360636+0000","last_peered":"2023-11-28T21:34:42.360636+0000","last_clean":"2023-11-28T21:34:42.360636+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:34:42.360636+0000","last_undegraded":"2023-11-28T21:34:42.360636+0000","last_fullsized":"2023-11-28T21:34:42.360636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:57.318311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.364131+0000","last_change":"2023-11-28T21:34:37.349181+0000","last_active":"2023-11-28T21:34:42.364131+0000","last_peered":"2023-11-28T21:34:42.364131+0000","last_clean":"2023-11-28T21:34:42.364131+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:34:42.364131+0000","last_undegraded":"2023-11-28T21:34:42.364131+0000","last_fullsized":"2023-11-28T21:34:42.364131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:27:59.989299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.362529+0000","last_change":"2023-11-28T21:34:37.336668+0000","last_active":"2023-11-28T21:34:42.362529+0000","last_peered":"2023-11-28T21:34:42.362529+0000","last_clean":"2023-11-28T21:34:42.362529+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:34:42.362529+0000","last_undegraded":"2023-11-28T21:34:42.362529+0000","last_fullsized":"2023-11-28T21:34:42.362529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:06.416717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.591421+0000","last_change":"2023-11-28T21:34:37.332790+0000","last_active":"2023-11-28T21:34:42.591421+0000","last_peered":"2023-11-28T21:34:42.591421+0000","last_clean":"2023-11-28T21:34:42.591421+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:34:42.591421+0000","last_undegraded":"2023-11-28T21:34:42.591421+0000","last_fullsized":"2023-11-28T21:34:42.591421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:32:45.920387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592031+0000","last_change":"2023-11-28T21:34:37.320794+0000","last_active":"2023-11-28T21:34:42.592031+0000","last_peered":"2023-11-28T21:34:42.592031+0000","last_clean":"2023-11-28T21:34:42.592031+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:34:42.592031+0000","last_undegraded":"2023-11-28T21:34:42.592031+0000","last_fullsized":"2023-11-28T21:34:42.592031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:16:46.347683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.360729+0000","last_change":"2023-11-28T21:34:37.337726+0000","last_active":"2023-11-28T21:34:42.360729+0000","last_peered":"2023-11-28T21:34:42.360729+0000","last_clean":"2023-11-28T21:34:42.360729+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:34:42.360729+0000","last_undegraded":"2023-11-28T21:34:42.360729+0000","last_fullsized":"2023-11-28T21:34:42.360729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:38:43.884668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597955+0000","last_change":"2023-11-28T21:34:37.334677+0000","last_active":"2023-11-28T21:34:42.597955+0000","last_peered":"2023-11-28T21:34:42.597955+0000","last_clean":"2023-11-28T21:34:42.597955+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:34:42.597955+0000","last_undegraded":"2023-11-28T21:34:42.597955+0000","last_fullsized":"2023-11-28T21:34:42.597955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:05:54.981899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.595140+0000","last_change":"2023-11-28T21:34:37.354861+0000","last_active":"2023-11-28T21:34:42.595140+0000","last_peered":"2023-11-28T21:34:42.595140+0000","last_clean":"2023-11-28T21:34:42.595140+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:34:42.595140+0000","last_undegraded":"2023-11-28T21:34:42.595140+0000","last_fullsized":"2023-11-28T21:34:42.595140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:30.602284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.594790+0000","last_change":"2023-11-28T21:34:37.347675+0000","last_active":"2023-11-28T21:34:42.594790+0000","last_peered":"2023-11-28T21:34:42.594790+0000","last_clean":"2023-11-28T21:34:42.594790+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:34:42.594790+0000","last_undegraded":"2023-11-28T21:34:42.594790+0000","last_fullsized":"2023-11-28T21:34:42.594790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:51:38.803896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"peering","last_fresh":"2023-11-28T21:34:42.599231+0000","last_change":"2023-11-28T21:35:24.589218+0000","last_active":"2023-11-28T21:35:24.589218+0000","last_peered":"2023-11-28T21:34:42.599231+0000","last_clean":"2023-11-28T21:34:42.599231+0000","last_became_active":"2023-11-28T21:34:37.357173+0000","last_became_peered":"2023-11-28T21:34:37.357173+0000","last_unstale":"2023-11-28T21:34:42.599231+0000","last_undegraded":"2023-11-28T21:34:42.599231+0000","last_fullsized":"2023-11-28T21:34:42.599231+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.359842+0000","last_change":"2023-11-28T21:34:37.358851+0000","last_active":"2023-11-28T21:34:42.359842+0000","last_peered":"2023-11-28T21:34:42.359842+0000","last_clean":"2023-11-28T21:34:42.359842+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:34:42.359842+0000","last_undegraded":"2023-11-28T21:34:42.359842+0000","last_fullsized":"2023-11-28T21:34:42.359842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.378519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.596821+0000","last_change":"2023-11-28T21:34:37.343972+0000","last_active":"2023-11-28T21:34:42.596821+0000","last_peered":"2023-11-28T21:34:42.596821+0000","last_clean":"2023-11-28T21:34:42.596821+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:34:42.596821+0000","last_undegraded":"2023-11-28T21:34:42.596821+0000","last_fullsized":"2023-11-28T21:34:42.596821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:56:00.591888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.362934+0000","last_change":"2023-11-28T21:34:37.346299+0000","last_active":"2023-11-28T21:34:42.362934+0000","last_peered":"2023-11-28T21:34:42.362934+0000","last_clean":"2023-11-28T21:34:42.362934+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:34:42.362934+0000","last_undegraded":"2023-11-28T21:34:42.362934+0000","last_fullsized":"2023-11-28T21:34:42.362934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:06.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.597196+0000","last_change":"2023-11-28T21:34:37.343410+0000","last_active":"2023-11-28T21:34:42.597196+0000","last_peered":"2023-11-28T21:34:42.597196+0000","last_clean":"2023-11-28T21:34:42.597196+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:34:42.597196+0000","last_undegraded":"2023-11-28T21:34:42.597196+0000","last_fullsized":"2023-11-28T21:34:42.597196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:29:45.070478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.363576+0000","last_change":"2023-11-28T21:34:37.348270+0000","last_active":"2023-11-28T21:34:42.363576+0000","last_peered":"2023-11-28T21:34:42.363576+0000","last_clean":"2023-11-28T21:34:42.363576+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:34:42.363576+0000","last_undegraded":"2023-11-28T21:34:42.363576+0000","last_fullsized":"2023-11-28T21:34:42.363576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:36:10.938478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.592223+0000","last_change":"2023-11-28T21:34:37.330786+0000","last_active":"2023-11-28T21:34:42.592223+0000","last_peered":"2023-11-28T21:34:42.592223+0000","last_clean":"2023-11-28T21:34:42.592223+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:34:42.592223+0000","last_undegraded":"2023-11-28T21:34:42.592223+0000","last_fullsized":"2023-11-28T21:34:42.592223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:52:36.738867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.593287+0000","last_change":"2023-11-28T21:34:37.318725+0000","last_active":"2023-11-28T21:34:42.593287+0000","last_peered":"2023-11-28T21:34:42.593287+0000","last_clean":"2023-11-28T21:34:42.593287+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:34:42.593287+0000","last_undegraded":"2023-11-28T21:34:42.593287+0000","last_fullsized":"2023-11-28T21:34:42.593287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:49:21.775066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T21:34:42.599013+0000","last_change":"2023-11-28T21:34:37.349829+0000","last_active":"2023-11-28T21:34:42.599013+0000","last_peered":"2023-11-28T21:34:42.599013+0000","last_clean":"2023-11-28T21:34:42.599013+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:34:42.599013+0000","last_undegraded":"2023-11-28T21:34:42.599013+0000","last_fullsized":"2023-11-28T21:34:42.599013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:07.752862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7451,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":14902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":339968,"data_stored":691476,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":111377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":108236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":100506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":104345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":100610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":107802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":696867,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:35:31.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:35:32.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6f5409640 1 Processor -- start 2023-11-28T21:35:32.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6f5409640 1 -- start start 2023-11-28T21:35:32.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6f5409640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb6f00ca110 0x7fb6f00cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6f5409640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f015acc0 con 0x7fb6f0153e40 2023-11-28T21:35:32.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6f5409640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f0159ea0 con 0x7fb6f00cfba0 2023-11-28T21:35:32.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6f5409640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f00ca520 con 0x7fb6f00ca110 2023-11-28T21:35:32.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6eeffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb6f0153e40 0x7fb6f015dc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45238/0 (socket says 172.21.15.12:45238) 2023-11-28T21:35:32.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6eeffd640 1 -- 172.21.15.12:0/439756288 learned_addr learned my addr 172.21.15.12:0/439756288 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:32.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6ee7fc640 1 --2- 172.21.15.12:0/439756288 >> v2:172.21.15.144:3300/0 conn(0x7fb6f00ca110 0x7fb6f00cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6edffb640 1 -- 172.21.15.12:0/439756288 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888515513 0 0) 0x7fb6f015acc0 con 0x7fb6f0153e40 2023-11-28T21:35:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6edffb640 1 -- 172.21.15.12:0/439756288 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6d8003530 con 0x7fb6f0153e40 2023-11-28T21:35:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6ee7fc640 1 -- 172.21.15.12:0/439756288 >> v1:172.21.15.144:6789/0 conn(0x7fb6f00cfba0 legacy=0x7fb6f00cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6ee7fc640 1 -- 172.21.15.12:0/439756288 >> v1:172.21.15.12:6789/0 conn(0x7fb6f0153e40 legacy=0x7fb6f015dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.012+0000 7fb6ee7fc640 1 -- 172.21.15.12:0/439756288 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f015a140 con 0x7fb6f00ca110 2023-11-28T21:35:32.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6ee7fc640 1 --2- 172.21.15.12:0/439756288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6f00ca110 0x7fb6f00cf660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40097c0 tx=0x7fb6e401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad4f284f9a851471 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:32.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6edffb640 1 -- 172.21.15.12:0/439756288 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40ab600 con 0x7fb6f00ca110 2023-11-28T21:35:32.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6edffb640 1 -- 172.21.15.12:0/439756288 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e40a7050 con 0x7fb6f00ca110 2023-11-28T21:35:32.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6edffb640 1 -- 172.21.15.12:0/439756288 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40ab960 con 0x7fb6f00ca110 2023-11-28T21:35:32.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6f5409640 1 -- 172.21.15.12:0/439756288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6f00ca110 msgr2=0x7fb6f00cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6f5409640 1 --2- 172.21.15.12:0/439756288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6f00ca110 0x7fb6f00cf660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40097c0 tx=0x7fb6e401c480 comp rx=0 tx=0).stop 2023-11-28T21:35:32.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6f5409640 1 -- 172.21.15.12:0/439756288 shutdown_connections 2023-11-28T21:35:32.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6f5409640 1 --2- 172.21.15.12:0/439756288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6f00ca110 0x7fb6f00cf660 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6f5409640 1 -- 172.21.15.12:0/439756288 >> 172.21.15.12:0/439756288 conn(0x7fb6f006b7b0 msgr2=0x7fb6f0066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:32.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6f5409640 1 -- 172.21.15.12:0/439756288 shutdown_connections 2023-11-28T21:35:32.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.016+0000 7fb6f5409640 1 -- 172.21.15.12:0/439756288 wait complete. 2023-11-28T21:35:32.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6f5409640 1 Processor -- start 2023-11-28T21:35:32.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6f5409640 1 -- start start 2023-11-28T21:35:32.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6f5409640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6f00ca110 0x7fb6f00b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6f5409640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6f00cfba0 0x7fb6f01f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6f5409640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f0153e40 0x7fb6f01f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6f5409640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f015d400 con 0x7fb6f0153e40 2023-11-28T21:35:32.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6f5409640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f014d710 con 0x7fb6f00ca110 2023-11-28T21:35:32.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6f5409640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f00b3c60 con 0x7fb6f00cfba0 2023-11-28T21:35:32.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f0153e40 0x7fb6f01f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f0153e40 0x7fb6f01f5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41710/0 (socket says 172.21.15.12:41710) 2023-11-28T21:35:32.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6ef7fe640 1 -- 172.21.15.12:0/3212177778 learned_addr learned my addr 172.21.15.12:0/3212177778 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:32.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6ee7fc640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6f00cfba0 0x7fb6f01f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6eeffd640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6f00ca110 0x7fb6f00b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6ef7fe640 1 -- 172.21.15.12:0/3212177778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6f00cfba0 msgr2=0x7fb6f01f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6ef7fe640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6f00cfba0 0x7fb6f01f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6ef7fe640 1 -- 172.21.15.12:0/3212177778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6f00ca110 msgr2=0x7fb6f00b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6ef7fe640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6f00ca110 0x7fb6f00b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6ef7fe640 1 -- 172.21.15.12:0/3212177778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e4009420 con 0x7fb6f0153e40 2023-11-28T21:35:32.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6ef7fe640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f0153e40 0x7fb6f01f5e30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c710 tx=0x7fb6e800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:32.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6d77fe640 1 -- 172.21.15.12:0/3212177778 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8006d60 con 0x7fb6f0153e40 2023-11-28T21:35:32.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6d77fe640 1 -- 172.21.15.12:0/3212177778 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e800ce20 con 0x7fb6f0153e40 2023-11-28T21:35:32.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6f5409640 1 -- 172.21.15.12:0/3212177778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f014ff50 con 0x7fb6f0153e40 2023-11-28T21:35:32.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6d77fe640 1 -- 172.21.15.12:0/3212177778 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8003bc0 con 0x7fb6f0153e40 2023-11-28T21:35:32.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.020+0000 7fb6f5409640 1 -- 172.21.15.12:0/3212177778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f00b3670 con 0x7fb6f0153e40 2023-11-28T21:35:32.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.024+0000 7fb6f5409640 1 -- 172.21.15.12:0/3212177778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f0074f60 con 0x7fb6f0153e40 2023-11-28T21:35:32.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.024+0000 7fb6d77fe640 1 -- 172.21.15.12:0/3212177778 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb6e8003d20 con 0x7fb6f0153e40 2023-11-28T21:35:32.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.024+0000 7fb6d77fe640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb6dc066470 0x7fb6dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.024+0000 7fb6d77fe640 1 -- 172.21.15.12:0/3212177778 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb6e8083d40 con 0x7fb6f0153e40 2023-11-28T21:35:32.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.024+0000 7fb6eeffd640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb6dc066470 0x7fb6dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.024+0000 7fb6eeffd640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb6dc066470 0x7fb6dc068930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00011f0 tx=0x7fb6e0007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:32.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.028+0000 7fb6d77fe640 1 -- 172.21.15.12:0/3212177778 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb6e8054930 con 0x7fb6f0153e40 2023-11-28T21:35:32.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.180+0000 7fb6f5409640 1 -- 172.21.15.12:0/3212177778 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6f006cc80 con 0x7fb6dc066470 2023-11-28T21:35:32.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.192+0000 7fb6d77fe640 1 -- 172.21.15.12:0/3212177778 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+351421 (secure 0 0 0) 0x7fb6e0005860 con 0x7fb6dc066470 2023-11-28T21:35:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:35:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 -- 172.21.15.12:0/3212177778 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb6dc066470 msgr2=0x7fb6dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb6dc066470 0x7fb6dc068930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00011f0 tx=0x7fb6e0007d40 comp rx=0 tx=0).stop 2023-11-28T21:35:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 -- 172.21.15.12:0/3212177778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f0153e40 msgr2=0x7fb6f01f5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f0153e40 0x7fb6f01f5e30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c710 tx=0x7fb6e800cbd0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 -- 172.21.15.12:0/3212177778 shutdown_connections 2023-11-28T21:35:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb6dc066470 0x7fb6dc068930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f0153e40 0x7fb6f01f5e30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6f00cfba0 0x7fb6f01f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 --2- 172.21.15.12:0/3212177778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6f00ca110 0x7fb6f00b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 -- 172.21.15.12:0/3212177778 >> 172.21.15.12:0/3212177778 conn(0x7fb6f006b7b0 msgr2=0x7fb6f0150cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:32.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 -- 172.21.15.12:0/3212177778 shutdown_connections 2023-11-28T21:35:32.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.196+0000 7fb6f5409640 1 -- 172.21.15.12:0/3212177778 wait complete. 2023-11-28T21:35:32.218 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2023-11-28T21:35:31.828476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598073,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":323,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218628,"kb_used_data":2460,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756092,"statfs":{"total":773094113280,"available":772870238208,"internally_reserved":0,"allocated":2519040,"data_stored":2105215,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":254,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001538"},"pg_stats":[{"pgid":"2.1f","version":"24'9","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.580953+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:35:24.580953+0000","last_peered":"2023-11-28T21:35:24.580953+0000","last_clean":"2023-11-28T21:35:24.580953+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:35:24.580953+0000","last_undegraded":"2023-11-28T21:35:24.580953+0000","last_fullsized":"2023-11-28T21:35:24.580953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:46.167511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570318+0000","last_change":"2023-11-28T21:34:37.335656+0000","last_active":"2023-11-28T21:35:24.570318+0000","last_peered":"2023-11-28T21:35:24.570318+0000","last_clean":"2023-11-28T21:35:24.570318+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:35:24.570318+0000","last_undegraded":"2023-11-28T21:35:24.570318+0000","last_fullsized":"2023-11-28T21:35:24.570318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:25:49.707378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569336+0000","last_change":"2023-11-28T21:34:41.483856+0000","last_active":"2023-11-28T21:35:24.569336+0000","last_peered":"2023-11-28T21:35:24.569336+0000","last_clean":"2023-11-28T21:35:24.569336+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:35:24.569336+0000","last_undegraded":"2023-11-28T21:35:24.569336+0000","last_fullsized":"2023-11-28T21:35:24.569336+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:50:56.005578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570516+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:35:24.570516+0000","last_peered":"2023-11-28T21:35:24.570516+0000","last_clean":"2023-11-28T21:35:24.570516+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:35:24.570516+0000","last_undegraded":"2023-11-28T21:35:24.570516+0000","last_fullsized":"2023-11-28T21:35:24.570516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.580664+0000","last_change":"2023-11-28T21:34:37.333649+0000","last_active":"2023-11-28T21:35:24.580664+0000","last_peered":"2023-11-28T21:35:24.580664+0000","last_clean":"2023-11-28T21:35:24.580664+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:35:24.580664+0000","last_undegraded":"2023-11-28T21:35:24.580664+0000","last_fullsized":"2023-11-28T21:35:24.580664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:21:42.041590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578771+0000","last_change":"2023-11-28T21:34:41.388232+0000","last_active":"2023-11-28T21:35:24.578771+0000","last_peered":"2023-11-28T21:35:24.578771+0000","last_clean":"2023-11-28T21:35:24.578771+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:35:24.578771+0000","last_undegraded":"2023-11-28T21:35:24.578771+0000","last_fullsized":"2023-11-28T21:35:24.578771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:22.960012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590922+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:35:24.590922+0000","last_peered":"2023-11-28T21:35:24.590922+0000","last_clean":"2023-11-28T21:35:24.590922+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:35:24.590922+0000","last_undegraded":"2023-11-28T21:35:24.590922+0000","last_fullsized":"2023-11-28T21:35:24.590922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.576038+0000","last_change":"2023-11-28T21:34:37.348126+0000","last_active":"2023-11-28T21:35:24.576038+0000","last_peered":"2023-11-28T21:35:24.576038+0000","last_clean":"2023-11-28T21:35:24.576038+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:35:24.576038+0000","last_undegraded":"2023-11-28T21:35:24.576038+0000","last_fullsized":"2023-11-28T21:35:24.576038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:47.982143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581423+0000","last_change":"2023-11-28T21:34:41.388539+0000","last_active":"2023-11-28T21:35:24.581423+0000","last_peered":"2023-11-28T21:35:24.581423+0000","last_clean":"2023-11-28T21:35:24.581423+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:35:24.581423+0000","last_undegraded":"2023-11-28T21:35:24.581423+0000","last_fullsized":"2023-11-28T21:35:24.581423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.589668+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:35:24.589668+0000","last_peered":"2023-11-28T21:35:24.589668+0000","last_clean":"2023-11-28T21:35:24.589668+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:35:24.589668+0000","last_undegraded":"2023-11-28T21:35:24.589668+0000","last_fullsized":"2023-11-28T21:35:24.589668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579763+0000","last_change":"2023-11-28T21:34:37.337266+0000","last_active":"2023-11-28T21:35:24.579763+0000","last_peered":"2023-11-28T21:35:24.579763+0000","last_clean":"2023-11-28T21:35:24.579763+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:35:24.579763+0000","last_undegraded":"2023-11-28T21:35:24.579763+0000","last_fullsized":"2023-11-28T21:35:24.579763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:32:48.848989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574284+0000","last_change":"2023-11-28T21:34:41.372957+0000","last_active":"2023-11-28T21:35:24.574284+0000","last_peered":"2023-11-28T21:35:24.574284+0000","last_clean":"2023-11-28T21:35:24.574284+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:35:24.574284+0000","last_undegraded":"2023-11-28T21:35:24.574284+0000","last_fullsized":"2023-11-28T21:35:24.574284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.583250+0000","last_change":"2023-11-28T21:34:37.353174+0000","last_active":"2023-11-28T21:35:24.583250+0000","last_peered":"2023-11-28T21:35:24.583250+0000","last_clean":"2023-11-28T21:35:24.583250+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:35:24.583250+0000","last_undegraded":"2023-11-28T21:35:24.583250+0000","last_fullsized":"2023-11-28T21:35:24.583250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:32.966564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590181+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:35:24.590181+0000","last_peered":"2023-11-28T21:35:24.590181+0000","last_clean":"2023-11-28T21:35:24.590181+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:35:24.590181+0000","last_undegraded":"2023-11-28T21:35:24.590181+0000","last_fullsized":"2023-11-28T21:35:24.590181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.618289+0000","last_change":"2023-11-28T21:35:25.618289+0000","last_active":"2023-11-28T21:35:25.618289+0000","last_peered":"2023-11-28T21:35:25.618289+0000","last_clean":"2023-11-28T21:35:25.618289+0000","last_became_active":"2023-11-28T21:35:25.616978+0000","last_became_peered":"2023-11-28T21:35:25.616978+0000","last_unstale":"2023-11-28T21:35:25.618289+0000","last_undegraded":"2023-11-28T21:35:25.618289+0000","last_fullsized":"2023-11-28T21:35:25.618289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:15:26.380543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570019+0000","last_change":"2023-11-28T21:34:37.313495+0000","last_active":"2023-11-28T21:35:24.570019+0000","last_peered":"2023-11-28T21:35:24.570019+0000","last_clean":"2023-11-28T21:35:24.570019+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:35:24.570019+0000","last_undegraded":"2023-11-28T21:35:24.570019+0000","last_fullsized":"2023-11-28T21:35:24.570019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:32:50.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"24'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573439+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:35:24.573439+0000","last_peered":"2023-11-28T21:35:24.573439+0000","last_clean":"2023-11-28T21:35:24.573439+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:35:24.573439+0000","last_undegraded":"2023-11-28T21:35:24.573439+0000","last_fullsized":"2023-11-28T21:35:24.573439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:53.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.792110+0000","last_change":"2023-11-28T21:35:25.792110+0000","last_active":"2023-11-28T21:35:25.792110+0000","last_peered":"2023-11-28T21:35:25.792110+0000","last_clean":"2023-11-28T21:35:25.792110+0000","last_became_active":"2023-11-28T21:35:25.790674+0000","last_became_peered":"2023-11-28T21:35:25.790674+0000","last_unstale":"2023-11-28T21:35:25.792110+0000","last_undegraded":"2023-11-28T21:35:25.792110+0000","last_fullsized":"2023-11-28T21:35:25.792110+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:22:43.106775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569482+0000","last_change":"2023-11-28T21:34:37.326569+0000","last_active":"2023-11-28T21:35:24.569482+0000","last_peered":"2023-11-28T21:35:24.569482+0000","last_clean":"2023-11-28T21:35:24.569482+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:35:24.569482+0000","last_undegraded":"2023-11-28T21:35:24.569482+0000","last_fullsized":"2023-11-28T21:35:24.569482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:39:14.264355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573928+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:35:24.573928+0000","last_peered":"2023-11-28T21:35:24.573928+0000","last_clean":"2023-11-28T21:35:24.573928+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:35:24.573928+0000","last_undegraded":"2023-11-28T21:35:24.573928+0000","last_fullsized":"2023-11-28T21:35:24.573928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574972+0000","last_change":"2023-11-28T21:34:41.483661+0000","last_active":"2023-11-28T21:35:24.574972+0000","last_peered":"2023-11-28T21:35:24.574972+0000","last_clean":"2023-11-28T21:35:24.574972+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:35:24.574972+0000","last_undegraded":"2023-11-28T21:35:24.574972+0000","last_fullsized":"2023-11-28T21:35:24.574972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574562+0000","last_change":"2023-11-28T21:34:37.336922+0000","last_active":"2023-11-28T21:35:24.574562+0000","last_peered":"2023-11-28T21:35:24.574562+0000","last_clean":"2023-11-28T21:35:24.574562+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:35:24.574562+0000","last_undegraded":"2023-11-28T21:35:24.574562+0000","last_fullsized":"2023-11-28T21:35:24.574562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:45.663475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"24'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588244+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:35:24.588244+0000","last_peered":"2023-11-28T21:35:24.588244+0000","last_clean":"2023-11-28T21:35:24.588244+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:35:24.588244+0000","last_undegraded":"2023-11-28T21:35:24.588244+0000","last_fullsized":"2023-11-28T21:35:24.588244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581947+0000","last_change":"2023-11-28T21:34:41.382756+0000","last_active":"2023-11-28T21:35:24.581947+0000","last_peered":"2023-11-28T21:35:24.581947+0000","last_clean":"2023-11-28T21:35:24.581947+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:35:24.581947+0000","last_undegraded":"2023-11-28T21:35:24.581947+0000","last_fullsized":"2023-11-28T21:35:24.581947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:07.402513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579759+0000","last_change":"2023-11-28T21:34:37.320446+0000","last_active":"2023-11-28T21:35:24.579759+0000","last_peered":"2023-11-28T21:35:24.579759+0000","last_clean":"2023-11-28T21:35:24.579759+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:35:24.579759+0000","last_undegraded":"2023-11-28T21:35:24.579759+0000","last_fullsized":"2023-11-28T21:35:24.579759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:33.404441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'4","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.587700+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:35:24.587700+0000","last_peered":"2023-11-28T21:35:24.587700+0000","last_clean":"2023-11-28T21:35:24.587700+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:35:24.587700+0000","last_undegraded":"2023-11-28T21:35:24.587700+0000","last_fullsized":"2023-11-28T21:35:24.587700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574174+0000","last_change":"2023-11-28T21:34:41.485107+0000","last_active":"2023-11-28T21:35:24.574174+0000","last_peered":"2023-11-28T21:35:24.574174+0000","last_clean":"2023-11-28T21:35:24.574174+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:35:24.574174+0000","last_undegraded":"2023-11-28T21:35:24.574174+0000","last_fullsized":"2023-11-28T21:35:24.574174+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570615+0000","last_change":"2023-11-28T21:34:37.345104+0000","last_active":"2023-11-28T21:35:24.570615+0000","last_peered":"2023-11-28T21:35:24.570615+0000","last_clean":"2023-11-28T21:35:24.570615+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:35:24.570615+0000","last_undegraded":"2023-11-28T21:35:24.570615+0000","last_fullsized":"2023-11-28T21:35:24.570615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:25:16.125179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574812+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:35:24.574812+0000","last_peered":"2023-11-28T21:35:24.574812+0000","last_clean":"2023-11-28T21:35:24.574812+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:35:24.574812+0000","last_undegraded":"2023-11-28T21:35:24.574812+0000","last_fullsized":"2023-11-28T21:35:24.574812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:36.968220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569173+0000","last_change":"2023-11-28T21:34:41.475704+0000","last_active":"2023-11-28T21:35:24.569173+0000","last_peered":"2023-11-28T21:35:24.569173+0000","last_clean":"2023-11-28T21:35:24.569173+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:35:24.569173+0000","last_undegraded":"2023-11-28T21:35:24.569173+0000","last_fullsized":"2023-11-28T21:35:24.569173+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:01:34.494307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574121+0000","last_change":"2023-11-28T21:34:37.330831+0000","last_active":"2023-11-28T21:35:24.574121+0000","last_peered":"2023-11-28T21:35:24.574121+0000","last_clean":"2023-11-28T21:35:24.574121+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:35:24.574121+0000","last_undegraded":"2023-11-28T21:35:24.574121+0000","last_fullsized":"2023-11-28T21:35:24.574121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:50:38.612360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"24'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590947+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:35:24.590947+0000","last_peered":"2023-11-28T21:35:24.590947+0000","last_clean":"2023-11-28T21:35:24.590947+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:35:24.590947+0000","last_undegraded":"2023-11-28T21:35:24.590947+0000","last_fullsized":"2023-11-28T21:35:24.590947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578635+0000","last_change":"2023-11-28T21:34:41.475481+0000","last_active":"2023-11-28T21:35:24.578635+0000","last_peered":"2023-11-28T21:35:24.578635+0000","last_clean":"2023-11-28T21:35:24.578635+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:35:24.578635+0000","last_undegraded":"2023-11-28T21:35:24.578635+0000","last_fullsized":"2023-11-28T21:35:24.578635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:22:18.635807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579777+0000","last_change":"2023-11-28T21:34:37.337846+0000","last_active":"2023-11-28T21:35:24.579777+0000","last_peered":"2023-11-28T21:35:24.579777+0000","last_clean":"2023-11-28T21:35:24.579777+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:35:24.579777+0000","last_undegraded":"2023-11-28T21:35:24.579777+0000","last_fullsized":"2023-11-28T21:35:24.579777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:46:21.358603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574182+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:35:24.574182+0000","last_peered":"2023-11-28T21:35:24.574182+0000","last_clean":"2023-11-28T21:35:24.574182+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:35:24.574182+0000","last_undegraded":"2023-11-28T21:35:24.574182+0000","last_fullsized":"2023-11-28T21:35:24.574182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:08:15.270352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.572984+0000","last_change":"2023-11-28T21:34:41.370410+0000","last_active":"2023-11-28T21:35:24.572984+0000","last_peered":"2023-11-28T21:35:24.572984+0000","last_clean":"2023-11-28T21:35:24.572984+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:35:24.572984+0000","last_undegraded":"2023-11-28T21:35:24.572984+0000","last_fullsized":"2023-11-28T21:35:24.572984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.582169+0000","last_change":"2023-11-28T21:34:37.324608+0000","last_active":"2023-11-28T21:35:24.582169+0000","last_peered":"2023-11-28T21:35:24.582169+0000","last_clean":"2023-11-28T21:35:24.582169+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:35:24.582169+0000","last_undegraded":"2023-11-28T21:35:24.582169+0000","last_fullsized":"2023-11-28T21:35:24.582169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:13:53.438473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"24'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.575152+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:35:24.575152+0000","last_peered":"2023-11-28T21:35:24.575152+0000","last_clean":"2023-11-28T21:35:24.575152+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:35:24.575152+0000","last_undegraded":"2023-11-28T21:35:24.575152+0000","last_fullsized":"2023-11-28T21:35:24.575152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574069+0000","last_change":"2023-11-28T21:34:41.380599+0000","last_active":"2023-11-28T21:35:24.574069+0000","last_peered":"2023-11-28T21:35:24.574069+0000","last_clean":"2023-11-28T21:35:24.574069+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:35:24.574069+0000","last_undegraded":"2023-11-28T21:35:24.574069+0000","last_fullsized":"2023-11-28T21:35:24.574069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573050+0000","last_change":"2023-11-28T21:34:37.330869+0000","last_active":"2023-11-28T21:35:24.573050+0000","last_peered":"2023-11-28T21:35:24.573050+0000","last_clean":"2023-11-28T21:35:24.573050+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:35:24.573050+0000","last_undegraded":"2023-11-28T21:35:24.573050+0000","last_fullsized":"2023-11-28T21:35:24.573050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:58:55.987235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"24'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578798+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:35:24.578798+0000","last_peered":"2023-11-28T21:35:24.578798+0000","last_clean":"2023-11-28T21:35:24.578798+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:35:24.578798+0000","last_undegraded":"2023-11-28T21:35:24.578798+0000","last_fullsized":"2023-11-28T21:35:24.578798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.575055+0000","last_change":"2023-11-28T21:34:41.371897+0000","last_active":"2023-11-28T21:35:24.575055+0000","last_peered":"2023-11-28T21:35:24.575055+0000","last_clean":"2023-11-28T21:35:24.575055+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:35:24.575055+0000","last_undegraded":"2023-11-28T21:35:24.575055+0000","last_fullsized":"2023-11-28T21:35:24.575055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578402+0000","last_change":"2023-11-28T21:34:37.319833+0000","last_active":"2023-11-28T21:35:24.578402+0000","last_peered":"2023-11-28T21:35:24.578402+0000","last_clean":"2023-11-28T21:35:24.578402+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:35:24.578402+0000","last_undegraded":"2023-11-28T21:35:24.578402+0000","last_fullsized":"2023-11-28T21:35:24.578402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:02:17.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588430+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:35:24.588430+0000","last_peered":"2023-11-28T21:35:24.588430+0000","last_clean":"2023-11-28T21:35:24.588430+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:35:24.588430+0000","last_undegraded":"2023-11-28T21:35:24.588430+0000","last_fullsized":"2023-11-28T21:35:24.588430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579573+0000","last_change":"2023-11-28T21:34:41.475053+0000","last_active":"2023-11-28T21:35:24.579573+0000","last_peered":"2023-11-28T21:35:24.579573+0000","last_clean":"2023-11-28T21:35:24.579573+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:35:24.579573+0000","last_undegraded":"2023-11-28T21:35:24.579573+0000","last_fullsized":"2023-11-28T21:35:24.579573+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:32.658001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569672+0000","last_change":"2023-11-28T21:34:37.358890+0000","last_active":"2023-11-28T21:35:24.569672+0000","last_peered":"2023-11-28T21:35:24.569672+0000","last_clean":"2023-11-28T21:35:24.569672+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:35:24.569672+0000","last_undegraded":"2023-11-28T21:35:24.569672+0000","last_fullsized":"2023-11-28T21:35:24.569672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:54.878141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.577419+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:35:24.577419+0000","last_peered":"2023-11-28T21:35:24.577419+0000","last_clean":"2023-11-28T21:35:24.577419+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:35:24.577419+0000","last_undegraded":"2023-11-28T21:35:24.577419+0000","last_fullsized":"2023-11-28T21:35:24.577419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588554+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:35:24.588554+0000","last_peered":"2023-11-28T21:35:24.588554+0000","last_clean":"2023-11-28T21:35:24.588554+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:35:24.588554+0000","last_undegraded":"2023-11-28T21:35:24.588554+0000","last_fullsized":"2023-11-28T21:35:24.588554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.586735+0000","last_change":"2023-11-28T21:34:41.477522+0000","last_active":"2023-11-28T21:35:24.586735+0000","last_peered":"2023-11-28T21:35:24.586735+0000","last_clean":"2023-11-28T21:35:24.586735+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:35:24.586735+0000","last_undegraded":"2023-11-28T21:35:24.586735+0000","last_fullsized":"2023-11-28T21:35:24.586735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:36:36.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590606+0000","last_change":"2023-11-28T21:34:37.350441+0000","last_active":"2023-11-28T21:35:24.590606+0000","last_peered":"2023-11-28T21:35:24.590606+0000","last_clean":"2023-11-28T21:35:24.590606+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:35:24.590606+0000","last_undegraded":"2023-11-28T21:35:24.590606+0000","last_fullsized":"2023-11-28T21:35:24.590606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:54.775495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579251+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:35:24.579251+0000","last_peered":"2023-11-28T21:35:24.579251+0000","last_clean":"2023-11-28T21:35:24.579251+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:35:24.579251+0000","last_undegraded":"2023-11-28T21:35:24.579251+0000","last_fullsized":"2023-11-28T21:35:24.579251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.583047+0000","last_change":"2023-11-28T21:34:41.485154+0000","last_active":"2023-11-28T21:35:24.583047+0000","last_peered":"2023-11-28T21:35:24.583047+0000","last_clean":"2023-11-28T21:35:24.583047+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:35:24.583047+0000","last_undegraded":"2023-11-28T21:35:24.583047+0000","last_fullsized":"2023-11-28T21:35:24.583047+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:35:21.006172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.576292+0000","last_change":"2023-11-28T21:34:37.345652+0000","last_active":"2023-11-28T21:35:24.576292+0000","last_peered":"2023-11-28T21:35:24.576292+0000","last_clean":"2023-11-28T21:35:24.576292+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:35:24.576292+0000","last_undegraded":"2023-11-28T21:35:24.576292+0000","last_fullsized":"2023-11-28T21:35:24.576292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:21:17.474200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574398+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:35:24.574398+0000","last_peered":"2023-11-28T21:35:24.574398+0000","last_clean":"2023-11-28T21:35:24.574398+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:35:24.574398+0000","last_undegraded":"2023-11-28T21:35:24.574398+0000","last_fullsized":"2023-11-28T21:35:24.574398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:48.073446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578747+0000","last_change":"2023-11-28T21:34:41.370764+0000","last_active":"2023-11-28T21:35:24.578747+0000","last_peered":"2023-11-28T21:35:24.578747+0000","last_clean":"2023-11-28T21:35:24.578747+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:35:24.578747+0000","last_undegraded":"2023-11-28T21:35:24.578747+0000","last_fullsized":"2023-11-28T21:35:24.578747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:06:42.297098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.594934+0000","last_change":"2023-11-28T21:35:25.594934+0000","last_active":"2023-11-28T21:35:25.594934+0000","last_peered":"2023-11-28T21:35:25.594934+0000","last_clean":"2023-11-28T21:35:25.594934+0000","last_became_active":"2023-11-28T21:35:25.594533+0000","last_became_peered":"2023-11-28T21:35:25.594533+0000","last_unstale":"2023-11-28T21:35:25.594934+0000","last_undegraded":"2023-11-28T21:35:25.594934+0000","last_fullsized":"2023-11-28T21:35:25.594934+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:05.168934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579639+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:35:24.579639+0000","last_peered":"2023-11-28T21:35:24.579639+0000","last_clean":"2023-11-28T21:35:24.579639+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:35:24.579639+0000","last_undegraded":"2023-11-28T21:35:24.579639+0000","last_fullsized":"2023-11-28T21:35:24.579639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573161+0000","last_change":"2023-11-28T21:34:41.477127+0000","last_active":"2023-11-28T21:35:24.573161+0000","last_peered":"2023-11-28T21:35:24.573161+0000","last_clean":"2023-11-28T21:35:24.573161+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:35:24.573161+0000","last_undegraded":"2023-11-28T21:35:24.573161+0000","last_fullsized":"2023-11-28T21:35:24.573161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:32.798817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590603+0000","last_change":"2023-11-28T21:34:37.355693+0000","last_active":"2023-11-28T21:35:24.590603+0000","last_peered":"2023-11-28T21:35:24.590603+0000","last_clean":"2023-11-28T21:35:24.590603+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:35:24.590603+0000","last_undegraded":"2023-11-28T21:35:24.590603+0000","last_fullsized":"2023-11-28T21:35:24.590603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:12.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.583538+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:35:24.583538+0000","last_peered":"2023-11-28T21:35:24.583538+0000","last_clean":"2023-11-28T21:35:24.583538+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:35:24.583538+0000","last_undegraded":"2023-11-28T21:35:24.583538+0000","last_fullsized":"2023-11-28T21:35:24.583538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573923+0000","last_change":"2023-11-28T21:34:41.382468+0000","last_active":"2023-11-28T21:35:24.573923+0000","last_peered":"2023-11-28T21:35:24.573923+0000","last_clean":"2023-11-28T21:35:24.573923+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:35:24.573923+0000","last_undegraded":"2023-11-28T21:35:24.573923+0000","last_fullsized":"2023-11-28T21:35:24.573923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:17.410943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588771+0000","last_change":"2023-11-28T21:34:37.353232+0000","last_active":"2023-11-28T21:35:24.588771+0000","last_peered":"2023-11-28T21:35:24.588771+0000","last_clean":"2023-11-28T21:35:24.588771+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:35:24.588771+0000","last_undegraded":"2023-11-28T21:35:24.588771+0000","last_fullsized":"2023-11-28T21:35:24.588771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:11.285378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'15","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579450+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:35:24.579450+0000","last_peered":"2023-11-28T21:35:24.579450+0000","last_clean":"2023-11-28T21:35:24.579450+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:35:24.579450+0000","last_undegraded":"2023-11-28T21:35:24.579450+0000","last_fullsized":"2023-11-28T21:35:24.579450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579344+0000","last_change":"2023-11-28T21:34:41.385400+0000","last_active":"2023-11-28T21:35:24.579344+0000","last_peered":"2023-11-28T21:35:24.579344+0000","last_clean":"2023-11-28T21:35:24.579344+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:35:24.579344+0000","last_undegraded":"2023-11-28T21:35:24.579344+0000","last_fullsized":"2023-11-28T21:35:24.579344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:36.433439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.584714+0000","last_change":"2023-11-28T21:34:37.333461+0000","last_active":"2023-11-28T21:35:24.584714+0000","last_peered":"2023-11-28T21:35:24.584714+0000","last_clean":"2023-11-28T21:35:24.584714+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:35:24.584714+0000","last_undegraded":"2023-11-28T21:35:24.584714+0000","last_fullsized":"2023-11-28T21:35:24.584714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:25.513524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.787524+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:35:25.787524+0000","last_peered":"2023-11-28T21:35:25.787524+0000","last_clean":"2023-11-28T21:35:25.787524+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:35:25.787524+0000","last_undegraded":"2023-11-28T21:35:25.787524+0000","last_fullsized":"2023-11-28T21:35:25.787524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":842,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570442+0000","last_change":"2023-11-28T21:34:41.479919+0000","last_active":"2023-11-28T21:35:24.570442+0000","last_peered":"2023-11-28T21:35:24.570442+0000","last_clean":"2023-11-28T21:35:24.570442+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:35:24.570442+0000","last_undegraded":"2023-11-28T21:35:24.570442+0000","last_fullsized":"2023-11-28T21:35:24.570442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:11.183117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.589994+0000","last_change":"2023-11-28T21:34:37.355729+0000","last_active":"2023-11-28T21:35:24.589994+0000","last_peered":"2023-11-28T21:35:24.589994+0000","last_clean":"2023-11-28T21:35:24.589994+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:35:24.589994+0000","last_undegraded":"2023-11-28T21:35:24.589994+0000","last_fullsized":"2023-11-28T21:35:24.589994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:39.178378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574908+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:35:24.574908+0000","last_peered":"2023-11-28T21:35:24.574908+0000","last_clean":"2023-11-28T21:35:24.574908+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:35:24.574908+0000","last_undegraded":"2023-11-28T21:35:24.574908+0000","last_fullsized":"2023-11-28T21:35:24.574908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569708+0000","last_change":"2023-11-28T21:34:41.478483+0000","last_active":"2023-11-28T21:35:24.569708+0000","last_peered":"2023-11-28T21:35:24.569708+0000","last_clean":"2023-11-28T21:35:24.569708+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:35:24.569708+0000","last_undegraded":"2023-11-28T21:35:24.569708+0000","last_fullsized":"2023-11-28T21:35:24.569708+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:25:54.342692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581071+0000","last_change":"2023-11-28T21:34:37.345476+0000","last_active":"2023-11-28T21:35:24.581071+0000","last_peered":"2023-11-28T21:35:24.581071+0000","last_clean":"2023-11-28T21:35:24.581071+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:35:24.581071+0000","last_undegraded":"2023-11-28T21:35:24.581071+0000","last_fullsized":"2023-11-28T21:35:24.581071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.660412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'6","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579023+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:35:24.579023+0000","last_peered":"2023-11-28T21:35:24.579023+0000","last_clean":"2023-11-28T21:35:24.579023+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:35:24.579023+0000","last_undegraded":"2023-11-28T21:35:24.579023+0000","last_fullsized":"2023-11-28T21:35:24.579023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:28.694184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579447+0000","last_change":"2023-11-28T21:34:41.477926+0000","last_active":"2023-11-28T21:35:24.579447+0000","last_peered":"2023-11-28T21:35:24.579447+0000","last_clean":"2023-11-28T21:35:24.579447+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:35:24.579447+0000","last_undegraded":"2023-11-28T21:35:24.579447+0000","last_fullsized":"2023-11-28T21:35:24.579447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:26.403506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590387+0000","last_change":"2023-11-28T21:34:37.343885+0000","last_active":"2023-11-28T21:35:24.590387+0000","last_peered":"2023-11-28T21:35:24.590387+0000","last_clean":"2023-11-28T21:35:24.590387+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:35:24.590387+0000","last_undegraded":"2023-11-28T21:35:24.590387+0000","last_fullsized":"2023-11-28T21:35:24.590387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:36.255834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'11","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.589106+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:35:24.589106+0000","last_peered":"2023-11-28T21:35:24.589106+0000","last_clean":"2023-11-28T21:35:24.589106+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:35:24.589106+0000","last_undegraded":"2023-11-28T21:35:24.589106+0000","last_fullsized":"2023-11-28T21:35:24.589106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.791874+0000","last_change":"2023-11-28T21:35:25.791874+0000","last_active":"2023-11-28T21:35:25.791874+0000","last_peered":"2023-11-28T21:35:25.791874+0000","last_clean":"2023-11-28T21:35:25.791874+0000","last_became_active":"2023-11-28T21:35:25.790293+0000","last_became_peered":"2023-11-28T21:35:25.790293+0000","last_unstale":"2023-11-28T21:35:25.791874+0000","last_undegraded":"2023-11-28T21:35:25.791874+0000","last_fullsized":"2023-11-28T21:35:25.791874+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:21:17.947428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.582575+0000","last_change":"2023-11-28T21:34:37.345635+0000","last_active":"2023-11-28T21:35:24.582575+0000","last_peered":"2023-11-28T21:35:24.582575+0000","last_clean":"2023-11-28T21:35:24.582575+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:35:24.582575+0000","last_undegraded":"2023-11-28T21:35:24.582575+0000","last_fullsized":"2023-11-28T21:35:24.582575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:52:12.144267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574361+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:35:24.574361+0000","last_peered":"2023-11-28T21:35:24.574361+0000","last_clean":"2023-11-28T21:35:24.574361+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:35:24.574361+0000","last_undegraded":"2023-11-28T21:35:24.574361+0000","last_fullsized":"2023-11-28T21:35:24.574361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:35.266650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581717+0000","last_change":"2023-11-28T21:34:41.365758+0000","last_active":"2023-11-28T21:35:24.581717+0000","last_peered":"2023-11-28T21:35:24.581717+0000","last_clean":"2023-11-28T21:35:24.581717+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:35:24.581717+0000","last_undegraded":"2023-11-28T21:35:24.581717+0000","last_fullsized":"2023-11-28T21:35:24.581717+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.580813+0000","last_change":"2023-11-28T21:34:37.333607+0000","last_active":"2023-11-28T21:35:24.580813+0000","last_peered":"2023-11-28T21:35:24.580813+0000","last_clean":"2023-11-28T21:35:24.580813+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:35:24.580813+0000","last_undegraded":"2023-11-28T21:35:24.580813+0000","last_fullsized":"2023-11-28T21:35:24.580813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:13:01.639908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"24'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.576877+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:35:24.576877+0000","last_peered":"2023-11-28T21:35:24.576877+0000","last_clean":"2023-11-28T21:35:24.576877+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:35:24.576877+0000","last_undegraded":"2023-11-28T21:35:24.576877+0000","last_fullsized":"2023-11-28T21:35:24.576877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569623+0000","last_change":"2023-11-28T21:34:41.380413+0000","last_active":"2023-11-28T21:35:24.569623+0000","last_peered":"2023-11-28T21:35:24.569623+0000","last_clean":"2023-11-28T21:35:24.569623+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:35:24.569623+0000","last_undegraded":"2023-11-28T21:35:24.569623+0000","last_fullsized":"2023-11-28T21:35:24.569623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:20:28.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.595606+0000","last_change":"2023-11-28T21:35:25.595606+0000","last_active":"2023-11-28T21:35:25.595606+0000","last_peered":"2023-11-28T21:35:25.595606+0000","last_clean":"2023-11-28T21:35:25.595606+0000","last_became_active":"2023-11-28T21:35:25.595386+0000","last_became_peered":"2023-11-28T21:35:25.595386+0000","last_unstale":"2023-11-28T21:35:25.595606+0000","last_undegraded":"2023-11-28T21:35:25.595606+0000","last_fullsized":"2023-11-28T21:35:25.595606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:16:44.887005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"24'3","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.587348+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:35:24.587348+0000","last_peered":"2023-11-28T21:35:24.587348+0000","last_clean":"2023-11-28T21:35:24.587348+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:35:24.587348+0000","last_undegraded":"2023-11-28T21:35:24.587348+0000","last_fullsized":"2023-11-28T21:35:24.587348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569699+0000","last_change":"2023-11-28T21:34:41.475185+0000","last_active":"2023-11-28T21:35:24.569699+0000","last_peered":"2023-11-28T21:35:24.569699+0000","last_clean":"2023-11-28T21:35:24.569699+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:35:24.569699+0000","last_undegraded":"2023-11-28T21:35:24.569699+0000","last_fullsized":"2023-11-28T21:35:24.569699+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:09.389881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578519+0000","last_change":"2023-11-28T21:34:41.479956+0000","last_active":"2023-11-28T21:35:24.578519+0000","last_peered":"2023-11-28T21:35:24.578519+0000","last_clean":"2023-11-28T21:35:24.578519+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:35:24.578519+0000","last_undegraded":"2023-11-28T21:35:24.578519+0000","last_fullsized":"2023-11-28T21:35:24.578519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:46.004852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.589823+0000","last_change":"2023-11-28T21:35:25.589823+0000","last_active":"2023-11-28T21:35:25.589823+0000","last_peered":"2023-11-28T21:35:25.589823+0000","last_clean":"2023-11-28T21:35:25.589823+0000","last_became_active":"2023-11-28T21:35:25.589378+0000","last_became_peered":"2023-11-28T21:35:25.589378+0000","last_unstale":"2023-11-28T21:35:25.589823+0000","last_undegraded":"2023-11-28T21:35:25.589823+0000","last_fullsized":"2023-11-28T21:35:25.589823+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:11.796380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"24'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573550+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:35:24.573550+0000","last_peered":"2023-11-28T21:35:24.573550+0000","last_clean":"2023-11-28T21:35:24.573550+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:35:24.573550+0000","last_undegraded":"2023-11-28T21:35:24.573550+0000","last_fullsized":"2023-11-28T21:35:24.573550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.586981+0000","last_change":"2023-11-28T21:34:41.385969+0000","last_active":"2023-11-28T21:35:24.586981+0000","last_peered":"2023-11-28T21:35:24.586981+0000","last_clean":"2023-11-28T21:35:24.586981+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:35:24.586981+0000","last_undegraded":"2023-11-28T21:35:24.586981+0000","last_fullsized":"2023-11-28T21:35:24.586981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:01.692549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569384+0000","last_change":"2023-11-28T21:34:37.314087+0000","last_active":"2023-11-28T21:35:24.569384+0000","last_peered":"2023-11-28T21:35:24.569384+0000","last_clean":"2023-11-28T21:35:24.569384+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:35:24.569384+0000","last_undegraded":"2023-11-28T21:35:24.569384+0000","last_fullsized":"2023-11-28T21:35:24.569384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:25:11.912313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"24'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570172+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:35:24.570172+0000","last_peered":"2023-11-28T21:35:24.570172+0000","last_clean":"2023-11-28T21:35:24.570172+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:35:24.570172+0000","last_undegraded":"2023-11-28T21:35:24.570172+0000","last_fullsized":"2023-11-28T21:35:24.570172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'3","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.583399+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:35:24.583399+0000","last_peered":"2023-11-28T21:35:24.583399+0000","last_clean":"2023-11-28T21:35:24.583399+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:35:24.583399+0000","last_undegraded":"2023-11-28T21:35:24.583399+0000","last_fullsized":"2023-11-28T21:35:24.583399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578925+0000","last_change":"2023-11-28T21:34:37.319801+0000","last_active":"2023-11-28T21:35:24.578925+0000","last_peered":"2023-11-28T21:35:24.578925+0000","last_clean":"2023-11-28T21:35:24.578925+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:35:24.578925+0000","last_undegraded":"2023-11-28T21:35:24.578925+0000","last_fullsized":"2023-11-28T21:35:24.578925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:40:13.251522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578964+0000","last_change":"2023-11-28T21:34:41.370523+0000","last_active":"2023-11-28T21:35:24.578964+0000","last_peered":"2023-11-28T21:35:24.578964+0000","last_clean":"2023-11-28T21:35:24.578964+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:35:24.578964+0000","last_undegraded":"2023-11-28T21:35:24.578964+0000","last_fullsized":"2023-11-28T21:35:24.578964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:17:36.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'5","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569956+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:35:24.569956+0000","last_peered":"2023-11-28T21:35:24.569956+0000","last_clean":"2023-11-28T21:35:24.569956+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:35:24.569956+0000","last_undegraded":"2023-11-28T21:35:24.569956+0000","last_fullsized":"2023-11-28T21:35:24.569956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581212+0000","last_change":"2023-11-28T21:34:37.328809+0000","last_active":"2023-11-28T21:35:24.581212+0000","last_peered":"2023-11-28T21:35:24.581212+0000","last_clean":"2023-11-28T21:35:24.581212+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:35:24.581212+0000","last_undegraded":"2023-11-28T21:35:24.581212+0000","last_fullsized":"2023-11-28T21:35:24.581212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:27.300090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570686+0000","last_change":"2023-11-28T21:34:41.363087+0000","last_active":"2023-11-28T21:35:24.570686+0000","last_peered":"2023-11-28T21:35:24.570686+0000","last_clean":"2023-11-28T21:35:24.570686+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:35:24.570686+0000","last_undegraded":"2023-11-28T21:35:24.570686+0000","last_fullsized":"2023-11-28T21:35:24.570686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:35.046132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.584352+0000","last_change":"2023-11-28T21:34:37.333670+0000","last_active":"2023-11-28T21:35:24.584352+0000","last_peered":"2023-11-28T21:35:24.584352+0000","last_clean":"2023-11-28T21:35:24.584352+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:35:24.584352+0000","last_undegraded":"2023-11-28T21:35:24.584352+0000","last_fullsized":"2023-11-28T21:35:24.584352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:32.598014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578736+0000","last_change":"2023-11-28T21:34:37.347101+0000","last_active":"2023-11-28T21:35:24.578736+0000","last_peered":"2023-11-28T21:35:24.578736+0000","last_clean":"2023-11-28T21:35:24.578736+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:35:24.578736+0000","last_undegraded":"2023-11-28T21:35:24.578736+0000","last_fullsized":"2023-11-28T21:35:24.578736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:39:43.186110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574495+0000","last_change":"2023-11-28T21:34:37.338576+0000","last_active":"2023-11-28T21:35:24.574495+0000","last_peered":"2023-11-28T21:35:24.574495+0000","last_clean":"2023-11-28T21:35:24.574495+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:35:24.574495+0000","last_undegraded":"2023-11-28T21:35:24.574495+0000","last_fullsized":"2023-11-28T21:35:24.574495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:53:02.356097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578831+0000","last_change":"2023-11-28T21:34:37.316270+0000","last_active":"2023-11-28T21:35:24.578831+0000","last_peered":"2023-11-28T21:35:24.578831+0000","last_clean":"2023-11-28T21:35:24.578831+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:35:24.578831+0000","last_undegraded":"2023-11-28T21:35:24.578831+0000","last_fullsized":"2023-11-28T21:35:24.578831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:43.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570075+0000","last_change":"2023-11-28T21:34:37.334973+0000","last_active":"2023-11-28T21:35:24.570075+0000","last_peered":"2023-11-28T21:35:24.570075+0000","last_clean":"2023-11-28T21:35:24.570075+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:35:24.570075+0000","last_undegraded":"2023-11-28T21:35:24.570075+0000","last_fullsized":"2023-11-28T21:35:24.570075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:55.050945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573304+0000","last_change":"2023-11-28T21:34:37.318155+0000","last_active":"2023-11-28T21:35:24.573304+0000","last_peered":"2023-11-28T21:35:24.573304+0000","last_clean":"2023-11-28T21:35:24.573304+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:35:24.573304+0000","last_undegraded":"2023-11-28T21:35:24.573304+0000","last_fullsized":"2023-11-28T21:35:24.573304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:29.840255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.587050+0000","last_change":"2023-11-28T21:34:37.345758+0000","last_active":"2023-11-28T21:35:24.587050+0000","last_peered":"2023-11-28T21:35:24.587050+0000","last_clean":"2023-11-28T21:35:24.587050+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:35:24.587050+0000","last_undegraded":"2023-11-28T21:35:24.587050+0000","last_fullsized":"2023-11-28T21:35:24.587050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:26.444654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570520+0000","last_change":"2023-11-28T21:34:37.334989+0000","last_active":"2023-11-28T21:35:24.570520+0000","last_peered":"2023-11-28T21:35:24.570520+0000","last_clean":"2023-11-28T21:35:24.570520+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:35:24.570520+0000","last_undegraded":"2023-11-28T21:35:24.570520+0000","last_fullsized":"2023-11-28T21:35:24.570520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:36:57.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573377+0000","last_change":"2023-11-28T21:34:37.333075+0000","last_active":"2023-11-28T21:35:24.573377+0000","last_peered":"2023-11-28T21:35:24.573377+0000","last_clean":"2023-11-28T21:35:24.573377+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:35:24.573377+0000","last_undegraded":"2023-11-28T21:35:24.573377+0000","last_fullsized":"2023-11-28T21:35:24.573377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:50:43.748349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570076+0000","last_change":"2023-11-28T21:34:37.337230+0000","last_active":"2023-11-28T21:35:24.570076+0000","last_peered":"2023-11-28T21:35:24.570076+0000","last_clean":"2023-11-28T21:35:24.570076+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:35:24.570076+0000","last_undegraded":"2023-11-28T21:35:24.570076+0000","last_fullsized":"2023-11-28T21:35:24.570076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:48:45.994559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579747+0000","last_change":"2023-11-28T21:34:37.349894+0000","last_active":"2023-11-28T21:35:24.579747+0000","last_peered":"2023-11-28T21:35:24.579747+0000","last_clean":"2023-11-28T21:35:24.579747+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:35:24.579747+0000","last_undegraded":"2023-11-28T21:35:24.579747+0000","last_fullsized":"2023-11-28T21:35:24.579747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:10.293911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578635+0000","last_change":"2023-11-28T21:34:37.344269+0000","last_active":"2023-11-28T21:35:24.578635+0000","last_peered":"2023-11-28T21:35:24.578635+0000","last_clean":"2023-11-28T21:35:24.578635+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:35:24.578635+0000","last_undegraded":"2023-11-28T21:35:24.578635+0000","last_fullsized":"2023-11-28T21:35:24.578635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:03.204740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588662+0000","last_change":"2023-11-28T21:34:37.353059+0000","last_active":"2023-11-28T21:35:24.588662+0000","last_peered":"2023-11-28T21:35:24.588662+0000","last_clean":"2023-11-28T21:35:24.588662+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:35:24.588662+0000","last_undegraded":"2023-11-28T21:35:24.588662+0000","last_fullsized":"2023-11-28T21:35:24.588662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:45:12.640797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590822+0000","last_change":"2023-11-28T21:34:37.352965+0000","last_active":"2023-11-28T21:35:24.590822+0000","last_peered":"2023-11-28T21:35:24.590822+0000","last_clean":"2023-11-28T21:35:24.590822+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:35:24.590822+0000","last_undegraded":"2023-11-28T21:35:24.590822+0000","last_fullsized":"2023-11-28T21:35:24.590822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:30:42.087715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579567+0000","last_change":"2023-11-28T21:34:37.336830+0000","last_active":"2023-11-28T21:35:24.579567+0000","last_peered":"2023-11-28T21:35:24.579567+0000","last_clean":"2023-11-28T21:35:24.579567+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:35:24.579567+0000","last_undegraded":"2023-11-28T21:35:24.579567+0000","last_fullsized":"2023-11-28T21:35:24.579567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:57.318311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578322+0000","last_change":"2023-11-28T21:34:37.349181+0000","last_active":"2023-11-28T21:35:24.578322+0000","last_peered":"2023-11-28T21:35:24.578322+0000","last_clean":"2023-11-28T21:35:24.578322+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:35:24.578322+0000","last_undegraded":"2023-11-28T21:35:24.578322+0000","last_fullsized":"2023-11-28T21:35:24.578322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:27:59.989299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573318+0000","last_change":"2023-11-28T21:34:37.336668+0000","last_active":"2023-11-28T21:35:24.573318+0000","last_peered":"2023-11-28T21:35:24.573318+0000","last_clean":"2023-11-28T21:35:24.573318+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:35:24.573318+0000","last_undegraded":"2023-11-28T21:35:24.573318+0000","last_fullsized":"2023-11-28T21:35:24.573318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:06.416717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573851+0000","last_change":"2023-11-28T21:34:37.332790+0000","last_active":"2023-11-28T21:35:24.573851+0000","last_peered":"2023-11-28T21:35:24.573851+0000","last_clean":"2023-11-28T21:35:24.573851+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:35:24.573851+0000","last_undegraded":"2023-11-28T21:35:24.573851+0000","last_fullsized":"2023-11-28T21:35:24.573851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:32:45.920387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581071+0000","last_change":"2023-11-28T21:34:37.320794+0000","last_active":"2023-11-28T21:35:24.581071+0000","last_peered":"2023-11-28T21:35:24.581071+0000","last_clean":"2023-11-28T21:35:24.581071+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:35:24.581071+0000","last_undegraded":"2023-11-28T21:35:24.581071+0000","last_fullsized":"2023-11-28T21:35:24.581071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:16:46.347683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569545+0000","last_change":"2023-11-28T21:34:37.337726+0000","last_active":"2023-11-28T21:35:24.569545+0000","last_peered":"2023-11-28T21:35:24.569545+0000","last_clean":"2023-11-28T21:35:24.569545+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:35:24.569545+0000","last_undegraded":"2023-11-28T21:35:24.569545+0000","last_fullsized":"2023-11-28T21:35:24.569545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:38:43.884668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.582359+0000","last_change":"2023-11-28T21:34:37.334677+0000","last_active":"2023-11-28T21:35:24.582359+0000","last_peered":"2023-11-28T21:35:24.582359+0000","last_clean":"2023-11-28T21:35:24.582359+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:35:24.582359+0000","last_undegraded":"2023-11-28T21:35:24.582359+0000","last_fullsized":"2023-11-28T21:35:24.582359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:05:54.981899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.587132+0000","last_change":"2023-11-28T21:34:37.354861+0000","last_active":"2023-11-28T21:35:24.587132+0000","last_peered":"2023-11-28T21:35:24.587132+0000","last_clean":"2023-11-28T21:35:24.587132+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:35:24.587132+0000","last_undegraded":"2023-11-28T21:35:24.587132+0000","last_fullsized":"2023-11-28T21:35:24.587132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:30.602284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.587935+0000","last_change":"2023-11-28T21:34:37.347675+0000","last_active":"2023-11-28T21:35:24.587935+0000","last_peered":"2023-11-28T21:35:24.587935+0000","last_clean":"2023-11-28T21:35:24.587935+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:35:24.587935+0000","last_undegraded":"2023-11-28T21:35:24.587935+0000","last_fullsized":"2023-11-28T21:35:24.587935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:51:38.803896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.599601+0000","last_change":"2023-11-28T21:35:25.599601+0000","last_active":"2023-11-28T21:35:25.599601+0000","last_peered":"2023-11-28T21:35:25.599601+0000","last_clean":"2023-11-28T21:35:25.599601+0000","last_became_active":"2023-11-28T21:35:25.598839+0000","last_became_peered":"2023-11-28T21:35:25.598839+0000","last_unstale":"2023-11-28T21:35:25.599601+0000","last_undegraded":"2023-11-28T21:35:25.599601+0000","last_fullsized":"2023-11-28T21:35:25.599601+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:51:09.247433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569279+0000","last_change":"2023-11-28T21:34:37.358851+0000","last_active":"2023-11-28T21:35:24.569279+0000","last_peered":"2023-11-28T21:35:24.569279+0000","last_clean":"2023-11-28T21:35:24.569279+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:35:24.569279+0000","last_undegraded":"2023-11-28T21:35:24.569279+0000","last_fullsized":"2023-11-28T21:35:24.569279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.378519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588726+0000","last_change":"2023-11-28T21:34:37.343972+0000","last_active":"2023-11-28T21:35:24.588726+0000","last_peered":"2023-11-28T21:35:24.588726+0000","last_clean":"2023-11-28T21:35:24.588726+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:35:24.588726+0000","last_undegraded":"2023-11-28T21:35:24.588726+0000","last_fullsized":"2023-11-28T21:35:24.588726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:56:00.591888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574264+0000","last_change":"2023-11-28T21:34:37.346299+0000","last_active":"2023-11-28T21:35:24.574264+0000","last_peered":"2023-11-28T21:35:24.574264+0000","last_clean":"2023-11-28T21:35:24.574264+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:35:24.574264+0000","last_undegraded":"2023-11-28T21:35:24.574264+0000","last_fullsized":"2023-11-28T21:35:24.574264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:06.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.589786+0000","last_change":"2023-11-28T21:34:37.343410+0000","last_active":"2023-11-28T21:35:24.589786+0000","last_peered":"2023-11-28T21:35:24.589786+0000","last_clean":"2023-11-28T21:35:24.589786+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:35:24.589786+0000","last_undegraded":"2023-11-28T21:35:24.589786+0000","last_fullsized":"2023-11-28T21:35:24.589786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:29:45.070478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.575841+0000","last_change":"2023-11-28T21:34:37.348270+0000","last_active":"2023-11-28T21:35:24.575841+0000","last_peered":"2023-11-28T21:35:24.575841+0000","last_clean":"2023-11-28T21:35:24.575841+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:35:24.575841+0000","last_undegraded":"2023-11-28T21:35:24.575841+0000","last_fullsized":"2023-11-28T21:35:24.575841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:36:10.938478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579215+0000","last_change":"2023-11-28T21:34:37.330786+0000","last_active":"2023-11-28T21:35:24.579215+0000","last_peered":"2023-11-28T21:35:24.579215+0000","last_clean":"2023-11-28T21:35:24.579215+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:35:24.579215+0000","last_undegraded":"2023-11-28T21:35:24.579215+0000","last_fullsized":"2023-11-28T21:35:24.579215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:52:36.738867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574738+0000","last_change":"2023-11-28T21:34:37.318725+0000","last_active":"2023-11-28T21:35:24.574738+0000","last_peered":"2023-11-28T21:35:24.574738+0000","last_clean":"2023-11-28T21:35:24.574738+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:35:24.574738+0000","last_undegraded":"2023-11-28T21:35:24.574738+0000","last_fullsized":"2023-11-28T21:35:24.574738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:49:21.775066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.591700+0000","last_change":"2023-11-28T21:34:37.349829+0000","last_active":"2023-11-28T21:35:24.591700+0000","last_peered":"2023-11-28T21:35:24.591700+0000","last_clean":"2023-11-28T21:35:24.591700+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:35:24.591700+0000","last_undegraded":"2023-11-28T21:35:24.591700+0000","last_fullsized":"2023-11-28T21:35:24.591700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:07.752862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7705,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":15410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":705445,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.52100000000000002}]},{"osd":1,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.71999999999999997}]},{"osd":3,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.5629999999999997,"5min":4.5629999999999997,"15min":4.5629999999999997},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.998}]},{"osd":4,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.93700000000000006}]},{"osd":5,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.86299999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0329999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":118534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":115139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":114729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":118314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":114579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":114705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":703770,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.33100000000000002}]},{"osd":2,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.875}]},{"osd":6,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.68400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:35:32.226 INFO:tasks.mds_thrash.ceph_manager:clean! 2023-11-28T21:35:32.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:35:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.400+0000 7efcd9009640 1 Processor -- start 2023-11-28T21:35:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.400+0000 7efcd9009640 1 -- start start 2023-11-28T21:35:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efcd9009640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd40cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efcd9009640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efcd40cc600 0x7efcd415a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efcd9009640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd415d370 con 0x7efcd4154120 2023-11-28T21:35:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efcd9009640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd40b3040 con 0x7efcd40cc600 2023-11-28T21:35:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efcd9009640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4154500 con 0x7efcd40cbba0 2023-11-28T21:35:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efccbfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd40cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efcd359e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efcd40cc600 0x7efcd415a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efccbfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd40cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:42286/0 (socket says 172.21.15.12:42286) 2023-11-28T21:35:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efccbfff640 1 -- 172.21.15.12:0/135668454 learned_addr learned my addr 172.21.15.12:0/135668454 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efccbfff640 1 -- 172.21.15.12:0/135668454 >> v2:172.21.15.144:3300/0 conn(0x7efcd40cc600 msgr2=0x7efcd415a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efccbfff640 1 --2- 172.21.15.12:0/135668454 >> v2:172.21.15.144:3300/0 conn(0x7efcd40cc600 0x7efcd415a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efccbfff640 1 -- 172.21.15.12:0/135668454 >> v1:172.21.15.144:6789/0 conn(0x7efcd4154120 legacy=0x7efcd40ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.404+0000 7efccbfff640 1 -- 172.21.15.12:0/135668454 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4159d80 con 0x7efcd40cbba0 2023-11-28T21:35:32.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efccbfff640 1 --2- 172.21.15.12:0/135668454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd40cc0c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009870 tx=0x7efcbc01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6da8296bbfcfb44c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:32.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd259c640 1 -- 172.21.15.12:0/135668454 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0ab620 con 0x7efcd40cbba0 2023-11-28T21:35:32.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd259c640 1 -- 172.21.15.12:0/135668454 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc0ab780 con 0x7efcd40cbba0 2023-11-28T21:35:32.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd259c640 1 -- 172.21.15.12:0/135668454 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0abb10 con 0x7efcd40cbba0 2023-11-28T21:35:32.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd9009640 1 -- 172.21.15.12:0/135668454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 msgr2=0x7efcd40cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd9009640 1 --2- 172.21.15.12:0/135668454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd40cc0c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efcbc009870 tx=0x7efcbc01c510 comp rx=0 tx=0).stop 2023-11-28T21:35:32.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd9009640 1 -- 172.21.15.12:0/135668454 shutdown_connections 2023-11-28T21:35:32.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd9009640 1 --2- 172.21.15.12:0/135668454 >> v2:172.21.15.144:3300/0 conn(0x7efcd40cc600 0x7efcd415a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd9009640 1 --2- 172.21.15.12:0/135668454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd40cc0c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd9009640 1 -- 172.21.15.12:0/135668454 >> 172.21.15.12:0/135668454 conn(0x7efcd4071ce0 msgr2=0x7efcd40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:32.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd9009640 1 -- 172.21.15.12:0/135668454 shutdown_connections 2023-11-28T21:35:32.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd9009640 1 -- 172.21.15.12:0/135668454 wait complete. 2023-11-28T21:35:32.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.408+0000 7efcd9009640 1 Processor -- start 2023-11-28T21:35:32.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd9009640 1 -- start start 2023-11-28T21:35:32.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd9009640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd9009640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcd40cc600 0x7efcd41496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd9009640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd4154120 0x7efcd41f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd9009640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4146930 con 0x7efcd40cc600 2023-11-28T21:35:32.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd9009640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4146ef0 con 0x7efcd4154120 2023-11-28T21:35:32.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd9009640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd415c840 con 0x7efcd40cbba0 2023-11-28T21:35:32.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcd40cc600 0x7efcd41496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcd40cc600 0x7efcd41496e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41714/0 (socket says 172.21.15.12:41714) 2023-11-28T21:35:32.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccbfff640 1 -- 172.21.15.12:0/157846195 learned_addr learned my addr 172.21.15.12:0/157846195 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:32.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd359e640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd4154120 0x7efcd41f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd2d9d640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccbfff640 1 -- 172.21.15.12:0/157846195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 msgr2=0x7efcd415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccbfff640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd415af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccbfff640 1 -- 172.21.15.12:0/157846195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd4154120 msgr2=0x7efcd41f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccbfff640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd4154120 0x7efcd41f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccbfff640 1 -- 172.21.15.12:0/157846195 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0094d0 con 0x7efcd40cc600 2023-11-28T21:35:32.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd2d9d640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:32.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd359e640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd4154120 0x7efcd41f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:32.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccbfff640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcd40cc600 0x7efcd41496e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efcbc01abc0 tx=0x7efcbc01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:32.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccb7fe640 1 -- 172.21.15.12:0/157846195 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0abe80 con 0x7efcd40cc600 2023-11-28T21:35:32.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccb7fe640 1 -- 172.21.15.12:0/157846195 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc0be3f0 con 0x7efcd40cc600 2023-11-28T21:35:32.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd9009640 1 -- 172.21.15.12:0/157846195 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd415ccc0 con 0x7efcd40cc600 2023-11-28T21:35:32.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efccb7fe640 1 -- 172.21.15.12:0/157846195 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0be780 con 0x7efcd40cc600 2023-11-28T21:35:32.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd9009640 1 -- 172.21.15.12:0/157846195 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd4146b80 con 0x7efcd40cc600 2023-11-28T21:35:32.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.412+0000 7efcd9009640 1 -- 172.21.15.12:0/157846195 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd4074f60 con 0x7efcd40cc600 2023-11-28T21:35:32.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.416+0000 7efccb7fe640 1 -- 172.21.15.12:0/157846195 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7efcbc0b43f0 con 0x7efcd40cc600 2023-11-28T21:35:32.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.416+0000 7efccb7fe640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efcb40664a0 0x7efcb4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.416+0000 7efccb7fe640 1 -- 172.21.15.12:0/157846195 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efcbc11e510 con 0x7efcd40cc600 2023-11-28T21:35:32.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.416+0000 7efcd2d9d640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efcb40664a0 0x7efcb4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.416+0000 7efcd2d9d640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efcb40664a0 0x7efcb4068960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efcc4000f70 tx=0x7efcc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:32.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.420+0000 7efccb7fe640 1 -- 172.21.15.12:0/157846195 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efcbc0eb670 con 0x7efcd40cc600 2023-11-28T21:35:32.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.572+0000 7efcd9009640 1 -- 172.21.15.12:0/157846195 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efcd415e120 con 0x7efcb40664a0 2023-11-28T21:35:32.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.584+0000 7efccb7fe640 1 -- 172.21.15.12:0/157846195 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+351421 (secure 0 0 0) 0x7efcc4001ca0 con 0x7efcb40664a0 2023-11-28T21:35:32.589 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:32.591 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:35:32.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 -- 172.21.15.12:0/157846195 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efcb40664a0 msgr2=0x7efcb4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efcb40664a0 0x7efcb4068960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efcc4000f70 tx=0x7efcc4009040 comp rx=0 tx=0).stop 2023-11-28T21:35:32.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 -- 172.21.15.12:0/157846195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcd40cc600 msgr2=0x7efcd41496e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcd40cc600 0x7efcd41496e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efcbc01abc0 tx=0x7efcbc01ad50 comp rx=0 tx=0).stop 2023-11-28T21:35:32.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 -- 172.21.15.12:0/157846195 shutdown_connections 2023-11-28T21:35:32.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efcb40664a0 0x7efcb4068960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd4154120 0x7efcd41f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcd40cc600 0x7efcd41496e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 --2- 172.21.15.12:0/157846195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcd40cbba0 0x7efcd415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 -- 172.21.15.12:0/157846195 >> 172.21.15.12:0/157846195 conn(0x7efcd4071ce0 msgr2=0x7efcd40c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:32.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 -- 172.21.15.12:0/157846195 shutdown_connections 2023-11-28T21:35:32.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.588+0000 7efcd9009640 1 -- 172.21.15.12:0/157846195 wait complete. 2023-11-28T21:35:32.609 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2023-11-28T21:35:31.828476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598073,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":323,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218628,"kb_used_data":2460,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756092,"statfs":{"total":773094113280,"available":772870238208,"internally_reserved":0,"allocated":2519040,"data_stored":2105215,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":254,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001538"},"pg_stats":[{"pgid":"2.1f","version":"24'9","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.580953+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:35:24.580953+0000","last_peered":"2023-11-28T21:35:24.580953+0000","last_clean":"2023-11-28T21:35:24.580953+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:35:24.580953+0000","last_undegraded":"2023-11-28T21:35:24.580953+0000","last_fullsized":"2023-11-28T21:35:24.580953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:46.167511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570318+0000","last_change":"2023-11-28T21:34:37.335656+0000","last_active":"2023-11-28T21:35:24.570318+0000","last_peered":"2023-11-28T21:35:24.570318+0000","last_clean":"2023-11-28T21:35:24.570318+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:35:24.570318+0000","last_undegraded":"2023-11-28T21:35:24.570318+0000","last_fullsized":"2023-11-28T21:35:24.570318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:25:49.707378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569336+0000","last_change":"2023-11-28T21:34:41.483856+0000","last_active":"2023-11-28T21:35:24.569336+0000","last_peered":"2023-11-28T21:35:24.569336+0000","last_clean":"2023-11-28T21:35:24.569336+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:35:24.569336+0000","last_undegraded":"2023-11-28T21:35:24.569336+0000","last_fullsized":"2023-11-28T21:35:24.569336+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:50:56.005578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570516+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:35:24.570516+0000","last_peered":"2023-11-28T21:35:24.570516+0000","last_clean":"2023-11-28T21:35:24.570516+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:35:24.570516+0000","last_undegraded":"2023-11-28T21:35:24.570516+0000","last_fullsized":"2023-11-28T21:35:24.570516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.580664+0000","last_change":"2023-11-28T21:34:37.333649+0000","last_active":"2023-11-28T21:35:24.580664+0000","last_peered":"2023-11-28T21:35:24.580664+0000","last_clean":"2023-11-28T21:35:24.580664+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:35:24.580664+0000","last_undegraded":"2023-11-28T21:35:24.580664+0000","last_fullsized":"2023-11-28T21:35:24.580664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:21:42.041590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578771+0000","last_change":"2023-11-28T21:34:41.388232+0000","last_active":"2023-11-28T21:35:24.578771+0000","last_peered":"2023-11-28T21:35:24.578771+0000","last_clean":"2023-11-28T21:35:24.578771+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:35:24.578771+0000","last_undegraded":"2023-11-28T21:35:24.578771+0000","last_fullsized":"2023-11-28T21:35:24.578771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:35:22.960012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590922+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:35:24.590922+0000","last_peered":"2023-11-28T21:35:24.590922+0000","last_clean":"2023-11-28T21:35:24.590922+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:35:24.590922+0000","last_undegraded":"2023-11-28T21:35:24.590922+0000","last_fullsized":"2023-11-28T21:35:24.590922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.576038+0000","last_change":"2023-11-28T21:34:37.348126+0000","last_active":"2023-11-28T21:35:24.576038+0000","last_peered":"2023-11-28T21:35:24.576038+0000","last_clean":"2023-11-28T21:35:24.576038+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:35:24.576038+0000","last_undegraded":"2023-11-28T21:35:24.576038+0000","last_fullsized":"2023-11-28T21:35:24.576038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:47.982143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581423+0000","last_change":"2023-11-28T21:34:41.388539+0000","last_active":"2023-11-28T21:35:24.581423+0000","last_peered":"2023-11-28T21:35:24.581423+0000","last_clean":"2023-11-28T21:35:24.581423+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:35:24.581423+0000","last_undegraded":"2023-11-28T21:35:24.581423+0000","last_fullsized":"2023-11-28T21:35:24.581423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.589668+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:35:24.589668+0000","last_peered":"2023-11-28T21:35:24.589668+0000","last_clean":"2023-11-28T21:35:24.589668+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:35:24.589668+0000","last_undegraded":"2023-11-28T21:35:24.589668+0000","last_fullsized":"2023-11-28T21:35:24.589668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579763+0000","last_change":"2023-11-28T21:34:37.337266+0000","last_active":"2023-11-28T21:35:24.579763+0000","last_peered":"2023-11-28T21:35:24.579763+0000","last_clean":"2023-11-28T21:35:24.579763+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:35:24.579763+0000","last_undegraded":"2023-11-28T21:35:24.579763+0000","last_fullsized":"2023-11-28T21:35:24.579763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:32:48.848989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574284+0000","last_change":"2023-11-28T21:34:41.372957+0000","last_active":"2023-11-28T21:35:24.574284+0000","last_peered":"2023-11-28T21:35:24.574284+0000","last_clean":"2023-11-28T21:35:24.574284+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:35:24.574284+0000","last_undegraded":"2023-11-28T21:35:24.574284+0000","last_fullsized":"2023-11-28T21:35:24.574284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:00:59.046422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.583250+0000","last_change":"2023-11-28T21:34:37.353174+0000","last_active":"2023-11-28T21:35:24.583250+0000","last_peered":"2023-11-28T21:35:24.583250+0000","last_clean":"2023-11-28T21:35:24.583250+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:35:24.583250+0000","last_undegraded":"2023-11-28T21:35:24.583250+0000","last_fullsized":"2023-11-28T21:35:24.583250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:54:32.966564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590181+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:35:24.590181+0000","last_peered":"2023-11-28T21:35:24.590181+0000","last_clean":"2023-11-28T21:35:24.590181+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:35:24.590181+0000","last_undegraded":"2023-11-28T21:35:24.590181+0000","last_fullsized":"2023-11-28T21:35:24.590181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.618289+0000","last_change":"2023-11-28T21:35:25.618289+0000","last_active":"2023-11-28T21:35:25.618289+0000","last_peered":"2023-11-28T21:35:25.618289+0000","last_clean":"2023-11-28T21:35:25.618289+0000","last_became_active":"2023-11-28T21:35:25.616978+0000","last_became_peered":"2023-11-28T21:35:25.616978+0000","last_unstale":"2023-11-28T21:35:25.618289+0000","last_undegraded":"2023-11-28T21:35:25.618289+0000","last_fullsized":"2023-11-28T21:35:25.618289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:15:26.380543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570019+0000","last_change":"2023-11-28T21:34:37.313495+0000","last_active":"2023-11-28T21:35:24.570019+0000","last_peered":"2023-11-28T21:35:24.570019+0000","last_clean":"2023-11-28T21:35:24.570019+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:35:24.570019+0000","last_undegraded":"2023-11-28T21:35:24.570019+0000","last_fullsized":"2023-11-28T21:35:24.570019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:32:50.702191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"24'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573439+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:35:24.573439+0000","last_peered":"2023-11-28T21:35:24.573439+0000","last_clean":"2023-11-28T21:35:24.573439+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:35:24.573439+0000","last_undegraded":"2023-11-28T21:35:24.573439+0000","last_fullsized":"2023-11-28T21:35:24.573439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:53.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.792110+0000","last_change":"2023-11-28T21:35:25.792110+0000","last_active":"2023-11-28T21:35:25.792110+0000","last_peered":"2023-11-28T21:35:25.792110+0000","last_clean":"2023-11-28T21:35:25.792110+0000","last_became_active":"2023-11-28T21:35:25.790674+0000","last_became_peered":"2023-11-28T21:35:25.790674+0000","last_unstale":"2023-11-28T21:35:25.792110+0000","last_undegraded":"2023-11-28T21:35:25.792110+0000","last_fullsized":"2023-11-28T21:35:25.792110+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:22:43.106775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569482+0000","last_change":"2023-11-28T21:34:37.326569+0000","last_active":"2023-11-28T21:35:24.569482+0000","last_peered":"2023-11-28T21:35:24.569482+0000","last_clean":"2023-11-28T21:35:24.569482+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:35:24.569482+0000","last_undegraded":"2023-11-28T21:35:24.569482+0000","last_fullsized":"2023-11-28T21:35:24.569482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:39:14.264355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573928+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:35:24.573928+0000","last_peered":"2023-11-28T21:35:24.573928+0000","last_clean":"2023-11-28T21:35:24.573928+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:35:24.573928+0000","last_undegraded":"2023-11-28T21:35:24.573928+0000","last_fullsized":"2023-11-28T21:35:24.573928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574972+0000","last_change":"2023-11-28T21:34:41.483661+0000","last_active":"2023-11-28T21:35:24.574972+0000","last_peered":"2023-11-28T21:35:24.574972+0000","last_clean":"2023-11-28T21:35:24.574972+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:35:24.574972+0000","last_undegraded":"2023-11-28T21:35:24.574972+0000","last_fullsized":"2023-11-28T21:35:24.574972+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:43:37.267639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574562+0000","last_change":"2023-11-28T21:34:37.336922+0000","last_active":"2023-11-28T21:35:24.574562+0000","last_peered":"2023-11-28T21:35:24.574562+0000","last_clean":"2023-11-28T21:35:24.574562+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:35:24.574562+0000","last_undegraded":"2023-11-28T21:35:24.574562+0000","last_fullsized":"2023-11-28T21:35:24.574562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:45.663475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"24'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588244+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:35:24.588244+0000","last_peered":"2023-11-28T21:35:24.588244+0000","last_clean":"2023-11-28T21:35:24.588244+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:35:24.588244+0000","last_undegraded":"2023-11-28T21:35:24.588244+0000","last_fullsized":"2023-11-28T21:35:24.588244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581947+0000","last_change":"2023-11-28T21:34:41.382756+0000","last_active":"2023-11-28T21:35:24.581947+0000","last_peered":"2023-11-28T21:35:24.581947+0000","last_clean":"2023-11-28T21:35:24.581947+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:35:24.581947+0000","last_undegraded":"2023-11-28T21:35:24.581947+0000","last_fullsized":"2023-11-28T21:35:24.581947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:07.402513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579759+0000","last_change":"2023-11-28T21:34:37.320446+0000","last_active":"2023-11-28T21:35:24.579759+0000","last_peered":"2023-11-28T21:35:24.579759+0000","last_clean":"2023-11-28T21:35:24.579759+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:35:24.579759+0000","last_undegraded":"2023-11-28T21:35:24.579759+0000","last_fullsized":"2023-11-28T21:35:24.579759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:33.404441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'4","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.587700+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:35:24.587700+0000","last_peered":"2023-11-28T21:35:24.587700+0000","last_clean":"2023-11-28T21:35:24.587700+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:35:24.587700+0000","last_undegraded":"2023-11-28T21:35:24.587700+0000","last_fullsized":"2023-11-28T21:35:24.587700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574174+0000","last_change":"2023-11-28T21:34:41.485107+0000","last_active":"2023-11-28T21:35:24.574174+0000","last_peered":"2023-11-28T21:35:24.574174+0000","last_clean":"2023-11-28T21:35:24.574174+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:35:24.574174+0000","last_undegraded":"2023-11-28T21:35:24.574174+0000","last_fullsized":"2023-11-28T21:35:24.574174+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570615+0000","last_change":"2023-11-28T21:34:37.345104+0000","last_active":"2023-11-28T21:35:24.570615+0000","last_peered":"2023-11-28T21:35:24.570615+0000","last_clean":"2023-11-28T21:35:24.570615+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:35:24.570615+0000","last_undegraded":"2023-11-28T21:35:24.570615+0000","last_fullsized":"2023-11-28T21:35:24.570615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:25:16.125179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574812+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:35:24.574812+0000","last_peered":"2023-11-28T21:35:24.574812+0000","last_clean":"2023-11-28T21:35:24.574812+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:35:24.574812+0000","last_undegraded":"2023-11-28T21:35:24.574812+0000","last_fullsized":"2023-11-28T21:35:24.574812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:36.968220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569173+0000","last_change":"2023-11-28T21:34:41.475704+0000","last_active":"2023-11-28T21:35:24.569173+0000","last_peered":"2023-11-28T21:35:24.569173+0000","last_clean":"2023-11-28T21:35:24.569173+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:35:24.569173+0000","last_undegraded":"2023-11-28T21:35:24.569173+0000","last_fullsized":"2023-11-28T21:35:24.569173+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:01:34.494307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574121+0000","last_change":"2023-11-28T21:34:37.330831+0000","last_active":"2023-11-28T21:35:24.574121+0000","last_peered":"2023-11-28T21:35:24.574121+0000","last_clean":"2023-11-28T21:35:24.574121+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:35:24.574121+0000","last_undegraded":"2023-11-28T21:35:24.574121+0000","last_fullsized":"2023-11-28T21:35:24.574121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:50:38.612360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"24'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590947+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:35:24.590947+0000","last_peered":"2023-11-28T21:35:24.590947+0000","last_clean":"2023-11-28T21:35:24.590947+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:35:24.590947+0000","last_undegraded":"2023-11-28T21:35:24.590947+0000","last_fullsized":"2023-11-28T21:35:24.590947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578635+0000","last_change":"2023-11-28T21:34:41.475481+0000","last_active":"2023-11-28T21:35:24.578635+0000","last_peered":"2023-11-28T21:35:24.578635+0000","last_clean":"2023-11-28T21:35:24.578635+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:35:24.578635+0000","last_undegraded":"2023-11-28T21:35:24.578635+0000","last_fullsized":"2023-11-28T21:35:24.578635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:22:18.635807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579777+0000","last_change":"2023-11-28T21:34:37.337846+0000","last_active":"2023-11-28T21:35:24.579777+0000","last_peered":"2023-11-28T21:35:24.579777+0000","last_clean":"2023-11-28T21:35:24.579777+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:35:24.579777+0000","last_undegraded":"2023-11-28T21:35:24.579777+0000","last_fullsized":"2023-11-28T21:35:24.579777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:46:21.358603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574182+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:35:24.574182+0000","last_peered":"2023-11-28T21:35:24.574182+0000","last_clean":"2023-11-28T21:35:24.574182+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:35:24.574182+0000","last_undegraded":"2023-11-28T21:35:24.574182+0000","last_fullsized":"2023-11-28T21:35:24.574182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:08:15.270352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.572984+0000","last_change":"2023-11-28T21:34:41.370410+0000","last_active":"2023-11-28T21:35:24.572984+0000","last_peered":"2023-11-28T21:35:24.572984+0000","last_clean":"2023-11-28T21:35:24.572984+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:35:24.572984+0000","last_undegraded":"2023-11-28T21:35:24.572984+0000","last_fullsized":"2023-11-28T21:35:24.572984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:58:03.139624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.582169+0000","last_change":"2023-11-28T21:34:37.324608+0000","last_active":"2023-11-28T21:35:24.582169+0000","last_peered":"2023-11-28T21:35:24.582169+0000","last_clean":"2023-11-28T21:35:24.582169+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:35:24.582169+0000","last_undegraded":"2023-11-28T21:35:24.582169+0000","last_fullsized":"2023-11-28T21:35:24.582169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:13:53.438473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"24'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.575152+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:35:24.575152+0000","last_peered":"2023-11-28T21:35:24.575152+0000","last_clean":"2023-11-28T21:35:24.575152+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:35:24.575152+0000","last_undegraded":"2023-11-28T21:35:24.575152+0000","last_fullsized":"2023-11-28T21:35:24.575152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574069+0000","last_change":"2023-11-28T21:34:41.380599+0000","last_active":"2023-11-28T21:35:24.574069+0000","last_peered":"2023-11-28T21:35:24.574069+0000","last_clean":"2023-11-28T21:35:24.574069+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:35:24.574069+0000","last_undegraded":"2023-11-28T21:35:24.574069+0000","last_fullsized":"2023-11-28T21:35:24.574069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:19.019612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573050+0000","last_change":"2023-11-28T21:34:37.330869+0000","last_active":"2023-11-28T21:35:24.573050+0000","last_peered":"2023-11-28T21:35:24.573050+0000","last_clean":"2023-11-28T21:35:24.573050+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:35:24.573050+0000","last_undegraded":"2023-11-28T21:35:24.573050+0000","last_fullsized":"2023-11-28T21:35:24.573050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:58:55.987235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"24'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578798+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:35:24.578798+0000","last_peered":"2023-11-28T21:35:24.578798+0000","last_clean":"2023-11-28T21:35:24.578798+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:35:24.578798+0000","last_undegraded":"2023-11-28T21:35:24.578798+0000","last_fullsized":"2023-11-28T21:35:24.578798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.575055+0000","last_change":"2023-11-28T21:34:41.371897+0000","last_active":"2023-11-28T21:35:24.575055+0000","last_peered":"2023-11-28T21:35:24.575055+0000","last_clean":"2023-11-28T21:35:24.575055+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:35:24.575055+0000","last_undegraded":"2023-11-28T21:35:24.575055+0000","last_fullsized":"2023-11-28T21:35:24.575055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578402+0000","last_change":"2023-11-28T21:34:37.319833+0000","last_active":"2023-11-28T21:35:24.578402+0000","last_peered":"2023-11-28T21:35:24.578402+0000","last_clean":"2023-11-28T21:35:24.578402+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:35:24.578402+0000","last_undegraded":"2023-11-28T21:35:24.578402+0000","last_fullsized":"2023-11-28T21:35:24.578402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:02:17.779459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588430+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:35:24.588430+0000","last_peered":"2023-11-28T21:35:24.588430+0000","last_clean":"2023-11-28T21:35:24.588430+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:35:24.588430+0000","last_undegraded":"2023-11-28T21:35:24.588430+0000","last_fullsized":"2023-11-28T21:35:24.588430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579573+0000","last_change":"2023-11-28T21:34:41.475053+0000","last_active":"2023-11-28T21:35:24.579573+0000","last_peered":"2023-11-28T21:35:24.579573+0000","last_clean":"2023-11-28T21:35:24.579573+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:35:24.579573+0000","last_undegraded":"2023-11-28T21:35:24.579573+0000","last_fullsized":"2023-11-28T21:35:24.579573+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:32.658001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569672+0000","last_change":"2023-11-28T21:34:37.358890+0000","last_active":"2023-11-28T21:35:24.569672+0000","last_peered":"2023-11-28T21:35:24.569672+0000","last_clean":"2023-11-28T21:35:24.569672+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:35:24.569672+0000","last_undegraded":"2023-11-28T21:35:24.569672+0000","last_fullsized":"2023-11-28T21:35:24.569672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:12:54.878141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.577419+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:35:24.577419+0000","last_peered":"2023-11-28T21:35:24.577419+0000","last_clean":"2023-11-28T21:35:24.577419+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:35:24.577419+0000","last_undegraded":"2023-11-28T21:35:24.577419+0000","last_fullsized":"2023-11-28T21:35:24.577419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":171,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588554+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:35:24.588554+0000","last_peered":"2023-11-28T21:35:24.588554+0000","last_clean":"2023-11-28T21:35:24.588554+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:35:24.588554+0000","last_undegraded":"2023-11-28T21:35:24.588554+0000","last_fullsized":"2023-11-28T21:35:24.588554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.586735+0000","last_change":"2023-11-28T21:34:41.477522+0000","last_active":"2023-11-28T21:35:24.586735+0000","last_peered":"2023-11-28T21:35:24.586735+0000","last_clean":"2023-11-28T21:35:24.586735+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:35:24.586735+0000","last_undegraded":"2023-11-28T21:35:24.586735+0000","last_fullsized":"2023-11-28T21:35:24.586735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:36:36.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590606+0000","last_change":"2023-11-28T21:34:37.350441+0000","last_active":"2023-11-28T21:35:24.590606+0000","last_peered":"2023-11-28T21:35:24.590606+0000","last_clean":"2023-11-28T21:35:24.590606+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:35:24.590606+0000","last_undegraded":"2023-11-28T21:35:24.590606+0000","last_fullsized":"2023-11-28T21:35:24.590606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:54.775495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579251+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:35:24.579251+0000","last_peered":"2023-11-28T21:35:24.579251+0000","last_clean":"2023-11-28T21:35:24.579251+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:35:24.579251+0000","last_undegraded":"2023-11-28T21:35:24.579251+0000","last_fullsized":"2023-11-28T21:35:24.579251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.583047+0000","last_change":"2023-11-28T21:34:41.485154+0000","last_active":"2023-11-28T21:35:24.583047+0000","last_peered":"2023-11-28T21:35:24.583047+0000","last_clean":"2023-11-28T21:35:24.583047+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:35:24.583047+0000","last_undegraded":"2023-11-28T21:35:24.583047+0000","last_fullsized":"2023-11-28T21:35:24.583047+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:35:21.006172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.576292+0000","last_change":"2023-11-28T21:34:37.345652+0000","last_active":"2023-11-28T21:35:24.576292+0000","last_peered":"2023-11-28T21:35:24.576292+0000","last_clean":"2023-11-28T21:35:24.576292+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:35:24.576292+0000","last_undegraded":"2023-11-28T21:35:24.576292+0000","last_fullsized":"2023-11-28T21:35:24.576292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:21:17.474200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574398+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:35:24.574398+0000","last_peered":"2023-11-28T21:35:24.574398+0000","last_clean":"2023-11-28T21:35:24.574398+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:35:24.574398+0000","last_undegraded":"2023-11-28T21:35:24.574398+0000","last_fullsized":"2023-11-28T21:35:24.574398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:48.073446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578747+0000","last_change":"2023-11-28T21:34:41.370764+0000","last_active":"2023-11-28T21:35:24.578747+0000","last_peered":"2023-11-28T21:35:24.578747+0000","last_clean":"2023-11-28T21:35:24.578747+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:35:24.578747+0000","last_undegraded":"2023-11-28T21:35:24.578747+0000","last_fullsized":"2023-11-28T21:35:24.578747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:06:42.297098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.594934+0000","last_change":"2023-11-28T21:35:25.594934+0000","last_active":"2023-11-28T21:35:25.594934+0000","last_peered":"2023-11-28T21:35:25.594934+0000","last_clean":"2023-11-28T21:35:25.594934+0000","last_became_active":"2023-11-28T21:35:25.594533+0000","last_became_peered":"2023-11-28T21:35:25.594533+0000","last_unstale":"2023-11-28T21:35:25.594934+0000","last_undegraded":"2023-11-28T21:35:25.594934+0000","last_fullsized":"2023-11-28T21:35:25.594934+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:05.168934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579639+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:35:24.579639+0000","last_peered":"2023-11-28T21:35:24.579639+0000","last_clean":"2023-11-28T21:35:24.579639+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:35:24.579639+0000","last_undegraded":"2023-11-28T21:35:24.579639+0000","last_fullsized":"2023-11-28T21:35:24.579639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573161+0000","last_change":"2023-11-28T21:34:41.477127+0000","last_active":"2023-11-28T21:35:24.573161+0000","last_peered":"2023-11-28T21:35:24.573161+0000","last_clean":"2023-11-28T21:35:24.573161+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:35:24.573161+0000","last_undegraded":"2023-11-28T21:35:24.573161+0000","last_fullsized":"2023-11-28T21:35:24.573161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:32.798817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590603+0000","last_change":"2023-11-28T21:34:37.355693+0000","last_active":"2023-11-28T21:35:24.590603+0000","last_peered":"2023-11-28T21:35:24.590603+0000","last_clean":"2023-11-28T21:35:24.590603+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:35:24.590603+0000","last_undegraded":"2023-11-28T21:35:24.590603+0000","last_fullsized":"2023-11-28T21:35:24.590603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:12.574669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.583538+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:35:24.583538+0000","last_peered":"2023-11-28T21:35:24.583538+0000","last_clean":"2023-11-28T21:35:24.583538+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:35:24.583538+0000","last_undegraded":"2023-11-28T21:35:24.583538+0000","last_fullsized":"2023-11-28T21:35:24.583538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573923+0000","last_change":"2023-11-28T21:34:41.382468+0000","last_active":"2023-11-28T21:35:24.573923+0000","last_peered":"2023-11-28T21:35:24.573923+0000","last_clean":"2023-11-28T21:35:24.573923+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:35:24.573923+0000","last_undegraded":"2023-11-28T21:35:24.573923+0000","last_fullsized":"2023-11-28T21:35:24.573923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:39:17.410943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588771+0000","last_change":"2023-11-28T21:34:37.353232+0000","last_active":"2023-11-28T21:35:24.588771+0000","last_peered":"2023-11-28T21:35:24.588771+0000","last_clean":"2023-11-28T21:35:24.588771+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:35:24.588771+0000","last_undegraded":"2023-11-28T21:35:24.588771+0000","last_fullsized":"2023-11-28T21:35:24.588771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:11.285378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'15","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579450+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:35:24.579450+0000","last_peered":"2023-11-28T21:35:24.579450+0000","last_clean":"2023-11-28T21:35:24.579450+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:35:24.579450+0000","last_undegraded":"2023-11-28T21:35:24.579450+0000","last_fullsized":"2023-11-28T21:35:24.579450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579344+0000","last_change":"2023-11-28T21:34:41.385400+0000","last_active":"2023-11-28T21:35:24.579344+0000","last_peered":"2023-11-28T21:35:24.579344+0000","last_clean":"2023-11-28T21:35:24.579344+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:35:24.579344+0000","last_undegraded":"2023-11-28T21:35:24.579344+0000","last_fullsized":"2023-11-28T21:35:24.579344+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:36.433439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.584714+0000","last_change":"2023-11-28T21:34:37.333461+0000","last_active":"2023-11-28T21:35:24.584714+0000","last_peered":"2023-11-28T21:35:24.584714+0000","last_clean":"2023-11-28T21:35:24.584714+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:35:24.584714+0000","last_undegraded":"2023-11-28T21:35:24.584714+0000","last_fullsized":"2023-11-28T21:35:24.584714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:30:25.513524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"26'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.787524+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:35:25.787524+0000","last_peered":"2023-11-28T21:35:25.787524+0000","last_clean":"2023-11-28T21:35:25.787524+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:35:25.787524+0000","last_undegraded":"2023-11-28T21:35:25.787524+0000","last_fullsized":"2023-11-28T21:35:25.787524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":842,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570442+0000","last_change":"2023-11-28T21:34:41.479919+0000","last_active":"2023-11-28T21:35:24.570442+0000","last_peered":"2023-11-28T21:35:24.570442+0000","last_clean":"2023-11-28T21:35:24.570442+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:35:24.570442+0000","last_undegraded":"2023-11-28T21:35:24.570442+0000","last_fullsized":"2023-11-28T21:35:24.570442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:11.183117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.589994+0000","last_change":"2023-11-28T21:34:37.355729+0000","last_active":"2023-11-28T21:35:24.589994+0000","last_peered":"2023-11-28T21:35:24.589994+0000","last_clean":"2023-11-28T21:35:24.589994+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:35:24.589994+0000","last_undegraded":"2023-11-28T21:35:24.589994+0000","last_fullsized":"2023-11-28T21:35:24.589994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:39.178378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"24'3","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574908+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:35:24.574908+0000","last_peered":"2023-11-28T21:35:24.574908+0000","last_clean":"2023-11-28T21:35:24.574908+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:35:24.574908+0000","last_undegraded":"2023-11-28T21:35:24.574908+0000","last_fullsized":"2023-11-28T21:35:24.574908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569708+0000","last_change":"2023-11-28T21:34:41.478483+0000","last_active":"2023-11-28T21:35:24.569708+0000","last_peered":"2023-11-28T21:35:24.569708+0000","last_clean":"2023-11-28T21:35:24.569708+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:35:24.569708+0000","last_undegraded":"2023-11-28T21:35:24.569708+0000","last_fullsized":"2023-11-28T21:35:24.569708+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:25:54.342692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581071+0000","last_change":"2023-11-28T21:34:37.345476+0000","last_active":"2023-11-28T21:35:24.581071+0000","last_peered":"2023-11-28T21:35:24.581071+0000","last_clean":"2023-11-28T21:35:24.581071+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:35:24.581071+0000","last_undegraded":"2023-11-28T21:35:24.581071+0000","last_fullsized":"2023-11-28T21:35:24.581071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.660412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'6","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579023+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:35:24.579023+0000","last_peered":"2023-11-28T21:35:24.579023+0000","last_clean":"2023-11-28T21:35:24.579023+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:35:24.579023+0000","last_undegraded":"2023-11-28T21:35:24.579023+0000","last_fullsized":"2023-11-28T21:35:24.579023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:28.694184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579447+0000","last_change":"2023-11-28T21:34:41.477926+0000","last_active":"2023-11-28T21:35:24.579447+0000","last_peered":"2023-11-28T21:35:24.579447+0000","last_clean":"2023-11-28T21:35:24.579447+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:35:24.579447+0000","last_undegraded":"2023-11-28T21:35:24.579447+0000","last_fullsized":"2023-11-28T21:35:24.579447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:26.403506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590387+0000","last_change":"2023-11-28T21:34:37.343885+0000","last_active":"2023-11-28T21:35:24.590387+0000","last_peered":"2023-11-28T21:35:24.590387+0000","last_clean":"2023-11-28T21:35:24.590387+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:35:24.590387+0000","last_undegraded":"2023-11-28T21:35:24.590387+0000","last_fullsized":"2023-11-28T21:35:24.590387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:36.255834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'11","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.589106+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:35:24.589106+0000","last_peered":"2023-11-28T21:35:24.589106+0000","last_clean":"2023-11-28T21:35:24.589106+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:35:24.589106+0000","last_undegraded":"2023-11-28T21:35:24.589106+0000","last_fullsized":"2023-11-28T21:35:24.589106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.791874+0000","last_change":"2023-11-28T21:35:25.791874+0000","last_active":"2023-11-28T21:35:25.791874+0000","last_peered":"2023-11-28T21:35:25.791874+0000","last_clean":"2023-11-28T21:35:25.791874+0000","last_became_active":"2023-11-28T21:35:25.790293+0000","last_became_peered":"2023-11-28T21:35:25.790293+0000","last_unstale":"2023-11-28T21:35:25.791874+0000","last_undegraded":"2023-11-28T21:35:25.791874+0000","last_fullsized":"2023-11-28T21:35:25.791874+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:21:17.947428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.582575+0000","last_change":"2023-11-28T21:34:37.345635+0000","last_active":"2023-11-28T21:35:24.582575+0000","last_peered":"2023-11-28T21:35:24.582575+0000","last_clean":"2023-11-28T21:35:24.582575+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:35:24.582575+0000","last_undegraded":"2023-11-28T21:35:24.582575+0000","last_fullsized":"2023-11-28T21:35:24.582575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:52:12.144267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'4","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574361+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:35:24.574361+0000","last_peered":"2023-11-28T21:35:24.574361+0000","last_clean":"2023-11-28T21:35:24.574361+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:35:24.574361+0000","last_undegraded":"2023-11-28T21:35:24.574361+0000","last_fullsized":"2023-11-28T21:35:24.574361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:35.266650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581717+0000","last_change":"2023-11-28T21:34:41.365758+0000","last_active":"2023-11-28T21:35:24.581717+0000","last_peered":"2023-11-28T21:35:24.581717+0000","last_clean":"2023-11-28T21:35:24.581717+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:35:24.581717+0000","last_undegraded":"2023-11-28T21:35:24.581717+0000","last_fullsized":"2023-11-28T21:35:24.581717+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:01:02.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.580813+0000","last_change":"2023-11-28T21:34:37.333607+0000","last_active":"2023-11-28T21:35:24.580813+0000","last_peered":"2023-11-28T21:35:24.580813+0000","last_clean":"2023-11-28T21:35:24.580813+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:35:24.580813+0000","last_undegraded":"2023-11-28T21:35:24.580813+0000","last_fullsized":"2023-11-28T21:35:24.580813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:13:01.639908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"24'2","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.576877+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:35:24.576877+0000","last_peered":"2023-11-28T21:35:24.576877+0000","last_clean":"2023-11-28T21:35:24.576877+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:35:24.576877+0000","last_undegraded":"2023-11-28T21:35:24.576877+0000","last_fullsized":"2023-11-28T21:35:24.576877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569623+0000","last_change":"2023-11-28T21:34:41.380413+0000","last_active":"2023-11-28T21:35:24.569623+0000","last_peered":"2023-11-28T21:35:24.569623+0000","last_clean":"2023-11-28T21:35:24.569623+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:35:24.569623+0000","last_undegraded":"2023-11-28T21:35:24.569623+0000","last_fullsized":"2023-11-28T21:35:24.569623+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:20:28.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.595606+0000","last_change":"2023-11-28T21:35:25.595606+0000","last_active":"2023-11-28T21:35:25.595606+0000","last_peered":"2023-11-28T21:35:25.595606+0000","last_clean":"2023-11-28T21:35:25.595606+0000","last_became_active":"2023-11-28T21:35:25.595386+0000","last_became_peered":"2023-11-28T21:35:25.595386+0000","last_unstale":"2023-11-28T21:35:25.595606+0000","last_undegraded":"2023-11-28T21:35:25.595606+0000","last_fullsized":"2023-11-28T21:35:25.595606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:16:44.887005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"24'3","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.587348+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:35:24.587348+0000","last_peered":"2023-11-28T21:35:24.587348+0000","last_clean":"2023-11-28T21:35:24.587348+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:35:24.587348+0000","last_undegraded":"2023-11-28T21:35:24.587348+0000","last_fullsized":"2023-11-28T21:35:24.587348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569699+0000","last_change":"2023-11-28T21:34:41.475185+0000","last_active":"2023-11-28T21:35:24.569699+0000","last_peered":"2023-11-28T21:35:24.569699+0000","last_clean":"2023-11-28T21:35:24.569699+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:35:24.569699+0000","last_undegraded":"2023-11-28T21:35:24.569699+0000","last_fullsized":"2023-11-28T21:35:24.569699+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:12:09.389881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578519+0000","last_change":"2023-11-28T21:34:41.479956+0000","last_active":"2023-11-28T21:35:24.578519+0000","last_peered":"2023-11-28T21:35:24.578519+0000","last_clean":"2023-11-28T21:35:24.578519+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:35:24.578519+0000","last_undegraded":"2023-11-28T21:35:24.578519+0000","last_fullsized":"2023-11-28T21:35:24.578519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:46.004852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.589823+0000","last_change":"2023-11-28T21:35:25.589823+0000","last_active":"2023-11-28T21:35:25.589823+0000","last_peered":"2023-11-28T21:35:25.589823+0000","last_clean":"2023-11-28T21:35:25.589823+0000","last_became_active":"2023-11-28T21:35:25.589378+0000","last_became_peered":"2023-11-28T21:35:25.589378+0000","last_unstale":"2023-11-28T21:35:25.589823+0000","last_undegraded":"2023-11-28T21:35:25.589823+0000","last_fullsized":"2023-11-28T21:35:25.589823+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:11.796380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"24'1","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573550+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:35:24.573550+0000","last_peered":"2023-11-28T21:35:24.573550+0000","last_clean":"2023-11-28T21:35:24.573550+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:35:24.573550+0000","last_undegraded":"2023-11-28T21:35:24.573550+0000","last_fullsized":"2023-11-28T21:35:24.573550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.586981+0000","last_change":"2023-11-28T21:34:41.385969+0000","last_active":"2023-11-28T21:35:24.586981+0000","last_peered":"2023-11-28T21:35:24.586981+0000","last_clean":"2023-11-28T21:35:24.586981+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:35:24.586981+0000","last_undegraded":"2023-11-28T21:35:24.586981+0000","last_fullsized":"2023-11-28T21:35:24.586981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:01.692549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569384+0000","last_change":"2023-11-28T21:34:37.314087+0000","last_active":"2023-11-28T21:35:24.569384+0000","last_peered":"2023-11-28T21:35:24.569384+0000","last_clean":"2023-11-28T21:35:24.569384+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:35:24.569384+0000","last_undegraded":"2023-11-28T21:35:24.569384+0000","last_fullsized":"2023-11-28T21:35:24.569384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:25:11.912313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"24'2","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570172+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:35:24.570172+0000","last_peered":"2023-11-28T21:35:24.570172+0000","last_clean":"2023-11-28T21:35:24.570172+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:35:24.570172+0000","last_undegraded":"2023-11-28T21:35:24.570172+0000","last_fullsized":"2023-11-28T21:35:24.570172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'3","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.583399+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:35:24.583399+0000","last_peered":"2023-11-28T21:35:24.583399+0000","last_clean":"2023-11-28T21:35:24.583399+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:35:24.583399+0000","last_undegraded":"2023-11-28T21:35:24.583399+0000","last_fullsized":"2023-11-28T21:35:24.583399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578925+0000","last_change":"2023-11-28T21:34:37.319801+0000","last_active":"2023-11-28T21:35:24.578925+0000","last_peered":"2023-11-28T21:35:24.578925+0000","last_clean":"2023-11-28T21:35:24.578925+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:35:24.578925+0000","last_undegraded":"2023-11-28T21:35:24.578925+0000","last_fullsized":"2023-11-28T21:35:24.578925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:40:13.251522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578964+0000","last_change":"2023-11-28T21:34:41.370523+0000","last_active":"2023-11-28T21:35:24.578964+0000","last_peered":"2023-11-28T21:35:24.578964+0000","last_clean":"2023-11-28T21:35:24.578964+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:35:24.578964+0000","last_undegraded":"2023-11-28T21:35:24.578964+0000","last_fullsized":"2023-11-28T21:35:24.578964+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:17:36.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'5","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569956+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:35:24.569956+0000","last_peered":"2023-11-28T21:35:24.569956+0000","last_clean":"2023-11-28T21:35:24.569956+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:35:24.569956+0000","last_undegraded":"2023-11-28T21:35:24.569956+0000","last_fullsized":"2023-11-28T21:35:24.569956+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581212+0000","last_change":"2023-11-28T21:34:37.328809+0000","last_active":"2023-11-28T21:35:24.581212+0000","last_peered":"2023-11-28T21:35:24.581212+0000","last_clean":"2023-11-28T21:35:24.581212+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:35:24.581212+0000","last_undegraded":"2023-11-28T21:35:24.581212+0000","last_fullsized":"2023-11-28T21:35:24.581212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:27.300090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570686+0000","last_change":"2023-11-28T21:34:41.363087+0000","last_active":"2023-11-28T21:35:24.570686+0000","last_peered":"2023-11-28T21:35:24.570686+0000","last_clean":"2023-11-28T21:35:24.570686+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:35:24.570686+0000","last_undegraded":"2023-11-28T21:35:24.570686+0000","last_fullsized":"2023-11-28T21:35:24.570686+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:06:35.046132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.584352+0000","last_change":"2023-11-28T21:34:37.333670+0000","last_active":"2023-11-28T21:35:24.584352+0000","last_peered":"2023-11-28T21:35:24.584352+0000","last_clean":"2023-11-28T21:35:24.584352+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:35:24.584352+0000","last_undegraded":"2023-11-28T21:35:24.584352+0000","last_fullsized":"2023-11-28T21:35:24.584352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:32.598014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578736+0000","last_change":"2023-11-28T21:34:37.347101+0000","last_active":"2023-11-28T21:35:24.578736+0000","last_peered":"2023-11-28T21:35:24.578736+0000","last_clean":"2023-11-28T21:35:24.578736+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:35:24.578736+0000","last_undegraded":"2023-11-28T21:35:24.578736+0000","last_fullsized":"2023-11-28T21:35:24.578736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:39:43.186110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574495+0000","last_change":"2023-11-28T21:34:37.338576+0000","last_active":"2023-11-28T21:35:24.574495+0000","last_peered":"2023-11-28T21:35:24.574495+0000","last_clean":"2023-11-28T21:35:24.574495+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:35:24.574495+0000","last_undegraded":"2023-11-28T21:35:24.574495+0000","last_fullsized":"2023-11-28T21:35:24.574495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:53:02.356097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578831+0000","last_change":"2023-11-28T21:34:37.316270+0000","last_active":"2023-11-28T21:35:24.578831+0000","last_peered":"2023-11-28T21:35:24.578831+0000","last_clean":"2023-11-28T21:35:24.578831+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:35:24.578831+0000","last_undegraded":"2023-11-28T21:35:24.578831+0000","last_fullsized":"2023-11-28T21:35:24.578831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:43.604671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570075+0000","last_change":"2023-11-28T21:34:37.334973+0000","last_active":"2023-11-28T21:35:24.570075+0000","last_peered":"2023-11-28T21:35:24.570075+0000","last_clean":"2023-11-28T21:35:24.570075+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:35:24.570075+0000","last_undegraded":"2023-11-28T21:35:24.570075+0000","last_fullsized":"2023-11-28T21:35:24.570075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:13:55.050945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573304+0000","last_change":"2023-11-28T21:34:37.318155+0000","last_active":"2023-11-28T21:35:24.573304+0000","last_peered":"2023-11-28T21:35:24.573304+0000","last_clean":"2023-11-28T21:35:24.573304+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:35:24.573304+0000","last_undegraded":"2023-11-28T21:35:24.573304+0000","last_fullsized":"2023-11-28T21:35:24.573304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:29.840255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.587050+0000","last_change":"2023-11-28T21:34:37.345758+0000","last_active":"2023-11-28T21:35:24.587050+0000","last_peered":"2023-11-28T21:35:24.587050+0000","last_clean":"2023-11-28T21:35:24.587050+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:35:24.587050+0000","last_undegraded":"2023-11-28T21:35:24.587050+0000","last_fullsized":"2023-11-28T21:35:24.587050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:26.444654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570520+0000","last_change":"2023-11-28T21:34:37.334989+0000","last_active":"2023-11-28T21:35:24.570520+0000","last_peered":"2023-11-28T21:35:24.570520+0000","last_clean":"2023-11-28T21:35:24.570520+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:35:24.570520+0000","last_undegraded":"2023-11-28T21:35:24.570520+0000","last_fullsized":"2023-11-28T21:35:24.570520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:36:57.421286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573377+0000","last_change":"2023-11-28T21:34:37.333075+0000","last_active":"2023-11-28T21:35:24.573377+0000","last_peered":"2023-11-28T21:35:24.573377+0000","last_clean":"2023-11-28T21:35:24.573377+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:35:24.573377+0000","last_undegraded":"2023-11-28T21:35:24.573377+0000","last_fullsized":"2023-11-28T21:35:24.573377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:50:43.748349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.570076+0000","last_change":"2023-11-28T21:34:37.337230+0000","last_active":"2023-11-28T21:35:24.570076+0000","last_peered":"2023-11-28T21:35:24.570076+0000","last_clean":"2023-11-28T21:35:24.570076+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:35:24.570076+0000","last_undegraded":"2023-11-28T21:35:24.570076+0000","last_fullsized":"2023-11-28T21:35:24.570076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:48:45.994559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579747+0000","last_change":"2023-11-28T21:34:37.349894+0000","last_active":"2023-11-28T21:35:24.579747+0000","last_peered":"2023-11-28T21:35:24.579747+0000","last_clean":"2023-11-28T21:35:24.579747+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:35:24.579747+0000","last_undegraded":"2023-11-28T21:35:24.579747+0000","last_fullsized":"2023-11-28T21:35:24.579747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:04:10.293911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578635+0000","last_change":"2023-11-28T21:34:37.344269+0000","last_active":"2023-11-28T21:35:24.578635+0000","last_peered":"2023-11-28T21:35:24.578635+0000","last_clean":"2023-11-28T21:35:24.578635+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:35:24.578635+0000","last_undegraded":"2023-11-28T21:35:24.578635+0000","last_fullsized":"2023-11-28T21:35:24.578635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:03.204740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588662+0000","last_change":"2023-11-28T21:34:37.353059+0000","last_active":"2023-11-28T21:35:24.588662+0000","last_peered":"2023-11-28T21:35:24.588662+0000","last_clean":"2023-11-28T21:35:24.588662+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:35:24.588662+0000","last_undegraded":"2023-11-28T21:35:24.588662+0000","last_fullsized":"2023-11-28T21:35:24.588662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:45:12.640797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.590822+0000","last_change":"2023-11-28T21:34:37.352965+0000","last_active":"2023-11-28T21:35:24.590822+0000","last_peered":"2023-11-28T21:35:24.590822+0000","last_clean":"2023-11-28T21:35:24.590822+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:35:24.590822+0000","last_undegraded":"2023-11-28T21:35:24.590822+0000","last_fullsized":"2023-11-28T21:35:24.590822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:30:42.087715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579567+0000","last_change":"2023-11-28T21:34:37.336830+0000","last_active":"2023-11-28T21:35:24.579567+0000","last_peered":"2023-11-28T21:35:24.579567+0000","last_clean":"2023-11-28T21:35:24.579567+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:35:24.579567+0000","last_undegraded":"2023-11-28T21:35:24.579567+0000","last_fullsized":"2023-11-28T21:35:24.579567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:57.318311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.578322+0000","last_change":"2023-11-28T21:34:37.349181+0000","last_active":"2023-11-28T21:35:24.578322+0000","last_peered":"2023-11-28T21:35:24.578322+0000","last_clean":"2023-11-28T21:35:24.578322+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:35:24.578322+0000","last_undegraded":"2023-11-28T21:35:24.578322+0000","last_fullsized":"2023-11-28T21:35:24.578322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:27:59.989299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573318+0000","last_change":"2023-11-28T21:34:37.336668+0000","last_active":"2023-11-28T21:35:24.573318+0000","last_peered":"2023-11-28T21:35:24.573318+0000","last_clean":"2023-11-28T21:35:24.573318+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:35:24.573318+0000","last_undegraded":"2023-11-28T21:35:24.573318+0000","last_fullsized":"2023-11-28T21:35:24.573318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:06.416717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.573851+0000","last_change":"2023-11-28T21:34:37.332790+0000","last_active":"2023-11-28T21:35:24.573851+0000","last_peered":"2023-11-28T21:35:24.573851+0000","last_clean":"2023-11-28T21:35:24.573851+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:35:24.573851+0000","last_undegraded":"2023-11-28T21:35:24.573851+0000","last_fullsized":"2023-11-28T21:35:24.573851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:32:45.920387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.581071+0000","last_change":"2023-11-28T21:34:37.320794+0000","last_active":"2023-11-28T21:35:24.581071+0000","last_peered":"2023-11-28T21:35:24.581071+0000","last_clean":"2023-11-28T21:35:24.581071+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:35:24.581071+0000","last_undegraded":"2023-11-28T21:35:24.581071+0000","last_fullsized":"2023-11-28T21:35:24.581071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:16:46.347683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569545+0000","last_change":"2023-11-28T21:34:37.337726+0000","last_active":"2023-11-28T21:35:24.569545+0000","last_peered":"2023-11-28T21:35:24.569545+0000","last_clean":"2023-11-28T21:35:24.569545+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:35:24.569545+0000","last_undegraded":"2023-11-28T21:35:24.569545+0000","last_fullsized":"2023-11-28T21:35:24.569545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:38:43.884668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.582359+0000","last_change":"2023-11-28T21:34:37.334677+0000","last_active":"2023-11-28T21:35:24.582359+0000","last_peered":"2023-11-28T21:35:24.582359+0000","last_clean":"2023-11-28T21:35:24.582359+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:35:24.582359+0000","last_undegraded":"2023-11-28T21:35:24.582359+0000","last_fullsized":"2023-11-28T21:35:24.582359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:05:54.981899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.587132+0000","last_change":"2023-11-28T21:34:37.354861+0000","last_active":"2023-11-28T21:35:24.587132+0000","last_peered":"2023-11-28T21:35:24.587132+0000","last_clean":"2023-11-28T21:35:24.587132+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:35:24.587132+0000","last_undegraded":"2023-11-28T21:35:24.587132+0000","last_fullsized":"2023-11-28T21:35:24.587132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:30.602284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.587935+0000","last_change":"2023-11-28T21:34:37.347675+0000","last_active":"2023-11-28T21:35:24.587935+0000","last_peered":"2023-11-28T21:35:24.587935+0000","last_clean":"2023-11-28T21:35:24.587935+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:35:24.587935+0000","last_undegraded":"2023-11-28T21:35:24.587935+0000","last_fullsized":"2023-11-28T21:35:24.587935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:51:38.803896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-11-28T21:35:25.599601+0000","last_change":"2023-11-28T21:35:25.599601+0000","last_active":"2023-11-28T21:35:25.599601+0000","last_peered":"2023-11-28T21:35:25.599601+0000","last_clean":"2023-11-28T21:35:25.599601+0000","last_became_active":"2023-11-28T21:35:25.598839+0000","last_became_peered":"2023-11-28T21:35:25.598839+0000","last_unstale":"2023-11-28T21:35:25.599601+0000","last_undegraded":"2023-11-28T21:35:25.599601+0000","last_fullsized":"2023-11-28T21:35:25.599601+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:51:09.247433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.569279+0000","last_change":"2023-11-28T21:34:37.358851+0000","last_active":"2023-11-28T21:35:24.569279+0000","last_peered":"2023-11-28T21:35:24.569279+0000","last_clean":"2023-11-28T21:35:24.569279+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:35:24.569279+0000","last_undegraded":"2023-11-28T21:35:24.569279+0000","last_fullsized":"2023-11-28T21:35:24.569279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:34.378519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.588726+0000","last_change":"2023-11-28T21:34:37.343972+0000","last_active":"2023-11-28T21:35:24.588726+0000","last_peered":"2023-11-28T21:35:24.588726+0000","last_clean":"2023-11-28T21:35:24.588726+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:35:24.588726+0000","last_undegraded":"2023-11-28T21:35:24.588726+0000","last_fullsized":"2023-11-28T21:35:24.588726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:56:00.591888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574264+0000","last_change":"2023-11-28T21:34:37.346299+0000","last_active":"2023-11-28T21:35:24.574264+0000","last_peered":"2023-11-28T21:35:24.574264+0000","last_clean":"2023-11-28T21:35:24.574264+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:35:24.574264+0000","last_undegraded":"2023-11-28T21:35:24.574264+0000","last_fullsized":"2023-11-28T21:35:24.574264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:35:06.065622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.589786+0000","last_change":"2023-11-28T21:34:37.343410+0000","last_active":"2023-11-28T21:35:24.589786+0000","last_peered":"2023-11-28T21:35:24.589786+0000","last_clean":"2023-11-28T21:35:24.589786+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:35:24.589786+0000","last_undegraded":"2023-11-28T21:35:24.589786+0000","last_fullsized":"2023-11-28T21:35:24.589786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:29:45.070478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.575841+0000","last_change":"2023-11-28T21:34:37.348270+0000","last_active":"2023-11-28T21:35:24.575841+0000","last_peered":"2023-11-28T21:35:24.575841+0000","last_clean":"2023-11-28T21:35:24.575841+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:35:24.575841+0000","last_undegraded":"2023-11-28T21:35:24.575841+0000","last_fullsized":"2023-11-28T21:35:24.575841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:36:10.938478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.579215+0000","last_change":"2023-11-28T21:34:37.330786+0000","last_active":"2023-11-28T21:35:24.579215+0000","last_peered":"2023-11-28T21:35:24.579215+0000","last_clean":"2023-11-28T21:35:24.579215+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:35:24.579215+0000","last_undegraded":"2023-11-28T21:35:24.579215+0000","last_fullsized":"2023-11-28T21:35:24.579215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:52:36.738867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.574738+0000","last_change":"2023-11-28T21:34:37.318725+0000","last_active":"2023-11-28T21:35:24.574738+0000","last_peered":"2023-11-28T21:35:24.574738+0000","last_clean":"2023-11-28T21:35:24.574738+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:35:24.574738+0000","last_undegraded":"2023-11-28T21:35:24.574738+0000","last_fullsized":"2023-11-28T21:35:24.574738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:49:21.775066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T21:35:24.591700+0000","last_change":"2023-11-28T21:34:37.349829+0000","last_active":"2023-11-28T21:35:24.591700+0000","last_peered":"2023-11-28T21:35:24.591700+0000","last_clean":"2023-11-28T21:35:24.591700+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:35:24.591700+0000","last_undegraded":"2023-11-28T21:35:24.591700+0000","last_fullsized":"2023-11-28T21:35:24.591700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:28:07.752862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7705,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":15410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":705445,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.52100000000000002}]},{"osd":1,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.71999999999999997}]},{"osd":3,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.5629999999999997,"5min":4.5629999999999997,"15min":4.5629999999999997},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.998}]},{"osd":4,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.93700000000000006}]},{"osd":5,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.86299999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.0329999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":118534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":115139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":114729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":118314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":114579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":114705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":703770,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.33100000000000002}]},{"osd":2,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.875}]},{"osd":6,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Tue Nov 28 21:35:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.68400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:35:32.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.796+0000 7fc1389d1640 1 Processor -- start 2023-11-28T21:35:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.796+0000 7fc1389d1640 1 -- start start 2023-11-28T21:35:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc1389d1640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc134150c20 0x7fc134156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc1389d1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc13406e300 0x7fc1340c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc1389d1640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc134067410 con 0x7fc13406cf40 2023-11-28T21:35:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc1389d1640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13406e090 con 0x7fc13406e300 2023-11-28T21:35:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc1389d1640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc134157f30 con 0x7fc134150c20 2023-11-28T21:35:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc12bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc13406e300 0x7fc1340c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc132d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc13406cf40 0x7fc134158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45242/0 (socket says 172.21.15.12:45242) 2023-11-28T21:35:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc132d9d640 1 -- 172.21.15.12:0/332015299 learned_addr learned my addr 172.21.15.12:0/332015299 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc12bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc13406e300 0x7fc1340c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41718/0 (socket says 172.21.15.12:41718) 2023-11-28T21:35:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc13259c640 1 --2- 172.21.15.12:0/332015299 >> v2:172.21.15.144:3300/0 conn(0x7fc134150c20 0x7fc134156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc12bfff640 1 -- 172.21.15.12:0/332015299 >> v2:172.21.15.144:3300/0 conn(0x7fc134150c20 msgr2=0x7fc134156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc12bfff640 1 --2- 172.21.15.12:0/332015299 >> v2:172.21.15.144:3300/0 conn(0x7fc134150c20 0x7fc134156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc12bfff640 1 -- 172.21.15.12:0/332015299 >> v1:172.21.15.12:6789/0 conn(0x7fc13406cf40 legacy=0x7fc134158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.800+0000 7fc12bfff640 1 -- 172.21.15.12:0/332015299 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc134151000 con 0x7fc13406e300 2023-11-28T21:35:32.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc12bfff640 1 --2- 172.21.15.12:0/332015299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13406e300 0x7fc1340c6dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc13406c6a0 tx=0x7fc11c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba38f55afa2dc0d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:32.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc131d9b640 1 -- 172.21.15.12:0/332015299 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c0ab600 con 0x7fc13406e300 2023-11-28T21:35:32.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc131d9b640 1 -- 172.21.15.12:0/332015299 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11c0a7050 con 0x7fc13406e300 2023-11-28T21:35:32.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc131d9b640 1 -- 172.21.15.12:0/332015299 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c0ab980 con 0x7fc13406e300 2023-11-28T21:35:32.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc1389d1640 1 -- 172.21.15.12:0/332015299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13406e300 msgr2=0x7fc1340c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc1389d1640 1 --2- 172.21.15.12:0/332015299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13406e300 0x7fc1340c6dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc13406c6a0 tx=0x7fc11c01c460 comp rx=0 tx=0).stop 2023-11-28T21:35:32.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc1389d1640 1 -- 172.21.15.12:0/332015299 shutdown_connections 2023-11-28T21:35:32.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc1389d1640 1 --2- 172.21.15.12:0/332015299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc13406e300 0x7fc1340c6dd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc1389d1640 1 --2- 172.21.15.12:0/332015299 >> v2:172.21.15.144:3300/0 conn(0x7fc134150c20 0x7fc134156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc1389d1640 1 -- 172.21.15.12:0/332015299 >> 172.21.15.12:0/332015299 conn(0x7fc134066360 msgr2=0x7fc134066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:32.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc1389d1640 1 -- 172.21.15.12:0/332015299 shutdown_connections 2023-11-28T21:35:32.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc1389d1640 1 -- 172.21.15.12:0/332015299 wait complete. 2023-11-28T21:35:32.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc1389d1640 1 Processor -- start 2023-11-28T21:35:32.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.804+0000 7fc1389d1640 1 -- start start 2023-11-28T21:35:32.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc1389d1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc13406cf40 0x7fc1341f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc1389d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc13406e300 0x7fc1341ef190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc1389d1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc134150c20 0x7fc13414c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc1389d1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1341f2e20 con 0x7fc134150c20 2023-11-28T21:35:32.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc1389d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13415c010 con 0x7fc13406e300 2023-11-28T21:35:32.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc1389d1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1341f27a0 con 0x7fc13406cf40 2023-11-28T21:35:32.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc132d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc134150c20 0x7fc13414c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc132d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc134150c20 0x7fc13414c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41722/0 (socket says 172.21.15.12:41722) 2023-11-28T21:35:32.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc132d9d640 1 -- 172.21.15.12:0/946918664 learned_addr learned my addr 172.21.15.12:0/946918664 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:32.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc13259c640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc13406cf40 0x7fc1341f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc12bfff640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc13406e300 0x7fc1341ef190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc132d9d640 1 -- 172.21.15.12:0/946918664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc13406cf40 msgr2=0x7fc1341f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc132d9d640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc13406cf40 0x7fc1341f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc132d9d640 1 -- 172.21.15.12:0/946918664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc13406e300 msgr2=0x7fc1341ef190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc132d9d640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc13406e300 0x7fc1341ef190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc132d9d640 1 -- 172.21.15.12:0/946918664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c009420 con 0x7fc134150c20 2023-11-28T21:35:32.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc13259c640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc13406cf40 0x7fc1341f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc132d9d640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc134150c20 0x7fc13414c450 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00cbc0 tx=0x7fc12c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc12affd640 1 -- 172.21.15.12:0/946918664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c00ad40 con 0x7fc134150c20 2023-11-28T21:35:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc12affd640 1 -- 172.21.15.12:0/946918664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12c012400 con 0x7fc134150c20 2023-11-28T21:35:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc12affd640 1 -- 172.21.15.12:0/946918664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c00aea0 con 0x7fc134150c20 2023-11-28T21:35:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc1389d1640 1 -- 172.21.15.12:0/946918664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13415b530 con 0x7fc134150c20 2023-11-28T21:35:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc1389d1640 1 -- 172.21.15.12:0/946918664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1341f2a40 con 0x7fc134150c20 2023-11-28T21:35:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc1389d1640 1 -- 172.21.15.12:0/946918664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1341f2f60 con 0x7fc134150c20 2023-11-28T21:35:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.808+0000 7fc12affd640 1 -- 172.21.15.12:0/946918664 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc12c007070 con 0x7fc134150c20 2023-11-28T21:35:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.812+0000 7fc12affd640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1140664a0 0x7fc114068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.812+0000 7fc12affd640 1 -- 172.21.15.12:0/946918664 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc12c084060 con 0x7fc134150c20 2023-11-28T21:35:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.812+0000 7fc13259c640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1140664a0 0x7fc114068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.812+0000 7fc13259c640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1140664a0 0x7fc114068960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc124006aa0 tx=0x7fc124006a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.816+0000 7fc12affd640 1 -- 172.21.15.12:0/946918664 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc12c0b5280 con 0x7fc134150c20 2023-11-28T21:35:32.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.972+0000 7fc1389d1640 1 -- 172.21.15.12:0/946918664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc134074f60 con 0x7fc134150c20 2023-11-28T21:35:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.972+0000 7fc12affd640 1 -- 172.21.15.12:0/946918664 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18746 (secure 0 0 0) 0x7fc12c017090 con 0x7fc134150c20 2023-11-28T21:35:32.976 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:32.976 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:35:25.566387+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 -- 172.21.15.12:0/946918664 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1140664a0 msgr2=0x7fc114068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1140664a0 0x7fc114068960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc124006aa0 tx=0x7fc124006a30 comp rx=0 tx=0).stop 2023-11-28T21:35:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 -- 172.21.15.12:0/946918664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc134150c20 msgr2=0x7fc13414c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc134150c20 0x7fc13414c450 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00cbc0 tx=0x7fc12c00a430 comp rx=0 tx=0).stop 2023-11-28T21:35:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 -- 172.21.15.12:0/946918664 shutdown_connections 2023-11-28T21:35:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1140664a0 0x7fc114068960 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc134150c20 0x7fc13414c450 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc13406e300 0x7fc1341ef190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 --2- 172.21.15.12:0/946918664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc13406cf40 0x7fc1341f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 -- 172.21.15.12:0/946918664 >> 172.21.15.12:0/946918664 conn(0x7fc134066360 msgr2=0x7fc1340714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 -- 172.21.15.12:0/946918664 shutdown_connections 2023-11-28T21:35:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:32.976+0000 7fc1389d1640 1 -- 172.21.15.12:0/946918664 wait complete. 2023-11-28T21:35:32.998 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T21:34:29.878190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T21:34:34.039032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T21:34:35.369379+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T21:34:39.373079+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T21:35:32.999 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T21:35:33.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc6a5c4640 1 Processor -- start 2023-11-28T21:35:33.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc6a5c4640 1 -- start start 2023-11-28T21:35:33.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc6a5c4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc640c6ca0 0x7fcc64159e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc6a5c4640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc64071120 con 0x7fcc64071fa0 2023-11-28T21:35:33.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc6a5c4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc64071400 con 0x7fcc641502f0 2023-11-28T21:35:33.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc6a5c4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640c7810 con 0x7fcc640c6ca0 2023-11-28T21:35:33.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc637fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcc64071fa0 0x7fcc641596f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45244/0 (socket says 172.21.15.12:45244) 2023-11-28T21:35:33.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc637fe640 1 -- 172.21.15.12:0/1581810650 learned_addr learned my addr 172.21.15.12:0/1581810650 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:33.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc68b61640 1 --2- 172.21.15.12:0/1581810650 >> v2:172.21.15.144:3300/0 conn(0x7fcc640c6ca0 0x7fcc64159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253718847 0 0) 0x7fcc64071120 con 0x7fcc64071fa0 2023-11-28T21:35:33.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc4c003530 con 0x7fcc64071fa0 2023-11-28T21:35:33.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268996799 0 0) 0x7fcc64071400 con 0x7fcc641502f0 2023-11-28T21:35:33.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc64071120 con 0x7fcc641502f0 2023-11-28T21:35:33.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1753620416 0 0) 0x7fcc4c003530 con 0x7fcc64071fa0 2023-11-28T21:35:33.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 >> v2:172.21.15.144:3300/0 conn(0x7fcc640c6ca0 msgr2=0x7fcc64159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:33.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 --2- 172.21.15.12:0/1581810650 >> v2:172.21.15.144:3300/0 conn(0x7fcc640c6ca0 0x7fcc64159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 >> v1:172.21.15.144:6789/0 conn(0x7fcc641502f0 legacy=0x7fcc64071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.150 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640c6aa0 con 0x7fcc64071fa0 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fcc58001800 con 0x7fcc64071fa0 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 >> v1:172.21.15.12:6789/0 conn(0x7fcc64071fa0 legacy=0x7fcc641596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c003b80 0x7fcc4c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc4c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c005190 0x7fcc4c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640c7810 con 0x7fcc4c0047d0 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c00bfe0 con 0x7fcc4c005190 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c00c1a0 con 0x7fcc4c003b80 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc637fe640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc4c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc63fff640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c003b80 0x7fcc4c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.144+0000 7fcc68b61640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c005190 0x7fcc4c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc637fe640 1 -- 172.21.15.12:0/1581810650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c003b80 msgr2=0x7fcc4c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:33.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc637fe640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4c003b80 0x7fcc4c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc637fe640 1 -- 172.21.15.12:0/1581810650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c005190 msgr2=0x7fcc4c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:33.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc637fe640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc4c005190 0x7fcc4c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc637fe640 1 -- 172.21.15.12:0/1581810650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c003530 con 0x7fcc4c0047d0 2023-11-28T21:35:33.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc68b61640 1 -- 172.21.15.12:0/1581810650 reap_dead start 2023-11-28T21:35:33.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc637fe640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc4c004c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c160 tx=0x7fcc5801f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3021928697afe30c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:33.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58007270 con 0x7fcc4c0047d0 2023-11-28T21:35:33.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc580ab070 con 0x7fcc4c0047d0 2023-11-28T21:35:33.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc62ffd640 1 -- 172.21.15.12:0/1581810650 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58007670 con 0x7fcc4c0047d0 2023-11-28T21:35:33.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/1581810650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 msgr2=0x7fcc4c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc6a5c4640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc4c004c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c160 tx=0x7fcc5801f2f0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/1581810650 shutdown_connections 2023-11-28T21:35:33.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc6a5c4640 1 --2- 172.21.15.12:0/1581810650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc4c004c50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/1581810650 >> 172.21.15.12:0/1581810650 conn(0x7fcc640c8450 msgr2=0x7fcc640c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:33.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/1581810650 shutdown_connections 2023-11-28T21:35:33.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.148+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/1581810650 wait complete. 2023-11-28T21:35:33.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc6a5c4640 1 Processor -- start 2023-11-28T21:35:33.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc6a5c4640 1 -- start start 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc6a5c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc641532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc6a5c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc64153800 0x7fcc64147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc6a5c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc640c3fb0 0x7fcc64147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc6a5c4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6414cf50 con 0x7fcc4c0047d0 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc6a5c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64147da0 con 0x7fcc640c3fb0 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc6a5c4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc641486c0 con 0x7fcc64153800 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc641532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc641532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41728/0 (socket says 172.21.15.12:41728) 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc63fff640 1 -- 172.21.15.12:0/2815679635 learned_addr learned my addr 172.21.15.12:0/2815679635 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc637fe640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc64153800 0x7fcc64147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc68b61640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc640c3fb0 0x7fcc64147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc63fff640 1 -- 172.21.15.12:0/2815679635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc64153800 msgr2=0x7fcc64147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc63fff640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc64153800 0x7fcc64147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc63fff640 1 -- 172.21.15.12:0/2815679635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc640c3fb0 msgr2=0x7fcc64147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc63fff640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc640c3fb0 0x7fcc64147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc63fff640 1 -- 172.21.15.12:0/2815679635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc641f08d0 con 0x7fcc4c0047d0 2023-11-28T21:35:33.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc63fff640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc641532c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400e4e0 tx=0x7fcc5400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:33.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.152+0000 7fcc60ff9640 1 -- 172.21.15.12:0/2815679635 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc540052e0 con 0x7fcc4c0047d0 2023-11-28T21:35:33.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.156+0000 7fcc60ff9640 1 -- 172.21.15.12:0/2815679635 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc54005440 con 0x7fcc4c0047d0 2023-11-28T21:35:33.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.156+0000 7fcc60ff9640 1 -- 172.21.15.12:0/2815679635 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54003840 con 0x7fcc4c0047d0 2023-11-28T21:35:33.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.156+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/2815679635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc64152500 con 0x7fcc4c0047d0 2023-11-28T21:35:33.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.156+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/2815679635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc64152ac0 con 0x7fcc4c0047d0 2023-11-28T21:35:33.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.156+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/2815679635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc64074f60 con 0x7fcc4c0047d0 2023-11-28T21:35:33.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.156+0000 7fcc60ff9640 1 -- 172.21.15.12:0/2815679635 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fcc540039a0 con 0x7fcc4c0047d0 2023-11-28T21:35:33.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.156+0000 7fcc60ff9640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcc50066470 0x7fcc50068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.156+0000 7fcc60ff9640 1 -- 172.21.15.12:0/2815679635 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcc54083ff0 con 0x7fcc4c0047d0 2023-11-28T21:35:33.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.156+0000 7fcc637fe640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcc50066470 0x7fcc50068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.156+0000 7fcc637fe640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcc50066470 0x7fcc50068930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcc580ae000 tx=0x7fcc58007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.160+0000 7fcc60ff9640 1 -- 172.21.15.12:0/2815679635 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcc540b5280 con 0x7fcc4c0047d0 2023-11-28T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/2815679635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc640c2560 con 0x7fcc4c0047d0 2023-11-28T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc60ff9640 1 -- 172.21.15.12:0/2815679635 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc54054ac0 con 0x7fcc4c0047d0 2023-11-28T21:35:33.322 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-11-28T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/2815679635 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcc50066470 msgr2=0x7fcc50068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcc50066470 0x7fcc50068930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcc580ae000 tx=0x7fcc58007040 comp rx=0 tx=0).stop 2023-11-28T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/2815679635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 msgr2=0x7fcc641532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc641532c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400e4e0 tx=0x7fcc5400e9a0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/2815679635 shutdown_connections 2023-11-28T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcc50066470 0x7fcc50068930 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc640c3fb0 0x7fcc64147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc64153800 0x7fcc64147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 --2- 172.21.15.12:0/2815679635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc4c0047d0 0x7fcc641532c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/2815679635 >> 172.21.15.12:0/2815679635 conn(0x7fcc640c8450 msgr2=0x7fcc6406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/2815679635 shutdown_connections 2023-11-28T21:35:33.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.320+0000 7fcc6a5c4640 1 -- 172.21.15.12:0/2815679635 wait complete. 2023-11-28T21:35:33.345 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T21:35:33.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7abb0f640 1 Processor -- start 2023-11-28T21:35:33.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7abb0f640 1 -- start start 2023-11-28T21:35:33.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7abb0f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb7a40c27c0 0x7fb7a4156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7abb0f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb7a406d7d0 0x7fb7a406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7abb0f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a40c6ce0 con 0x7fb7a41500d0 2023-11-28T21:35:33.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7abb0f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a406e120 con 0x7fb7a40c27c0 2023-11-28T21:35:33.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7abb0f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a4157030 con 0x7fb7a406d7d0 2023-11-28T21:35:33.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7a90aa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb7a40c27c0 0x7fb7a4156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7a90aa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb7a40c27c0 0x7fb7a4156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41742/0 (socket says 172.21.15.12:41742) 2023-11-28T21:35:33.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7a90aa640 1 -- 172.21.15.12:0/2219968123 learned_addr learned my addr 172.21.15.12:0/2219968123 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:33.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7aa0ac640 1 --2- 172.21.15.12:0/2219968123 >> v2:172.21.15.144:3300/0 conn(0x7fb7a406d7d0 0x7fb7a406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7a90aa640 1 -- 172.21.15.12:0/2219968123 >> v2:172.21.15.144:3300/0 conn(0x7fb7a406d7d0 msgr2=0x7fb7a406dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:33.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7a90aa640 1 --2- 172.21.15.12:0/2219968123 >> v2:172.21.15.144:3300/0 conn(0x7fb7a406d7d0 0x7fb7a406dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7a90aa640 1 -- 172.21.15.12:0/2219968123 >> v1:172.21.15.144:6789/0 conn(0x7fb7a41500d0 legacy=0x7fb7a40c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:33.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.488+0000 7fb7a90aa640 1 -- 172.21.15.12:0/2219968123 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a40c5df0 con 0x7fb7a40c27c0 2023-11-28T21:35:33.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7a90aa640 1 --2- 172.21.15.12:0/2219968123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 0x7fb7a4156af0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb7a41508e0 tx=0x7fb79401eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fe257420a3806a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:33.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7a88a9640 1 -- 172.21.15.12:0/2219968123 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940a79e0 con 0x7fb7a40c27c0 2023-11-28T21:35:33.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7a88a9640 1 -- 172.21.15.12:0/2219968123 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7940a7b40 con 0x7fb7a40c27c0 2023-11-28T21:35:33.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7a88a9640 1 -- 172.21.15.12:0/2219968123 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940af440 con 0x7fb7a40c27c0 2023-11-28T21:35:33.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7abb0f640 1 -- 172.21.15.12:0/2219968123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 msgr2=0x7fb7a4156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7abb0f640 1 --2- 172.21.15.12:0/2219968123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 0x7fb7a4156af0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb7a41508e0 tx=0x7fb79401eb90 comp rx=0 tx=0).stop 2023-11-28T21:35:33.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7abb0f640 1 -- 172.21.15.12:0/2219968123 shutdown_connections 2023-11-28T21:35:33.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7abb0f640 1 --2- 172.21.15.12:0/2219968123 >> v2:172.21.15.144:3300/0 conn(0x7fb7a406d7d0 0x7fb7a406dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7abb0f640 1 --2- 172.21.15.12:0/2219968123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 0x7fb7a4156af0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7abb0f640 1 -- 172.21.15.12:0/2219968123 >> 172.21.15.12:0/2219968123 conn(0x7fb7a4066360 msgr2=0x7fb7a4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:33.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7abb0f640 1 -- 172.21.15.12:0/2219968123 shutdown_connections 2023-11-28T21:35:33.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.492+0000 7fb7abb0f640 1 -- 172.21.15.12:0/2219968123 wait complete. 2023-11-28T21:35:33.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7abb0f640 1 Processor -- start 2023-11-28T21:35:33.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7abb0f640 1 -- start start 2023-11-28T21:35:33.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7abb0f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7a406d7d0 0x7fb7a406ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7abb0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 0x7fb7a4149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7abb0f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7a41500d0 0x7fb7a414a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7abb0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a41593f0 con 0x7fb7a40c27c0 2023-11-28T21:35:33.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7abb0f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a41f3130 con 0x7fb7a41500d0 2023-11-28T21:35:33.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7abb0f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a41f22d0 con 0x7fb7a406d7d0 2023-11-28T21:35:33.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7a90aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 0x7fb7a4149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7a90aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 0x7fb7a4149b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41754/0 (socket says 172.21.15.12:41754) 2023-11-28T21:35:33.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7a90aa640 1 -- 172.21.15.12:0/4226793739 learned_addr learned my addr 172.21.15.12:0/4226793739 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:33.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7a98ab640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7a406d7d0 0x7fb7a406ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7aa0ac640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7a41500d0 0x7fb7a414a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7a90aa640 1 -- 172.21.15.12:0/4226793739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7a406d7d0 msgr2=0x7fb7a406ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7a90aa640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7a406d7d0 0x7fb7a406ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7a90aa640 1 -- 172.21.15.12:0/4226793739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7a41500d0 msgr2=0x7fb7a414a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7a90aa640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7a41500d0 0x7fb7a414a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7a90aa640 1 -- 172.21.15.12:0/4226793739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79400b770 con 0x7fb7a40c27c0 2023-11-28T21:35:33.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7a90aa640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 0x7fb7a4149b50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb79401c7c0 tx=0x7fb79401b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:33.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb79a7fc640 1 -- 172.21.15.12:0/4226793739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940ba660 con 0x7fb7a40c27c0 2023-11-28T21:35:33.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb79a7fc640 1 -- 172.21.15.12:0/4226793739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7940ba7c0 con 0x7fb7a40c27c0 2023-11-28T21:35:33.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb79a7fc640 1 -- 172.21.15.12:0/4226793739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940bab70 con 0x7fb7a40c27c0 2023-11-28T21:35:33.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7abb0f640 1 -- 172.21.15.12:0/4226793739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a41f25b0 con 0x7fb7a40c27c0 2023-11-28T21:35:33.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.496+0000 7fb7abb0f640 1 -- 172.21.15.12:0/4226793739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a41f2b70 con 0x7fb7a40c27c0 2023-11-28T21:35:33.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.500+0000 7fb7abb0f640 1 -- 172.21.15.12:0/4226793739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a4074f60 con 0x7fb7a40c27c0 2023-11-28T21:35:33.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.500+0000 7fb79a7fc640 1 -- 172.21.15.12:0/4226793739 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb7940c2360 con 0x7fb7a40c27c0 2023-11-28T21:35:33.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.500+0000 7fb79a7fc640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7880664a0 0x7fb788068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.500+0000 7fb79a7fc640 1 -- 172.21.15.12:0/4226793739 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb79411f0f0 con 0x7fb7a40c27c0 2023-11-28T21:35:33.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.500+0000 7fb7a98ab640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7880664a0 0x7fb788068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.500+0000 7fb7a98ab640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7880664a0 0x7fb788068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0004e30 tx=0x7fb7a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:33.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.504+0000 7fb79a7fc640 1 -- 172.21.15.12:0/4226793739 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb7940ec1a0 con 0x7fb7a40c27c0 2023-11-28T21:35:33.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.664+0000 7fb7abb0f640 1 -- 172.21.15.12:0/4226793739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7a40c7320 con 0x7fb7a40c27c0 2023-11-28T21:35:33.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.664+0000 7fb79a7fc640 1 -- 172.21.15.12:0/4226793739 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7940b4020 con 0x7fb7a40c27c0 2023-11-28T21:35:33.669 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:35:33.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 -- 172.21.15.12:0/4226793739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7880664a0 msgr2=0x7fb788068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7880664a0 0x7fb788068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0004e30 tx=0x7fb7a0009040 comp rx=0 tx=0).stop 2023-11-28T21:35:33.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 -- 172.21.15.12:0/4226793739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 msgr2=0x7fb7a4149b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 0x7fb7a4149b50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb79401c7c0 tx=0x7fb79401b930 comp rx=0 tx=0).stop 2023-11-28T21:35:33.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 -- 172.21.15.12:0/4226793739 shutdown_connections 2023-11-28T21:35:33.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7880664a0 0x7fb788068960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7a41500d0 0x7fb7a414a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7a40c27c0 0x7fb7a4149b50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 --2- 172.21.15.12:0/4226793739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7a406d7d0 0x7fb7a406ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 -- 172.21.15.12:0/4226793739 >> 172.21.15.12:0/4226793739 conn(0x7fb7a4066360 msgr2=0x7fb7a4154560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:33.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 -- 172.21.15.12:0/4226793739 shutdown_connections 2023-11-28T21:35:33.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.668+0000 7fb7abb0f640 1 -- 172.21.15.12:0/4226793739 wait complete. 2023-11-28T21:35:33.690 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T21:35:33.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ffa9aa640 1 Processor -- start 2023-11-28T21:35:33.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ffa9aa640 1 -- start start 2023-11-28T21:35:33.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ffa9aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ff415d320 0x7f5ff406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ffa9aa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5ff406aaf0 0x7f5ff40c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ffa9aa640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff415e270 con 0x7f5ff4154100 2023-11-28T21:35:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ffa9aa640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff40c4e60 con 0x7f5ff406aaf0 2023-11-28T21:35:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ffa9aa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff415d740 con 0x7f5ff415d320 2023-11-28T21:35:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ff8f47640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5ff406aaf0 0x7f5ff40c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ff8f47640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5ff406aaf0 0x7f5ff40c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41764/0 (socket says 172.21.15.12:41764) 2023-11-28T21:35:33.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ff8f47640 1 -- 172.21.15.12:0/1332007878 learned_addr learned my addr 172.21.15.12:0/1332007878 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:33.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ff37fe640 1 --2- 172.21.15.12:0/1332007878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ff415d320 0x7f5ff406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ff8f47640 1 -- 172.21.15.12:0/1332007878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ff415d320 msgr2=0x7f5ff406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ff8f47640 1 --2- 172.21.15.12:0/1332007878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ff415d320 0x7f5ff406a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ff8f47640 1 -- 172.21.15.12:0/1332007878 >> v1:172.21.15.144:6789/0 conn(0x7f5ff4154100 legacy=0x7f5ff40c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.836+0000 7f5ff8f47640 1 -- 172.21.15.12:0/1332007878 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff41544e0 con 0x7f5ff406aaf0 2023-11-28T21:35:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ff8f47640 1 --2- 172.21.15.12:0/1332007878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff406aaf0 0x7f5ff40c52f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009e50 tx=0x7f5fec01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabe8cd09d26c639 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ff2ffd640 1 -- 172.21.15.12:0/1332007878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0a58a0 con 0x7f5ff406aaf0 2023-11-28T21:35:33.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ff2ffd640 1 -- 172.21.15.12:0/1332007878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec0a5a00 con 0x7f5ff406aaf0 2023-11-28T21:35:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ff2ffd640 1 -- 172.21.15.12:0/1332007878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0a5d90 con 0x7f5ff406aaf0 2023-11-28T21:35:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/1332007878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff406aaf0 msgr2=0x7f5ff40c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ffa9aa640 1 --2- 172.21.15.12:0/1332007878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff406aaf0 0x7f5ff40c52f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009e50 tx=0x7f5fec01c9e0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/1332007878 shutdown_connections 2023-11-28T21:35:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ffa9aa640 1 --2- 172.21.15.12:0/1332007878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff406aaf0 0x7f5ff40c52f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ffa9aa640 1 --2- 172.21.15.12:0/1332007878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ff415d320 0x7f5ff406a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/1332007878 >> 172.21.15.12:0/1332007878 conn(0x7f5ff406c690 msgr2=0x7f5ff4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:33.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/1332007878 shutdown_connections 2023-11-28T21:35:33.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.840+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/1332007878 wait complete. 2023-11-28T21:35:33.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ffa9aa640 1 Processor -- start 2023-11-28T21:35:33.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ffa9aa640 1 -- start start 2023-11-28T21:35:33.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ffa9aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ff406aaf0 0x7f5ff413ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ffa9aa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff4154100 0x7f5ff40cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ffa9aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff415d320 0x7f5ff40cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ffa9aa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40b32f0 con 0x7f5ff415d320 2023-11-28T21:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ffa9aa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40b3d80 con 0x7f5ff4154100 2023-11-28T21:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ffa9aa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40cdfc0 con 0x7f5ff406aaf0 2023-11-28T21:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff8f47640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff415d320 0x7f5ff40cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff8f47640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff415d320 0x7f5ff40cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41768/0 (socket says 172.21.15.12:41768) 2023-11-28T21:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff8f47640 1 -- 172.21.15.12:0/2291165751 learned_addr learned my addr 172.21.15.12:0/2291165751 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff37fe640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff4154100 0x7f5ff40cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff3fff640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ff406aaf0 0x7f5ff413ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff8f47640 1 -- 172.21.15.12:0/2291165751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ff406aaf0 msgr2=0x7f5ff413ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff8f47640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ff406aaf0 0x7f5ff413ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff8f47640 1 -- 172.21.15.12:0/2291165751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff4154100 msgr2=0x7f5ff40cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff8f47640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff4154100 0x7f5ff40cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff8f47640 1 -- 172.21.15.12:0/2291165751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec009b30 con 0x7f5ff415d320 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff8f47640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff415d320 0x7f5ff40cbcd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5fec01a610 tx=0x7f5fec019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff0ff9640 1 -- 172.21.15.12:0/2291165751 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0b8670 con 0x7f5ff415d320 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff0ff9640 1 -- 172.21.15.12:0/2291165751 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec0b87d0 con 0x7f5ff415d320 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ff0ff9640 1 -- 172.21.15.12:0/2291165751 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0b8b80 con 0x7f5ff415d320 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/2291165751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff413d5c0 con 0x7f5ff415d320 2023-11-28T21:35:33.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.844+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/2291165751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff40b4d50 con 0x7f5ff415d320 2023-11-28T21:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.848+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/2291165751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff40b3430 con 0x7f5ff415d320 2023-11-28T21:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.848+0000 7f5ff0ff9640 1 -- 172.21.15.12:0/2291165751 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5fec0b8ce0 con 0x7f5ff415d320 2023-11-28T21:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.848+0000 7f5ff0ff9640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5fe0066470 0x7f5fe0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.848+0000 7f5ff0ff9640 1 -- 172.21.15.12:0/2291165751 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5fec11f6c0 con 0x7f5ff415d320 2023-11-28T21:35:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.848+0000 7f5ff3fff640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5fe0066470 0x7f5fe0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:33.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.848+0000 7f5ff3fff640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5fe0066470 0x7f5fe0068930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007fd0 tx=0x7f5fe4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:33.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:33.852+0000 7f5ff0ff9640 1 -- 172.21.15.12:0/2291165751 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ff40b3430 con 0x7f5ff415d320 2023-11-28T21:35:34.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/2291165751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ff40b3610 con 0x7f5ff415d320 2023-11-28T21:35:34.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ff0ff9640 1 -- 172.21.15.12:0/2291165751 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ff40b3610 con 0x7f5ff415d320 2023-11-28T21:35:34.014 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-11-28T21:35:34.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/2291165751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5fe0066470 msgr2=0x7f5fe0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5fe0066470 0x7f5fe0068930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007fd0 tx=0x7f5fe4009040 comp rx=0 tx=0).stop 2023-11-28T21:35:34.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/2291165751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff415d320 msgr2=0x7f5ff40cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff415d320 0x7f5ff40cbcd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5fec01a610 tx=0x7f5fec019780 comp rx=0 tx=0).stop 2023-11-28T21:35:34.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/2291165751 shutdown_connections 2023-11-28T21:35:34.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5fe0066470 0x7f5fe0068930 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ff415d320 0x7f5ff40cbcd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ff4154100 0x7f5ff40cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 --2- 172.21.15.12:0/2291165751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ff406aaf0 0x7f5ff413ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/2291165751 >> 172.21.15.12:0/2291165751 conn(0x7f5ff406c690 msgr2=0x7f5ff40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:34.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/2291165751 shutdown_connections 2023-11-28T21:35:34.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.012+0000 7f5ffa9aa640 1 -- 172.21.15.12:0/2291165751 wait complete. 2023-11-28T21:35:34.035 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T21:35:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.176+0000 7fe9f039d640 1 Processor -- start 2023-11-28T21:35:34.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.176+0000 7fe9f039d640 1 -- start start 2023-11-28T21:35:34.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9f039d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe9e806e300 0x7fe9e8156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9f039d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe9e806cf40 0x7fe9e8158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9f039d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e8067410 con 0x7fe9e8150c20 2023-11-28T21:35:34.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9f039d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e806e090 con 0x7fe9e806cf40 2023-11-28T21:35:34.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9f039d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e8157f30 con 0x7fe9e806e300 2023-11-28T21:35:34.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9ee139640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe9e8150c20 0x7fe9e80c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45254/0 (socket says 172.21.15.12:45254) 2023-11-28T21:35:34.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9ee139640 1 -- 172.21.15.12:0/1737312698 learned_addr learned my addr 172.21.15.12:0/1737312698 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:34.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9ee93a640 1 --2- 172.21.15.12:0/1737312698 >> v2:172.21.15.12:3300/0 conn(0x7fe9e806cf40 0x7fe9e8158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9ed938640 1 --2- 172.21.15.12:0/1737312698 >> v2:172.21.15.144:3300/0 conn(0x7fe9e806e300 0x7fe9e8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9ee93a640 1 -- 172.21.15.12:0/1737312698 >> v2:172.21.15.144:3300/0 conn(0x7fe9e806e300 msgr2=0x7fe9e8156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:34.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9ee93a640 1 --2- 172.21.15.12:0/1737312698 >> v2:172.21.15.144:3300/0 conn(0x7fe9e806e300 0x7fe9e8156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9ed137640 1 -- 172.21.15.12:0/1737312698 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248748876 0 0) 0x7fe9e8067410 con 0x7fe9e8150c20 2023-11-28T21:35:34.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9ee93a640 1 -- 172.21.15.12:0/1737312698 >> v1:172.21.15.12:6789/0 conn(0x7fe9e8150c20 legacy=0x7fe9e80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.180+0000 7fe9ee93a640 1 -- 172.21.15.12:0/1737312698 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8151000 con 0x7fe9e806cf40 2023-11-28T21:35:34.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9ee93a640 1 --2- 172.21.15.12:0/1737312698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806cf40 0x7fe9e8158410 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80c8e60 tx=0x7fe9e001eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c28dd150cf30c33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9ed137640 1 -- 172.21.15.12:0/1737312698 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000b2d0 con 0x7fe9e806cf40 2023-11-28T21:35:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9ed137640 1 -- 172.21.15.12:0/1737312698 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e00a77d0 con 0x7fe9e806cf40 2023-11-28T21:35:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9ed137640 1 -- 172.21.15.12:0/1737312698 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e00a7b60 con 0x7fe9e806cf40 2023-11-28T21:35:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9f039d640 1 -- 172.21.15.12:0/1737312698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806cf40 msgr2=0x7fe9e8158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9f039d640 1 --2- 172.21.15.12:0/1737312698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806cf40 0x7fe9e8158410 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80c8e60 tx=0x7fe9e001eb10 comp rx=0 tx=0).stop 2023-11-28T21:35:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9f039d640 1 -- 172.21.15.12:0/1737312698 shutdown_connections 2023-11-28T21:35:34.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9f039d640 1 --2- 172.21.15.12:0/1737312698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806cf40 0x7fe9e8158410 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9f039d640 1 --2- 172.21.15.12:0/1737312698 >> v2:172.21.15.144:3300/0 conn(0x7fe9e806e300 0x7fe9e8156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9f039d640 1 -- 172.21.15.12:0/1737312698 >> 172.21.15.12:0/1737312698 conn(0x7fe9e8066360 msgr2=0x7fe9e8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9f039d640 1 -- 172.21.15.12:0/1737312698 shutdown_connections 2023-11-28T21:35:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9f039d640 1 -- 172.21.15.12:0/1737312698 wait complete. 2023-11-28T21:35:34.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9f039d640 1 Processor -- start 2023-11-28T21:35:34.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.184+0000 7fe9f039d640 1 -- start start 2023-11-28T21:35:34.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9f039d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9e806cf40 0x7fe9e8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9f039d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806e300 0x7fe9e81f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9f039d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8150c20 0x7fe9e80c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9f039d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e81f3720 con 0x7fe9e806e300 2023-11-28T21:35:34.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9f039d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e80c5030 con 0x7fe9e8150c20 2023-11-28T21:35:34.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9f039d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e80c5310 con 0x7fe9e806cf40 2023-11-28T21:35:34.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ed938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806e300 0x7fe9e81f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ed938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806e300 0x7fe9e81f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41772/0 (socket says 172.21.15.12:41772) 2023-11-28T21:35:34.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ed938640 1 -- 172.21.15.12:0/745902565 learned_addr learned my addr 172.21.15.12:0/745902565 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:34.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ee93a640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8150c20 0x7fe9e80c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ee139640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9e806cf40 0x7fe9e8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ed938640 1 -- 172.21.15.12:0/745902565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9e806cf40 msgr2=0x7fe9e8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ed938640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9e806cf40 0x7fe9e8151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ed938640 1 -- 172.21.15.12:0/745902565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8150c20 msgr2=0x7fe9e80c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ed938640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8150c20 0x7fe9e80c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ed938640 1 -- 172.21.15.12:0/745902565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e000bc80 con 0x7fe9e806e300 2023-11-28T21:35:34.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9ed938640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806e300 0x7fe9e81f31e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800d7d0 tx=0x7fe9d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9deffd640 1 -- 172.21.15.12:0/745902565 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8018070 con 0x7fe9e806e300 2023-11-28T21:35:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9deffd640 1 -- 172.21.15.12:0/745902565 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d8004020 con 0x7fe9e806e300 2023-11-28T21:35:34.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9deffd640 1 -- 172.21.15.12:0/745902565 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d80041d0 con 0x7fe9e806e300 2023-11-28T21:35:34.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9f039d640 1 -- 172.21.15.12:0/745902565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e81512e0 con 0x7fe9e806e300 2023-11-28T21:35:34.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9f039d640 1 -- 172.21.15.12:0/745902565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e81518a0 con 0x7fe9e806e300 2023-11-28T21:35:34.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9f039d640 1 -- 172.21.15.12:0/745902565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e8152300 con 0x7fe9e806e300 2023-11-28T21:35:34.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.188+0000 7fe9deffd640 1 -- 172.21.15.12:0/745902565 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe9d8004330 con 0x7fe9e806e300 2023-11-28T21:35:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.192+0000 7fe9deffd640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe9cc066470 0x7fe9cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.192+0000 7fe9deffd640 1 -- 172.21.15.12:0/745902565 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe9d8084040 con 0x7fe9e806e300 2023-11-28T21:35:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.192+0000 7fe9ee139640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe9cc066470 0x7fe9cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.192+0000 7fe9ee139640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe9cc066470 0x7fe9cc068930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4000f70 tx=0x7fe9e4007b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:34.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.196+0000 7fe9deffd640 1 -- 172.21.15.12:0/745902565 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe9d8051100 con 0x7fe9e806e300 2023-11-28T21:35:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.352+0000 7fe9f039d640 1 -- 172.21.15.12:0/745902565 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe9e8151a60 con 0x7fe9e806e300 2023-11-28T21:35:34.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.352+0000 7fe9deffd640 1 -- 172.21.15.12:0/745902565 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe9d8054720 con 0x7fe9e806e300 2023-11-28T21:35:34.355 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:35:34.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.352+0000 7fe9f039d640 1 -- 172.21.15.12:0/745902565 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe9cc066470 msgr2=0x7fe9cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.352+0000 7fe9f039d640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe9cc066470 0x7fe9cc068930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4000f70 tx=0x7fe9e4007b60 comp rx=0 tx=0).stop 2023-11-28T21:35:34.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.352+0000 7fe9f039d640 1 -- 172.21.15.12:0/745902565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806e300 msgr2=0x7fe9e81f31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.352+0000 7fe9f039d640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806e300 0x7fe9e81f31e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800d7d0 tx=0x7fe9d800dc90 comp rx=0 tx=0).stop 2023-11-28T21:35:34.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.352+0000 7fe9f039d640 1 -- 172.21.15.12:0/745902565 shutdown_connections 2023-11-28T21:35:34.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.352+0000 7fe9f039d640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe9cc066470 0x7fe9cc068930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.352+0000 7fe9f039d640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9e8150c20 0x7fe9e80c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.356+0000 7fe9f039d640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9e806e300 0x7fe9e81f31e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.356+0000 7fe9f039d640 1 --2- 172.21.15.12:0/745902565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9e806cf40 0x7fe9e8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.356+0000 7fe9f039d640 1 -- 172.21.15.12:0/745902565 >> 172.21.15.12:0/745902565 conn(0x7fe9e8066360 msgr2=0x7fe9e81e7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:34.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.356+0000 7fe9f039d640 1 -- 172.21.15.12:0/745902565 shutdown_connections 2023-11-28T21:35:34.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.356+0000 7fe9f039d640 1 -- 172.21.15.12:0/745902565 wait complete. 2023-11-28T21:35:34.377 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:35:34.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.516+0000 7f4c78c9c640 1 Processor -- start 2023-11-28T21:35:34.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.516+0000 7f4c78c9c640 1 -- start start 2023-11-28T21:35:34.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c78c9c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4c74153ee0 0x7f4c7415a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c78c9c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c7406a5b0 0x7f4c740c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c78c9c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c740c6350 con 0x7f4c74070bd0 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c78c9c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c740c6b90 con 0x7f4c74153ee0 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c78c9c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74072410 con 0x7f4c7406a5b0 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c7259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4c74153ee0 0x7f4c7415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c7259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4c74153ee0 0x7f4c7415a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41778/0 (socket says 172.21.15.12:41778) 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c7259c640 1 -- 172.21.15.12:0/3279300653 learned_addr learned my addr 172.21.15.12:0/3279300653 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c72d9d640 1 --1- 172.21.15.12:0/3279300653 >> v1:172.21.15.12:6789/0 conn(0x7f4c74070bd0 0x7f4c74071060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45258/0 (socket says 172.21.15.12:45258) 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c71d9b640 1 --2- 172.21.15.12:0/3279300653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c7406a5b0 0x7f4c740c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c7259c640 1 -- 172.21.15.12:0/3279300653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c7406a5b0 msgr2=0x7f4c740c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c7259c640 1 --2- 172.21.15.12:0/3279300653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c7406a5b0 0x7f4c740c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c7259c640 1 -- 172.21.15.12:0/3279300653 >> v1:172.21.15.12:6789/0 conn(0x7f4c74070bd0 legacy=0x7f4c74071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.520+0000 7f4c7259c640 1 -- 172.21.15.12:0/3279300653 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740c65d0 con 0x7f4c74153ee0 2023-11-28T21:35:34.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c7259c640 1 --2- 172.21.15.12:0/3279300653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74153ee0 0x7f4c7415a680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009870 tx=0x7f4c6401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caddb46c976d0d71 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:34.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c7159a640 1 -- 172.21.15.12:0/3279300653 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c640a8a30 con 0x7f4c74153ee0 2023-11-28T21:35:34.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c7159a640 1 -- 172.21.15.12:0/3279300653 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c640a8b90 con 0x7f4c74153ee0 2023-11-28T21:35:34.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c7159a640 1 -- 172.21.15.12:0/3279300653 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c640b0440 con 0x7f4c74153ee0 2023-11-28T21:35:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c78c9c640 1 -- 172.21.15.12:0/3279300653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74153ee0 msgr2=0x7f4c7415a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c78c9c640 1 --2- 172.21.15.12:0/3279300653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74153ee0 0x7f4c7415a680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009870 tx=0x7f4c6401ed30 comp rx=0 tx=0).stop 2023-11-28T21:35:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c78c9c640 1 -- 172.21.15.12:0/3279300653 shutdown_connections 2023-11-28T21:35:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c78c9c640 1 --2- 172.21.15.12:0/3279300653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c7406a5b0 0x7f4c740c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c78c9c640 1 --2- 172.21.15.12:0/3279300653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74153ee0 0x7f4c7415a680 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c78c9c640 1 -- 172.21.15.12:0/3279300653 >> 172.21.15.12:0/3279300653 conn(0x7f4c740c8450 msgr2=0x7f4c740c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c78c9c640 1 -- 172.21.15.12:0/3279300653 shutdown_connections 2023-11-28T21:35:34.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c78c9c640 1 -- 172.21.15.12:0/3279300653 wait complete. 2023-11-28T21:35:34.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c78c9c640 1 Processor -- start 2023-11-28T21:35:34.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.524+0000 7f4c78c9c640 1 -- start start 2023-11-28T21:35:34.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c78c9c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c7406a5b0 0x7f4c74157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c78c9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74070bd0 0x7f4c7414d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c78c9c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c74153ee0 0x7f4c741554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c78c9c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74154540 con 0x7f4c74070bd0 2023-11-28T21:35:34.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c78c9c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7414c8f0 con 0x7f4c7406a5b0 2023-11-28T21:35:34.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c78c9c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74154720 con 0x7f4c74153ee0 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c71d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74070bd0 0x7f4c7414d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c71d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74070bd0 0x7f4c7414d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41788/0 (socket says 172.21.15.12:41788) 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c71d9b640 1 -- 172.21.15.12:0/419211584 learned_addr learned my addr 172.21.15.12:0/419211584 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c7259c640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c7406a5b0 0x7f4c74157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c72d9d640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c74153ee0 0x7f4c741554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c71d9b640 1 -- 172.21.15.12:0/419211584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c74153ee0 msgr2=0x7f4c741554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c71d9b640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c74153ee0 0x7f4c741554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c71d9b640 1 -- 172.21.15.12:0/419211584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c7406a5b0 msgr2=0x7f4c74157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c71d9b640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c7406a5b0 0x7f4c74157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c71d9b640 1 -- 172.21.15.12:0/419211584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c640094d0 con 0x7f4c74070bd0 2023-11-28T21:35:34.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c7259c640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c7406a5b0 0x7f4c74157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:34.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c71d9b640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74070bd0 0x7f4c7414d110 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800d5d0 tx=0x7f4c6800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:34.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c62ffd640 1 -- 172.21.15.12:0/419211584 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68018070 con 0x7f4c74070bd0 2023-11-28T21:35:34.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c62ffd640 1 -- 172.21.15.12:0/419211584 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c680059a0 con 0x7f4c74070bd0 2023-11-28T21:35:34.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c62ffd640 1 -- 172.21.15.12:0/419211584 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c68005b30 con 0x7f4c74070bd0 2023-11-28T21:35:34.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c78c9c640 1 -- 172.21.15.12:0/419211584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c74154a00 con 0x7f4c74070bd0 2023-11-28T21:35:34.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c78c9c640 1 -- 172.21.15.12:0/419211584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c74154fc0 con 0x7f4c74070bd0 2023-11-28T21:35:34.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c78c9c640 1 -- 172.21.15.12:0/419211584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c74155180 con 0x7f4c74070bd0 2023-11-28T21:35:34.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.528+0000 7f4c62ffd640 1 -- 172.21.15.12:0/419211584 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4c68005c90 con 0x7f4c74070bd0 2023-11-28T21:35:34.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.532+0000 7f4c62ffd640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c540664a0 0x7f4c54068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.532+0000 7f4c62ffd640 1 -- 172.21.15.12:0/419211584 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4c68084130 con 0x7f4c74070bd0 2023-11-28T21:35:34.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.532+0000 7f4c7259c640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c540664a0 0x7f4c54068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.532+0000 7f4c7259c640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c540664a0 0x7f4c54068960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c640b2000 tx=0x7f4c64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:34.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.536+0000 7f4c62ffd640 1 -- 172.21.15.12:0/419211584 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4c680510c0 con 0x7f4c74070bd0 2023-11-28T21:35:34.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.724+0000 7f4c78c9c640 1 -- 172.21.15.12:0/419211584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c74074f60 con 0x7f4c74070bd0 2023-11-28T21:35:34.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.724+0000 7f4c62ffd640 1 -- 172.21.15.12:0/419211584 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6529 (secure 0 0 0) 0x7f4c680546e0 con 0x7f4c74070bd0 2023-11-28T21:35:34.728 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:34.729 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:35:14.337396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:35:34.729 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2023-11-28T21:35:34.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 -- 172.21.15.12:0/419211584 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c540664a0 msgr2=0x7f4c54068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c540664a0 0x7f4c54068960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c640b2000 tx=0x7f4c64008040 comp rx=0 tx=0).stop 2023-11-28T21:35:34.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 -- 172.21.15.12:0/419211584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74070bd0 msgr2=0x7f4c7414d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74070bd0 0x7f4c7414d110 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800d5d0 tx=0x7f4c6800da90 comp rx=0 tx=0).stop 2023-11-28T21:35:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 -- 172.21.15.12:0/419211584 shutdown_connections 2023-11-28T21:35:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c540664a0 0x7f4c54068960 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c74153ee0 0x7f4c741554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c74070bd0 0x7f4c7414d110 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 --2- 172.21.15.12:0/419211584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c7406a5b0 0x7f4c74157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 -- 172.21.15.12:0/419211584 >> 172.21.15.12:0/419211584 conn(0x7f4c740c8450 msgr2=0x7f4c740ca110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 -- 172.21.15.12:0/419211584 shutdown_connections 2023-11-28T21:35:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.728+0000 7f4c78c9c640 1 -- 172.21.15.12:0/419211584 wait complete. 2023-11-28T21:35:34.750 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:35:34.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee5b98d640 1 Processor -- start 2023-11-28T21:35:34.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee5b98d640 1 -- start start 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee5b98d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee54150c20 0x7fee54156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee5b98d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee5406cf40 0x7fee54158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee5b98d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee54067410 con 0x7fee5406e300 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee5b98d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5406e090 con 0x7fee54150c20 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee5b98d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee54157f30 con 0x7fee5406cf40 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee58f28640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fee5406e300 0x7fee540c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33554/0 (socket says 172.21.15.12:33554) 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee58f28640 1 -- 172.21.15.12:0/4001830977 learned_addr learned my addr 172.21.15.12:0/4001830977 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee59729640 1 --2- 172.21.15.12:0/4001830977 >> v2:172.21.15.144:3300/0 conn(0x7fee54150c20 0x7fee54156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee59f2a640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee5406cf40 0x7fee54158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844151230 0 0) 0x7fee54067410 con 0x7fee5406e300 2023-11-28T21:35:34.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee40003530 con 0x7fee5406e300 2023-11-28T21:35:34.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2089727834 0 0) 0x7fee40003530 con 0x7fee5406e300 2023-11-28T21:35:34.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee5406cf40 msgr2=0x7fee54158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:34.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee4bfff640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee5406cf40 0x7fee54158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 >> v2:172.21.15.144:3300/0 conn(0x7fee54150c20 msgr2=0x7fee54156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:34.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.892+0000 7fee4bfff640 1 --2- 172.21.15.12:0/4001830977 >> v2:172.21.15.144:3300/0 conn(0x7fee54150c20 0x7fee54156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54151000 con 0x7fee5406e300 2023-11-28T21:35:34.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fee44001680 con 0x7fee5406e300 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 >> v1:172.21.15.12:6789/0 conn(0x7fee5406e300 legacy=0x7fee540c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee5406cf40 0x7fee400039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee40003f70 0x7fee400047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee40004d10 0x7fee40005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5406e090 con 0x7fee40004d10 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee54157f30 con 0x7fee40003f70 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4000b9c0 con 0x7fee5406cf40 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee59f2a640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee40004d10 0x7fee40005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee58f28640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee40003f70 0x7fee400047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee59729640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee5406cf40 0x7fee400039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee59f2a640 1 -- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee5406cf40 msgr2=0x7fee400039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee59f2a640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee5406cf40 0x7fee400039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee59f2a640 1 -- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee40003f70 msgr2=0x7fee400047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:35:34.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee59f2a640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee40003f70 0x7fee400047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee59f2a640 1 -- 172.21.15.12:0/4001830977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee40003530 con 0x7fee40004d10 2023-11-28T21:35:34.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee59f2a640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee40004d10 0x7fee40005190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009fc0 tx=0x7fee4c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b21fa42fae36052 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:34.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0a5c70 con 0x7fee40004d10 2023-11-28T21:35:34.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c00f040 con 0x7fee40004d10 2023-11-28T21:35:34.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.896+0000 7fee4bfff640 1 -- 172.21.15.12:0/4001830977 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0ad440 con 0x7fee40004d10 2023-11-28T21:35:34.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 -- 172.21.15.12:0/4001830977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee40004d10 msgr2=0x7fee40005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 --2- 172.21.15.12:0/4001830977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee40004d10 0x7fee40005190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009fc0 tx=0x7fee4c01cc60 comp rx=0 tx=0).stop 2023-11-28T21:35:34.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee59f2a640 1 -- 172.21.15.12:0/4001830977 reap_dead start 2023-11-28T21:35:34.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 -- 172.21.15.12:0/4001830977 shutdown_connections 2023-11-28T21:35:34.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 -- 172.21.15.12:0/4001830977 >> 172.21.15.12:0/4001830977 conn(0x7fee54066360 msgr2=0x7fee54066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:34.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 -- 172.21.15.12:0/4001830977 shutdown_connections 2023-11-28T21:35:34.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 -- 172.21.15.12:0/4001830977 wait complete. 2023-11-28T21:35:34.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 Processor -- start 2023-11-28T21:35:34.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 -- start start 2023-11-28T21:35:34.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee541e43b0 0x7fee541ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee54071ca0 0x7fee541efe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee541e4790 0x7fee5414a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee541ef4a0 con 0x7fee541e43b0 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee541f2b90 con 0x7fee541e4790 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee5b98d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee541f2e30 con 0x7fee54071ca0 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee59729640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee541e43b0 0x7fee541ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee59729640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee541e43b0 0x7fee541ef8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45984/0 (socket says 172.21.15.12:45984) 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee59729640 1 -- 172.21.15.12:0/3128695455 learned_addr learned my addr 172.21.15.12:0/3128695455 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee58f28640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee54071ca0 0x7fee541efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.900+0000 7fee59f2a640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee541e4790 0x7fee5414a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee59729640 1 -- 172.21.15.12:0/3128695455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee54071ca0 msgr2=0x7fee541efe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee59729640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee54071ca0 0x7fee541efe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee59729640 1 -- 172.21.15.12:0/3128695455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee541e4790 msgr2=0x7fee5414a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:34.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee59729640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee541e4790 0x7fee5414a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:34.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee59729640 1 -- 172.21.15.12:0/3128695455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee541f3110 con 0x7fee541e43b0 2023-11-28T21:35:34.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee58f28640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee54071ca0 0x7fee541efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:34.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee59f2a640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee541e4790 0x7fee5414a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:35:34.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee59729640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee541e43b0 0x7fee541ef8f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fee500116a0 tx=0x7fee50011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:34.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee49ffb640 1 -- 172.21.15.12:0/3128695455 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000f9c0 con 0x7fee541e43b0 2023-11-28T21:35:34.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee49ffb640 1 -- 172.21.15.12:0/3128695455 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5000fb20 con 0x7fee541e43b0 2023-11-28T21:35:34.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee49ffb640 1 -- 172.21.15.12:0/3128695455 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000fe90 con 0x7fee541e43b0 2023-11-28T21:35:34.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee5b98d640 1 -- 172.21.15.12:0/3128695455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee541f33f0 con 0x7fee541e43b0 2023-11-28T21:35:34.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee5b98d640 1 -- 172.21.15.12:0/3128695455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee54154020 con 0x7fee541e43b0 2023-11-28T21:35:34.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee5b98d640 1 -- 172.21.15.12:0/3128695455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee541ef740 con 0x7fee541e43b0 2023-11-28T21:35:34.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee49ffb640 1 -- 172.21.15.12:0/3128695455 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fee50015460 con 0x7fee541e43b0 2023-11-28T21:35:34.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.904+0000 7fee49ffb640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee38066470 0x7fee38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:35:34.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.908+0000 7fee49ffb640 1 -- 172.21.15.12:0/3128695455 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fee50084c70 con 0x7fee541e43b0 2023-11-28T21:35:34.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.908+0000 7fee58f28640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee38066470 0x7fee38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:35:34.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.908+0000 7fee58f28640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee38066470 0x7fee38068930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fee54070cd0 tx=0x7fee44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:35:34.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:34.908+0000 7fee49ffb640 1 -- 172.21.15.12:0/3128695455 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee50051c90 con 0x7fee541e43b0 2023-11-28T21:35:35.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.068+0000 7fee5b98d640 1 -- 172.21.15.12:0/3128695455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee54074f60 con 0x7fee541e43b0 2023-11-28T21:35:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee49ffb640 1 -- 172.21.15.12:0/3128695455 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18746 (secure 0 0 0) 0x7fee50022090 con 0x7fee541e43b0 2023-11-28T21:35:35.073 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:35:35.074 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:35:25.566387+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:35:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 -- 172.21.15.12:0/3128695455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee38066470 msgr2=0x7fee38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:35.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee38066470 0x7fee38068930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fee54070cd0 tx=0x7fee44009040 comp rx=0 tx=0).stop 2023-11-28T21:35:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 -- 172.21.15.12:0/3128695455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee541e43b0 msgr2=0x7fee541ef8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:35:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee541e43b0 0x7fee541ef8f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fee500116a0 tx=0x7fee50011b60 comp rx=0 tx=0).stop 2023-11-28T21:35:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 -- 172.21.15.12:0/3128695455 shutdown_connections 2023-11-28T21:35:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee38066470 0x7fee38068930 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee541e4790 0x7fee5414a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee54071ca0 0x7fee541efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 --2- 172.21.15.12:0/3128695455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee541e43b0 0x7fee541ef8f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:35:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 -- 172.21.15.12:0/3128695455 >> 172.21.15.12:0/3128695455 conn(0x7fee54066360 msgr2=0x7fee54154710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:35:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 -- 172.21.15.12:0/3128695455 shutdown_connections 2023-11-28T21:35:35.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:35:35.072+0000 7fee5b98d640 1 -- 172.21.15.12:0/3128695455 wait complete. 2023-11-28T21:35:35.095 DEBUG:tasks.mds_thrash:Yielding 2023-11-28T21:35:35.095 INFO:teuthology.run_tasks:Running task workunit... 2023-11-28T21:35:35.108 INFO:tasks.workunit:Pulling workunits from ref 43dd94c190073cf16cff1e711dd7f5a7b1cbc883 2023-11-28T21:35:35.108 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-11-28T21:35:35.108 INFO:tasks.workunit:timeout=6h 2023-11-28T21:35:35.108 INFO:tasks.workunit:cleanup=True 2023-11-28T21:35:35.108 DEBUG:teuthology.orchestra.run.smithi012:> stat -- /home/ubuntu/cephtest/mnt.0 2023-11-28T21:35:35.109 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2023-11-28T21:35:35.109 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 87 secs before thrashing 2023-11-28T21:35:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-11-28T21:35:35.118 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-11-28T21:35:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 2bh/43d Inode: 1 Links: 2 2023-11-28T21:35:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T21:35:35.118 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2023-11-28T21:35:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2023-11-28 21:34:42.327982642 +0000 2023-11-28T21:35:35.119 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2023-11-28 21:34:50.644657330 +0000 2023-11-28T21:35:35.119 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2023-11-28 21:34:42.327982642 +0000 2023-11-28T21:35:35.119 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-11-28T21:35:35.119 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-11-28T21:35:36.044 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 43dd94c190073cf16cff1e711dd7f5a7b1cbc883 2023-11-28T21:35:36.056 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-11-28T21:35:36.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:35:36.628 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:36:06.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:36:06.711 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:36:36.761 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:36:36.767 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:36:42.802 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 64% (7862/12117) Updating files: 65% (7877/12117) Updating files: 66% (7998/12117) Updating files: 67% (8119/12117) Updating files: 68% (8240/12117) Updating files: 69% (8361/12117) Updating files: 70% (8482/12117) Updating files: 71% (8604/12117) Updating files: 72% (8725/12117) Updating files: 73% (8846/12117) Updating files: 74% (8967/12117) Updating files: 75% (9088/12117) Updating files: 76% (9209/12117) Updating files: 77% (9331/12117) Updating files: 78% (9452/12117) Updating files: 79% (9573/12117) Updating files: 80% (9694/12117) Updating files: 81% (9815/12117) Updating files: 82% (9936/12117) Updating files: 83% (10058/12117) Updating files: 84% (10179/12117) Updating files: 85% (10300/12117) Updating files: 86% (10421/12117) Updating files: 87% (10542/12117) Updating files: 88% (10663/12117) Updating files: 89% (10785/12117) Updating files: 90% (10906/12117) Updating files: 91% (11027/12117) Updating files: 92% (11148/12117) Updating files: 93% (11269/12117) Updating files: 94% (11390/12117) Updating files: 95% (11512/12117) Updating files: 96% (11633/12117) Updating files: 97% (11754/12117) Updating files: 98% (11875/12117) Updating files: 99% (11996/12117) Updating files: 100% (12117/12117) Updating files: 100% (12117/12117), done. 2023-11-28T21:36:43.349 INFO:tasks.workunit.client.0.smithi012.stderr:Note: switching to '43dd94c190073cf16cff1e711dd7f5a7b1cbc883'. 2023-11-28T21:36:43.349 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-11-28T21:36:43.349 INFO:tasks.workunit.client.0.smithi012.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr:changes and commit them, and you can discard any commits you make in this 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr:state without impacting any branches by switching back to a branch. 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr:If you want to create a new branch to retain commits you create, you may 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr:do so (now or later) by using -c with the switch command. Example: 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr: git switch -c 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr:Or undo this operation with: 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr: git switch - 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-11-28T21:36:43.350 INFO:tasks.workunit.client.0.smithi012.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-11-28T21:36:43.351 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-11-28T21:36:43.351 INFO:tasks.workunit.client.0.smithi012.stderr:HEAD is now at 43dd94c1900 Merge PR #52623 into wip-vshankar-testing-20231127.102914 2023-11-28T21:36:43.352 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-11-28T21:36:43.362 INFO:tasks.workunit.client.0.smithi012.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-11-28T21:36:43.365 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-28T21:36:43.365 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-11-28T21:36:44.096 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-11-28T21:36:44.180 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-11-28T21:36:44.237 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-28T21:36:44.239 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-28T21:36:44.239 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-11-28T21:36:44.295 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-28T21:36:44.301 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:36:44.301 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-11-28T21:36:44.350 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2023-11-28T21:36:44.352 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2023-11-28T21:36:44.352 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2023-11-28T21:36:44.413 INFO:tasks.workunit.client.0.smithi012.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2023-11-28T21:36:46.853 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir foo 2023-11-28T21:36:46.863 INFO:tasks.workunit.client.0.smithi012.stderr:+ cp linux-2.6.33.tar.bz2 foo 2023-11-28T21:36:47.355 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir foo/.snap/barsnap 2023-11-28T21:36:47.373 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm foo/linux-2.6.33.tar.bz2 2023-11-28T21:36:48.377 INFO:tasks.workunit.client.0.smithi012.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2023-11-28T21:36:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:passed: files are identical 2023-11-28T21:36:48.833 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo passed: files are identical 2023-11-28T21:36:48.833 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir foo/.snap/barsnap 2023-11-28T21:36:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-11-28T21:36:48.858 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-11-28T21:36:48.858 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-28T21:36:48.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-28T21:36:48.955 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2023-11-28T21:36:48.955 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2023-11-28T21:36:52.391 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 1 2023-11-28T21:36:52.394 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 2 2023-11-28T21:36:52.394 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 3 2023-11-28T21:36:52.394 INFO:tasks.workunit.client.0.smithi012.stderr:+ '[' -e file4 ']' 2023-11-28T21:36:52.394 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir .snap/snap1 2023-11-28T21:36:52.416 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 4 2023-11-28T21:36:52.419 INFO:tasks.workunit.client.0.smithi012.stderr:++ ls 2023-11-28T21:36:52.427 INFO:tasks.workunit.client.0.smithi012.stderr:+ now='file1 2023-11-28T21:36:52.427 INFO:tasks.workunit.client.0.smithi012.stderr:file2 2023-11-28T21:36:52.427 INFO:tasks.workunit.client.0.smithi012.stderr:file3 2023-11-28T21:36:52.428 INFO:tasks.workunit.client.0.smithi012.stderr:file4' 2023-11-28T21:36:52.428 INFO:tasks.workunit.client.0.smithi012.stderr:++ ls .snap/snap1 2023-11-28T21:36:52.435 INFO:tasks.workunit.client.0.smithi012.stderr:+ then='file1 2023-11-28T21:36:52.435 INFO:tasks.workunit.client.0.smithi012.stderr:file2 2023-11-28T21:36:52.436 INFO:tasks.workunit.client.0.smithi012.stderr:file3' 2023-11-28T21:36:52.436 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir .snap/snap1 2023-11-28T21:36:52.447 INFO:tasks.workunit.client.0.smithi012.stderr:+ '[' 'file1 2023-11-28T21:36:52.447 INFO:tasks.workunit.client.0.smithi012.stderr:file2 2023-11-28T21:36:52.447 INFO:tasks.workunit.client.0.smithi012.stderr:file3 2023-11-28T21:36:52.447 INFO:tasks.workunit.client.0.smithi012.stderr:file4' = 'file1 2023-11-28T21:36:52.447 INFO:tasks.workunit.client.0.smithi012.stderr:file2 2023-11-28T21:36:52.447 INFO:tasks.workunit.client.0.smithi012.stderr:file3' ']' 2023-11-28T21:36:52.447 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 1 2023-11-28T21:36:52.466 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 2 2023-11-28T21:36:52.508 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 3 2023-11-28T21:36:52.531 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir .snap/snap1 2023-11-28T21:36:52.555 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo 4 2023-11-28T21:36:52.566 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir .snap/snap1 2023-11-28T21:36:53.355 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm file1 file2 file3 file4 2023-11-28T21:36:53.370 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-11-28T21:36:53.371 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-11-28T21:36:53.372 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-28T21:36:53.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-28T21:36:53.407 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2023-11-28T21:36:53.407 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2023-11-28T21:36:57.393 INFO:tasks.workunit.client.0.smithi012.stdout:Create dir 100 to 199 ... 2023-11-28T21:36:57.394 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 100 2023-11-28T21:36:57.404 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 101 2023-11-28T21:36:57.426 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 102 2023-11-28T21:36:57.447 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 103 2023-11-28T21:36:57.469 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 104 2023-11-28T21:36:57.493 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 105 2023-11-28T21:36:57.521 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 106 2023-11-28T21:36:57.781 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 107 2023-11-28T21:36:58.046 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 108 2023-11-28T21:36:58.067 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 109 2023-11-28T21:36:58.090 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 110 2023-11-28T21:36:58.109 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 111 2023-11-28T21:36:58.129 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 112 2023-11-28T21:36:58.147 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 113 2023-11-28T21:36:58.607 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 114 2023-11-28T21:36:58.629 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 115 2023-11-28T21:36:58.650 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 116 2023-11-28T21:36:58.671 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 117 2023-11-28T21:36:59.014 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 118 2023-11-28T21:36:59.450 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 119 2023-11-28T21:36:59.487 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 120 2023-11-28T21:36:59.509 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 121 2023-11-28T21:36:59.532 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 122 2023-11-28T21:36:59.554 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 123 2023-11-28T21:37:00.162 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 124 2023-11-28T21:37:00.186 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 125 2023-11-28T21:37:00.453 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 126 2023-11-28T21:37:00.475 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 127 2023-11-28T21:37:00.498 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 128 2023-11-28T21:37:00.519 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 129 2023-11-28T21:37:01.176 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 130 2023-11-28T21:37:01.198 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 131 2023-11-28T21:37:01.220 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 132 2023-11-28T21:37:02.110 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:02.140 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 133 2023-11-28T21:37:02.161 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 134 2023-11-28T21:37:02.182 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 135 2023-11-28T21:37:02.204 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 136 2023-11-28T21:37:02.224 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 137 2023-11-28T21:37:02.241 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 138 2023-11-28T21:37:02.259 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 139 2023-11-28T21:37:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e658d6640 1 Processor -- start 2023-11-28T21:37:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e658d6640 1 -- start start 2023-11-28T21:37:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e658d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 0x7f5e60156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e658d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e60067410 con 0x7f5e6006cf40 2023-11-28T21:37:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e658d6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6006e090 con 0x7f5e6006e300 2023-11-28T21:37:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e658d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e60157f30 con 0x7f5e60150c20 2023-11-28T21:37:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e5f7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5e6006cf40 0x7f5e60158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39424/0 (socket says 172.21.15.12:39424) 2023-11-28T21:37:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e5f7fe640 1 -- 172.21.15.12:0/3031924600 learned_addr learned my addr 172.21.15.12:0/3031924600 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e5effd640 1 --2- 172.21.15.12:0/3031924600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 0x7f5e60156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e5e7fc640 1 -- 172.21.15.12:0/3031924600 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044148816 0 0) 0x7f5e60067410 con 0x7f5e6006cf40 2023-11-28T21:37:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e5e7fc640 1 -- 172.21.15.12:0/3031924600 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e4c003530 con 0x7f5e6006cf40 2023-11-28T21:37:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e5e7fc640 1 -- 172.21.15.12:0/3031924600 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724023734 0 0) 0x7f5e6006e090 con 0x7f5e6006e300 2023-11-28T21:37:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e5e7fc640 1 -- 172.21.15.12:0/3031924600 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e60067410 con 0x7f5e6006e300 2023-11-28T21:37:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e5effd640 1 -- 172.21.15.12:0/3031924600 >> v1:172.21.15.144:6789/0 conn(0x7f5e6006e300 legacy=0x7f5e600c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e5effd640 1 -- 172.21.15.12:0/3031924600 >> v1:172.21.15.12:6789/0 conn(0x7f5e6006cf40 legacy=0x7f5e60158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.263+0000 7f5e5effd640 1 -- 172.21.15.12:0/3031924600 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e60151000 con 0x7f5e60150c20 2023-11-28T21:37:02.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.267+0000 7f5e5effd640 1 --2- 172.21.15.12:0/3031924600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 0x7f5e60156910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5e500097f0 tx=0x7f5e5001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843fdeb4e96f10df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:02.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.267+0000 7f5e5e7fc640 1 -- 172.21.15.12:0/3031924600 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e500ab600 con 0x7f5e60150c20 2023-11-28T21:37:02.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.267+0000 7f5e5e7fc640 1 -- 172.21.15.12:0/3031924600 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e500a7050 con 0x7f5e60150c20 2023-11-28T21:37:02.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.267+0000 7f5e5e7fc640 1 -- 172.21.15.12:0/3031924600 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e500ab980 con 0x7f5e60150c20 2023-11-28T21:37:02.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.267+0000 7f5e658d6640 1 -- 172.21.15.12:0/3031924600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 msgr2=0x7f5e60156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.267+0000 7f5e658d6640 1 --2- 172.21.15.12:0/3031924600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 0x7f5e60156910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5e500097f0 tx=0x7f5e5001c4c0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 -- 172.21.15.12:0/3031924600 shutdown_connections 2023-11-28T21:37:02.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 --2- 172.21.15.12:0/3031924600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 0x7f5e60156910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 -- 172.21.15.12:0/3031924600 >> 172.21.15.12:0/3031924600 conn(0x7f5e60066360 msgr2=0x7f5e60066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:02.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 -- 172.21.15.12:0/3031924600 shutdown_connections 2023-11-28T21:37:02.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 -- 172.21.15.12:0/3031924600 wait complete. 2023-11-28T21:37:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 Processor -- start 2023-11-28T21:37:02.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 -- start start 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6006cf40 0x7f5e600c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6006e300 0x7f5e60072620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 0x7f5e601f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e60072420 con 0x7f5e6006cf40 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e601f6ca0 con 0x7f5e6006e300 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e658d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e601f6f80 con 0x7f5e60150c20 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e5effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6006cf40 0x7f5e600c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e5effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6006cf40 0x7f5e600c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50716/0 (socket says 172.21.15.12:50716) 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e5effd640 1 -- 172.21.15.12:0/3123784558 learned_addr learned my addr 172.21.15.12:0/3123784558 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e5f7fe640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 0x7f5e601f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e57fff640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6006e300 0x7f5e60072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.278 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 140 2023-11-28T21:37:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e5effd640 1 -- 172.21.15.12:0/3123784558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 msgr2=0x7f5e601f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e5effd640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 0x7f5e601f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e5effd640 1 -- 172.21.15.12:0/3123784558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6006e300 msgr2=0x7f5e60072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e5effd640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6006e300 0x7f5e60072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e5effd640 1 -- 172.21.15.12:0/3123784558 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e50009420 con 0x7f5e6006cf40 2023-11-28T21:37:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.271+0000 7f5e57fff640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6006e300 0x7f5e60072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e5effd640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6006cf40 0x7f5e600c5020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5e500097f0 tx=0x7f5e50019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e5cff9640 1 -- 172.21.15.12:0/3123784558 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50006e20 con 0x7f5e6006cf40 2023-11-28T21:37:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e5cff9640 1 -- 172.21.15.12:0/3123784558 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e500b4400 con 0x7f5e6006cf40 2023-11-28T21:37:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e5cff9640 1 -- 172.21.15.12:0/3123784558 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e500b4790 con 0x7f5e6006cf40 2023-11-28T21:37:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e658d6640 1 -- 172.21.15.12:0/3123784558 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e600b8f80 con 0x7f5e6006cf40 2023-11-28T21:37:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e658d6640 1 -- 172.21.15.12:0/3123784558 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e600c48b0 con 0x7f5e6006cf40 2023-11-28T21:37:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e658d6640 1 -- 172.21.15.12:0/3123784558 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6006b910 con 0x7f5e6006cf40 2023-11-28T21:37:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e5cff9640 1 -- 172.21.15.12:0/3123784558 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5e500b2040 con 0x7f5e6006cf40 2023-11-28T21:37:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e5cff9640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e400664a0 0x7f5e40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e5cff9640 1 -- 172.21.15.12:0/3123784558 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f5e5011dfa0 con 0x7f5e6006cf40 2023-11-28T21:37:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.275+0000 7f5e57fff640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e400664a0 0x7f5e40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.279+0000 7f5e57fff640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e400664a0 0x7f5e40068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5e601f07c0 tx=0x7f5e48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:02.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.279+0000 7f5e5cff9640 1 -- 172.21.15.12:0/3123784558 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5e500eb030 con 0x7f5e6006cf40 2023-11-28T21:37:02.291 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 141 2023-11-28T21:37:02.307 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 142 2023-11-28T21:37:02.321 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 143 2023-11-28T21:37:02.335 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 144 2023-11-28T21:37:02.348 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 145 2023-11-28T21:37:02.360 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 146 2023-11-28T21:37:02.380 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 147 2023-11-28T21:37:02.392 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 148 2023-11-28T21:37:02.420 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 149 2023-11-28T21:37:02.451 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 150 2023-11-28T21:37:02.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 -- 172.21.15.12:0/3123784558 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e60074f60 con 0x7f5e6006cf40 2023-11-28T21:37:02.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e5cff9640 1 -- 172.21.15.12:0/3123784558 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6529 (secure 0 0 0) 0x7f5e500b0090 con 0x7f5e6006cf40 2023-11-28T21:37:02.479 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:02.480 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:35:14.337396+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:02.480 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2023-11-28T21:37:02.481 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 151 2023-11-28T21:37:02.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 -- 172.21.15.12:0/3123784558 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e400664a0 msgr2=0x7f5e40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e400664a0 0x7f5e40068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5e601f07c0 tx=0x7f5e48009040 comp rx=0 tx=0).stop 2023-11-28T21:37:02.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 -- 172.21.15.12:0/3123784558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6006cf40 msgr2=0x7f5e600c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6006cf40 0x7f5e600c5020 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5e500097f0 tx=0x7f5e50019260 comp rx=0 tx=0).stop 2023-11-28T21:37:02.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 -- 172.21.15.12:0/3123784558 shutdown_connections 2023-11-28T21:37:02.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e400664a0 0x7f5e40068960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e60150c20 0x7f5e601f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e6006e300 0x7f5e60072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 --2- 172.21.15.12:0/3123784558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e6006cf40 0x7f5e600c5020 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 -- 172.21.15.12:0/3123784558 >> 172.21.15.12:0/3123784558 conn(0x7f5e60066360 msgr2=0x7f5e60073a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:02.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 -- 172.21.15.12:0/3123784558 shutdown_connections 2023-11-28T21:37:02.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.475+0000 7f5e658d6640 1 -- 172.21.15.12:0/3123784558 wait complete. 2023-11-28T21:37:02.502 INFO:tasks.mds_thrash.fs.[cephfs]:thrashing max_mds: 3 -> 1 2023-11-28T21:37:02.502 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2023-11-28T21:37:02.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.703+0000 7fb04b1aa640 1 Processor -- start 2023-11-28T21:37:02.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.703+0000 7fb04b1aa640 1 -- start start 2023-11-28T21:37:02.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.703+0000 7fb04b1aa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb044150c20 0x7fb044156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb04b1aa640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb04406e300 0x7fb0440c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb04b1aa640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb044067410 con 0x7fb04406cf40 2023-11-28T21:37:02.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb04b1aa640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04406e090 con 0x7fb044150c20 2023-11-28T21:37:02.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb04b1aa640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb044157f30 con 0x7fb04406e300 2023-11-28T21:37:02.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb048f46640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb044150c20 0x7fb044156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb048f46640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb044150c20 0x7fb044156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50722/0 (socket says 172.21.15.12:50722) 2023-11-28T21:37:02.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb048f46640 1 -- 172.21.15.12:0/353912979 learned_addr learned my addr 172.21.15.12:0/353912979 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:02.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb043fff640 1 --2- 172.21.15.12:0/353912979 >> v2:172.21.15.144:3300/0 conn(0x7fb04406e300 0x7fb0440c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb048f46640 1 -- 172.21.15.12:0/353912979 >> v2:172.21.15.144:3300/0 conn(0x7fb04406e300 msgr2=0x7fb0440c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:02.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb048f46640 1 --2- 172.21.15.12:0/353912979 >> v2:172.21.15.144:3300/0 conn(0x7fb04406e300 0x7fb0440c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb048f46640 1 -- 172.21.15.12:0/353912979 >> v1:172.21.15.12:6789/0 conn(0x7fb04406cf40 legacy=0x7fb044158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb048f46640 1 -- 172.21.15.12:0/353912979 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044151000 con 0x7fb044150c20 2023-11-28T21:37:02.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb048f46640 1 --2- 172.21.15.12:0/353912979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb044150c20 0x7fb044156910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0440c8e60 tx=0x7fb03801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ebcdf82e932a89 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:02.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb0437fe640 1 -- 172.21.15.12:0/353912979 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0380ab610 con 0x7fb044150c20 2023-11-28T21:37:02.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb0437fe640 1 -- 172.21.15.12:0/353912979 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0380ab770 con 0x7fb044150c20 2023-11-28T21:37:02.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.707+0000 7fb0437fe640 1 -- 172.21.15.12:0/353912979 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0380abb00 con 0x7fb044150c20 2023-11-28T21:37:02.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 -- 172.21.15.12:0/353912979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb044150c20 msgr2=0x7fb044156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 --2- 172.21.15.12:0/353912979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb044150c20 0x7fb044156910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0440c8e60 tx=0x7fb03801c500 comp rx=0 tx=0).stop 2023-11-28T21:37:02.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 -- 172.21.15.12:0/353912979 shutdown_connections 2023-11-28T21:37:02.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 --2- 172.21.15.12:0/353912979 >> v2:172.21.15.144:3300/0 conn(0x7fb04406e300 0x7fb0440c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 --2- 172.21.15.12:0/353912979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb044150c20 0x7fb044156910 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 -- 172.21.15.12:0/353912979 >> 172.21.15.12:0/353912979 conn(0x7fb044066360 msgr2=0x7fb044066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:02.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 -- 172.21.15.12:0/353912979 shutdown_connections 2023-11-28T21:37:02.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 -- 172.21.15.12:0/353912979 wait complete. 2023-11-28T21:37:02.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 Processor -- start 2023-11-28T21:37:02.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 -- start start 2023-11-28T21:37:02.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb04406cf40 0x7fb0440c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04406e300 0x7fb044142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb044150c20 0x7fb044143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb044144b10 con 0x7fb04406e300 2023-11-28T21:37:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb044143ed0 con 0x7fb044150c20 2023-11-28T21:37:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb04b1aa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb044145140 con 0x7fb04406cf40 2023-11-28T21:37:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb043fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04406e300 0x7fb044142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb043fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04406e300 0x7fb044142ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50736/0 (socket says 172.21.15.12:50736) 2023-11-28T21:37:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb043fff640 1 -- 172.21.15.12:0/1655052146 learned_addr learned my addr 172.21.15.12:0/1655052146 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:02.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb049747640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb044150c20 0x7fb044143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb048f46640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb04406cf40 0x7fb0440c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb043fff640 1 -- 172.21.15.12:0/1655052146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb04406cf40 msgr2=0x7fb0440c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb043fff640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb04406cf40 0x7fb0440c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb043fff640 1 -- 172.21.15.12:0/1655052146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb044150c20 msgr2=0x7fb044143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:02.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb043fff640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb044150c20 0x7fb044143400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:02.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb043fff640 1 -- 172.21.15.12:0/1655052146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0380094d0 con 0x7fb04406e300 2023-11-28T21:37:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.711+0000 7fb048f46640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb04406cf40 0x7fb0440c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.715+0000 7fb043fff640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04406e300 0x7fb044142ec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e900 tx=0x7fb03000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.715+0000 7fb0417fa640 1 -- 172.21.15.12:0/1655052146 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030009690 con 0x7fb04406e300 2023-11-28T21:37:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.715+0000 7fb0417fa640 1 -- 172.21.15.12:0/1655052146 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03000ca00 con 0x7fb04406e300 2023-11-28T21:37:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.715+0000 7fb0417fa640 1 -- 172.21.15.12:0/1655052146 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000cd90 con 0x7fb04406e300 2023-11-28T21:37:02.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.715+0000 7fb04b1aa640 1 -- 172.21.15.12:0/1655052146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb044144170 con 0x7fb04406e300 2023-11-28T21:37:02.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.715+0000 7fb04b1aa640 1 -- 172.21.15.12:0/1655052146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb044144730 con 0x7fb04406e300 2023-11-28T21:37:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.715+0000 7fb04b1aa640 1 -- 172.21.15.12:0/1655052146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044074f60 con 0x7fb04406e300 2023-11-28T21:37:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.715+0000 7fb0417fa640 1 -- 172.21.15.12:0/1655052146 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb030013470 con 0x7fb04406e300 2023-11-28T21:37:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.715+0000 7fb0417fa640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb028066470 0x7fb028068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.715+0000 7fb0417fa640 1 -- 172.21.15.12:0/1655052146 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fb0300843a0 con 0x7fb04406e300 2023-11-28T21:37:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.719+0000 7fb048f46640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb028066470 0x7fb028068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:02.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.719+0000 7fb048f46640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb028066470 0x7fb028068930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb0380af000 tx=0x7fb0380a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:02.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.723+0000 7fb0417fa640 1 -- 172.21.15.12:0/1655052146 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb0300514e0 con 0x7fb04406e300 2023-11-28T21:37:02.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:02.911+0000 7fb04b1aa640 1 -- 172.21.15.12:0/1655052146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7fb0440c2c30 con 0x7fb04406e300 2023-11-28T21:37:03.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.323+0000 7fb0417fa640 1 -- 172.21.15.12:0/1655052146 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v14) v1 ==== 105+0+0 (secure 0 0 0) 0x7fb030054b00 con 0x7fb04406e300 2023-11-28T21:37:03.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 -- 172.21.15.12:0/1655052146 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb028066470 msgr2=0x7fb028068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb028066470 0x7fb028068930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb0380af000 tx=0x7fb0380a7000 comp rx=0 tx=0).stop 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 -- 172.21.15.12:0/1655052146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04406e300 msgr2=0x7fb044142ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04406e300 0x7fb044142ec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e900 tx=0x7fb03000edc0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 -- 172.21.15.12:0/1655052146 shutdown_connections 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb028066470 0x7fb028068930 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb044150c20 0x7fb044143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb04406e300 0x7fb044142ec0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 --2- 172.21.15.12:0/1655052146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb04406cf40 0x7fb0440c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 -- 172.21.15.12:0/1655052146 >> 172.21.15.12:0/1655052146 conn(0x7fb044066360 msgr2=0x7fb0440c5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 -- 172.21.15.12:0/1655052146 shutdown_connections 2023-11-28T21:37:03.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.327+0000 7fb04b1aa640 1 -- 172.21.15.12:0/1655052146 wait complete. 2023-11-28T21:37:03.356 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:37:03.356 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:03.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.495+0000 7f943a9d8640 1 Processor -- start 2023-11-28T21:37:03.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.495+0000 7f943a9d8640 1 -- start start 2023-11-28T21:37:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f943a9d8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9434151af0 0x7f94340c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f943a9d8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f943406e300 0x7f9434072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f943a9d8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9434157b70 con 0x7f9434151ed0 2023-11-28T21:37:03.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f943a9d8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94340707f0 con 0x7f9434151af0 2023-11-28T21:37:03.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f943a9d8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94340c7b90 con 0x7f943406e300 2023-11-28T21:37:03.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f9433fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9434151af0 0x7f94340c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:03.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f9433fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9434151af0 0x7f94340c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50746/0 (socket says 172.21.15.12:50746) 2023-11-28T21:37:03.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f9433fff640 1 -- 172.21.15.12:0/4118532234 learned_addr learned my addr 172.21.15.12:0/4118532234 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:03.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f9438f75640 1 --2- 172.21.15.12:0/4118532234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f943406e300 0x7f9434072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:03.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f9433fff640 1 -- 172.21.15.12:0/4118532234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f943406e300 msgr2=0x7f9434072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:03.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f9433fff640 1 --2- 172.21.15.12:0/4118532234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f943406e300 0x7f9434072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f9433fff640 1 -- 172.21.15.12:0/4118532234 >> v1:172.21.15.144:6789/0 conn(0x7f9434151ed0 legacy=0x7f94340c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:03.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.499+0000 7f9433fff640 1 -- 172.21.15.12:0/4118532234 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9434157e50 con 0x7f9434151af0 2023-11-28T21:37:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f9433fff640 1 --2- 172.21.15.12:0/4118532234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9434151af0 0x7f94340c7650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9424009870 tx=0x7f942401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4032bc6b8516c206 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f9432ffd640 1 -- 172.21.15.12:0/4118532234 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240ab600 con 0x7f9434151af0 2023-11-28T21:37:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f9432ffd640 1 -- 172.21.15.12:0/4118532234 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94240a7050 con 0x7f9434151af0 2023-11-28T21:37:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f9432ffd640 1 -- 172.21.15.12:0/4118532234 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240ab980 con 0x7f9434151af0 2023-11-28T21:37:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 -- 172.21.15.12:0/4118532234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9434151af0 msgr2=0x7f94340c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 --2- 172.21.15.12:0/4118532234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9434151af0 0x7f94340c7650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9424009870 tx=0x7f942401c500 comp rx=0 tx=0).stop 2023-11-28T21:37:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 -- 172.21.15.12:0/4118532234 shutdown_connections 2023-11-28T21:37:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 --2- 172.21.15.12:0/4118532234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f943406e300 0x7f9434072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 --2- 172.21.15.12:0/4118532234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9434151af0 0x7f94340c7650 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 -- 172.21.15.12:0/4118532234 >> 172.21.15.12:0/4118532234 conn(0x7f9434066360 msgr2=0x7f9434066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:03.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 -- 172.21.15.12:0/4118532234 shutdown_connections 2023-11-28T21:37:03.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 -- 172.21.15.12:0/4118532234 wait complete. 2023-11-28T21:37:03.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 Processor -- start 2023-11-28T21:37:03.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 -- start start 2023-11-28T21:37:03.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943406e300 0x7f94341f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:03.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434151af0 0x7f94341f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:03.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9434142f90 0x7f9434141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:03.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9434153290 con 0x7f943406e300 2023-11-28T21:37:03.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94341f6fd0 con 0x7f9434142f90 2023-11-28T21:37:03.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.503+0000 7f943a9d8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9434143430 con 0x7f9434151af0 2023-11-28T21:37:03.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9433fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943406e300 0x7f94341f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:03.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9433fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943406e300 0x7f94341f6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50760/0 (socket says 172.21.15.12:50760) 2023-11-28T21:37:03.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9433fff640 1 -- 172.21.15.12:0/2016837709 learned_addr learned my addr 172.21.15.12:0/2016837709 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:03.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f94337fe640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434151af0 0x7f94341f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:03.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9438f75640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9434142f90 0x7f9434141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:03.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9433fff640 1 -- 172.21.15.12:0/2016837709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434151af0 msgr2=0x7f94341f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:03.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9433fff640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434151af0 0x7f94341f69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9433fff640 1 -- 172.21.15.12:0/2016837709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9434142f90 msgr2=0x7f9434141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:03.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9433fff640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9434142f90 0x7f9434141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9433fff640 1 -- 172.21.15.12:0/2016837709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94240094d0 con 0x7f943406e300 2023-11-28T21:37:03.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9433fff640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943406e300 0x7f94341f6460 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f942401abb0 tx=0x7f942401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:03.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9430ff9640 1 -- 172.21.15.12:0/2016837709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240b6420 con 0x7f943406e300 2023-11-28T21:37:03.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9430ff9640 1 -- 172.21.15.12:0/2016837709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94240b65a0 con 0x7f943406e300 2023-11-28T21:37:03.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9430ff9640 1 -- 172.21.15.12:0/2016837709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240b6950 con 0x7f943406e300 2023-11-28T21:37:03.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f943a9d8640 1 -- 172.21.15.12:0/2016837709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94341436d0 con 0x7f943406e300 2023-11-28T21:37:03.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f943a9d8640 1 -- 172.21.15.12:0/2016837709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9434152fb0 con 0x7f943406e300 2023-11-28T21:37:03.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f943a9d8640 1 -- 172.21.15.12:0/2016837709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9434074f60 con 0x7f943406e300 2023-11-28T21:37:03.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.507+0000 7f9430ff9640 1 -- 172.21.15.12:0/2016837709 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f94240b2040 con 0x7f943406e300 2023-11-28T21:37:03.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.511+0000 7f9430ff9640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9420066470 0x7f9420068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:03.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.511+0000 7f9430ff9640 1 -- 172.21.15.12:0/2016837709 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f942411e610 con 0x7f943406e300 2023-11-28T21:37:03.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.511+0000 7f94337fe640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9420066470 0x7f9420068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:03.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.511+0000 7f94337fe640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9420066470 0x7f9420068930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94341415f0 tx=0x7f9428008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:03.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.515+0000 7f9430ff9640 1 -- 172.21.15.12:0/2016837709 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f94240eb570 con 0x7f943406e300 2023-11-28T21:37:03.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 -- 172.21.15.12:0/2016837709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9434142d70 con 0x7f943406e300 2023-11-28T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f9430ff9640 1 -- 172.21.15.12:0/2016837709 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7f94240eeb90 con 0x7f943406e300 2023-11-28T21:37:03.711 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:03.712 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:03.712 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 -- 172.21.15.12:0/2016837709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9420066470 msgr2=0x7f9420068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:03.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9420066470 0x7f9420068930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94341415f0 tx=0x7f9428008040 comp rx=0 tx=0).stop 2023-11-28T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 -- 172.21.15.12:0/2016837709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943406e300 msgr2=0x7f94341f6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943406e300 0x7f94341f6460 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f942401abb0 tx=0x7f942401ad40 comp rx=0 tx=0).stop 2023-11-28T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 -- 172.21.15.12:0/2016837709 shutdown_connections 2023-11-28T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9420066470 0x7f9420068930 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9434142f90 0x7f9434141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434151af0 0x7f94341f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 --2- 172.21.15.12:0/2016837709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f943406e300 0x7f94341f6460 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.707+0000 7f943a9d8640 1 -- 172.21.15.12:0/2016837709 >> 172.21.15.12:0/2016837709 conn(0x7f9434066360 msgr2=0x7f943406cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.711+0000 7f943a9d8640 1 -- 172.21.15.12:0/2016837709 shutdown_connections 2023-11-28T21:37:03.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:03.711+0000 7f943a9d8640 1 -- 172.21.15.12:0/2016837709 wait complete. 2023-11-28T21:37:03.733 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-11-28T21:37:03.733 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 14, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 14, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.12:6838/949980653", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 949980653, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 949980653, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4247, "incarnation": 12, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:stopping", "state_seq": 13 }, "gid_4250": { "addr": "172.21.15.12:6839/3027655738", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3027655738, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3027655738, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4250, "incarnation": 8, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 11 }, "gid_4261": { "addr": "172.21.15.144:6833/3053317078", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 3053317078, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 3053317078, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4261, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:37:03.325643+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4261, "mds_1": 4250, "mds_2": 4247 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:37:03.788 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 152 2023-11-28T21:37:03.813 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 153 2023-11-28T21:37:04.077 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 154 2023-11-28T21:37:04.342 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 155 2023-11-28T21:37:04.610 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 156 2023-11-28T21:37:04.632 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 157 2023-11-28T21:37:04.654 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 158 2023-11-28T21:37:04.676 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 159 2023-11-28T21:37:04.698 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 160 2023-11-28T21:37:04.716 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 161 2023-11-28T21:37:04.734 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 162 2023-11-28T21:37:04.750 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 163 2023-11-28T21:37:04.768 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 164 2023-11-28T21:37:04.783 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 165 2023-11-28T21:37:04.798 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 166 2023-11-28T21:37:05.061 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 167 2023-11-28T21:37:05.077 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 168 2023-11-28T21:37:05.099 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 169 2023-11-28T21:37:05.121 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 170 2023-11-28T21:37:05.141 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 171 2023-11-28T21:37:05.162 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 172 2023-11-28T21:37:05.181 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 173 2023-11-28T21:37:05.198 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 174 2023-11-28T21:37:05.213 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 175 2023-11-28T21:37:05.229 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 176 2023-11-28T21:37:05.244 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 177 2023-11-28T21:37:05.258 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 178 2023-11-28T21:37:05.271 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 179 2023-11-28T21:37:05.284 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 180 2023-11-28T21:37:05.296 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 181 2023-11-28T21:37:05.308 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 182 2023-11-28T21:37:05.320 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 183 2023-11-28T21:37:05.332 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 184 2023-11-28T21:37:05.343 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 185 2023-11-28T21:37:05.735 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:05.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f653c209640 1 Processor -- start 2023-11-28T21:37:05.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f653c209640 1 -- start start 2023-11-28T21:37:05.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f653c209640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6534151af0 0x7f65340c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:05.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f653c209640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6534151ed0 0x7f65340c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:05.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f653c209640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6534157b70 con 0x7f653406e300 2023-11-28T21:37:05.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f653c209640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65340707f0 con 0x7f6534151af0 2023-11-28T21:37:05.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f653c209640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65340c7b90 con 0x7f6534151ed0 2023-11-28T21:37:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f6539fa5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6534151af0 0x7f65340c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f653a7a6640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f653406e300 0x7f6534072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35030/0 (socket says 172.21.15.12:35030) 2023-11-28T21:37:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f653a7a6640 1 -- 172.21.15.12:0/3051653827 learned_addr learned my addr 172.21.15.12:0/3051653827 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f65397a4640 1 --2- 172.21.15.12:0/3051653827 >> v2:172.21.15.144:3300/0 conn(0x7f6534151ed0 0x7f65340c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f6539fa5640 1 -- 172.21.15.12:0/3051653827 >> v2:172.21.15.144:3300/0 conn(0x7f6534151ed0 msgr2=0x7f65340c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f6539fa5640 1 --2- 172.21.15.12:0/3051653827 >> v2:172.21.15.144:3300/0 conn(0x7f6534151ed0 0x7f65340c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f6539fa5640 1 -- 172.21.15.12:0/3051653827 >> v1:172.21.15.12:6789/0 conn(0x7f653406e300 legacy=0x7f6534072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f6539fa5640 1 -- 172.21.15.12:0/3051653827 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6534157e50 con 0x7f6534151af0 2023-11-28T21:37:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.883+0000 7f6538fa3640 1 -- 172.21.15.12:0/3051653827 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011137787 0 0) 0x7f6534157b70 con 0x7f653406e300 2023-11-28T21:37:05.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f6539fa5640 1 --2- 172.21.15.12:0/3051653827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 0x7f65340c7650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f65340c8e60 tx=0x7f653001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282e2c15bde86b3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:05.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f6538fa3640 1 -- 172.21.15.12:0/3051653827 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65300ab600 con 0x7f6534151af0 2023-11-28T21:37:05.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f6538fa3640 1 -- 172.21.15.12:0/3051653827 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65300a7050 con 0x7f6534151af0 2023-11-28T21:37:05.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f6538fa3640 1 -- 172.21.15.12:0/3051653827 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65300ab980 con 0x7f6534151af0 2023-11-28T21:37:05.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f653c209640 1 -- 172.21.15.12:0/3051653827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 msgr2=0x7f65340c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:05.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f653c209640 1 --2- 172.21.15.12:0/3051653827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 0x7f65340c7650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f65340c8e60 tx=0x7f653001c460 comp rx=0 tx=0).stop 2023-11-28T21:37:05.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f653c209640 1 -- 172.21.15.12:0/3051653827 shutdown_connections 2023-11-28T21:37:05.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f653c209640 1 --2- 172.21.15.12:0/3051653827 >> v2:172.21.15.144:3300/0 conn(0x7f6534151ed0 0x7f65340c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:05.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f653c209640 1 --2- 172.21.15.12:0/3051653827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 0x7f65340c7650 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:05.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f653c209640 1 -- 172.21.15.12:0/3051653827 >> 172.21.15.12:0/3051653827 conn(0x7f6534066360 msgr2=0x7f6534066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:05.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f653c209640 1 -- 172.21.15.12:0/3051653827 shutdown_connections 2023-11-28T21:37:05.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.887+0000 7f653c209640 1 -- 172.21.15.12:0/3051653827 wait complete. 2023-11-28T21:37:05.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 Processor -- start 2023-11-28T21:37:05.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 -- start start 2023-11-28T21:37:05.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f653406e300 0x7f65341f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:05.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 0x7f6534148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:05.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6534144c70 0x7f65341451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:05.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653414a530 con 0x7f6534151af0 2023-11-28T21:37:05.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65341f3640 con 0x7f6534144c70 2023-11-28T21:37:05.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65340c5df0 con 0x7f653406e300 2023-11-28T21:37:05.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f65397a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 0x7f6534148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:05.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f65397a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 0x7f6534148d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49300/0 (socket says 172.21.15.12:49300) 2023-11-28T21:37:05.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f65397a4640 1 -- 172.21.15.12:0/1860542574 learned_addr learned my addr 172.21.15.12:0/1860542574 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:05.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f6539fa5640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f653406e300 0x7f65341f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:05.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653a7a6640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6534144c70 0x7f65341451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:05.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f65397a4640 1 -- 172.21.15.12:0/1860542574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f653406e300 msgr2=0x7f65341f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:05.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f65397a4640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f653406e300 0x7f65341f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:05.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f65397a4640 1 -- 172.21.15.12:0/1860542574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6534144c70 msgr2=0x7f65341451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:05.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f65397a4640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6534144c70 0x7f65341451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:05.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f65397a4640 1 -- 172.21.15.12:0/1860542574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6530009420 con 0x7f6534151af0 2023-11-28T21:37:05.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f65397a4640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 0x7f6534148d00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f652400d5c0 tx=0x7f652400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:05.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f652a7fc640 1 -- 172.21.15.12:0/1860542574 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524004240 con 0x7f6534151af0 2023-11-28T21:37:05.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f652a7fc640 1 -- 172.21.15.12:0/1860542574 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65240043a0 con 0x7f6534151af0 2023-11-28T21:37:05.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 -- 172.21.15.12:0/1860542574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6534145700 con 0x7f6534151af0 2023-11-28T21:37:05.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f652a7fc640 1 -- 172.21.15.12:0/1860542574 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524003840 con 0x7f6534151af0 2023-11-28T21:37:05.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 -- 172.21.15.12:0/1860542574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653414b750 con 0x7f6534151af0 2023-11-28T21:37:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.891+0000 7f653c209640 1 -- 172.21.15.12:0/1860542574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65341f37f0 con 0x7f6534151af0 2023-11-28T21:37:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.895+0000 7f652a7fc640 1 -- 172.21.15.12:0/1860542574 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f65240039a0 con 0x7f6534151af0 2023-11-28T21:37:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.895+0000 7f652a7fc640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 0x7f6518068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.895+0000 7f652a7fc640 1 -- 172.21.15.12:0/1860542574 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f65240845a0 con 0x7f6534151af0 2023-11-28T21:37:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.895+0000 7f6539fa5640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 0x7f6518068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:05.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.895+0000 7f6539fa5640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 0x7f6518068930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f65300ad000 tx=0x7f6530008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:05.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:05.899+0000 7f652a7fc640 1 -- 172.21.15.12:0/1860542574 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f65341f37f0 con 0x7f6534151af0 2023-11-28T21:37:06.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.086+0000 7f653c209640 1 -- 172.21.15.12:0/1860542574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65341f37f0 con 0x7f6534151af0 2023-11-28T21:37:06.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.086+0000 7f652a7fc640 1 -- 172.21.15.12:0/1860542574 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7f65341f37f0 con 0x7f6534151af0 2023-11-28T21:37:06.093 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:06.093 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 -- 172.21.15.12:0/1860542574 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 msgr2=0x7f6518068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:06.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 0x7f6518068930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f65300ad000 tx=0x7f6530008040 comp rx=0 tx=0).stop 2023-11-28T21:37:06.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 -- 172.21.15.12:0/1860542574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 msgr2=0x7f6534148d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:06.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 0x7f6534148d00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f652400d5c0 tx=0x7f652400da80 comp rx=0 tx=0).stop 2023-11-28T21:37:06.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 -- 172.21.15.12:0/1860542574 shutdown_connections 2023-11-28T21:37:06.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 0x7f6518068930 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:06.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6534144c70 0x7f65341451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:06.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534151af0 0x7f6534148d00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:06.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 --2- 172.21.15.12:0/1860542574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f653406e300 0x7f65341f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:06.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 -- 172.21.15.12:0/1860542574 >> 172.21.15.12:0/1860542574 conn(0x7f6534066360 msgr2=0x7f653406faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:06.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 -- 172.21.15.12:0/1860542574 shutdown_connections 2023-11-28T21:37:06.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:06.090+0000 7f653c209640 1 -- 172.21.15.12:0/1860542574 wait complete. 2023-11-28T21:37:06.113 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:06.305 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 186 2023-11-28T21:37:06.319 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 187 2023-11-28T21:37:06.335 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 188 2023-11-28T21:37:06.353 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 189 2023-11-28T21:37:06.371 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 190 2023-11-28T21:37:06.634 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 191 2023-11-28T21:37:06.663 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 192 2023-11-28T21:37:06.694 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 193 2023-11-28T21:37:06.732 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 194 2023-11-28T21:37:06.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:37:06.834 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:37:07.733 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 195 2023-11-28T21:37:08.115 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:08.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.262+0000 7f8c4ecd6640 1 Processor -- start 2023-11-28T21:37:08.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.262+0000 7f8c4ecd6640 1 -- start start 2023-11-28T21:37:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c4ecd6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c48151af0 0x7f8c480c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c4ecd6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8c48151ed0 0x7f8c480c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c4ecd6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c48157b70 con 0x7f8c4806e300 2023-11-28T21:37:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c4ecd6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c480707f0 con 0x7f8c48151ed0 2023-11-28T21:37:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c4ecd6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480c7b90 con 0x7f8c48151af0 2023-11-28T21:37:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c47fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8c48151ed0 0x7f8c480c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c47fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8c48151ed0 0x7f8c480c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49308/0 (socket says 172.21.15.12:49308) 2023-11-28T21:37:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c4d273640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8c4806e300 0x7f8c48072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35040/0 (socket says 172.21.15.12:35040) 2023-11-28T21:37:08.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c47fff640 1 -- 172.21.15.12:0/799970292 learned_addr learned my addr 172.21.15.12:0/799970292 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c4ca72640 1 --2- 172.21.15.12:0/799970292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c48151af0 0x7f8c480c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c47fff640 1 -- 172.21.15.12:0/799970292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c48151af0 msgr2=0x7f8c480c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c47fff640 1 --2- 172.21.15.12:0/799970292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c48151af0 0x7f8c480c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c47fff640 1 -- 172.21.15.12:0/799970292 >> v1:172.21.15.12:6789/0 conn(0x7f8c4806e300 legacy=0x7f8c48072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:08.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.266+0000 7f8c47fff640 1 -- 172.21.15.12:0/799970292 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48157e50 con 0x7f8c48151ed0 2023-11-28T21:37:08.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c47fff640 1 --2- 172.21.15.12:0/799970292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151ed0 0x7f8c480c49d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009870 tx=0x7f8c3401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db321a10ffd5ad3e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:08.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c477fe640 1 -- 172.21.15.12:0/799970292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c340ab600 con 0x7f8c48151ed0 2023-11-28T21:37:08.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c477fe640 1 -- 172.21.15.12:0/799970292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c340a7050 con 0x7f8c48151ed0 2023-11-28T21:37:08.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c477fe640 1 -- 172.21.15.12:0/799970292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c340ab980 con 0x7f8c48151ed0 2023-11-28T21:37:08.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/799970292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151ed0 msgr2=0x7f8c480c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:08.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c4ecd6640 1 --2- 172.21.15.12:0/799970292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151ed0 0x7f8c480c49d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009870 tx=0x7f8c3401c540 comp rx=0 tx=0).stop 2023-11-28T21:37:08.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/799970292 shutdown_connections 2023-11-28T21:37:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c4ecd6640 1 --2- 172.21.15.12:0/799970292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151ed0 0x7f8c480c49d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c4ecd6640 1 --2- 172.21.15.12:0/799970292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c48151af0 0x7f8c480c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/799970292 >> 172.21.15.12:0/799970292 conn(0x7f8c48066360 msgr2=0x7f8c48066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/799970292 shutdown_connections 2023-11-28T21:37:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/799970292 wait complete. 2023-11-28T21:37:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c4ecd6640 1 Processor -- start 2023-11-28T21:37:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.270+0000 7f8c4ecd6640 1 -- start start 2023-11-28T21:37:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4ecd6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c4806e300 0x7f8c4814cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4ecd6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151af0 0x7f8c4813ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4ecd6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4814db40 0x7f8c481f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4ecd6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481439b0 con 0x7f8c48151af0 2023-11-28T21:37:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4ecd6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4814d3b0 con 0x7f8c4814db40 2023-11-28T21:37:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4ecd6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c48143080 con 0x7f8c4806e300 2023-11-28T21:37:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c47fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151af0 0x7f8c4813ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c47fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151af0 0x7f8c4813ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49312/0 (socket says 172.21.15.12:49312) 2023-11-28T21:37:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c47fff640 1 -- 172.21.15.12:0/2757461157 learned_addr learned my addr 172.21.15.12:0/2757461157 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4ca72640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c4806e300 0x7f8c4814cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4d273640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4814db40 0x7f8c481f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c47fff640 1 -- 172.21.15.12:0/2757461157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c4806e300 msgr2=0x7f8c4814cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c47fff640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c4806e300 0x7f8c4814cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c47fff640 1 -- 172.21.15.12:0/2757461157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4814db40 msgr2=0x7f8c481f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c47fff640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4814db40 0x7f8c481f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c47fff640 1 -- 172.21.15.12:0/2757461157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c340094d0 con 0x7f8c48151af0 2023-11-28T21:37:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4d273640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4814db40 0x7f8c481f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c47fff640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151af0 0x7f8c4813ea50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8c3401a170 tx=0x7f8c340192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c457fa640 1 -- 172.21.15.12:0/2757461157 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c340aa7b0 con 0x7f8c48151af0 2023-11-28T21:37:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c457fa640 1 -- 172.21.15.12:0/2757461157 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c340aa910 con 0x7f8c48151af0 2023-11-28T21:37:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/2757461157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c48143360 con 0x7f8c48151af0 2023-11-28T21:37:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c457fa640 1 -- 172.21.15.12:0/2757461157 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c340aac80 con 0x7f8c48151af0 2023-11-28T21:37:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/2757461157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4814d670 con 0x7f8c48151af0 2023-11-28T21:37:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.274+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/2757461157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c48143810 con 0x7f8c48151af0 2023-11-28T21:37:08.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.278+0000 7f8c457fa640 1 -- 172.21.15.12:0/2757461157 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8c340b2070 con 0x7f8c48151af0 2023-11-28T21:37:08.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.278+0000 7f8c457fa640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c2c066470 0x7f8c2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:08.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.278+0000 7f8c457fa640 1 -- 172.21.15.12:0/2757461157 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f8c3411dc90 con 0x7f8c48151af0 2023-11-28T21:37:08.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.278+0000 7f8c4ca72640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c2c066470 0x7f8c2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:08.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.278+0000 7f8c4ca72640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c2c066470 0x7f8c2c068930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c007a60 tx=0x7f8c3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:08.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.282+0000 7f8c457fa640 1 -- 172.21.15.12:0/2757461157 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8c48143810 con 0x7f8c48151af0 2023-11-28T21:37:08.353 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 196 2023-11-28T21:37:08.373 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 197 2023-11-28T21:37:08.398 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 198 2023-11-28T21:37:08.421 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 199 2023-11-28T21:37:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.478+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/2757461157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c48143b60 con 0x7f8c48151af0 2023-11-28T21:37:08.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.478+0000 7f8c457fa640 1 -- 172.21.15.12:0/2757461157 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7f8c48143b60 con 0x7f8c48151af0 2023-11-28T21:37:08.485 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:08.485 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:08.485 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:08.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/2757461157 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c2c066470 msgr2=0x7f8c2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c2c066470 0x7f8c2c068930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c007a60 tx=0x7f8c3c008040 comp rx=0 tx=0).stop 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/2757461157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151af0 msgr2=0x7f8c4813ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151af0 0x7f8c4813ea50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8c3401a170 tx=0x7f8c340192e0 comp rx=0 tx=0).stop 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/2757461157 shutdown_connections 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c2c066470 0x7f8c2c068930 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c4814db40 0x7f8c481f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c48151af0 0x7f8c4813ea50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 --2- 172.21.15.12:0/2757461157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c4806e300 0x7f8c4814cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/2757461157 >> 172.21.15.12:0/2757461157 conn(0x7f8c48066360 msgr2=0x7f8c48073a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/2757461157 shutdown_connections 2023-11-28T21:37:08.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:08.482+0000 7f8c4ecd6640 1 -- 172.21.15.12:0/2757461157 wait complete. 2023-11-28T21:37:08.505 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:08.620 INFO:tasks.workunit.client.0.smithi012.stdout:Take first snapshot .snap/test1 2023-11-28T21:37:08.960 INFO:tasks.workunit.client.0.smithi012.stdout:Create dir 200 to 299 ... 2023-11-28T21:37:08.962 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 200 2023-11-28T21:37:08.971 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 201 2023-11-28T21:37:08.993 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 202 2023-11-28T21:37:09.015 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 203 2023-11-28T21:37:09.036 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 204 2023-11-28T21:37:09.054 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 205 2023-11-28T21:37:09.070 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 206 2023-11-28T21:37:09.086 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 207 2023-11-28T21:37:09.101 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 208 2023-11-28T21:37:09.115 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 209 2023-11-28T21:37:09.128 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 210 2023-11-28T21:37:09.142 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 211 2023-11-28T21:37:09.408 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 212 2023-11-28T21:37:09.424 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 213 2023-11-28T21:37:09.689 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 214 2023-11-28T21:37:09.708 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 215 2023-11-28T21:37:09.728 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 216 2023-11-28T21:37:09.749 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 217 2023-11-28T21:37:09.768 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 218 2023-11-28T21:37:09.787 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 219 2023-11-28T21:37:09.804 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 220 2023-11-28T21:37:09.821 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 221 2023-11-28T21:37:09.836 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 222 2023-11-28T21:37:09.850 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 223 2023-11-28T21:37:10.506 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:10.636 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 224 2023-11-28T21:37:10.652 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 225 2023-11-28T21:37:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd450596640 1 Processor -- start 2023-11-28T21:37:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd450596640 1 -- start start 2023-11-28T21:37:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd450596640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd448151af0 0x7fd4480c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd450596640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd448151ed0 0x7fd4480c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd450596640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd448157b70 con 0x7fd44806e300 2023-11-28T21:37:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd450596640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4480707f0 con 0x7fd448151af0 2023-11-28T21:37:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd450596640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480c7b90 con 0x7fd448151ed0 2023-11-28T21:37:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44eb33640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd44806e300 0x7fd448072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35042/0 (socket says 172.21.15.12:35042) 2023-11-28T21:37:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44eb33640 1 -- 172.21.15.12:0/810385277 learned_addr learned my addr 172.21.15.12:0/810385277 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44e332640 1 --2- 172.21.15.12:0/810385277 >> v2:172.21.15.144:3300/0 conn(0x7fd448151af0 0x7fd4480c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44db31640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd448151ed0 0x7fd4480c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511098155 0 0) 0x7fd448157b70 con 0x7fd44806e300 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd434003530 con 0x7fd44806e300 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4228328996 0 0) 0x7fd434003530 con 0x7fd44806e300 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd448151ed0 msgr2=0x7fd4480c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd448151ed0 0x7fd4480c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 >> v2:172.21.15.144:3300/0 conn(0x7fd448151af0 msgr2=0x7fd4480c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 --2- 172.21.15.12:0/810385277 >> v2:172.21.15.144:3300/0 conn(0x7fd448151af0 0x7fd4480c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd448157e50 con 0x7fd44806e300 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fd440001a00 con 0x7fd44806e300 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 >> v1:172.21.15.12:6789/0 conn(0x7fd44806e300 legacy=0x7fd448072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4340039f0 0x7fd434003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4340047d0 0x7fd434004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd434005190 0x7fd43400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480707f0 con 0x7fd4340039f0 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4480c7b90 con 0x7fd4340047d0 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340045e0 con 0x7fd434005190 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44e332640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4340039f0 0x7fd434003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44db31640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4340047d0 0x7fd434004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44eb33640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd434005190 0x7fd43400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44e332640 1 -- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd434005190 msgr2=0x7fd43400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44e332640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd434005190 0x7fd43400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44e332640 1 -- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4340047d0 msgr2=0x7fd434004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44e332640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4340047d0 0x7fd434004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44e332640 1 -- 172.21.15.12:0/810385277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd434003530 con 0x7fd4340039f0 2023-11-28T21:37:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.658+0000 7fd44eb33640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd434005190 0x7fd43400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:37:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd44e332640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4340039f0 0x7fd434003e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd444009ae0 tx=0x7fd44401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a0b378684ced4e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4440a6470 con 0x7fd4340039f0 2023-11-28T21:37:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4440a65d0 con 0x7fd4340039f0 2023-11-28T21:37:10.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd44d330640 1 -- 172.21.15.12:0/810385277 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4440a6980 con 0x7fd4340039f0 2023-11-28T21:37:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd450596640 1 -- 172.21.15.12:0/810385277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4340039f0 msgr2=0x7fd434003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd450596640 1 --2- 172.21.15.12:0/810385277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4340039f0 0x7fd434003e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd444009ae0 tx=0x7fd44401ca30 comp rx=0 tx=0).stop 2023-11-28T21:37:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd44eb33640 1 -- 172.21.15.12:0/810385277 reap_dead start 2023-11-28T21:37:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd450596640 1 -- 172.21.15.12:0/810385277 shutdown_connections 2023-11-28T21:37:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd450596640 1 -- 172.21.15.12:0/810385277 >> 172.21.15.12:0/810385277 conn(0x7fd448066360 msgr2=0x7fd448066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:10.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd450596640 1 -- 172.21.15.12:0/810385277 shutdown_connections 2023-11-28T21:37:10.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.662+0000 7fd450596640 1 -- 172.21.15.12:0/810385277 wait complete. 2023-11-28T21:37:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd450596640 1 Processor -- start 2023-11-28T21:37:10.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd450596640 1 -- start start 2023-11-28T21:37:10.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd450596640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4340039f0 0x7fd448147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd450596640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4481595a0 0x7fd448146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd450596640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd448146dd0 0x7fd4481f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd450596640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44815ab30 con 0x7fd4481595a0 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd450596640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4481e5740 con 0x7fd4340039f0 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd450596640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd448147430 con 0x7fd448146dd0 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44db31640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4481595a0 0x7fd448146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44db31640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4481595a0 0x7fd448146890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49332/0 (socket says 172.21.15.12:49332) 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44db31640 1 -- 172.21.15.12:0/432183336 learned_addr learned my addr 172.21.15.12:0/432183336 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44e332640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4340039f0 0x7fd448147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44eb33640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd448146dd0 0x7fd4481f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44db31640 1 -- 172.21.15.12:0/432183336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd448146dd0 msgr2=0x7fd4481f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44db31640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd448146dd0 0x7fd4481f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44db31640 1 -- 172.21.15.12:0/432183336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4340039f0 msgr2=0x7fd448147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44db31640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4340039f0 0x7fd448147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44db31640 1 -- 172.21.15.12:0/432183336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd448159a40 con 0x7fd4481595a0 2023-11-28T21:37:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd44db31640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4481595a0 0x7fd448146890 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd438011930 tx=0x7fd438011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd43effd640 1 -- 172.21.15.12:0/432183336 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4380095b0 con 0x7fd4481595a0 2023-11-28T21:37:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd43effd640 1 -- 172.21.15.12:0/432183336 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43800faf0 con 0x7fd4481595a0 2023-11-28T21:37:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd43effd640 1 -- 172.21.15.12:0/432183336 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43800fe80 con 0x7fd4481595a0 2023-11-28T21:37:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.666+0000 7fd450596640 1 -- 172.21.15.12:0/432183336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44814dcd0 con 0x7fd4481595a0 2023-11-28T21:37:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.670+0000 7fd450596640 1 -- 172.21.15.12:0/432183336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4481479b0 con 0x7fd4481595a0 2023-11-28T21:37:10.675 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 226 2023-11-28T21:37:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.670+0000 7fd450596640 1 -- 172.21.15.12:0/432183336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4481471e0 con 0x7fd4481595a0 2023-11-28T21:37:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.670+0000 7fd43effd640 1 -- 172.21.15.12:0/432183336 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd438014440 con 0x7fd4481595a0 2023-11-28T21:37:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.670+0000 7fd43effd640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd42c066400 0x7fd42c0688c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.670+0000 7fd43effd640 1 -- 172.21.15.12:0/432183336 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fd438084bf0 con 0x7fd4481595a0 2023-11-28T21:37:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.670+0000 7fd44e332640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd42c066400 0x7fd42c0688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.674+0000 7fd44e332640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd42c066400 0x7fd42c0688c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd444009ae0 tx=0x7fd4440b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:10.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.674+0000 7fd43effd640 1 -- 172.21.15.12:0/432183336 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd438051b50 con 0x7fd4481595a0 2023-11-28T21:37:10.694 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 227 2023-11-28T21:37:10.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 -- 172.21.15.12:0/432183336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd448074f60 con 0x7fd4481595a0 2023-11-28T21:37:10.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd43effd640 1 -- 172.21.15.12:0/432183336 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7fd438022020 con 0x7fd4481595a0 2023-11-28T21:37:10.875 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:10.875 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:10.875 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 -- 172.21.15.12:0/432183336 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd42c066400 msgr2=0x7fd42c0688c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd42c066400 0x7fd42c0688c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd444009ae0 tx=0x7fd4440b3040 comp rx=0 tx=0).stop 2023-11-28T21:37:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 -- 172.21.15.12:0/432183336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4481595a0 msgr2=0x7fd448146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4481595a0 0x7fd448146890 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd438011930 tx=0x7fd438011df0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 -- 172.21.15.12:0/432183336 shutdown_connections 2023-11-28T21:37:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd42c066400 0x7fd42c0688c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd448146dd0 0x7fd4481f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4481595a0 0x7fd448146890 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 --2- 172.21.15.12:0/432183336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4340039f0 0x7fd448147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 -- 172.21.15.12:0/432183336 >> 172.21.15.12:0/432183336 conn(0x7fd448066360 msgr2=0x7fd4480c3510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 -- 172.21.15.12:0/432183336 shutdown_connections 2023-11-28T21:37:10.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:10.870+0000 7fd450596640 1 -- 172.21.15.12:0/432183336 wait complete. 2023-11-28T21:37:10.896 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:11.562 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 228 2023-11-28T21:37:11.583 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 229 2023-11-28T21:37:11.605 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 230 2023-11-28T21:37:11.626 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 231 2023-11-28T21:37:11.646 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 232 2023-11-28T21:37:11.666 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 233 2023-11-28T21:37:11.683 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 234 2023-11-28T21:37:11.700 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 235 2023-11-28T21:37:11.714 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 236 2023-11-28T21:37:12.510 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 237 2023-11-28T21:37:12.620 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 238 2023-11-28T21:37:12.652 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 239 2023-11-28T21:37:12.686 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 240 2023-11-28T21:37:12.721 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 241 2023-11-28T21:37:12.756 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 242 2023-11-28T21:37:12.792 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 243 2023-11-28T21:37:12.828 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 244 2023-11-28T21:37:12.863 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 245 2023-11-28T21:37:12.897 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:12.901 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 246 2023-11-28T21:37:12.936 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 247 2023-11-28T21:37:12.959 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 248 2023-11-28T21:37:12.977 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 249 2023-11-28T21:37:12.993 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 250 2023-11-28T21:37:13.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.038+0000 7f99ece17640 1 Processor -- start 2023-11-28T21:37:13.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.038+0000 7f99ece17640 1 -- start start 2023-11-28T21:37:13.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99ece17640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f99e80cac40 0x7f99e806b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:13.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99ece17640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f99e80c4980 0x7f99e80cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:13.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99ece17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99e80ca030 0x7f99e8159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:13.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99ece17640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e80cb560 con 0x7f99e80cac40 2023-11-28T21:37:13.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99ece17640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e80c4da0 con 0x7f99e80c4980 2023-11-28T21:37:13.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99ece17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e815a5e0 con 0x7f99e80ca030 2023-11-28T21:37:13.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99e659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f99e80cac40 0x7f99e806b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:13.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99e659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f99e80cac40 0x7f99e806b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49342/0 (socket says 172.21.15.12:49342) 2023-11-28T21:37:13.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99e659c640 1 -- 172.21.15.12:0/2248640380 learned_addr learned my addr 172.21.15.12:0/2248640380 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:13.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99e5d9b640 1 --2- 172.21.15.12:0/2248640380 >> v2:172.21.15.144:3300/0 conn(0x7f99e80c4980 0x7f99e80cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:13.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99e6d9d640 1 --2- 172.21.15.12:0/2248640380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99e80ca030 0x7f99e8159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:13.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99e659c640 1 -- 172.21.15.12:0/2248640380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99e80ca030 msgr2=0x7f99e8159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:13.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99e659c640 1 --2- 172.21.15.12:0/2248640380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99e80ca030 0x7f99e8159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99e659c640 1 -- 172.21.15.12:0/2248640380 >> v2:172.21.15.144:3300/0 conn(0x7f99e80c4980 msgr2=0x7f99e80cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:13.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99e659c640 1 --2- 172.21.15.12:0/2248640380 >> v2:172.21.15.144:3300/0 conn(0x7f99e80c4980 0x7f99e80cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.042+0000 7f99e659c640 1 -- 172.21.15.12:0/2248640380 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e815a8c0 con 0x7f99e80cac40 2023-11-28T21:37:13.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99e659c640 1 --2- 172.21.15.12:0/2248640380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80cac40 0x7f99e806b130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f99e80c9cf0 tx=0x7f99d801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a693f8055eb2f880 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:13.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99e559a640 1 -- 172.21.15.12:0/2248640380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80a89b0 con 0x7f99e80cac40 2023-11-28T21:37:13.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99e559a640 1 -- 172.21.15.12:0/2248640380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d80a8b10 con 0x7f99e80cac40 2023-11-28T21:37:13.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99e559a640 1 -- 172.21.15.12:0/2248640380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d80a8ea0 con 0x7f99e80cac40 2023-11-28T21:37:13.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 -- 172.21.15.12:0/2248640380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80cac40 msgr2=0x7f99e806b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:13.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 --2- 172.21.15.12:0/2248640380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80cac40 0x7f99e806b130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f99e80c9cf0 tx=0x7f99d801ecf0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 -- 172.21.15.12:0/2248640380 shutdown_connections 2023-11-28T21:37:13.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 --2- 172.21.15.12:0/2248640380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99e80ca030 0x7f99e8159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 --2- 172.21.15.12:0/2248640380 >> v2:172.21.15.144:3300/0 conn(0x7f99e80c4980 0x7f99e80cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 --2- 172.21.15.12:0/2248640380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80cac40 0x7f99e806b130 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 -- 172.21.15.12:0/2248640380 >> 172.21.15.12:0/2248640380 conn(0x7f99e80720e0 msgr2=0x7f99e80724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:13.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 -- 172.21.15.12:0/2248640380 shutdown_connections 2023-11-28T21:37:13.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 -- 172.21.15.12:0/2248640380 wait complete. 2023-11-28T21:37:13.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 Processor -- start 2023-11-28T21:37:13.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.046+0000 7f99ece17640 1 -- start start 2023-11-28T21:37:13.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99ece17640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99e80c4980 0x7f99e81e3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:13.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99ece17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80ca030 0x7f99e814a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:13.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99ece17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99e80cac40 0x7f99e8150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99ece17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e81554c0 con 0x7f99e80ca030 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99ece17640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e8154d30 con 0x7f99e80c4980 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99ece17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e81576b0 con 0x7f99e80cac40 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80ca030 0x7f99e814a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80ca030 0x7f99e814a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49352/0 (socket says 172.21.15.12:49352) 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e5d9b640 1 -- 172.21.15.12:0/3409664373 learned_addr learned my addr 172.21.15.12:0/3409664373 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e659c640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99e80c4980 0x7f99e81e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e6d9d640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99e80cac40 0x7f99e8150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e5d9b640 1 -- 172.21.15.12:0/3409664373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99e80cac40 msgr2=0x7f99e8150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e5d9b640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99e80cac40 0x7f99e8150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e5d9b640 1 -- 172.21.15.12:0/3409664373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99e80c4980 msgr2=0x7f99e81e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e5d9b640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99e80c4980 0x7f99e81e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e5d9b640 1 -- 172.21.15.12:0/3409664373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d80094d0 con 0x7f99e80ca030 2023-11-28T21:37:13.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99e5d9b640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80ca030 0x7f99e814a680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00e8a0 tx=0x7f99dc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:13.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99d6ffd640 1 -- 172.21.15.12:0/3409664373 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc00ee90 con 0x7f99e80ca030 2023-11-28T21:37:13.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99d6ffd640 1 -- 172.21.15.12:0/3409664373 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc00c940 con 0x7f99e80ca030 2023-11-28T21:37:13.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99d6ffd640 1 -- 172.21.15.12:0/3409664373 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc00ccf0 con 0x7f99e80ca030 2023-11-28T21:37:13.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99ece17640 1 -- 172.21.15.12:0/3409664373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e8154770 con 0x7f99e80ca030 2023-11-28T21:37:13.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99ece17640 1 -- 172.21.15.12:0/3409664373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e81e5880 con 0x7f99e80ca030 2023-11-28T21:37:13.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99ece17640 1 -- 172.21.15.12:0/3409664373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e8067c70 con 0x7f99e80ca030 2023-11-28T21:37:13.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.050+0000 7f99d6ffd640 1 -- 172.21.15.12:0/3409664373 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f99dc013470 con 0x7f99e80ca030 2023-11-28T21:37:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.054+0000 7f99d6ffd640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99c8066470 0x7f99c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.054+0000 7f99d6ffd640 1 -- 172.21.15.12:0/3409664373 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f99dc084250 con 0x7f99e80ca030 2023-11-28T21:37:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.054+0000 7f99e659c640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99c8066470 0x7f99c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:13.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.054+0000 7f99e659c640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99c8066470 0x7f99c8068930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99d80b2000 tx=0x7f99d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:13.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.058+0000 7f99d6ffd640 1 -- 172.21.15.12:0/3409664373 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f99dc051390 con 0x7f99e80ca030 2023-11-28T21:37:13.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 -- 172.21.15.12:0/3409664373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99e806a0c0 con 0x7f99e80ca030 2023-11-28T21:37:13.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99d6ffd640 1 -- 172.21.15.12:0/3409664373 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7f99dc0549b0 con 0x7f99e80ca030 2023-11-28T21:37:13.251 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:13.252 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:13.252 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:13.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 -- 172.21.15.12:0/3409664373 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99c8066470 msgr2=0x7f99c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:13.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99c8066470 0x7f99c8068930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f99d80b2000 tx=0x7f99d8009040 comp rx=0 tx=0).stop 2023-11-28T21:37:13.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 -- 172.21.15.12:0/3409664373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80ca030 msgr2=0x7f99e814a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:13.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80ca030 0x7f99e814a680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00e8a0 tx=0x7f99dc00ed60 comp rx=0 tx=0).stop 2023-11-28T21:37:13.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 -- 172.21.15.12:0/3409664373 shutdown_connections 2023-11-28T21:37:13.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99c8066470 0x7f99c8068930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99e80cac40 0x7f99e8150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99e80ca030 0x7f99e814a680 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 --2- 172.21.15.12:0/3409664373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99e80c4980 0x7f99e81e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:13.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 -- 172.21.15.12:0/3409664373 >> 172.21.15.12:0/3409664373 conn(0x7f99e80720e0 msgr2=0x7f99e80b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:13.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 -- 172.21.15.12:0/3409664373 shutdown_connections 2023-11-28T21:37:13.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:13.246+0000 7f99ece17640 1 -- 172.21.15.12:0/3409664373 wait complete. 2023-11-28T21:37:13.272 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:13.293 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 251 2023-11-28T21:37:13.317 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 252 2023-11-28T21:37:13.340 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 253 2023-11-28T21:37:13.358 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 254 2023-11-28T21:37:13.374 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 255 2023-11-28T21:37:13.394 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 256 2023-11-28T21:37:13.409 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 257 2023-11-28T21:37:13.423 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 258 2023-11-28T21:37:13.437 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 259 2023-11-28T21:37:13.449 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 260 2023-11-28T21:37:13.462 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 261 2023-11-28T21:37:13.473 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 262 2023-11-28T21:37:13.486 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 263 2023-11-28T21:37:13.497 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 264 2023-11-28T21:37:13.508 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 265 2023-11-28T21:37:13.566 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 266 2023-11-28T21:37:13.578 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 267 2023-11-28T21:37:13.590 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 268 2023-11-28T21:37:13.603 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 269 2023-11-28T21:37:13.616 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 270 2023-11-28T21:37:13.629 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 271 2023-11-28T21:37:13.641 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 272 2023-11-28T21:37:13.652 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 273 2023-11-28T21:37:13.664 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 274 2023-11-28T21:37:13.675 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 275 2023-11-28T21:37:13.686 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 276 2023-11-28T21:37:13.697 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 277 2023-11-28T21:37:13.707 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 278 2023-11-28T21:37:13.718 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 279 2023-11-28T21:37:13.728 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 280 2023-11-28T21:37:13.748 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 281 2023-11-28T21:37:13.759 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 282 2023-11-28T21:37:13.770 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 283 2023-11-28T21:37:13.781 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 284 2023-11-28T21:37:13.791 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 285 2023-11-28T21:37:13.802 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 286 2023-11-28T21:37:13.812 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 287 2023-11-28T21:37:13.822 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 288 2023-11-28T21:37:14.374 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 289 2023-11-28T21:37:14.390 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 290 2023-11-28T21:37:14.411 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 291 2023-11-28T21:37:14.434 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 292 2023-11-28T21:37:14.702 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 293 2023-11-28T21:37:14.729 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 294 2023-11-28T21:37:14.763 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 295 2023-11-28T21:37:14.799 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 296 2023-11-28T21:37:14.833 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 297 2023-11-28T21:37:14.871 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 298 2023-11-28T21:37:15.273 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.466+0000 7f2d766b4640 1 Processor -- start 2023-11-28T21:37:15.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.466+0000 7f2d766b4640 1 -- start start 2023-11-28T21:37:15.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.466+0000 7f2d766b4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2d70153fb0 0x7f2d700ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:15.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.466+0000 7f2d766b4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d7015a980 0x7f2d700caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.466+0000 7f2d766b4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2d700b3ad0 0x7f2d700b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d766b4640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d700b2b60 con 0x7f2d70153fb0 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d766b4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7015e1b0 con 0x7f2d700b3ad0 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d766b4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7015d390 con 0x7f2d7015a980 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d756b2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2d70153fb0 0x7f2d700ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d756b2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2d70153fb0 0x7f2d700ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52842/0 (socket says 172.21.15.12:52842) 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d756b2640 1 -- 172.21.15.12:0/2217392001 learned_addr learned my addr 172.21.15.12:0/2217392001 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d74eb1640 1 --2- 172.21.15.12:0/2217392001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d7015a980 0x7f2d700caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d75eb3640 1 --2- 172.21.15.12:0/2217392001 >> v2:172.21.15.144:3300/0 conn(0x7f2d700b3ad0 0x7f2d700b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d756b2640 1 -- 172.21.15.12:0/2217392001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d7015a980 msgr2=0x7f2d700caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d756b2640 1 --2- 172.21.15.12:0/2217392001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d7015a980 0x7f2d700caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d756b2640 1 -- 172.21.15.12:0/2217392001 >> v2:172.21.15.144:3300/0 conn(0x7f2d700b3ad0 msgr2=0x7f2d700b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:15.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d756b2640 1 --2- 172.21.15.12:0/2217392001 >> v2:172.21.15.144:3300/0 conn(0x7f2d700b3ad0 0x7f2d700b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d756b2640 1 -- 172.21.15.12:0/2217392001 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7015d630 con 0x7f2d70153fb0 2023-11-28T21:37:15.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.470+0000 7f2d75eb3640 1 --2- 172.21.15.12:0/2217392001 >> v2:172.21.15.144:3300/0 conn(0x7f2d700b3ad0 0x7f2d700b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:37:15.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d756b2640 1 --2- 172.21.15.12:0/2217392001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 0x7f2d700ca4f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2d700c9f70 tx=0x7f2d6c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577d1c51cfa9860b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:15.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d67fff640 1 -- 172.21.15.12:0/2217392001 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0aa5f0 con 0x7f2d70153fb0 2023-11-28T21:37:15.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d67fff640 1 -- 172.21.15.12:0/2217392001 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c0aa750 con 0x7f2d70153fb0 2023-11-28T21:37:15.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d67fff640 1 -- 172.21.15.12:0/2217392001 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0aaae0 con 0x7f2d70153fb0 2023-11-28T21:37:15.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d766b4640 1 -- 172.21.15.12:0/2217392001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 msgr2=0x7f2d700ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:15.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d766b4640 1 --2- 172.21.15.12:0/2217392001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 0x7f2d700ca4f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2d700c9f70 tx=0x7f2d6c01c4d0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d766b4640 1 -- 172.21.15.12:0/2217392001 shutdown_connections 2023-11-28T21:37:15.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d766b4640 1 --2- 172.21.15.12:0/2217392001 >> v2:172.21.15.144:3300/0 conn(0x7f2d700b3ad0 0x7f2d700b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d766b4640 1 --2- 172.21.15.12:0/2217392001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d7015a980 0x7f2d700caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d766b4640 1 --2- 172.21.15.12:0/2217392001 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 0x7f2d700ca4f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d766b4640 1 -- 172.21.15.12:0/2217392001 >> 172.21.15.12:0/2217392001 conn(0x7f2d70071ce0 msgr2=0x7f2d700720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:15.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d766b4640 1 -- 172.21.15.12:0/2217392001 shutdown_connections 2023-11-28T21:37:15.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.474+0000 7f2d766b4640 1 -- 172.21.15.12:0/2217392001 wait complete. 2023-11-28T21:37:15.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d766b4640 1 Processor -- start 2023-11-28T21:37:15.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d766b4640 1 -- start start 2023-11-28T21:37:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d766b4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d700b3ad0 0x7f2d701f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d766b4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 0x7f2d70145340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d766b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d7015a980 0x7f2d70144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d766b4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701566b0 con 0x7f2d70153fb0 2023-11-28T21:37:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d766b4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d701f3900 con 0x7f2d7015a980 2023-11-28T21:37:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d766b4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d70152940 con 0x7f2d700b3ad0 2023-11-28T21:37:15.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d74eb1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 0x7f2d70145340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d74eb1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 0x7f2d70145340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52858/0 (socket says 172.21.15.12:52858) 2023-11-28T21:37:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.478+0000 7f2d74eb1640 1 -- 172.21.15.12:0/936321235 learned_addr learned my addr 172.21.15.12:0/936321235 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d756b2640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d700b3ad0 0x7f2d701f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d74eb1640 1 -- 172.21.15.12:0/936321235 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d700b3ad0 msgr2=0x7f2d701f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d74eb1640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d700b3ad0 0x7f2d701f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d74eb1640 1 -- 172.21.15.12:0/936321235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d7015a980 msgr2=0x7f2d70144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d74eb1640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d7015a980 0x7f2d70144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d74eb1640 1 -- 172.21.15.12:0/936321235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0094d0 con 0x7f2d70153fb0 2023-11-28T21:37:15.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d74eb1640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 0x7f2d70145340 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000d800 tx=0x7f2d6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:15.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d65ffb640 1 -- 172.21.15.12:0/936321235 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000b6c0 con 0x7f2d70153fb0 2023-11-28T21:37:15.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d65ffb640 1 -- 172.21.15.12:0/936321235 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6000b820 con 0x7f2d70153fb0 2023-11-28T21:37:15.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d766b4640 1 -- 172.21.15.12:0/936321235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d70145880 con 0x7f2d70153fb0 2023-11-28T21:37:15.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d65ffb640 1 -- 172.21.15.12:0/936321235 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000bbb0 con 0x7f2d70153fb0 2023-11-28T21:37:15.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d766b4640 1 -- 172.21.15.12:0/936321235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d70152c00 con 0x7f2d70153fb0 2023-11-28T21:37:15.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.482+0000 7f2d766b4640 1 -- 172.21.15.12:0/936321235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d70074f60 con 0x7f2d70153fb0 2023-11-28T21:37:15.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.486+0000 7f2d65ffb640 1 -- 172.21.15.12:0/936321235 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2d6000bd10 con 0x7f2d70153fb0 2023-11-28T21:37:15.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.486+0000 7f2d65ffb640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 0x7f2d54068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:15.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.486+0000 7f2d65ffb640 1 -- 172.21.15.12:0/936321235 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f2d60084720 con 0x7f2d70153fb0 2023-11-28T21:37:15.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.486+0000 7f2d756b2640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 0x7f2d54068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:15.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.490+0000 7f2d756b2640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 0x7f2d54068930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0ae000 tx=0x7f2d6c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:15.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.490+0000 7f2d65ffb640 1 -- 172.21.15.12:0/936321235 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d600517b0 con 0x7f2d70153fb0 2023-11-28T21:37:15.596 INFO:tasks.workunit.client.0.smithi012.stdout: create dir 299 2023-11-28T21:37:15.617 INFO:tasks.workunit.client.0.smithi012.stdout:Create a snapshot in every first level dir ... 2023-11-28T21:37:15.700 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/.snap/snap-subdir-test 2023-11-28T21:37:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.702+0000 7f2d766b4640 1 -- 172.21.15.12:0/936321235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d700cb810 con 0x7f2d70153fb0 2023-11-28T21:37:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.702+0000 7f2d65ffb640 1 -- 172.21.15.12:0/936321235 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7f2d60054dd0 con 0x7f2d70153fb0 2023-11-28T21:37:15.709 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:15.709 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 -- 172.21.15.12:0/936321235 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 msgr2=0x7f2d54068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 0x7f2d54068930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0ae000 tx=0x7f2d6c0a7000 comp rx=0 tx=0).stop 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 -- 172.21.15.12:0/936321235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 msgr2=0x7f2d70145340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 0x7f2d70145340 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000d800 tx=0x7f2d6000dcc0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 -- 172.21.15.12:0/936321235 shutdown_connections 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 0x7f2d54068930 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d7015a980 0x7f2d70144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d70153fb0 0x7f2d70145340 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 --2- 172.21.15.12:0/936321235 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d700b3ad0 0x7f2d701f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 -- 172.21.15.12:0/936321235 >> 172.21.15.12:0/936321235 conn(0x7f2d70071ce0 msgr2=0x7f2d700cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 -- 172.21.15.12:0/936321235 shutdown_connections 2023-11-28T21:37:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:15.706+0000 7f2d766b4640 1 -- 172.21.15.12:0/936321235 wait complete. 2023-11-28T21:37:15.730 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-11-28T21:37:15.731 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 14, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 14, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.12:6838/949980653", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 949980653, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 949980653, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4247, "incarnation": 12, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:stopping", "state_seq": 13 }, "gid_4250": { "addr": "172.21.15.12:6839/3027655738", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3027655738, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3027655738, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4250, "incarnation": 8, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 11 }, "gid_4261": { "addr": "172.21.15.144:6833/3053317078", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 3053317078, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 3053317078, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4261, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:37:03.325643+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4261, "mds_1": 4250, "mds_2": 4247 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:37:15.738 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/30 file after the snapshot 2023-11-28T21:37:15.740 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/31 file after the snapshot 2023-11-28T21:37:15.740 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/32 file after the snapshot 2023-11-28T21:37:15.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/33 file after the snapshot 2023-11-28T21:37:15.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/34 file after the snapshot 2023-11-28T21:37:15.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/35 file after the snapshot 2023-11-28T21:37:15.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/36 file after the snapshot 2023-11-28T21:37:15.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/37 file after the snapshot 2023-11-28T21:37:15.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/38 file after the snapshot 2023-11-28T21:37:15.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 100/39 file after the snapshot 2023-11-28T21:37:15.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/.snap/snap-subdir-test 2023-11-28T21:37:15.794 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/30 file after the snapshot 2023-11-28T21:37:15.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/31 file after the snapshot 2023-11-28T21:37:15.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/32 file after the snapshot 2023-11-28T21:37:15.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/33 file after the snapshot 2023-11-28T21:37:15.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/34 file after the snapshot 2023-11-28T21:37:15.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/35 file after the snapshot 2023-11-28T21:37:15.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/36 file after the snapshot 2023-11-28T21:37:15.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/37 file after the snapshot 2023-11-28T21:37:15.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/38 file after the snapshot 2023-11-28T21:37:15.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 101/39 file after the snapshot 2023-11-28T21:37:15.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/.snap/snap-subdir-test 2023-11-28T21:37:15.840 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/30 file after the snapshot 2023-11-28T21:37:15.843 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/31 file after the snapshot 2023-11-28T21:37:15.843 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/32 file after the snapshot 2023-11-28T21:37:15.843 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/33 file after the snapshot 2023-11-28T21:37:15.843 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/34 file after the snapshot 2023-11-28T21:37:15.843 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/35 file after the snapshot 2023-11-28T21:37:15.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/36 file after the snapshot 2023-11-28T21:37:15.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/37 file after the snapshot 2023-11-28T21:37:15.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/38 file after the snapshot 2023-11-28T21:37:15.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 102/39 file after the snapshot 2023-11-28T21:37:15.844 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/.snap/snap-subdir-test 2023-11-28T21:37:15.886 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/30 file after the snapshot 2023-11-28T21:37:15.889 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/31 file after the snapshot 2023-11-28T21:37:15.889 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/32 file after the snapshot 2023-11-28T21:37:15.889 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/33 file after the snapshot 2023-11-28T21:37:15.889 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/34 file after the snapshot 2023-11-28T21:37:15.889 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/35 file after the snapshot 2023-11-28T21:37:15.889 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/36 file after the snapshot 2023-11-28T21:37:15.889 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/37 file after the snapshot 2023-11-28T21:37:15.889 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/38 file after the snapshot 2023-11-28T21:37:15.889 INFO:tasks.workunit.client.0.smithi012.stdout: create 103/39 file after the snapshot 2023-11-28T21:37:15.890 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/.snap/snap-subdir-test 2023-11-28T21:37:15.956 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/30 file after the snapshot 2023-11-28T21:37:15.962 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/31 file after the snapshot 2023-11-28T21:37:15.962 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/32 file after the snapshot 2023-11-28T21:37:15.962 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/33 file after the snapshot 2023-11-28T21:37:15.962 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/34 file after the snapshot 2023-11-28T21:37:15.962 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/35 file after the snapshot 2023-11-28T21:37:15.962 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/36 file after the snapshot 2023-11-28T21:37:15.962 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/37 file after the snapshot 2023-11-28T21:37:15.962 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/38 file after the snapshot 2023-11-28T21:37:15.963 INFO:tasks.workunit.client.0.smithi012.stdout: create 104/39 file after the snapshot 2023-11-28T21:37:15.963 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/.snap/snap-subdir-test 2023-11-28T21:37:16.023 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/30 file after the snapshot 2023-11-28T21:37:16.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/31 file after the snapshot 2023-11-28T21:37:16.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/32 file after the snapshot 2023-11-28T21:37:16.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/33 file after the snapshot 2023-11-28T21:37:16.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/34 file after the snapshot 2023-11-28T21:37:16.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/35 file after the snapshot 2023-11-28T21:37:16.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/36 file after the snapshot 2023-11-28T21:37:16.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/37 file after the snapshot 2023-11-28T21:37:16.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/38 file after the snapshot 2023-11-28T21:37:16.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 105/39 file after the snapshot 2023-11-28T21:37:16.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/.snap/snap-subdir-test 2023-11-28T21:37:16.090 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/30 file after the snapshot 2023-11-28T21:37:16.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/31 file after the snapshot 2023-11-28T21:37:16.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/32 file after the snapshot 2023-11-28T21:37:16.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/33 file after the snapshot 2023-11-28T21:37:16.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/34 file after the snapshot 2023-11-28T21:37:16.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/35 file after the snapshot 2023-11-28T21:37:16.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/36 file after the snapshot 2023-11-28T21:37:16.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/37 file after the snapshot 2023-11-28T21:37:16.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/38 file after the snapshot 2023-11-28T21:37:16.094 INFO:tasks.workunit.client.0.smithi012.stdout: create 106/39 file after the snapshot 2023-11-28T21:37:16.094 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/.snap/snap-subdir-test 2023-11-28T21:37:16.142 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/30 file after the snapshot 2023-11-28T21:37:16.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/31 file after the snapshot 2023-11-28T21:37:16.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/32 file after the snapshot 2023-11-28T21:37:16.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/33 file after the snapshot 2023-11-28T21:37:16.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/34 file after the snapshot 2023-11-28T21:37:16.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/35 file after the snapshot 2023-11-28T21:37:16.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/36 file after the snapshot 2023-11-28T21:37:16.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/37 file after the snapshot 2023-11-28T21:37:16.145 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/38 file after the snapshot 2023-11-28T21:37:16.145 INFO:tasks.workunit.client.0.smithi012.stdout: create 107/39 file after the snapshot 2023-11-28T21:37:16.145 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/.snap/snap-subdir-test 2023-11-28T21:37:16.193 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/30 file after the snapshot 2023-11-28T21:37:16.196 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/31 file after the snapshot 2023-11-28T21:37:16.196 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/32 file after the snapshot 2023-11-28T21:37:16.196 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/33 file after the snapshot 2023-11-28T21:37:16.196 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/34 file after the snapshot 2023-11-28T21:37:16.196 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/35 file after the snapshot 2023-11-28T21:37:16.196 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/36 file after the snapshot 2023-11-28T21:37:16.197 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/37 file after the snapshot 2023-11-28T21:37:16.197 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/38 file after the snapshot 2023-11-28T21:37:16.197 INFO:tasks.workunit.client.0.smithi012.stdout: create 108/39 file after the snapshot 2023-11-28T21:37:16.197 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/.snap/snap-subdir-test 2023-11-28T21:37:16.532 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/30 file after the snapshot 2023-11-28T21:37:16.542 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/31 file after the snapshot 2023-11-28T21:37:16.542 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/32 file after the snapshot 2023-11-28T21:37:16.543 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/33 file after the snapshot 2023-11-28T21:37:16.543 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/34 file after the snapshot 2023-11-28T21:37:16.543 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/35 file after the snapshot 2023-11-28T21:37:16.543 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/36 file after the snapshot 2023-11-28T21:37:16.543 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/37 file after the snapshot 2023-11-28T21:37:16.543 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/38 file after the snapshot 2023-11-28T21:37:16.543 INFO:tasks.workunit.client.0.smithi012.stdout: create 109/39 file after the snapshot 2023-11-28T21:37:16.543 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/.snap/snap-subdir-test 2023-11-28T21:37:17.318 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/30 file after the snapshot 2023-11-28T21:37:17.321 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/31 file after the snapshot 2023-11-28T21:37:17.321 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/32 file after the snapshot 2023-11-28T21:37:17.322 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/33 file after the snapshot 2023-11-28T21:37:17.322 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/34 file after the snapshot 2023-11-28T21:37:17.322 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/35 file after the snapshot 2023-11-28T21:37:17.322 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/36 file after the snapshot 2023-11-28T21:37:17.322 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/37 file after the snapshot 2023-11-28T21:37:17.322 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/38 file after the snapshot 2023-11-28T21:37:17.323 INFO:tasks.workunit.client.0.smithi012.stdout: create 110/39 file after the snapshot 2023-11-28T21:37:17.323 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/.snap/snap-subdir-test 2023-11-28T21:37:17.388 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/30 file after the snapshot 2023-11-28T21:37:17.396 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/31 file after the snapshot 2023-11-28T21:37:17.396 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/32 file after the snapshot 2023-11-28T21:37:17.396 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/33 file after the snapshot 2023-11-28T21:37:17.397 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/34 file after the snapshot 2023-11-28T21:37:17.397 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/35 file after the snapshot 2023-11-28T21:37:17.397 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/36 file after the snapshot 2023-11-28T21:37:17.397 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/37 file after the snapshot 2023-11-28T21:37:17.397 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/38 file after the snapshot 2023-11-28T21:37:17.397 INFO:tasks.workunit.client.0.smithi012.stdout: create 111/39 file after the snapshot 2023-11-28T21:37:17.397 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/.snap/snap-subdir-test 2023-11-28T21:37:17.474 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/30 file after the snapshot 2023-11-28T21:37:17.477 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/31 file after the snapshot 2023-11-28T21:37:17.477 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/32 file after the snapshot 2023-11-28T21:37:17.477 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/33 file after the snapshot 2023-11-28T21:37:17.477 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/34 file after the snapshot 2023-11-28T21:37:17.478 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/35 file after the snapshot 2023-11-28T21:37:17.478 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/36 file after the snapshot 2023-11-28T21:37:17.478 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/37 file after the snapshot 2023-11-28T21:37:17.478 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/38 file after the snapshot 2023-11-28T21:37:17.478 INFO:tasks.workunit.client.0.smithi012.stdout: create 112/39 file after the snapshot 2023-11-28T21:37:17.478 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/.snap/snap-subdir-test 2023-11-28T21:37:17.541 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/30 file after the snapshot 2023-11-28T21:37:17.544 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/31 file after the snapshot 2023-11-28T21:37:17.544 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/32 file after the snapshot 2023-11-28T21:37:17.544 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/33 file after the snapshot 2023-11-28T21:37:17.544 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/34 file after the snapshot 2023-11-28T21:37:17.545 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/35 file after the snapshot 2023-11-28T21:37:17.545 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/36 file after the snapshot 2023-11-28T21:37:17.545 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/37 file after the snapshot 2023-11-28T21:37:17.545 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/38 file after the snapshot 2023-11-28T21:37:17.545 INFO:tasks.workunit.client.0.smithi012.stdout: create 113/39 file after the snapshot 2023-11-28T21:37:17.545 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/.snap/snap-subdir-test 2023-11-28T21:37:17.606 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/30 file after the snapshot 2023-11-28T21:37:17.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/31 file after the snapshot 2023-11-28T21:37:17.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/32 file after the snapshot 2023-11-28T21:37:17.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/33 file after the snapshot 2023-11-28T21:37:17.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/34 file after the snapshot 2023-11-28T21:37:17.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/35 file after the snapshot 2023-11-28T21:37:17.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/36 file after the snapshot 2023-11-28T21:37:17.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/37 file after the snapshot 2023-11-28T21:37:17.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/38 file after the snapshot 2023-11-28T21:37:17.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 114/39 file after the snapshot 2023-11-28T21:37:17.611 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/.snap/snap-subdir-test 2023-11-28T21:37:17.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/30 file after the snapshot 2023-11-28T21:37:17.675 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/31 file after the snapshot 2023-11-28T21:37:17.675 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/32 file after the snapshot 2023-11-28T21:37:17.675 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/33 file after the snapshot 2023-11-28T21:37:17.676 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/34 file after the snapshot 2023-11-28T21:37:17.676 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/35 file after the snapshot 2023-11-28T21:37:17.676 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/36 file after the snapshot 2023-11-28T21:37:17.676 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/37 file after the snapshot 2023-11-28T21:37:17.676 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/38 file after the snapshot 2023-11-28T21:37:17.677 INFO:tasks.workunit.client.0.smithi012.stdout: create 115/39 file after the snapshot 2023-11-28T21:37:17.677 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/.snap/snap-subdir-test 2023-11-28T21:37:17.728 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/30 file after the snapshot 2023-11-28T21:37:17.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/31 file after the snapshot 2023-11-28T21:37:17.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/32 file after the snapshot 2023-11-28T21:37:17.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/33 file after the snapshot 2023-11-28T21:37:17.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/34 file after the snapshot 2023-11-28T21:37:17.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/35 file after the snapshot 2023-11-28T21:37:17.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/36 file after the snapshot 2023-11-28T21:37:17.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/37 file after the snapshot 2023-11-28T21:37:17.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/38 file after the snapshot 2023-11-28T21:37:17.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 116/39 file after the snapshot 2023-11-28T21:37:17.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/.snap/snap-subdir-test 2023-11-28T21:37:17.732 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:17.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.882+0000 7fd49039d640 1 Processor -- start 2023-11-28T21:37:17.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.882+0000 7fd49039d640 1 -- start start 2023-11-28T21:37:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.882+0000 7fd49039d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd488151af0 0x7fd4880c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.882+0000 7fd49039d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd488151ed0 0x7fd4880c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.882+0000 7fd49039d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd488157b70 con 0x7fd48806e300 2023-11-28T21:37:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.882+0000 7fd49039d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4880707f0 con 0x7fd488151af0 2023-11-28T21:37:17.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.882+0000 7fd49039d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880c7b90 con 0x7fd488151ed0 2023-11-28T21:37:17.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.882+0000 7fd48e139640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd488151af0 0x7fd4880c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:17.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.882+0000 7fd48d938640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd488151ed0 0x7fd4880c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:17.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.886+0000 7fd48d938640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd488151ed0 0x7fd4880c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:60864/0 (socket says 172.21.15.12:60864) 2023-11-28T21:37:17.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.886+0000 7fd48d938640 1 -- 172.21.15.12:0/4289993312 learned_addr learned my addr 172.21.15.12:0/4289993312 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:17.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.886+0000 7fd48d938640 1 -- 172.21.15.12:0/4289993312 >> v2:172.21.15.144:3300/0 conn(0x7fd488151af0 msgr2=0x7fd4880c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:17.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.886+0000 7fd48d938640 1 --2- 172.21.15.12:0/4289993312 >> v2:172.21.15.144:3300/0 conn(0x7fd488151af0 0x7fd4880c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:17.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.886+0000 7fd48d938640 1 -- 172.21.15.12:0/4289993312 >> v1:172.21.15.144:6789/0 conn(0x7fd48806e300 legacy=0x7fd488072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:17.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.886+0000 7fd48d938640 1 -- 172.21.15.12:0/4289993312 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd488157e50 con 0x7fd488151ed0 2023-11-28T21:37:17.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.886+0000 7fd48d938640 1 --2- 172.21.15.12:0/4289993312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd488151ed0 0x7fd4880c49d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd478009870 tx=0x7fd47801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7891a98763dd1b96 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:17.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.886+0000 7fd48d137640 1 -- 172.21.15.12:0/4289993312 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780ab600 con 0x7fd488151ed0 2023-11-28T21:37:17.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.886+0000 7fd48d137640 1 -- 172.21.15.12:0/4289993312 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4780a7050 con 0x7fd488151ed0 2023-11-28T21:37:17.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.886+0000 7fd48d137640 1 -- 172.21.15.12:0/4289993312 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780ab980 con 0x7fd488151ed0 2023-11-28T21:37:17.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 -- 172.21.15.12:0/4289993312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd488151ed0 msgr2=0x7fd4880c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:17.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 --2- 172.21.15.12:0/4289993312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd488151ed0 0x7fd4880c49d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd478009870 tx=0x7fd47801c540 comp rx=0 tx=0).stop 2023-11-28T21:37:17.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 -- 172.21.15.12:0/4289993312 shutdown_connections 2023-11-28T21:37:17.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 --2- 172.21.15.12:0/4289993312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd488151ed0 0x7fd4880c49d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:17.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 --2- 172.21.15.12:0/4289993312 >> v2:172.21.15.144:3300/0 conn(0x7fd488151af0 0x7fd4880c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:17.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 -- 172.21.15.12:0/4289993312 >> 172.21.15.12:0/4289993312 conn(0x7fd488066360 msgr2=0x7fd488066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:17.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 -- 172.21.15.12:0/4289993312 shutdown_connections 2023-11-28T21:37:17.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 -- 172.21.15.12:0/4289993312 wait complete. 2023-11-28T21:37:17.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 Processor -- start 2023-11-28T21:37:17.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 -- start start 2023-11-28T21:37:17.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48806e300 0x7fd48814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:17.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd488151af0 0x7fd4881433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:17.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4881e9e10 0x7fd488143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:17.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488144930 con 0x7fd488151af0 2023-11-28T21:37:17.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488145120 con 0x7fd4881e9e10 2023-11-28T21:37:17.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd49039d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd488150d20 con 0x7fd48806e300 2023-11-28T21:37:17.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd48d938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd488151af0 0x7fd4881433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:17.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd48d938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd488151af0 0x7fd4881433f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52860/0 (socket says 172.21.15.12:52860) 2023-11-28T21:37:17.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd48d938640 1 -- 172.21.15.12:0/1467708723 learned_addr learned my addr 172.21.15.12:0/1467708723 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:17.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd48e139640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48806e300 0x7fd48814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:17.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.890+0000 7fd48e93a640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4881e9e10 0x7fd488143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd48d938640 1 -- 172.21.15.12:0/1467708723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48806e300 msgr2=0x7fd48814ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd48d938640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48806e300 0x7fd48814ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd48d938640 1 -- 172.21.15.12:0/1467708723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4881e9e10 msgr2=0x7fd488143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd48d938640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4881e9e10 0x7fd488143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:17.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd48d938640 1 -- 172.21.15.12:0/1467708723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4780094d0 con 0x7fd488151af0 2023-11-28T21:37:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd48e139640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48806e300 0x7fd48814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd48e93a640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4881e9e10 0x7fd488143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd48d938640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd488151af0 0x7fd4881433f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd47801a170 tx=0x7fd4780192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd47effd640 1 -- 172.21.15.12:0/1467708723 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780b4420 con 0x7fd488151af0 2023-11-28T21:37:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd47effd640 1 -- 172.21.15.12:0/1467708723 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4780b4580 con 0x7fd488151af0 2023-11-28T21:37:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd49039d640 1 -- 172.21.15.12:0/1467708723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488145be0 con 0x7fd488151af0 2023-11-28T21:37:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd47effd640 1 -- 172.21.15.12:0/1467708723 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780b48f0 con 0x7fd488151af0 2023-11-28T21:37:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd49039d640 1 -- 172.21.15.12:0/1467708723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48814fb20 con 0x7fd488151af0 2023-11-28T21:37:17.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd49039d640 1 -- 172.21.15.12:0/1467708723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd488145d70 con 0x7fd488151af0 2023-11-28T21:37:17.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd47effd640 1 -- 172.21.15.12:0/1467708723 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd4780b2070 con 0x7fd488151af0 2023-11-28T21:37:17.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd47effd640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd46c066470 0x7fd46c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:17.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd47effd640 1 -- 172.21.15.12:0/1467708723 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fd47811d790 con 0x7fd488151af0 2023-11-28T21:37:17.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.894+0000 7fd48e139640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd46c066470 0x7fd46c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:17.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.898+0000 7fd48e139640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd46c066470 0x7fd46c068930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd484009bb0 tx=0x7fd484008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:17.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:17.898+0000 7fd47effd640 1 -- 172.21.15.12:0/1467708723 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd4780ea810 con 0x7fd488151af0 2023-11-28T21:37:18.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 -- 172.21.15.12:0/1467708723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4880c79f0 con 0x7fd488151af0 2023-11-28T21:37:18.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd47effd640 1 -- 172.21.15.12:0/1467708723 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7fd4880c79f0 con 0x7fd488151af0 2023-11-28T21:37:18.099 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:18.099 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:18.099 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:18.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 -- 172.21.15.12:0/1467708723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd46c066470 msgr2=0x7fd46c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:18.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd46c066470 0x7fd46c068930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd484009bb0 tx=0x7fd484008040 comp rx=0 tx=0).stop 2023-11-28T21:37:18.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 -- 172.21.15.12:0/1467708723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd488151af0 msgr2=0x7fd4881433f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:18.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd488151af0 0x7fd4881433f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd47801a170 tx=0x7fd4780192e0 comp rx=0 tx=0).stop 2023-11-28T21:37:18.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 -- 172.21.15.12:0/1467708723 shutdown_connections 2023-11-28T21:37:18.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd46c066470 0x7fd46c068930 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:18.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4881e9e10 0x7fd488143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:18.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd488151af0 0x7fd4881433f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:18.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 --2- 172.21.15.12:0/1467708723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd48806e300 0x7fd48814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:18.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 -- 172.21.15.12:0/1467708723 >> 172.21.15.12:0/1467708723 conn(0x7fd488066360 msgr2=0x7fd48806ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:18.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 -- 172.21.15.12:0/1467708723 shutdown_connections 2023-11-28T21:37:18.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:18.094+0000 7fd49039d640 1 -- 172.21.15.12:0/1467708723 wait complete. 2023-11-28T21:37:18.121 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:18.464 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/30 file after the snapshot 2023-11-28T21:37:18.467 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/31 file after the snapshot 2023-11-28T21:37:18.467 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/32 file after the snapshot 2023-11-28T21:37:18.467 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/33 file after the snapshot 2023-11-28T21:37:18.468 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/34 file after the snapshot 2023-11-28T21:37:18.468 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/35 file after the snapshot 2023-11-28T21:37:18.468 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/36 file after the snapshot 2023-11-28T21:37:18.468 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/37 file after the snapshot 2023-11-28T21:37:18.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/38 file after the snapshot 2023-11-28T21:37:18.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 117/39 file after the snapshot 2023-11-28T21:37:18.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/.snap/snap-subdir-test 2023-11-28T21:37:19.203 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/30 file after the snapshot 2023-11-28T21:37:19.207 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/31 file after the snapshot 2023-11-28T21:37:19.207 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/32 file after the snapshot 2023-11-28T21:37:19.207 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/33 file after the snapshot 2023-11-28T21:37:19.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/34 file after the snapshot 2023-11-28T21:37:19.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/35 file after the snapshot 2023-11-28T21:37:19.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/36 file after the snapshot 2023-11-28T21:37:19.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/37 file after the snapshot 2023-11-28T21:37:19.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/38 file after the snapshot 2023-11-28T21:37:19.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 118/39 file after the snapshot 2023-11-28T21:37:19.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/.snap/snap-subdir-test 2023-11-28T21:37:19.303 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/30 file after the snapshot 2023-11-28T21:37:19.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/31 file after the snapshot 2023-11-28T21:37:19.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/32 file after the snapshot 2023-11-28T21:37:19.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/33 file after the snapshot 2023-11-28T21:37:19.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/34 file after the snapshot 2023-11-28T21:37:19.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/35 file after the snapshot 2023-11-28T21:37:19.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/36 file after the snapshot 2023-11-28T21:37:19.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/37 file after the snapshot 2023-11-28T21:37:19.307 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/38 file after the snapshot 2023-11-28T21:37:19.308 INFO:tasks.workunit.client.0.smithi012.stdout: create 119/39 file after the snapshot 2023-11-28T21:37:19.308 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/.snap/snap-subdir-test 2023-11-28T21:37:19.382 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/30 file after the snapshot 2023-11-28T21:37:19.385 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/31 file after the snapshot 2023-11-28T21:37:19.385 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/32 file after the snapshot 2023-11-28T21:37:19.385 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/33 file after the snapshot 2023-11-28T21:37:19.385 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/34 file after the snapshot 2023-11-28T21:37:19.385 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/35 file after the snapshot 2023-11-28T21:37:19.385 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/36 file after the snapshot 2023-11-28T21:37:19.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/37 file after the snapshot 2023-11-28T21:37:19.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/38 file after the snapshot 2023-11-28T21:37:19.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 120/39 file after the snapshot 2023-11-28T21:37:19.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/.snap/snap-subdir-test 2023-11-28T21:37:19.443 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/30 file after the snapshot 2023-11-28T21:37:19.447 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/31 file after the snapshot 2023-11-28T21:37:19.447 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/32 file after the snapshot 2023-11-28T21:37:19.447 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/33 file after the snapshot 2023-11-28T21:37:19.447 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/34 file after the snapshot 2023-11-28T21:37:19.447 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/35 file after the snapshot 2023-11-28T21:37:19.447 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/36 file after the snapshot 2023-11-28T21:37:19.447 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/37 file after the snapshot 2023-11-28T21:37:19.447 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/38 file after the snapshot 2023-11-28T21:37:19.447 INFO:tasks.workunit.client.0.smithi012.stdout: create 121/39 file after the snapshot 2023-11-28T21:37:19.447 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/.snap/snap-subdir-test 2023-11-28T21:37:19.514 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/30 file after the snapshot 2023-11-28T21:37:19.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/31 file after the snapshot 2023-11-28T21:37:19.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/32 file after the snapshot 2023-11-28T21:37:19.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/33 file after the snapshot 2023-11-28T21:37:19.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/34 file after the snapshot 2023-11-28T21:37:19.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/35 file after the snapshot 2023-11-28T21:37:19.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/36 file after the snapshot 2023-11-28T21:37:19.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/37 file after the snapshot 2023-11-28T21:37:19.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/38 file after the snapshot 2023-11-28T21:37:19.517 INFO:tasks.workunit.client.0.smithi012.stdout: create 122/39 file after the snapshot 2023-11-28T21:37:19.518 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/.snap/snap-subdir-test 2023-11-28T21:37:19.601 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/30 file after the snapshot 2023-11-28T21:37:19.603 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/31 file after the snapshot 2023-11-28T21:37:19.604 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/32 file after the snapshot 2023-11-28T21:37:19.604 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/33 file after the snapshot 2023-11-28T21:37:19.604 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/34 file after the snapshot 2023-11-28T21:37:19.604 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/35 file after the snapshot 2023-11-28T21:37:19.605 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/36 file after the snapshot 2023-11-28T21:37:19.605 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/37 file after the snapshot 2023-11-28T21:37:19.605 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/38 file after the snapshot 2023-11-28T21:37:19.605 INFO:tasks.workunit.client.0.smithi012.stdout: create 123/39 file after the snapshot 2023-11-28T21:37:19.605 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/.snap/snap-subdir-test 2023-11-28T21:37:19.661 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/30 file after the snapshot 2023-11-28T21:37:19.665 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/31 file after the snapshot 2023-11-28T21:37:19.665 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/32 file after the snapshot 2023-11-28T21:37:19.665 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/33 file after the snapshot 2023-11-28T21:37:19.665 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/34 file after the snapshot 2023-11-28T21:37:19.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/35 file after the snapshot 2023-11-28T21:37:19.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/36 file after the snapshot 2023-11-28T21:37:19.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/37 file after the snapshot 2023-11-28T21:37:19.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/38 file after the snapshot 2023-11-28T21:37:19.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 124/39 file after the snapshot 2023-11-28T21:37:19.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/.snap/snap-subdir-test 2023-11-28T21:37:19.731 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/30 file after the snapshot 2023-11-28T21:37:19.735 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/31 file after the snapshot 2023-11-28T21:37:19.735 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/32 file after the snapshot 2023-11-28T21:37:19.735 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/33 file after the snapshot 2023-11-28T21:37:19.736 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/34 file after the snapshot 2023-11-28T21:37:19.736 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/35 file after the snapshot 2023-11-28T21:37:19.736 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/36 file after the snapshot 2023-11-28T21:37:19.736 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/37 file after the snapshot 2023-11-28T21:37:19.736 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/38 file after the snapshot 2023-11-28T21:37:19.736 INFO:tasks.workunit.client.0.smithi012.stdout: create 125/39 file after the snapshot 2023-11-28T21:37:19.737 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/.snap/snap-subdir-test 2023-11-28T21:37:19.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/30 file after the snapshot 2023-11-28T21:37:19.802 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/31 file after the snapshot 2023-11-28T21:37:19.802 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/32 file after the snapshot 2023-11-28T21:37:19.802 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/33 file after the snapshot 2023-11-28T21:37:19.803 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/34 file after the snapshot 2023-11-28T21:37:19.803 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/35 file after the snapshot 2023-11-28T21:37:19.803 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/36 file after the snapshot 2023-11-28T21:37:19.804 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/37 file after the snapshot 2023-11-28T21:37:19.804 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/38 file after the snapshot 2023-11-28T21:37:19.804 INFO:tasks.workunit.client.0.smithi012.stdout: create 126/39 file after the snapshot 2023-11-28T21:37:19.804 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/.snap/snap-subdir-test 2023-11-28T21:37:19.882 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/30 file after the snapshot 2023-11-28T21:37:19.885 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/31 file after the snapshot 2023-11-28T21:37:19.886 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/32 file after the snapshot 2023-11-28T21:37:19.886 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/33 file after the snapshot 2023-11-28T21:37:19.886 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/34 file after the snapshot 2023-11-28T21:37:19.886 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/35 file after the snapshot 2023-11-28T21:37:19.886 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/36 file after the snapshot 2023-11-28T21:37:19.887 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/37 file after the snapshot 2023-11-28T21:37:19.887 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/38 file after the snapshot 2023-11-28T21:37:19.887 INFO:tasks.workunit.client.0.smithi012.stdout: create 127/39 file after the snapshot 2023-11-28T21:37:19.887 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/.snap/snap-subdir-test 2023-11-28T21:37:19.939 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/30 file after the snapshot 2023-11-28T21:37:19.942 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/31 file after the snapshot 2023-11-28T21:37:19.943 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/32 file after the snapshot 2023-11-28T21:37:19.943 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/33 file after the snapshot 2023-11-28T21:37:19.943 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/34 file after the snapshot 2023-11-28T21:37:19.943 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/35 file after the snapshot 2023-11-28T21:37:19.943 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/36 file after the snapshot 2023-11-28T21:37:19.943 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/37 file after the snapshot 2023-11-28T21:37:19.944 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/38 file after the snapshot 2023-11-28T21:37:19.944 INFO:tasks.workunit.client.0.smithi012.stdout: create 128/39 file after the snapshot 2023-11-28T21:37:19.944 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/.snap/snap-subdir-test 2023-11-28T21:37:19.993 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/30 file after the snapshot 2023-11-28T21:37:19.997 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/31 file after the snapshot 2023-11-28T21:37:19.998 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/32 file after the snapshot 2023-11-28T21:37:19.998 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/33 file after the snapshot 2023-11-28T21:37:19.998 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/34 file after the snapshot 2023-11-28T21:37:19.998 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/35 file after the snapshot 2023-11-28T21:37:19.998 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/36 file after the snapshot 2023-11-28T21:37:19.999 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/37 file after the snapshot 2023-11-28T21:37:19.999 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/38 file after the snapshot 2023-11-28T21:37:19.999 INFO:tasks.workunit.client.0.smithi012.stdout: create 129/39 file after the snapshot 2023-11-28T21:37:19.999 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/.snap/snap-subdir-test 2023-11-28T21:37:20.061 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/30 file after the snapshot 2023-11-28T21:37:20.064 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/31 file after the snapshot 2023-11-28T21:37:20.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/32 file after the snapshot 2023-11-28T21:37:20.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/33 file after the snapshot 2023-11-28T21:37:20.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/34 file after the snapshot 2023-11-28T21:37:20.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/35 file after the snapshot 2023-11-28T21:37:20.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/36 file after the snapshot 2023-11-28T21:37:20.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/37 file after the snapshot 2023-11-28T21:37:20.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/38 file after the snapshot 2023-11-28T21:37:20.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 130/39 file after the snapshot 2023-11-28T21:37:20.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/.snap/snap-subdir-test 2023-11-28T21:37:20.114 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/30 file after the snapshot 2023-11-28T21:37:20.117 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/31 file after the snapshot 2023-11-28T21:37:20.117 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/32 file after the snapshot 2023-11-28T21:37:20.117 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/33 file after the snapshot 2023-11-28T21:37:20.117 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/34 file after the snapshot 2023-11-28T21:37:20.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/35 file after the snapshot 2023-11-28T21:37:20.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/36 file after the snapshot 2023-11-28T21:37:20.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/37 file after the snapshot 2023-11-28T21:37:20.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/38 file after the snapshot 2023-11-28T21:37:20.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 131/39 file after the snapshot 2023-11-28T21:37:20.118 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/.snap/snap-subdir-test 2023-11-28T21:37:20.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:20.180 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/30 file after the snapshot 2023-11-28T21:37:20.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/31 file after the snapshot 2023-11-28T21:37:20.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/32 file after the snapshot 2023-11-28T21:37:20.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/33 file after the snapshot 2023-11-28T21:37:20.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/34 file after the snapshot 2023-11-28T21:37:20.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/35 file after the snapshot 2023-11-28T21:37:20.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/36 file after the snapshot 2023-11-28T21:37:20.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/37 file after the snapshot 2023-11-28T21:37:20.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/38 file after the snapshot 2023-11-28T21:37:20.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 132/39 file after the snapshot 2023-11-28T21:37:20.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/.snap/snap-subdir-test 2023-11-28T21:37:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f966429c640 1 Processor -- start 2023-11-28T21:37:20.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f966429c640 1 -- start start 2023-11-28T21:37:20.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f966429c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c0cb180 0x7f965c0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:20.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f966429c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c15acc0 con 0x7f965c0ca110 2023-11-28T21:37:20.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f966429c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c159ea0 con 0x7f965c153e40 2023-11-28T21:37:20.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f966429c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0ca520 con 0x7f965c0cb180 2023-11-28T21:37:20.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661837640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f965c0ca110 0x7f965c0cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56320/0 (socket says 172.21.15.12:56320) 2023-11-28T21:37:20.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661837640 1 -- 172.21.15.12:0/966299607 learned_addr learned my addr 172.21.15.12:0/966299607 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:20.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9662839640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c0cb180 0x7f965c0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:20.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593390588 0 0) 0x7f965c15acc0 con 0x7f965c0ca110 2023-11-28T21:37:20.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9648003530 con 0x7f965c0ca110 2023-11-28T21:37:20.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19678655 0 0) 0x7f965c159ea0 con 0x7f965c153e40 2023-11-28T21:37:20.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f965c15acc0 con 0x7f965c153e40 2023-11-28T21:37:20.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1511644412 0 0) 0x7f9648003530 con 0x7f965c0ca110 2023-11-28T21:37:20.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c0cb180 msgr2=0x7f965c0cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:20.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c0cb180 0x7f965c0cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:20.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 >> v1:172.21.15.144:6789/0 conn(0x7f965c153e40 legacy=0x7f965c15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:20.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c0cbb10 con 0x7f965c0ca110 2023-11-28T21:37:20.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f964c001680 con 0x7f965c0ca110 2023-11-28T21:37:20.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 >> v1:172.21.15.12:6789/0 conn(0x7f965c0ca110 legacy=0x7f965c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:20.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c0cb180 0x7f9648003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:20.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9648004160 0x7f96480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:20.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9648004d10 0x7f9648005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:20.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0ca520 con 0x7f9648004d10 2023-11-28T21:37:20.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964800bb20 con 0x7f9648004160 2023-11-28T21:37:20.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964800bce0 con 0x7f965c0cb180 2023-11-28T21:37:20.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9662839640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9648004d10 0x7f9648005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:20.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9662038640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c0cb180 0x7f9648003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:20.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9661837640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9648004160 0x7f96480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9662839640 1 -- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c0cb180 msgr2=0x7f9648003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9662839640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c0cb180 0x7f9648003bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9662839640 1 -- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9648004160 msgr2=0x7f96480047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9662839640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9648004160 0x7f96480047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9662839640 1 -- 172.21.15.12:0/966299607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648003530 con 0x7f9648004d10 2023-11-28T21:37:20.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.266+0000 7f9662038640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c0cb180 0x7f9648003bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:20.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f9662839640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9648004d10 0x7f9648005190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9654009fc0 tx=0x7f965401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2aa278e97700ebc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:20.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540a5c70 con 0x7f9648004d10 2023-11-28T21:37:20.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965400f040 con 0x7f9648004d10 2023-11-28T21:37:20.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f9661036640 1 -- 172.21.15.12:0/966299607 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540ad440 con 0x7f9648004d10 2023-11-28T21:37:20.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f966429c640 1 -- 172.21.15.12:0/966299607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9648004d10 msgr2=0x7f9648005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f966429c640 1 --2- 172.21.15.12:0/966299607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9648004d10 0x7f9648005190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9654009fc0 tx=0x7f965401cc60 comp rx=0 tx=0).stop 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f9662839640 1 -- 172.21.15.12:0/966299607 reap_dead start 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f966429c640 1 -- 172.21.15.12:0/966299607 shutdown_connections 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f966429c640 1 -- 172.21.15.12:0/966299607 >> 172.21.15.12:0/966299607 conn(0x7f965c06b7b0 msgr2=0x7f965c066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f966429c640 1 -- 172.21.15.12:0/966299607 shutdown_connections 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.270+0000 7f966429c640 1 -- 172.21.15.12:0/966299607 wait complete. 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f966429c640 1 Processor -- start 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f966429c640 1 -- start start 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f966429c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9648004d10 0x7f965c158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f966429c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965c14c8a0 0x7f965c147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f966429c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c1475c0 0x7f965c1f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f966429c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c159630 con 0x7f965c14c8a0 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f966429c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c150080 con 0x7f9648004d10 2023-11-28T21:37:20.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f966429c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c1583a0 con 0x7f965c1475c0 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9661837640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965c14c8a0 0x7f965c147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9661837640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965c14c8a0 0x7f965c147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52872/0 (socket says 172.21.15.12:52872) 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9661837640 1 -- 172.21.15.12:0/2423572024 learned_addr learned my addr 172.21.15.12:0/2423572024 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9662038640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9648004d10 0x7f965c158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9662839640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c1475c0 0x7f965c1f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9661837640 1 -- 172.21.15.12:0/2423572024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c1475c0 msgr2=0x7f965c1f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9661837640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c1475c0 0x7f965c1f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9661837640 1 -- 172.21.15.12:0/2423572024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9648004d10 msgr2=0x7f965c158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9661837640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9648004d10 0x7f965c158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9661837640 1 -- 172.21.15.12:0/2423572024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c1598a0 con 0x7f965c14c8a0 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9662038640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9648004d10 0x7f965c158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9661837640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965c14c8a0 0x7f965c147080 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f964c001210 tx=0x7f964c011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:20.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9652ffd640 1 -- 172.21.15.12:0/2423572024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00fb60 con 0x7f965c14c8a0 2023-11-28T21:37:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9652ffd640 1 -- 172.21.15.12:0/2423572024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c00fcc0 con 0x7f965c14c8a0 2023-11-28T21:37:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f9652ffd640 1 -- 172.21.15.12:0/2423572024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c015630 con 0x7f965c14c8a0 2023-11-28T21:37:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f966429c640 1 -- 172.21.15.12:0/2423572024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965c158680 con 0x7f965c14c8a0 2023-11-28T21:37:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.274+0000 7f966429c640 1 -- 172.21.15.12:0/2423572024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965c14d2e0 con 0x7f965c14c8a0 2023-11-28T21:37:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.278+0000 7f966429c640 1 -- 172.21.15.12:0/2423572024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c159a30 con 0x7f965c14c8a0 2023-11-28T21:37:20.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.278+0000 7f9652ffd640 1 -- 172.21.15.12:0/2423572024 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f964c015790 con 0x7f965c14c8a0 2023-11-28T21:37:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.278+0000 7f9652ffd640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9640066470 0x7f9640068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.278+0000 7f9652ffd640 1 -- 172.21.15.12:0/2423572024 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f964c087680 con 0x7f965c14c8a0 2023-11-28T21:37:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.278+0000 7f9662038640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9640066470 0x7f9640068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:20.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.278+0000 7f9662038640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9640066470 0x7f9640068930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9658007af0 tx=0x7f9658009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:20.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.282+0000 7f9652ffd640 1 -- 172.21.15.12:0/2423572024 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f964c054700 con 0x7f965c14c8a0 2023-11-28T21:37:20.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.474+0000 7f966429c640 1 -- 172.21.15.12:0/2423572024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f965c074f60 con 0x7f965c14c8a0 2023-11-28T21:37:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.474+0000 7f9652ffd640 1 -- 172.21.15.12:0/2423572024 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7f964c057d20 con 0x7f965c14c8a0 2023-11-28T21:37:20.480 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:20.480 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:20.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 -- 172.21.15.12:0/2423572024 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9640066470 msgr2=0x7f9640068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:20.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9640066470 0x7f9640068930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9658007af0 tx=0x7f9658009040 comp rx=0 tx=0).stop 2023-11-28T21:37:20.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 -- 172.21.15.12:0/2423572024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965c14c8a0 msgr2=0x7f965c147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965c14c8a0 0x7f965c147080 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f964c001210 tx=0x7f964c011d70 comp rx=0 tx=0).stop 2023-11-28T21:37:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 -- 172.21.15.12:0/2423572024 shutdown_connections 2023-11-28T21:37:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9640066470 0x7f9640068930 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f965c1475c0 0x7f965c1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965c14c8a0 0x7f965c147080 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 --2- 172.21.15.12:0/2423572024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9648004d10 0x7f965c158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 -- 172.21.15.12:0/2423572024 >> 172.21.15.12:0/2423572024 conn(0x7f965c06b7b0 msgr2=0x7f965c066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 -- 172.21.15.12:0/2423572024 shutdown_connections 2023-11-28T21:37:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:20.478+0000 7f966429c640 1 -- 172.21.15.12:0/2423572024 wait complete. 2023-11-28T21:37:20.501 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:20.519 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/30 file after the snapshot 2023-11-28T21:37:20.524 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/31 file after the snapshot 2023-11-28T21:37:20.524 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/32 file after the snapshot 2023-11-28T21:37:20.524 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/33 file after the snapshot 2023-11-28T21:37:20.524 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/34 file after the snapshot 2023-11-28T21:37:20.524 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/35 file after the snapshot 2023-11-28T21:37:20.524 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/36 file after the snapshot 2023-11-28T21:37:20.524 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/37 file after the snapshot 2023-11-28T21:37:20.524 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/38 file after the snapshot 2023-11-28T21:37:20.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 133/39 file after the snapshot 2023-11-28T21:37:20.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/.snap/snap-subdir-test 2023-11-28T21:37:20.592 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/30 file after the snapshot 2023-11-28T21:37:20.596 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/31 file after the snapshot 2023-11-28T21:37:20.596 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/32 file after the snapshot 2023-11-28T21:37:20.596 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/33 file after the snapshot 2023-11-28T21:37:20.596 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/34 file after the snapshot 2023-11-28T21:37:20.596 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/35 file after the snapshot 2023-11-28T21:37:20.596 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/36 file after the snapshot 2023-11-28T21:37:20.597 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/37 file after the snapshot 2023-11-28T21:37:20.597 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/38 file after the snapshot 2023-11-28T21:37:20.597 INFO:tasks.workunit.client.0.smithi012.stdout: create 134/39 file after the snapshot 2023-11-28T21:37:20.597 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/.snap/snap-subdir-test 2023-11-28T21:37:20.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/30 file after the snapshot 2023-11-28T21:37:20.661 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/31 file after the snapshot 2023-11-28T21:37:20.661 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/32 file after the snapshot 2023-11-28T21:37:20.661 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/33 file after the snapshot 2023-11-28T21:37:20.661 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/34 file after the snapshot 2023-11-28T21:37:20.662 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/35 file after the snapshot 2023-11-28T21:37:20.662 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/36 file after the snapshot 2023-11-28T21:37:20.662 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/37 file after the snapshot 2023-11-28T21:37:20.662 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/38 file after the snapshot 2023-11-28T21:37:20.662 INFO:tasks.workunit.client.0.smithi012.stdout: create 135/39 file after the snapshot 2023-11-28T21:37:20.662 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/.snap/snap-subdir-test 2023-11-28T21:37:20.711 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/30 file after the snapshot 2023-11-28T21:37:20.714 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/31 file after the snapshot 2023-11-28T21:37:20.714 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/32 file after the snapshot 2023-11-28T21:37:20.714 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/33 file after the snapshot 2023-11-28T21:37:20.714 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/34 file after the snapshot 2023-11-28T21:37:20.714 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/35 file after the snapshot 2023-11-28T21:37:20.714 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/36 file after the snapshot 2023-11-28T21:37:20.714 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/37 file after the snapshot 2023-11-28T21:37:20.715 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/38 file after the snapshot 2023-11-28T21:37:20.715 INFO:tasks.workunit.client.0.smithi012.stdout: create 136/39 file after the snapshot 2023-11-28T21:37:20.715 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/.snap/snap-subdir-test 2023-11-28T21:37:20.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/30 file after the snapshot 2023-11-28T21:37:20.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/31 file after the snapshot 2023-11-28T21:37:20.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/32 file after the snapshot 2023-11-28T21:37:20.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/33 file after the snapshot 2023-11-28T21:37:20.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/34 file after the snapshot 2023-11-28T21:37:20.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/35 file after the snapshot 2023-11-28T21:37:20.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/36 file after the snapshot 2023-11-28T21:37:20.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/37 file after the snapshot 2023-11-28T21:37:20.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/38 file after the snapshot 2023-11-28T21:37:20.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 137/39 file after the snapshot 2023-11-28T21:37:20.772 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/.snap/snap-subdir-test 2023-11-28T21:37:20.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/30 file after the snapshot 2023-11-28T21:37:20.841 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/31 file after the snapshot 2023-11-28T21:37:20.841 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/32 file after the snapshot 2023-11-28T21:37:20.841 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/33 file after the snapshot 2023-11-28T21:37:20.841 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/34 file after the snapshot 2023-11-28T21:37:20.841 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/35 file after the snapshot 2023-11-28T21:37:20.841 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/36 file after the snapshot 2023-11-28T21:37:20.841 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/37 file after the snapshot 2023-11-28T21:37:20.841 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/38 file after the snapshot 2023-11-28T21:37:20.842 INFO:tasks.workunit.client.0.smithi012.stdout: create 138/39 file after the snapshot 2023-11-28T21:37:20.842 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/.snap/snap-subdir-test 2023-11-28T21:37:20.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/30 file after the snapshot 2023-11-28T21:37:20.930 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/31 file after the snapshot 2023-11-28T21:37:20.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/32 file after the snapshot 2023-11-28T21:37:20.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/33 file after the snapshot 2023-11-28T21:37:20.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/34 file after the snapshot 2023-11-28T21:37:20.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/35 file after the snapshot 2023-11-28T21:37:20.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/36 file after the snapshot 2023-11-28T21:37:20.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/37 file after the snapshot 2023-11-28T21:37:20.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/38 file after the snapshot 2023-11-28T21:37:20.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 139/39 file after the snapshot 2023-11-28T21:37:20.931 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/.snap/snap-subdir-test 2023-11-28T21:37:21.228 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/30 file after the snapshot 2023-11-28T21:37:21.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/31 file after the snapshot 2023-11-28T21:37:21.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/32 file after the snapshot 2023-11-28T21:37:21.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/33 file after the snapshot 2023-11-28T21:37:21.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/34 file after the snapshot 2023-11-28T21:37:21.232 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/35 file after the snapshot 2023-11-28T21:37:21.232 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/36 file after the snapshot 2023-11-28T21:37:21.232 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/37 file after the snapshot 2023-11-28T21:37:21.232 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/38 file after the snapshot 2023-11-28T21:37:21.232 INFO:tasks.workunit.client.0.smithi012.stdout: create 140/39 file after the snapshot 2023-11-28T21:37:21.232 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/.snap/snap-subdir-test 2023-11-28T21:37:21.299 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/30 file after the snapshot 2023-11-28T21:37:21.303 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/31 file after the snapshot 2023-11-28T21:37:21.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/32 file after the snapshot 2023-11-28T21:37:21.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/33 file after the snapshot 2023-11-28T21:37:21.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/34 file after the snapshot 2023-11-28T21:37:21.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/35 file after the snapshot 2023-11-28T21:37:21.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/36 file after the snapshot 2023-11-28T21:37:21.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/37 file after the snapshot 2023-11-28T21:37:21.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/38 file after the snapshot 2023-11-28T21:37:21.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 141/39 file after the snapshot 2023-11-28T21:37:21.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/.snap/snap-subdir-test 2023-11-28T21:37:21.373 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/30 file after the snapshot 2023-11-28T21:37:21.376 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/31 file after the snapshot 2023-11-28T21:37:21.376 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/32 file after the snapshot 2023-11-28T21:37:21.376 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/33 file after the snapshot 2023-11-28T21:37:21.377 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/34 file after the snapshot 2023-11-28T21:37:21.377 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/35 file after the snapshot 2023-11-28T21:37:21.377 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/36 file after the snapshot 2023-11-28T21:37:21.377 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/37 file after the snapshot 2023-11-28T21:37:21.377 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/38 file after the snapshot 2023-11-28T21:37:21.378 INFO:tasks.workunit.client.0.smithi012.stdout: create 142/39 file after the snapshot 2023-11-28T21:37:21.378 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/.snap/snap-subdir-test 2023-11-28T21:37:21.453 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/30 file after the snapshot 2023-11-28T21:37:21.456 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/31 file after the snapshot 2023-11-28T21:37:21.457 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/32 file after the snapshot 2023-11-28T21:37:21.457 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/33 file after the snapshot 2023-11-28T21:37:21.457 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/34 file after the snapshot 2023-11-28T21:37:21.457 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/35 file after the snapshot 2023-11-28T21:37:21.457 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/36 file after the snapshot 2023-11-28T21:37:21.457 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/37 file after the snapshot 2023-11-28T21:37:21.457 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/38 file after the snapshot 2023-11-28T21:37:21.457 INFO:tasks.workunit.client.0.smithi012.stdout: create 143/39 file after the snapshot 2023-11-28T21:37:21.457 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/.snap/snap-subdir-test 2023-11-28T21:37:21.521 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/30 file after the snapshot 2023-11-28T21:37:21.524 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/31 file after the snapshot 2023-11-28T21:37:21.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/32 file after the snapshot 2023-11-28T21:37:21.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/33 file after the snapshot 2023-11-28T21:37:21.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/34 file after the snapshot 2023-11-28T21:37:21.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/35 file after the snapshot 2023-11-28T21:37:21.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/36 file after the snapshot 2023-11-28T21:37:21.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/37 file after the snapshot 2023-11-28T21:37:21.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/38 file after the snapshot 2023-11-28T21:37:21.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 144/39 file after the snapshot 2023-11-28T21:37:21.525 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/.snap/snap-subdir-test 2023-11-28T21:37:21.588 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/30 file after the snapshot 2023-11-28T21:37:21.590 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/31 file after the snapshot 2023-11-28T21:37:21.591 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/32 file after the snapshot 2023-11-28T21:37:21.591 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/33 file after the snapshot 2023-11-28T21:37:21.591 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/34 file after the snapshot 2023-11-28T21:37:21.591 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/35 file after the snapshot 2023-11-28T21:37:21.591 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/36 file after the snapshot 2023-11-28T21:37:21.592 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/37 file after the snapshot 2023-11-28T21:37:21.592 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/38 file after the snapshot 2023-11-28T21:37:21.592 INFO:tasks.workunit.client.0.smithi012.stdout: create 145/39 file after the snapshot 2023-11-28T21:37:21.592 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/.snap/snap-subdir-test 2023-11-28T21:37:21.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/30 file after the snapshot 2023-11-28T21:37:21.645 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/31 file after the snapshot 2023-11-28T21:37:21.645 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/32 file after the snapshot 2023-11-28T21:37:21.645 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/33 file after the snapshot 2023-11-28T21:37:21.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/34 file after the snapshot 2023-11-28T21:37:21.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/35 file after the snapshot 2023-11-28T21:37:21.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/36 file after the snapshot 2023-11-28T21:37:21.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/37 file after the snapshot 2023-11-28T21:37:21.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/38 file after the snapshot 2023-11-28T21:37:21.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 146/39 file after the snapshot 2023-11-28T21:37:21.646 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/.snap/snap-subdir-test 2023-11-28T21:37:22.330 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/30 file after the snapshot 2023-11-28T21:37:22.333 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/31 file after the snapshot 2023-11-28T21:37:22.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/32 file after the snapshot 2023-11-28T21:37:22.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/33 file after the snapshot 2023-11-28T21:37:22.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/34 file after the snapshot 2023-11-28T21:37:22.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/35 file after the snapshot 2023-11-28T21:37:22.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/36 file after the snapshot 2023-11-28T21:37:22.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/37 file after the snapshot 2023-11-28T21:37:22.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/38 file after the snapshot 2023-11-28T21:37:22.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 147/39 file after the snapshot 2023-11-28T21:37:22.334 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/.snap/snap-subdir-test 2023-11-28T21:37:22.414 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/30 file after the snapshot 2023-11-28T21:37:22.423 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/31 file after the snapshot 2023-11-28T21:37:22.423 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/32 file after the snapshot 2023-11-28T21:37:22.423 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/33 file after the snapshot 2023-11-28T21:37:22.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/34 file after the snapshot 2023-11-28T21:37:22.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/35 file after the snapshot 2023-11-28T21:37:22.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/36 file after the snapshot 2023-11-28T21:37:22.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/37 file after the snapshot 2023-11-28T21:37:22.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/38 file after the snapshot 2023-11-28T21:37:22.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 148/39 file after the snapshot 2023-11-28T21:37:22.424 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/.snap/snap-subdir-test 2023-11-28T21:37:22.491 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/30 file after the snapshot 2023-11-28T21:37:22.495 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/31 file after the snapshot 2023-11-28T21:37:22.495 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/32 file after the snapshot 2023-11-28T21:37:22.495 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/33 file after the snapshot 2023-11-28T21:37:22.495 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/34 file after the snapshot 2023-11-28T21:37:22.497 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/35 file after the snapshot 2023-11-28T21:37:22.497 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/36 file after the snapshot 2023-11-28T21:37:22.497 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/37 file after the snapshot 2023-11-28T21:37:22.497 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/38 file after the snapshot 2023-11-28T21:37:22.497 INFO:tasks.workunit.client.0.smithi012.stdout: create 149/39 file after the snapshot 2023-11-28T21:37:22.497 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/.snap/snap-subdir-test 2023-11-28T21:37:22.502 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:22.557 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/30 file after the snapshot 2023-11-28T21:37:22.562 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/31 file after the snapshot 2023-11-28T21:37:22.562 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/32 file after the snapshot 2023-11-28T21:37:22.562 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/33 file after the snapshot 2023-11-28T21:37:22.562 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/34 file after the snapshot 2023-11-28T21:37:22.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/35 file after the snapshot 2023-11-28T21:37:22.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/36 file after the snapshot 2023-11-28T21:37:22.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/37 file after the snapshot 2023-11-28T21:37:22.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/38 file after the snapshot 2023-11-28T21:37:22.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 150/39 file after the snapshot 2023-11-28T21:37:22.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/.snap/snap-subdir-test 2023-11-28T21:37:22.629 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/30 file after the snapshot 2023-11-28T21:37:22.632 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/31 file after the snapshot 2023-11-28T21:37:22.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/32 file after the snapshot 2023-11-28T21:37:22.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/33 file after the snapshot 2023-11-28T21:37:22.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/34 file after the snapshot 2023-11-28T21:37:22.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/35 file after the snapshot 2023-11-28T21:37:22.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/36 file after the snapshot 2023-11-28T21:37:22.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/37 file after the snapshot 2023-11-28T21:37:22.635 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/38 file after the snapshot 2023-11-28T21:37:22.635 INFO:tasks.workunit.client.0.smithi012.stdout: create 151/39 file after the snapshot 2023-11-28T21:37:22.636 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/.snap/snap-subdir-test 2023-11-28T21:37:22.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.654+0000 7f91034d6640 1 Processor -- start 2023-11-28T21:37:22.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.654+0000 7f91034d6640 1 -- start start 2023-11-28T21:37:22.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f91034d6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f90fc154100 0x7f90fc15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:22.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f91034d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90fc0ca3f0 0x7f90fc0c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:22.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f91034d6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90fc15e270 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f91034d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90fc0c4e60 con 0x7f90fc0ca3f0 2023-11-28T21:37:22.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f91034d6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90fc1544e0 con 0x7f90fc154100 2023-11-28T21:37:22.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f9101a73640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90fc0ca3f0 0x7f90fc0c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:22.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f9101a73640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90fc0ca3f0 0x7f90fc0c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52888/0 (socket says 172.21.15.12:52888) 2023-11-28T21:37:22.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f9101a73640 1 -- 172.21.15.12:0/4082519484 learned_addr learned my addr 172.21.15.12:0/4082519484 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:22.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f9101272640 1 --2- 172.21.15.12:0/4082519484 >> v2:172.21.15.144:3300/0 conn(0x7f90fc154100 0x7f90fc15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:22.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f9101a73640 1 -- 172.21.15.12:0/4082519484 >> v2:172.21.15.144:3300/0 conn(0x7f90fc154100 msgr2=0x7f90fc15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:22.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f9101a73640 1 --2- 172.21.15.12:0/4082519484 >> v2:172.21.15.144:3300/0 conn(0x7f90fc154100 0x7f90fc15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f9101a73640 1 -- 172.21.15.12:0/4082519484 >> v1:172.21.15.144:6789/0 conn(0x7f90fc0c9fd0 legacy=0x7f90fc06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:22.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.658+0000 7f9101a73640 1 -- 172.21.15.12:0/4082519484 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc15d9d0 con 0x7f90fc0ca3f0 2023-11-28T21:37:22.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f9101a73640 1 --2- 172.21.15.12:0/4082519484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0ca3f0 0x7f90fc0c52f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f90fc067120 tx=0x7f90f401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf138c7edcb8066 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:22.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f90f3fff640 1 -- 172.21.15.12:0/4082519484 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f40a5860 con 0x7f90fc0ca3f0 2023-11-28T21:37:22.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f90f3fff640 1 -- 172.21.15.12:0/4082519484 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f40a59c0 con 0x7f90fc0ca3f0 2023-11-28T21:37:22.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f90f3fff640 1 -- 172.21.15.12:0/4082519484 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f40a5d50 con 0x7f90fc0ca3f0 2023-11-28T21:37:22.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f91034d6640 1 -- 172.21.15.12:0/4082519484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0ca3f0 msgr2=0x7f90fc0c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:22.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f91034d6640 1 --2- 172.21.15.12:0/4082519484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0ca3f0 0x7f90fc0c52f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f90fc067120 tx=0x7f90f401c9a0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f91034d6640 1 -- 172.21.15.12:0/4082519484 shutdown_connections 2023-11-28T21:37:22.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f91034d6640 1 --2- 172.21.15.12:0/4082519484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0ca3f0 0x7f90fc0c52f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f91034d6640 1 --2- 172.21.15.12:0/4082519484 >> v2:172.21.15.144:3300/0 conn(0x7f90fc154100 0x7f90fc15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f91034d6640 1 -- 172.21.15.12:0/4082519484 >> 172.21.15.12:0/4082519484 conn(0x7f90fc06c690 msgr2=0x7f90fc066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:22.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f91034d6640 1 -- 172.21.15.12:0/4082519484 shutdown_connections 2023-11-28T21:37:22.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f91034d6640 1 -- 172.21.15.12:0/4082519484 wait complete. 2023-11-28T21:37:22.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f91034d6640 1 Processor -- start 2023-11-28T21:37:22.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.662+0000 7f91034d6640 1 -- start start 2023-11-28T21:37:22.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f91034d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0c9fd0 0x7f90fc144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:22.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f91034d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90fc0ca3f0 0x7f90fc1f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:22.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f91034d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90fc154100 0x7f90fc0b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:22.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f91034d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc15aee0 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f91034d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc156940 con 0x7f90fc0ca3f0 2023-11-28T21:37:22.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f91034d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc1f6d70 con 0x7f90fc154100 2023-11-28T21:37:22.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101272640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0c9fd0 0x7f90fc144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:22.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101272640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0c9fd0 0x7f90fc144330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52904/0 (socket says 172.21.15.12:52904) 2023-11-28T21:37:22.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101272640 1 -- 172.21.15.12:0/3883734557 learned_addr learned my addr 172.21.15.12:0/3883734557 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:22.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101a73640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90fc154100 0x7f90fc0b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:22.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9100a71640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90fc0ca3f0 0x7f90fc1f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:22.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101272640 1 -- 172.21.15.12:0/3883734557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90fc154100 msgr2=0x7f90fc0b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:22.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101272640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90fc154100 0x7f90fc0b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101272640 1 -- 172.21.15.12:0/3883734557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90fc0ca3f0 msgr2=0x7f90fc1f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:22.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101272640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90fc0ca3f0 0x7f90fc1f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101272640 1 -- 172.21.15.12:0/3883734557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4009b30 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101a73640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90fc154100 0x7f90fc0b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:22.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f9101272640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0c9fd0 0x7f90fc144330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f90f800d5f0 tx=0x7f90f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:22.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f90f1ffb640 1 -- 172.21.15.12:0/3883734557 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f80125f0 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f90f1ffb640 1 -- 172.21.15.12:0/3883734557 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f8012750 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f90f1ffb640 1 -- 172.21.15.12:0/3883734557 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8012ae0 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f91034d6640 1 -- 172.21.15.12:0/3883734557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc1f7200 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f91034d6640 1 -- 172.21.15.12:0/3883734557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc15ac00 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.666+0000 7f91034d6640 1 -- 172.21.15.12:0/3883734557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90fc067c40 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.670+0000 7f90f1ffb640 1 -- 172.21.15.12:0/3883734557 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f90f8012c40 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.670+0000 7f90f1ffb640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90e00664a0 0x7f90e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:22.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.670+0000 7f90f1ffb640 1 -- 172.21.15.12:0/3883734557 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f90f8084720 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.670+0000 7f9100a71640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90e00664a0 0x7f90e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:22.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.670+0000 7f9100a71640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90e00664a0 0x7f90e0068960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0cc940 tx=0x7f90ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:22.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.674+0000 7f90f1ffb640 1 -- 172.21.15.12:0/3883734557 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f90f80517a0 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.701 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/30 file after the snapshot 2023-11-28T21:37:22.704 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/31 file after the snapshot 2023-11-28T21:37:22.704 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/32 file after the snapshot 2023-11-28T21:37:22.704 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/33 file after the snapshot 2023-11-28T21:37:22.704 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/34 file after the snapshot 2023-11-28T21:37:22.704 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/35 file after the snapshot 2023-11-28T21:37:22.705 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/36 file after the snapshot 2023-11-28T21:37:22.705 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/37 file after the snapshot 2023-11-28T21:37:22.705 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/38 file after the snapshot 2023-11-28T21:37:22.705 INFO:tasks.workunit.client.0.smithi012.stdout: create 152/39 file after the snapshot 2023-11-28T21:37:22.705 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/.snap/snap-subdir-test 2023-11-28T21:37:22.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.866+0000 7f91034d6640 1 -- 172.21.15.12:0/3883734557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90fc074f60 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.866+0000 7f90f1ffb640 1 -- 172.21.15.12:0/3883734557 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7f90f8007380 con 0x7f90fc0c9fd0 2023-11-28T21:37:22.873 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:22.873 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:22.873 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 -- 172.21.15.12:0/3883734557 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90e00664a0 msgr2=0x7f90e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90e00664a0 0x7f90e0068960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0cc940 tx=0x7f90ec009040 comp rx=0 tx=0).stop 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 -- 172.21.15.12:0/3883734557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0c9fd0 msgr2=0x7f90fc144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0c9fd0 0x7f90fc144330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f90f800d5f0 tx=0x7f90f800dab0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 -- 172.21.15.12:0/3883734557 shutdown_connections 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90e00664a0 0x7f90e0068960 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90fc154100 0x7f90fc0b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90fc0ca3f0 0x7f90fc1f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 --2- 172.21.15.12:0/3883734557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90fc0c9fd0 0x7f90fc144330 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 -- 172.21.15.12:0/3883734557 >> 172.21.15.12:0/3883734557 conn(0x7f90fc06c690 msgr2=0x7f90fc147760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:22.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 -- 172.21.15.12:0/3883734557 shutdown_connections 2023-11-28T21:37:22.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:22.870+0000 7f91034d6640 1 -- 172.21.15.12:0/3883734557 wait complete. 2023-11-28T21:37:22.894 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:23.666 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/30 file after the snapshot 2023-11-28T21:37:23.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/31 file after the snapshot 2023-11-28T21:37:23.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/32 file after the snapshot 2023-11-28T21:37:23.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/33 file after the snapshot 2023-11-28T21:37:23.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/34 file after the snapshot 2023-11-28T21:37:23.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/35 file after the snapshot 2023-11-28T21:37:23.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/36 file after the snapshot 2023-11-28T21:37:23.672 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/37 file after the snapshot 2023-11-28T21:37:23.673 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/38 file after the snapshot 2023-11-28T21:37:23.673 INFO:tasks.workunit.client.0.smithi012.stdout: create 153/39 file after the snapshot 2023-11-28T21:37:23.673 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/.snap/snap-subdir-test 2023-11-28T21:37:23.738 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/30 file after the snapshot 2023-11-28T21:37:23.743 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/31 file after the snapshot 2023-11-28T21:37:23.743 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/32 file after the snapshot 2023-11-28T21:37:23.743 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/33 file after the snapshot 2023-11-28T21:37:23.743 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/34 file after the snapshot 2023-11-28T21:37:23.743 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/35 file after the snapshot 2023-11-28T21:37:23.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/36 file after the snapshot 2023-11-28T21:37:23.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/37 file after the snapshot 2023-11-28T21:37:23.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/38 file after the snapshot 2023-11-28T21:37:23.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 154/39 file after the snapshot 2023-11-28T21:37:23.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/.snap/snap-subdir-test 2023-11-28T21:37:23.807 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/30 file after the snapshot 2023-11-28T21:37:23.812 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/31 file after the snapshot 2023-11-28T21:37:23.812 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/32 file after the snapshot 2023-11-28T21:37:23.812 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/33 file after the snapshot 2023-11-28T21:37:23.812 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/34 file after the snapshot 2023-11-28T21:37:23.812 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/35 file after the snapshot 2023-11-28T21:37:23.812 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/36 file after the snapshot 2023-11-28T21:37:23.812 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/37 file after the snapshot 2023-11-28T21:37:23.812 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/38 file after the snapshot 2023-11-28T21:37:23.813 INFO:tasks.workunit.client.0.smithi012.stdout: create 155/39 file after the snapshot 2023-11-28T21:37:23.813 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/.snap/snap-subdir-test 2023-11-28T21:37:23.875 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/30 file after the snapshot 2023-11-28T21:37:23.879 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/31 file after the snapshot 2023-11-28T21:37:23.879 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/32 file after the snapshot 2023-11-28T21:37:23.879 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/33 file after the snapshot 2023-11-28T21:37:23.879 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/34 file after the snapshot 2023-11-28T21:37:23.879 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/35 file after the snapshot 2023-11-28T21:37:23.879 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/36 file after the snapshot 2023-11-28T21:37:23.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/37 file after the snapshot 2023-11-28T21:37:23.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/38 file after the snapshot 2023-11-28T21:37:23.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 156/39 file after the snapshot 2023-11-28T21:37:23.880 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/.snap/snap-subdir-test 2023-11-28T21:37:23.935 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/30 file after the snapshot 2023-11-28T21:37:23.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/31 file after the snapshot 2023-11-28T21:37:23.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/32 file after the snapshot 2023-11-28T21:37:23.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/33 file after the snapshot 2023-11-28T21:37:23.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/34 file after the snapshot 2023-11-28T21:37:23.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/35 file after the snapshot 2023-11-28T21:37:23.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/36 file after the snapshot 2023-11-28T21:37:23.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/37 file after the snapshot 2023-11-28T21:37:23.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/38 file after the snapshot 2023-11-28T21:37:23.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 157/39 file after the snapshot 2023-11-28T21:37:23.938 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/.snap/snap-subdir-test 2023-11-28T21:37:23.989 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/30 file after the snapshot 2023-11-28T21:37:23.991 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/31 file after the snapshot 2023-11-28T21:37:23.992 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/32 file after the snapshot 2023-11-28T21:37:23.992 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/33 file after the snapshot 2023-11-28T21:37:23.992 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/34 file after the snapshot 2023-11-28T21:37:23.993 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/35 file after the snapshot 2023-11-28T21:37:23.993 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/36 file after the snapshot 2023-11-28T21:37:23.993 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/37 file after the snapshot 2023-11-28T21:37:23.993 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/38 file after the snapshot 2023-11-28T21:37:23.993 INFO:tasks.workunit.client.0.smithi012.stdout: create 158/39 file after the snapshot 2023-11-28T21:37:23.993 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/.snap/snap-subdir-test 2023-11-28T21:37:24.062 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/30 file after the snapshot 2023-11-28T21:37:24.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/31 file after the snapshot 2023-11-28T21:37:24.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/32 file after the snapshot 2023-11-28T21:37:24.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/33 file after the snapshot 2023-11-28T21:37:24.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/34 file after the snapshot 2023-11-28T21:37:24.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/35 file after the snapshot 2023-11-28T21:37:24.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/36 file after the snapshot 2023-11-28T21:37:24.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/37 file after the snapshot 2023-11-28T21:37:24.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/38 file after the snapshot 2023-11-28T21:37:24.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 159/39 file after the snapshot 2023-11-28T21:37:24.065 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/.snap/snap-subdir-test 2023-11-28T21:37:24.120 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/30 file after the snapshot 2023-11-28T21:37:24.123 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/31 file after the snapshot 2023-11-28T21:37:24.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/32 file after the snapshot 2023-11-28T21:37:24.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/33 file after the snapshot 2023-11-28T21:37:24.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/34 file after the snapshot 2023-11-28T21:37:24.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/35 file after the snapshot 2023-11-28T21:37:24.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/36 file after the snapshot 2023-11-28T21:37:24.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/37 file after the snapshot 2023-11-28T21:37:24.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/38 file after the snapshot 2023-11-28T21:37:24.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 160/39 file after the snapshot 2023-11-28T21:37:24.124 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/.snap/snap-subdir-test 2023-11-28T21:37:24.186 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/30 file after the snapshot 2023-11-28T21:37:24.189 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/31 file after the snapshot 2023-11-28T21:37:24.189 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/32 file after the snapshot 2023-11-28T21:37:24.189 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/33 file after the snapshot 2023-11-28T21:37:24.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/34 file after the snapshot 2023-11-28T21:37:24.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/35 file after the snapshot 2023-11-28T21:37:24.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/36 file after the snapshot 2023-11-28T21:37:24.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/37 file after the snapshot 2023-11-28T21:37:24.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/38 file after the snapshot 2023-11-28T21:37:24.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 161/39 file after the snapshot 2023-11-28T21:37:24.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/.snap/snap-subdir-test 2023-11-28T21:37:24.507 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/30 file after the snapshot 2023-11-28T21:37:24.511 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/31 file after the snapshot 2023-11-28T21:37:24.511 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/32 file after the snapshot 2023-11-28T21:37:24.512 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/33 file after the snapshot 2023-11-28T21:37:24.512 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/34 file after the snapshot 2023-11-28T21:37:24.512 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/35 file after the snapshot 2023-11-28T21:37:24.512 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/36 file after the snapshot 2023-11-28T21:37:24.512 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/37 file after the snapshot 2023-11-28T21:37:24.512 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/38 file after the snapshot 2023-11-28T21:37:24.512 INFO:tasks.workunit.client.0.smithi012.stdout: create 162/39 file after the snapshot 2023-11-28T21:37:24.512 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/.snap/snap-subdir-test 2023-11-28T21:37:24.593 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/30 file after the snapshot 2023-11-28T21:37:24.598 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/31 file after the snapshot 2023-11-28T21:37:24.599 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/32 file after the snapshot 2023-11-28T21:37:24.599 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/33 file after the snapshot 2023-11-28T21:37:24.599 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/34 file after the snapshot 2023-11-28T21:37:24.599 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/35 file after the snapshot 2023-11-28T21:37:24.599 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/36 file after the snapshot 2023-11-28T21:37:24.599 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/37 file after the snapshot 2023-11-28T21:37:24.599 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/38 file after the snapshot 2023-11-28T21:37:24.599 INFO:tasks.workunit.client.0.smithi012.stdout: create 163/39 file after the snapshot 2023-11-28T21:37:24.599 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/.snap/snap-subdir-test 2023-11-28T21:37:24.654 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/30 file after the snapshot 2023-11-28T21:37:24.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/31 file after the snapshot 2023-11-28T21:37:24.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/32 file after the snapshot 2023-11-28T21:37:24.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/33 file after the snapshot 2023-11-28T21:37:24.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/34 file after the snapshot 2023-11-28T21:37:24.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/35 file after the snapshot 2023-11-28T21:37:24.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/36 file after the snapshot 2023-11-28T21:37:24.659 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/37 file after the snapshot 2023-11-28T21:37:24.659 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/38 file after the snapshot 2023-11-28T21:37:24.659 INFO:tasks.workunit.client.0.smithi012.stdout: create 164/39 file after the snapshot 2023-11-28T21:37:24.659 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/.snap/snap-subdir-test 2023-11-28T21:37:24.896 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:25.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc226c9640 1 Processor -- start 2023-11-28T21:37:25.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc226c9640 1 -- start start 2023-11-28T21:37:25.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc226c9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdc1c151af0 0x7fdc1c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:25.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc226c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151ed0 0x7fdc1c0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc226c9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdc1c06e300 0x7fdc1c0c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc226c9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0707f0 con 0x7fdc1c151af0 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc226c9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0c7b90 con 0x7fdc1c06e300 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc226c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c071c20 con 0x7fdc1c151ed0 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc1bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdc1c151af0 0x7fdc1c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc1bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdc1c151af0 0x7fdc1c0c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34276/0 (socket says 172.21.15.12:34276) 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc1bfff640 1 -- 172.21.15.12:0/2278576968 learned_addr learned my addr 172.21.15.12:0/2278576968 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc20c66640 1 --2- 172.21.15.12:0/2278576968 >> v2:172.21.15.144:3300/0 conn(0x7fdc1c06e300 0x7fdc1c0c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc1bfff640 1 -- 172.21.15.12:0/2278576968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151ed0 msgr2=0x7fdc1c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.042+0000 7fdc1b7fe640 1 --2- 172.21.15.12:0/2278576968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151ed0 0x7fdc1c0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:25.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/2278576968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151ed0 0x7fdc1c0c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc1bfff640 1 -- 172.21.15.12:0/2278576968 >> v2:172.21.15.144:3300/0 conn(0x7fdc1c06e300 msgr2=0x7fdc1c0c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/2278576968 >> v2:172.21.15.144:3300/0 conn(0x7fdc1c06e300 0x7fdc1c0c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc1bfff640 1 -- 172.21.15.12:0/2278576968 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c157b70 con 0x7fdc1c151af0 2023-11-28T21:37:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc20c66640 1 --2- 172.21.15.12:0/2278576968 >> v2:172.21.15.144:3300/0 conn(0x7fdc1c06e300 0x7fdc1c0c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:37:25.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc1b7fe640 1 --2- 172.21.15.12:0/2278576968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151ed0 0x7fdc1c0c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:37:25.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/2278576968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c151af0 0x7fdc1c0c7650 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c06b940 tx=0x7fdc0c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c51ffcea59c4422 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:25.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc1affd640 1 -- 172.21.15.12:0/2278576968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0ac410 con 0x7fdc1c151af0 2023-11-28T21:37:25.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc1affd640 1 -- 172.21.15.12:0/2278576968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c0ac570 con 0x7fdc1c151af0 2023-11-28T21:37:25.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc1affd640 1 -- 172.21.15.12:0/2278576968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0ac900 con 0x7fdc1c151af0 2023-11-28T21:37:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc226c9640 1 -- 172.21.15.12:0/2278576968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c151af0 msgr2=0x7fdc1c0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc226c9640 1 --2- 172.21.15.12:0/2278576968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c151af0 0x7fdc1c0c7650 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c06b940 tx=0x7fdc0c01c500 comp rx=0 tx=0).stop 2023-11-28T21:37:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc226c9640 1 -- 172.21.15.12:0/2278576968 shutdown_connections 2023-11-28T21:37:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc226c9640 1 --2- 172.21.15.12:0/2278576968 >> v2:172.21.15.144:3300/0 conn(0x7fdc1c06e300 0x7fdc1c0c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc226c9640 1 --2- 172.21.15.12:0/2278576968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151ed0 0x7fdc1c0c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc226c9640 1 --2- 172.21.15.12:0/2278576968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c151af0 0x7fdc1c0c7650 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc226c9640 1 -- 172.21.15.12:0/2278576968 >> 172.21.15.12:0/2278576968 conn(0x7fdc1c066360 msgr2=0x7fdc1c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc226c9640 1 -- 172.21.15.12:0/2278576968 shutdown_connections 2023-11-28T21:37:25.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.046+0000 7fdc226c9640 1 -- 172.21.15.12:0/2278576968 wait complete. 2023-11-28T21:37:25.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc226c9640 1 Processor -- start 2023-11-28T21:37:25.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc226c9640 1 -- start start 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc226c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c06e300 0x7fdc1c1f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc226c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151af0 0x7fdc1c145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc226c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c14cfd0 0x7fdc1c158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc226c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c146780 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc226c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1f17f0 con 0x7fdc1c06e300 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc226c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1464f0 con 0x7fdc1c151af0 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc20c66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c14cfd0 0x7fdc1c158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc20c66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c14cfd0 0x7fdc1c158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34280/0 (socket says 172.21.15.12:34280) 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc20c66640 1 -- 172.21.15.12:0/592175678 learned_addr learned my addr 172.21.15.12:0/592175678 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c06e300 0x7fdc1c1f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:25.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc1b7fe640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151af0 0x7fdc1c145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc20c66640 1 -- 172.21.15.12:0/592175678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151af0 msgr2=0x7fdc1c145fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc20c66640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151af0 0x7fdc1c145fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc20c66640 1 -- 172.21.15.12:0/592175678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c06e300 msgr2=0x7fdc1c1f12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc20c66640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c06e300 0x7fdc1c1f12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc20c66640 1 -- 172.21.15.12:0/592175678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c0094d0 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.050+0000 7fdc20c66640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c14cfd0 0x7fdc1c158c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400ce40 tx=0x7fdc1400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:25.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc18ff9640 1 -- 172.21.15.12:0/592175678 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400d840 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc18ff9640 1 -- 172.21.15.12:0/592175678 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1400d9a0 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc226c9640 1 -- 172.21.15.12:0/592175678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c0ca180 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc18ff9640 1 -- 172.21.15.12:0/592175678 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400dd50 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc226c9640 1 -- 172.21.15.12:0/592175678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c0ca700 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc226c9640 1 -- 172.21.15.12:0/592175678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c074f60 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc18ff9640 1 -- 172.21.15.12:0/592175678 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdc1401e3b0 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc18ff9640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc080664a0 0x7fdc08068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc18ff9640 1 -- 172.21.15.12:0/592175678 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fdc14084d20 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc080664a0 0x7fdc08068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:25.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.054+0000 7fdc1bfff640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc080664a0 0x7fdc08068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0ae000 tx=0x7fdc0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:25.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.058+0000 7fdc18ff9640 1 -- 172.21.15.12:0/592175678 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdc14051c80 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 -- 172.21.15.12:0/592175678 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc1c072c10 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc18ff9640 1 -- 172.21.15.12:0/592175678 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7fdc1401a070 con 0x7fdc1c14cfd0 2023-11-28T21:37:25.255 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:25.255 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:25.255 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 -- 172.21.15.12:0/592175678 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc080664a0 msgr2=0x7fdc08068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc080664a0 0x7fdc08068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0ae000 tx=0x7fdc0c008040 comp rx=0 tx=0).stop 2023-11-28T21:37:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 -- 172.21.15.12:0/592175678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c14cfd0 msgr2=0x7fdc1c158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c14cfd0 0x7fdc1c158c20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400ce40 tx=0x7fdc1400a710 comp rx=0 tx=0).stop 2023-11-28T21:37:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 -- 172.21.15.12:0/592175678 shutdown_connections 2023-11-28T21:37:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc080664a0 0x7fdc08068960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1c14cfd0 0x7fdc1c158c20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1c151af0 0x7fdc1c145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 --2- 172.21.15.12:0/592175678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc1c06e300 0x7fdc1c1f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 -- 172.21.15.12:0/592175678 >> 172.21.15.12:0/592175678 conn(0x7fdc1c066360 msgr2=0x7fdc1c06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 -- 172.21.15.12:0/592175678 shutdown_connections 2023-11-28T21:37:25.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:25.250+0000 7fdc226c9640 1 -- 172.21.15.12:0/592175678 wait complete. 2023-11-28T21:37:25.277 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:25.528 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/30 file after the snapshot 2023-11-28T21:37:25.530 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/31 file after the snapshot 2023-11-28T21:37:25.530 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/32 file after the snapshot 2023-11-28T21:37:25.531 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/33 file after the snapshot 2023-11-28T21:37:25.531 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/34 file after the snapshot 2023-11-28T21:37:25.531 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/35 file after the snapshot 2023-11-28T21:37:25.532 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/36 file after the snapshot 2023-11-28T21:37:25.532 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/37 file after the snapshot 2023-11-28T21:37:25.532 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/38 file after the snapshot 2023-11-28T21:37:25.532 INFO:tasks.workunit.client.0.smithi012.stdout: create 165/39 file after the snapshot 2023-11-28T21:37:25.532 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/.snap/snap-subdir-test 2023-11-28T21:37:25.606 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/30 file after the snapshot 2023-11-28T21:37:25.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/31 file after the snapshot 2023-11-28T21:37:25.610 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/32 file after the snapshot 2023-11-28T21:37:25.611 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/33 file after the snapshot 2023-11-28T21:37:25.611 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/34 file after the snapshot 2023-11-28T21:37:25.611 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/35 file after the snapshot 2023-11-28T21:37:25.611 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/36 file after the snapshot 2023-11-28T21:37:25.611 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/37 file after the snapshot 2023-11-28T21:37:25.611 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/38 file after the snapshot 2023-11-28T21:37:25.611 INFO:tasks.workunit.client.0.smithi012.stdout: create 166/39 file after the snapshot 2023-11-28T21:37:25.611 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/.snap/snap-subdir-test 2023-11-28T21:37:25.814 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/30 file after the snapshot 2023-11-28T21:37:25.824 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/31 file after the snapshot 2023-11-28T21:37:25.824 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/32 file after the snapshot 2023-11-28T21:37:25.825 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/33 file after the snapshot 2023-11-28T21:37:25.825 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/34 file after the snapshot 2023-11-28T21:37:25.825 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/35 file after the snapshot 2023-11-28T21:37:25.825 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/36 file after the snapshot 2023-11-28T21:37:25.825 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/37 file after the snapshot 2023-11-28T21:37:25.825 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/38 file after the snapshot 2023-11-28T21:37:25.825 INFO:tasks.workunit.client.0.smithi012.stdout: create 167/39 file after the snapshot 2023-11-28T21:37:25.825 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/.snap/snap-subdir-test 2023-11-28T21:37:25.908 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/30 file after the snapshot 2023-11-28T21:37:25.914 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/31 file after the snapshot 2023-11-28T21:37:25.914 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/32 file after the snapshot 2023-11-28T21:37:25.914 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/33 file after the snapshot 2023-11-28T21:37:25.914 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/34 file after the snapshot 2023-11-28T21:37:25.914 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/35 file after the snapshot 2023-11-28T21:37:25.914 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/36 file after the snapshot 2023-11-28T21:37:25.914 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/37 file after the snapshot 2023-11-28T21:37:25.914 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/38 file after the snapshot 2023-11-28T21:37:25.914 INFO:tasks.workunit.client.0.smithi012.stdout: create 168/39 file after the snapshot 2023-11-28T21:37:25.915 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/.snap/snap-subdir-test 2023-11-28T21:37:25.975 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/30 file after the snapshot 2023-11-28T21:37:25.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/31 file after the snapshot 2023-11-28T21:37:25.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/32 file after the snapshot 2023-11-28T21:37:25.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/33 file after the snapshot 2023-11-28T21:37:25.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/34 file after the snapshot 2023-11-28T21:37:25.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/35 file after the snapshot 2023-11-28T21:37:25.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/36 file after the snapshot 2023-11-28T21:37:25.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/37 file after the snapshot 2023-11-28T21:37:25.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/38 file after the snapshot 2023-11-28T21:37:25.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 169/39 file after the snapshot 2023-11-28T21:37:25.979 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/.snap/snap-subdir-test 2023-11-28T21:37:26.042 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/30 file after the snapshot 2023-11-28T21:37:26.047 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/31 file after the snapshot 2023-11-28T21:37:26.047 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/32 file after the snapshot 2023-11-28T21:37:26.047 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/33 file after the snapshot 2023-11-28T21:37:26.047 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/34 file after the snapshot 2023-11-28T21:37:26.047 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/35 file after the snapshot 2023-11-28T21:37:26.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/36 file after the snapshot 2023-11-28T21:37:26.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/37 file after the snapshot 2023-11-28T21:37:26.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/38 file after the snapshot 2023-11-28T21:37:26.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 170/39 file after the snapshot 2023-11-28T21:37:26.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/.snap/snap-subdir-test 2023-11-28T21:37:26.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/30 file after the snapshot 2023-11-28T21:37:26.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/31 file after the snapshot 2023-11-28T21:37:26.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/32 file after the snapshot 2023-11-28T21:37:26.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/33 file after the snapshot 2023-11-28T21:37:26.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/34 file after the snapshot 2023-11-28T21:37:26.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/35 file after the snapshot 2023-11-28T21:37:26.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/36 file after the snapshot 2023-11-28T21:37:26.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/37 file after the snapshot 2023-11-28T21:37:26.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/38 file after the snapshot 2023-11-28T21:37:26.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 171/39 file after the snapshot 2023-11-28T21:37:26.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/.snap/snap-subdir-test 2023-11-28T21:37:26.832 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/30 file after the snapshot 2023-11-28T21:37:26.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/31 file after the snapshot 2023-11-28T21:37:26.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/32 file after the snapshot 2023-11-28T21:37:26.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/33 file after the snapshot 2023-11-28T21:37:26.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/34 file after the snapshot 2023-11-28T21:37:26.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/35 file after the snapshot 2023-11-28T21:37:26.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/36 file after the snapshot 2023-11-28T21:37:26.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/37 file after the snapshot 2023-11-28T21:37:26.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/38 file after the snapshot 2023-11-28T21:37:26.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 172/39 file after the snapshot 2023-11-28T21:37:26.838 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/.snap/snap-subdir-test 2023-11-28T21:37:27.278 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:27.310 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/30 file after the snapshot 2023-11-28T21:37:27.314 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/31 file after the snapshot 2023-11-28T21:37:27.314 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/32 file after the snapshot 2023-11-28T21:37:27.314 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/33 file after the snapshot 2023-11-28T21:37:27.315 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/34 file after the snapshot 2023-11-28T21:37:27.315 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/35 file after the snapshot 2023-11-28T21:37:27.315 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/36 file after the snapshot 2023-11-28T21:37:27.315 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/37 file after the snapshot 2023-11-28T21:37:27.315 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/38 file after the snapshot 2023-11-28T21:37:27.315 INFO:tasks.workunit.client.0.smithi012.stdout: create 173/39 file after the snapshot 2023-11-28T21:37:27.315 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/.snap/snap-subdir-test 2023-11-28T21:37:27.380 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/30 file after the snapshot 2023-11-28T21:37:27.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/31 file after the snapshot 2023-11-28T21:37:27.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/32 file after the snapshot 2023-11-28T21:37:27.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/33 file after the snapshot 2023-11-28T21:37:27.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/34 file after the snapshot 2023-11-28T21:37:27.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/35 file after the snapshot 2023-11-28T21:37:27.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/36 file after the snapshot 2023-11-28T21:37:27.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/37 file after the snapshot 2023-11-28T21:37:27.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/38 file after the snapshot 2023-11-28T21:37:27.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 174/39 file after the snapshot 2023-11-28T21:37:27.384 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/.snap/snap-subdir-test 2023-11-28T21:37:27.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f8264817640 1 Processor -- start 2023-11-28T21:37:30.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f8264817640 1 -- start start 2023-11-28T21:37:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f8264817640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8260151af0 0x7f82600c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f8264817640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8260157b70 con 0x7f826006e300 2023-11-28T21:37:30.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f8264817640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8260157e50 con 0x7f8260151ed0 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f8264817640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82600707f0 con 0x7f8260151af0 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f825e59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8260151af0 0x7f82600c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f825e59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8260151af0 0x7f82600c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34290/0 (socket says 172.21.15.12:34290) 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f825e59c640 1 -- 172.21.15.12:0/3822289904 learned_addr learned my addr 172.21.15.12:0/3822289904 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f825e59c640 1 -- 172.21.15.12:0/3822289904 >> v1:172.21.15.144:6789/0 conn(0x7f8260151ed0 legacy=0x7f82600c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.434+0000 7f825e59c640 1 -- 172.21.15.12:0/3822289904 >> v1:172.21.15.12:6789/0 conn(0x7f826006e300 legacy=0x7f8260072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.438+0000 7f825dd9b640 1 -- 172.21.15.12:0/3822289904 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518162050 0 0) 0x7f8260157b70 con 0x7f826006e300 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.438+0000 7f825e59c640 1 -- 172.21.15.12:0/3822289904 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8260158130 con 0x7f8260151af0 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.438+0000 7f825e59c640 1 --2- 172.21.15.12:0/3822289904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8260151af0 0x7f82600c7650 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8250009870 tx=0x7f825001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfe6b051c51902b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f825dd9b640 1 -- 172.21.15.12:0/3822289904 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500ab600 con 0x7f8260151af0 2023-11-28T21:37:30.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f825dd9b640 1 -- 172.21.15.12:0/3822289904 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82500a7050 con 0x7f8260151af0 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f825dd9b640 1 -- 172.21.15.12:0/3822289904 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500ab980 con 0x7f8260151af0 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f8264817640 1 -- 172.21.15.12:0/3822289904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8260151af0 msgr2=0x7f82600c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f8264817640 1 --2- 172.21.15.12:0/3822289904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8260151af0 0x7f82600c7650 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8250009870 tx=0x7f825001c540 comp rx=0 tx=0).stop 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f8264817640 1 -- 172.21.15.12:0/3822289904 shutdown_connections 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f8264817640 1 --2- 172.21.15.12:0/3822289904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8260151af0 0x7f82600c7650 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f8264817640 1 -- 172.21.15.12:0/3822289904 >> 172.21.15.12:0/3822289904 conn(0x7f8260066360 msgr2=0x7f8260066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f8264817640 1 -- 172.21.15.12:0/3822289904 shutdown_connections 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f8264817640 1 -- 172.21.15.12:0/3822289904 wait complete. 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f8264817640 1 Processor -- start 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.442+0000 7f8264817640 1 -- start start 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8264817640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f826006e300 0x7f826014d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8264817640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8260151af0 0x7f8260148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:30.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8264817640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8260143550 0x7f8260140a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8264817640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600c4850 con 0x7f826006e300 2023-11-28T21:37:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8264817640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82601f70d0 con 0x7f8260151af0 2023-11-28T21:37:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8264817640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82601f6350 con 0x7f8260143550 2023-11-28T21:37:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f825e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f826006e300 0x7f826014d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f825e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f826006e300 0x7f826014d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34302/0 (socket says 172.21.15.12:34302) 2023-11-28T21:37:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f825e59c640 1 -- 172.21.15.12:0/1313049991 learned_addr learned my addr 172.21.15.12:0/1313049991 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f825e59c640 1 -- 172.21.15.12:0/1313049991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8260143550 msgr2=0x7f8260140a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f825e59c640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8260143550 0x7f8260140a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f825e59c640 1 -- 172.21.15.12:0/1313049991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8260151af0 msgr2=0x7f8260148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:30.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f825e59c640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8260151af0 0x7f8260148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:30.725 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/30 file after the snapshot 2023-11-28T21:37:30.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f825e59c640 1 -- 172.21.15.12:0/1313049991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82500094d0 con 0x7f826006e300 2023-11-28T21:37:30.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f825e59c640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f826006e300 0x7f826014d720 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8250019150 tx=0x7f82500192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:30.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8256ffd640 1 -- 172.21.15.12:0/1313049991 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500b4420 con 0x7f826006e300 2023-11-28T21:37:30.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8256ffd640 1 -- 172.21.15.12:0/1313049991 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82500b4580 con 0x7f826006e300 2023-11-28T21:37:30.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8264817640 1 -- 172.21.15.12:0/1313049991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82600c4570 con 0x7f826006e300 2023-11-28T21:37:30.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8256ffd640 1 -- 172.21.15.12:0/1313049991 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500b48f0 con 0x7f826006e300 2023-11-28T21:37:30.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8264817640 1 -- 172.21.15.12:0/1313049991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82601f68d0 con 0x7f826006e300 2023-11-28T21:37:30.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8264817640 1 -- 172.21.15.12:0/1313049991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8260074f60 con 0x7f826006e300 2023-11-28T21:37:30.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.446+0000 7f8256ffd640 1 -- 172.21.15.12:0/1313049991 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f82500b2070 con 0x7f826006e300 2023-11-28T21:37:30.728 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/31 file after the snapshot 2023-11-28T21:37:30.728 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/32 file after the snapshot 2023-11-28T21:37:30.728 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/33 file after the snapshot 2023-11-28T21:37:30.728 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/34 file after the snapshot 2023-11-28T21:37:30.728 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/35 file after the snapshot 2023-11-28T21:37:30.728 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/36 file after the snapshot 2023-11-28T21:37:30.728 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/37 file after the snapshot 2023-11-28T21:37:30.729 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/38 file after the snapshot 2023-11-28T21:37:30.729 INFO:tasks.workunit.client.0.smithi012.stdout: create 175/39 file after the snapshot 2023-11-28T21:37:30.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.450+0000 7f8256ffd640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8240066470 0x7f8240068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:30.729 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/.snap/snap-subdir-test 2023-11-28T21:37:30.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.450+0000 7f8256ffd640 1 -- 172.21.15.12:0/1313049991 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f825011dc50 con 0x7f826006e300 2023-11-28T21:37:30.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.450+0000 7f8257fff640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8240066470 0x7f8240068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:30.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.450+0000 7f8257fff640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8240066470 0x7f8240068930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8248000f70 tx=0x7f82480079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:30.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.454+0000 7f8256ffd640 1 -- 172.21.15.12:0/1313049991 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82500eac40 con 0x7f826006e300 2023-11-28T21:37:30.733 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/30 file after the snapshot 2023-11-28T21:37:30.733 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/31 file after the snapshot 2023-11-28T21:37:30.733 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/32 file after the snapshot 2023-11-28T21:37:30.733 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/33 file after the snapshot 2023-11-28T21:37:30.733 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/34 file after the snapshot 2023-11-28T21:37:30.733 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/35 file after the snapshot 2023-11-28T21:37:30.733 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/36 file after the snapshot 2023-11-28T21:37:30.733 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/37 file after the snapshot 2023-11-28T21:37:30.733 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/38 file after the snapshot 2023-11-28T21:37:30.733 INFO:tasks.workunit.client.0.smithi012.stdout: create 176/39 file after the snapshot 2023-11-28T21:37:30.734 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/.snap/snap-subdir-test 2023-11-28T21:37:30.734 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/30 file after the snapshot 2023-11-28T21:37:30.734 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/31 file after the snapshot 2023-11-28T21:37:30.734 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/32 file after the snapshot 2023-11-28T21:37:30.734 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/33 file after the snapshot 2023-11-28T21:37:30.734 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/34 file after the snapshot 2023-11-28T21:37:30.734 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/35 file after the snapshot 2023-11-28T21:37:30.734 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/36 file after the snapshot 2023-11-28T21:37:30.734 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/37 file after the snapshot 2023-11-28T21:37:30.734 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/38 file after the snapshot 2023-11-28T21:37:30.735 INFO:tasks.workunit.client.0.smithi012.stdout: create 177/39 file after the snapshot 2023-11-28T21:37:30.735 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/.snap/snap-subdir-test 2023-11-28T21:37:30.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 -- 172.21.15.12:0/1313049991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8260072c10 con 0x7f826006e300 2023-11-28T21:37:30.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8256ffd640 1 -- 172.21.15.12:0/1313049991 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7f82500ee260 con 0x7f826006e300 2023-11-28T21:37:30.735 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:30.736 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 -- 172.21.15.12:0/1313049991 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8240066470 msgr2=0x7f8240068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8240066470 0x7f8240068930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8248000f70 tx=0x7f82480079c0 comp rx=0 tx=0).stop 2023-11-28T21:37:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 -- 172.21.15.12:0/1313049991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f826006e300 msgr2=0x7f826014d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f826006e300 0x7f826014d720 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8250019150 tx=0x7f82500192e0 comp rx=0 tx=0).stop 2023-11-28T21:37:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 -- 172.21.15.12:0/1313049991 shutdown_connections 2023-11-28T21:37:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8240066470 0x7f8240068930 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8260143550 0x7f8260140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8260151af0 0x7f8260148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:30.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 --2- 172.21.15.12:0/1313049991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f826006e300 0x7f826014d720 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.654+0000 7f8264817640 1 -- 172.21.15.12:0/1313049991 >> 172.21.15.12:0/1313049991 conn(0x7f8260066360 msgr2=0x7f826014f540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.658+0000 7f8264817640 1 -- 172.21.15.12:0/1313049991 shutdown_connections 2023-11-28T21:37:30.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:27.658+0000 7f8264817640 1 -- 172.21.15.12:0/1313049991 wait complete. 2023-11-28T21:37:30.739 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-11-28T21:37:30.740 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 14, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 14, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.12:6838/949980653", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 949980653, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 949980653, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4247, "incarnation": 12, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:stopping", "state_seq": 13 }, "gid_4250": { "addr": "172.21.15.12:6839/3027655738", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3027655738, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3027655738, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4250, "incarnation": 8, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 11 }, "gid_4261": { "addr": "172.21.15.144:6833/3053317078", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 3053317078, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 3053317078, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4261, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:37:03.325643+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4261, "mds_1": 4250, "mds_2": 4247 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:37:30.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/30 file after the snapshot 2023-11-28T21:37:30.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/31 file after the snapshot 2023-11-28T21:37:30.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/32 file after the snapshot 2023-11-28T21:37:30.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/33 file after the snapshot 2023-11-28T21:37:30.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/34 file after the snapshot 2023-11-28T21:37:30.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/35 file after the snapshot 2023-11-28T21:37:30.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/36 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/37 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/38 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 178/39 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/.snap/snap-subdir-test 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/30 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/31 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/32 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/33 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/34 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/35 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/36 file after the snapshot 2023-11-28T21:37:30.756 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/37 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/38 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 179/39 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/.snap/snap-subdir-test 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/30 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/31 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/32 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/33 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/34 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/35 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/36 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/37 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/38 file after the snapshot 2023-11-28T21:37:30.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 180/39 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/.snap/snap-subdir-test 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/30 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/31 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/32 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/33 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/34 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/35 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/36 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/37 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/38 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 181/39 file after the snapshot 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/.snap/snap-subdir-test 2023-11-28T21:37:30.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/30 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/31 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/32 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/33 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/34 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/35 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/36 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/37 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/38 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 182/39 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/.snap/snap-subdir-test 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/30 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/31 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/32 file after the snapshot 2023-11-28T21:37:30.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/33 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/34 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/35 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/36 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/37 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/38 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 183/39 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/.snap/snap-subdir-test 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/30 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/31 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/32 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/33 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/34 file after the snapshot 2023-11-28T21:37:30.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/35 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/36 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/37 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/38 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 184/39 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/.snap/snap-subdir-test 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/30 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/31 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/32 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/33 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/34 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/35 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/36 file after the snapshot 2023-11-28T21:37:30.761 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/37 file after the snapshot 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/38 file after the snapshot 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 185/39 file after the snapshot 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/.snap/snap-subdir-test 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/30 file after the snapshot 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/31 file after the snapshot 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/32 file after the snapshot 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/33 file after the snapshot 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/34 file after the snapshot 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/35 file after the snapshot 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/36 file after the snapshot 2023-11-28T21:37:30.762 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/37 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/38 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 186/39 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/.snap/snap-subdir-test 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/30 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/31 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/32 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/33 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/34 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/35 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/36 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/37 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/38 file after the snapshot 2023-11-28T21:37:30.763 INFO:tasks.workunit.client.0.smithi012.stdout: create 187/39 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/.snap/snap-subdir-test 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/30 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/31 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/32 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/33 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/34 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/35 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/36 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/37 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/38 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 188/39 file after the snapshot 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/.snap/snap-subdir-test 2023-11-28T21:37:30.764 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/30 file after the snapshot 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/31 file after the snapshot 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/32 file after the snapshot 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/33 file after the snapshot 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/34 file after the snapshot 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/35 file after the snapshot 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/36 file after the snapshot 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/37 file after the snapshot 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/38 file after the snapshot 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 189/39 file after the snapshot 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/.snap/snap-subdir-test 2023-11-28T21:37:30.765 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/30 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/31 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/32 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/33 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/34 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/35 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/36 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/37 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/38 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 190/39 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/.snap/snap-subdir-test 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/30 file after the snapshot 2023-11-28T21:37:30.766 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/31 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/32 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/33 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/34 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/35 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/36 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/37 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/38 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 191/39 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/.snap/snap-subdir-test 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/30 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/31 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/32 file after the snapshot 2023-11-28T21:37:30.767 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/33 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/34 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/35 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/36 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/37 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/38 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 192/39 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/.snap/snap-subdir-test 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/30 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/31 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/32 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/33 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/34 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/35 file after the snapshot 2023-11-28T21:37:30.768 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/36 file after the snapshot 2023-11-28T21:37:30.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/37 file after the snapshot 2023-11-28T21:37:30.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/38 file after the snapshot 2023-11-28T21:37:30.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 193/39 file after the snapshot 2023-11-28T21:37:30.769 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/.snap/snap-subdir-test 2023-11-28T21:37:30.793 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/30 file after the snapshot 2023-11-28T21:37:30.797 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/31 file after the snapshot 2023-11-28T21:37:30.797 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/32 file after the snapshot 2023-11-28T21:37:30.797 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/33 file after the snapshot 2023-11-28T21:37:30.797 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/34 file after the snapshot 2023-11-28T21:37:30.797 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/35 file after the snapshot 2023-11-28T21:37:30.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/36 file after the snapshot 2023-11-28T21:37:30.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/37 file after the snapshot 2023-11-28T21:37:30.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/38 file after the snapshot 2023-11-28T21:37:30.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 194/39 file after the snapshot 2023-11-28T21:37:30.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/.snap/snap-subdir-test 2023-11-28T21:37:30.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/30 file after the snapshot 2023-11-28T21:37:30.874 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/31 file after the snapshot 2023-11-28T21:37:30.874 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/32 file after the snapshot 2023-11-28T21:37:30.874 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/33 file after the snapshot 2023-11-28T21:37:30.874 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/34 file after the snapshot 2023-11-28T21:37:30.874 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/35 file after the snapshot 2023-11-28T21:37:30.874 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/36 file after the snapshot 2023-11-28T21:37:30.874 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/37 file after the snapshot 2023-11-28T21:37:30.874 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/38 file after the snapshot 2023-11-28T21:37:30.875 INFO:tasks.workunit.client.0.smithi012.stdout: create 195/39 file after the snapshot 2023-11-28T21:37:30.875 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/.snap/snap-subdir-test 2023-11-28T21:37:30.959 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/30 file after the snapshot 2023-11-28T21:37:30.963 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/31 file after the snapshot 2023-11-28T21:37:30.964 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/32 file after the snapshot 2023-11-28T21:37:30.964 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/33 file after the snapshot 2023-11-28T21:37:30.965 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/34 file after the snapshot 2023-11-28T21:37:30.965 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/35 file after the snapshot 2023-11-28T21:37:30.965 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/36 file after the snapshot 2023-11-28T21:37:30.965 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/37 file after the snapshot 2023-11-28T21:37:30.966 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/38 file after the snapshot 2023-11-28T21:37:30.966 INFO:tasks.workunit.client.0.smithi012.stdout: create 196/39 file after the snapshot 2023-11-28T21:37:30.966 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/.snap/snap-subdir-test 2023-11-28T21:37:31.318 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/30 file after the snapshot 2023-11-28T21:37:31.328 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/31 file after the snapshot 2023-11-28T21:37:31.328 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/32 file after the snapshot 2023-11-28T21:37:31.328 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/33 file after the snapshot 2023-11-28T21:37:31.328 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/34 file after the snapshot 2023-11-28T21:37:31.328 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/35 file after the snapshot 2023-11-28T21:37:31.329 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/36 file after the snapshot 2023-11-28T21:37:31.329 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/37 file after the snapshot 2023-11-28T21:37:31.329 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/38 file after the snapshot 2023-11-28T21:37:31.329 INFO:tasks.workunit.client.0.smithi012.stdout: create 197/39 file after the snapshot 2023-11-28T21:37:31.329 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/.snap/snap-subdir-test 2023-11-28T21:37:31.383 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/30 file after the snapshot 2023-11-28T21:37:31.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/31 file after the snapshot 2023-11-28T21:37:31.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/32 file after the snapshot 2023-11-28T21:37:31.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/33 file after the snapshot 2023-11-28T21:37:31.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/34 file after the snapshot 2023-11-28T21:37:31.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/35 file after the snapshot 2023-11-28T21:37:31.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/36 file after the snapshot 2023-11-28T21:37:31.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/37 file after the snapshot 2023-11-28T21:37:31.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/38 file after the snapshot 2023-11-28T21:37:31.386 INFO:tasks.workunit.client.0.smithi012.stdout: create 198/39 file after the snapshot 2023-11-28T21:37:31.387 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/.snap/snap-subdir-test 2023-11-28T21:37:31.445 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/30 file after the snapshot 2023-11-28T21:37:31.448 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/31 file after the snapshot 2023-11-28T21:37:31.448 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/32 file after the snapshot 2023-11-28T21:37:31.448 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/33 file after the snapshot 2023-11-28T21:37:31.448 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/34 file after the snapshot 2023-11-28T21:37:31.448 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/35 file after the snapshot 2023-11-28T21:37:31.448 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/36 file after the snapshot 2023-11-28T21:37:31.448 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/37 file after the snapshot 2023-11-28T21:37:31.448 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/38 file after the snapshot 2023-11-28T21:37:31.448 INFO:tasks.workunit.client.0.smithi012.stdout: create 199/39 file after the snapshot 2023-11-28T21:37:31.449 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/.snap/snap-subdir-test 2023-11-28T21:37:31.505 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/30 file after the snapshot 2023-11-28T21:37:31.508 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/31 file after the snapshot 2023-11-28T21:37:31.508 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/32 file after the snapshot 2023-11-28T21:37:31.508 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/33 file after the snapshot 2023-11-28T21:37:31.509 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/34 file after the snapshot 2023-11-28T21:37:31.509 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/35 file after the snapshot 2023-11-28T21:37:31.509 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/36 file after the snapshot 2023-11-28T21:37:31.509 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/37 file after the snapshot 2023-11-28T21:37:31.509 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/38 file after the snapshot 2023-11-28T21:37:31.509 INFO:tasks.workunit.client.0.smithi012.stdout: create 200/39 file after the snapshot 2023-11-28T21:37:31.509 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/.snap/snap-subdir-test 2023-11-28T21:37:31.559 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/30 file after the snapshot 2023-11-28T21:37:31.562 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/31 file after the snapshot 2023-11-28T21:37:31.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/32 file after the snapshot 2023-11-28T21:37:31.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/33 file after the snapshot 2023-11-28T21:37:31.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/34 file after the snapshot 2023-11-28T21:37:31.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/35 file after the snapshot 2023-11-28T21:37:31.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/36 file after the snapshot 2023-11-28T21:37:31.563 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/37 file after the snapshot 2023-11-28T21:37:31.564 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/38 file after the snapshot 2023-11-28T21:37:31.564 INFO:tasks.workunit.client.0.smithi012.stdout: create 201/39 file after the snapshot 2023-11-28T21:37:31.564 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/.snap/snap-subdir-test 2023-11-28T21:37:31.624 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/30 file after the snapshot 2023-11-28T21:37:31.627 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/31 file after the snapshot 2023-11-28T21:37:31.627 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/32 file after the snapshot 2023-11-28T21:37:31.627 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/33 file after the snapshot 2023-11-28T21:37:31.627 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/34 file after the snapshot 2023-11-28T21:37:31.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/35 file after the snapshot 2023-11-28T21:37:31.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/36 file after the snapshot 2023-11-28T21:37:31.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/37 file after the snapshot 2023-11-28T21:37:31.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/38 file after the snapshot 2023-11-28T21:37:31.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 202/39 file after the snapshot 2023-11-28T21:37:31.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/.snap/snap-subdir-test 2023-11-28T21:37:31.930 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/30 file after the snapshot 2023-11-28T21:37:31.933 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/31 file after the snapshot 2023-11-28T21:37:31.933 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/32 file after the snapshot 2023-11-28T21:37:31.933 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/33 file after the snapshot 2023-11-28T21:37:31.933 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/34 file after the snapshot 2023-11-28T21:37:31.933 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/35 file after the snapshot 2023-11-28T21:37:31.934 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/36 file after the snapshot 2023-11-28T21:37:31.934 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/37 file after the snapshot 2023-11-28T21:37:31.934 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/38 file after the snapshot 2023-11-28T21:37:31.935 INFO:tasks.workunit.client.0.smithi012.stdout: create 203/39 file after the snapshot 2023-11-28T21:37:31.935 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/.snap/snap-subdir-test 2023-11-28T21:37:31.993 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/30 file after the snapshot 2023-11-28T21:37:31.996 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/31 file after the snapshot 2023-11-28T21:37:31.996 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/32 file after the snapshot 2023-11-28T21:37:31.996 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/33 file after the snapshot 2023-11-28T21:37:31.997 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/34 file after the snapshot 2023-11-28T21:37:31.997 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/35 file after the snapshot 2023-11-28T21:37:31.997 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/36 file after the snapshot 2023-11-28T21:37:31.997 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/37 file after the snapshot 2023-11-28T21:37:31.997 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/38 file after the snapshot 2023-11-28T21:37:31.997 INFO:tasks.workunit.client.0.smithi012.stdout: create 204/39 file after the snapshot 2023-11-28T21:37:31.997 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/.snap/snap-subdir-test 2023-11-28T21:37:32.071 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/30 file after the snapshot 2023-11-28T21:37:32.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/31 file after the snapshot 2023-11-28T21:37:32.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/32 file after the snapshot 2023-11-28T21:37:32.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/33 file after the snapshot 2023-11-28T21:37:32.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/34 file after the snapshot 2023-11-28T21:37:32.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/35 file after the snapshot 2023-11-28T21:37:32.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/36 file after the snapshot 2023-11-28T21:37:32.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/37 file after the snapshot 2023-11-28T21:37:32.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/38 file after the snapshot 2023-11-28T21:37:32.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 205/39 file after the snapshot 2023-11-28T21:37:32.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/.snap/snap-subdir-test 2023-11-28T21:37:32.130 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/30 file after the snapshot 2023-11-28T21:37:32.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/31 file after the snapshot 2023-11-28T21:37:32.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/32 file after the snapshot 2023-11-28T21:37:32.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/33 file after the snapshot 2023-11-28T21:37:32.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/34 file after the snapshot 2023-11-28T21:37:32.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/35 file after the snapshot 2023-11-28T21:37:32.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/36 file after the snapshot 2023-11-28T21:37:32.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/37 file after the snapshot 2023-11-28T21:37:32.134 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/38 file after the snapshot 2023-11-28T21:37:32.134 INFO:tasks.workunit.client.0.smithi012.stdout: create 206/39 file after the snapshot 2023-11-28T21:37:32.134 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/.snap/snap-subdir-test 2023-11-28T21:37:32.190 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/30 file after the snapshot 2023-11-28T21:37:32.193 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/31 file after the snapshot 2023-11-28T21:37:32.193 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/32 file after the snapshot 2023-11-28T21:37:32.194 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/33 file after the snapshot 2023-11-28T21:37:32.194 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/34 file after the snapshot 2023-11-28T21:37:32.194 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/35 file after the snapshot 2023-11-28T21:37:32.194 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/36 file after the snapshot 2023-11-28T21:37:32.194 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/37 file after the snapshot 2023-11-28T21:37:32.194 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/38 file after the snapshot 2023-11-28T21:37:32.194 INFO:tasks.workunit.client.0.smithi012.stdout: create 207/39 file after the snapshot 2023-11-28T21:37:32.194 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/.snap/snap-subdir-test 2023-11-28T21:37:32.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:32.868 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/30 file after the snapshot 2023-11-28T21:37:32.872 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/31 file after the snapshot 2023-11-28T21:37:32.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/32 file after the snapshot 2023-11-28T21:37:32.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/33 file after the snapshot 2023-11-28T21:37:32.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/34 file after the snapshot 2023-11-28T21:37:32.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/35 file after the snapshot 2023-11-28T21:37:32.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/36 file after the snapshot 2023-11-28T21:37:32.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/37 file after the snapshot 2023-11-28T21:37:32.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/38 file after the snapshot 2023-11-28T21:37:32.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 208/39 file after the snapshot 2023-11-28T21:37:32.873 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/.snap/snap-subdir-test 2023-11-28T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.894+0000 7f5ed249e640 1 Processor -- start 2023-11-28T21:37:32.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.894+0000 7f5ed249e640 1 -- start start 2023-11-28T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ed249e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5ecc06e300 0x7f5ecc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ed249e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ecc157b70 con 0x7f5ecc151af0 2023-11-28T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ed249e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ecc157e50 con 0x7f5ecc151ed0 2023-11-28T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ed249e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ecc0707f0 con 0x7f5ecc06e300 2023-11-28T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ecbfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5ecc151af0 0x7f5ecc0c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:32814/0 (socket says 172.21.15.12:32814) 2023-11-28T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ecbfff640 1 -- 172.21.15.12:0/2364220315 learned_addr learned my addr 172.21.15.12:0/2364220315 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ed0a3b640 1 --2- 172.21.15.12:0/2364220315 >> v2:172.21.15.12:3300/0 conn(0x7f5ecc06e300 0x7f5ecc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ecaffd640 1 -- 172.21.15.12:0/2364220315 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214418757 0 0) 0x7f5ecc157b70 con 0x7f5ecc151af0 2023-11-28T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ed0a3b640 1 -- 172.21.15.12:0/2364220315 >> v1:172.21.15.144:6789/0 conn(0x7f5ecc151ed0 legacy=0x7f5ecc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ed0a3b640 1 -- 172.21.15.12:0/2364220315 >> v1:172.21.15.12:6789/0 conn(0x7f5ecc151af0 legacy=0x7f5ecc0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:32.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.898+0000 7f5ed0a3b640 1 -- 172.21.15.12:0/2364220315 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc158130 con 0x7f5ecc06e300 2023-11-28T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed0a3b640 1 --2- 172.21.15.12:0/2364220315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 0x7f5ecc0c7650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009d90 tx=0x7f5ec401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9f36a915020bad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ecaffd640 1 -- 172.21.15.12:0/2364220315 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40a5810 con 0x7f5ecc06e300 2023-11-28T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ecaffd640 1 -- 172.21.15.12:0/2364220315 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec40a5970 con 0x7f5ecc06e300 2023-11-28T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ecaffd640 1 -- 172.21.15.12:0/2364220315 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40a5d00 con 0x7f5ecc06e300 2023-11-28T21:37:32.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 -- 172.21.15.12:0/2364220315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 msgr2=0x7f5ecc0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 --2- 172.21.15.12:0/2364220315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 0x7f5ecc0c7650 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009d90 tx=0x7f5ec401c950 comp rx=0 tx=0).stop 2023-11-28T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 -- 172.21.15.12:0/2364220315 shutdown_connections 2023-11-28T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 --2- 172.21.15.12:0/2364220315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 0x7f5ecc0c7650 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 -- 172.21.15.12:0/2364220315 >> 172.21.15.12:0/2364220315 conn(0x7f5ecc066360 msgr2=0x7f5ecc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 -- 172.21.15.12:0/2364220315 shutdown_connections 2023-11-28T21:37:32.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 -- 172.21.15.12:0/2364220315 wait complete. 2023-11-28T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 Processor -- start 2023-11-28T21:37:32.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 -- start start 2023-11-28T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 0x7f5ecc06d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:32.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ecc151af0 0x7f5ecc146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1496d0 0x7f5ecc146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc145da0 con 0x7f5ecc06e300 2023-11-28T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0c93f0 con 0x7f5ecc1496d0 2023-11-28T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.902+0000 7f5ed249e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0ca1c0 con 0x7f5ecc151af0 2023-11-28T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 0x7f5ecc06d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 0x7f5ecc06d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34328/0 (socket says 172.21.15.12:34328) 2023-11-28T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ecbfff640 1 -- 172.21.15.12:0/3315627587 learned_addr learned my addr 172.21.15.12:0/3315627587 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ed0a3b640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1496d0 0x7f5ecc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:32.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ecb7fe640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ecc151af0 0x7f5ecc146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:32.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ecbfff640 1 -- 172.21.15.12:0/3315627587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ecc151af0 msgr2=0x7f5ecc146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ecbfff640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ecc151af0 0x7f5ecc146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ecbfff640 1 -- 172.21.15.12:0/3315627587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1496d0 msgr2=0x7f5ecc146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ecbfff640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1496d0 0x7f5ecc146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ecbfff640 1 -- 172.21.15.12:0/3315627587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec4009a70 con 0x7f5ecc06e300 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ecbfff640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 0x7f5ecc06d980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c4a0 tx=0x7f5ebc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ec8ff9640 1 -- 172.21.15.12:0/3315627587 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0074e0 con 0x7f5ecc06e300 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ec8ff9640 1 -- 172.21.15.12:0/3315627587 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc007640 con 0x7f5ecc06e300 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ec8ff9640 1 -- 172.21.15.12:0/3315627587 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc004a30 con 0x7f5ecc06e300 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ed249e640 1 -- 172.21.15.12:0/3315627587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc0c9640 con 0x7f5ecc06e300 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ed249e640 1 -- 172.21.15.12:0/3315627587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc0c9c00 con 0x7f5ecc06e300 2023-11-28T21:37:32.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ed249e640 1 -- 172.21.15.12:0/3315627587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc074f60 con 0x7f5ecc06e300 2023-11-28T21:37:32.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.906+0000 7f5ec8ff9640 1 -- 172.21.15.12:0/3315627587 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5ebc004b90 con 0x7f5ecc06e300 2023-11-28T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.910+0000 7f5ec8ff9640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5eb8066470 0x7f5eb8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.910+0000 7f5ec8ff9640 1 -- 172.21.15.12:0/3315627587 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f5ebc083360 con 0x7f5ecc06e300 2023-11-28T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.910+0000 7f5ecb7fe640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5eb8066470 0x7f5eb8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:32.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.910+0000 7f5ecb7fe640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5eb8066470 0x7f5eb8068930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc145c00 tx=0x7f5ec0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:32.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:32.914+0000 7f5ec8ff9640 1 -- 172.21.15.12:0/3315627587 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ebc050530 con 0x7f5ecc06e300 2023-11-28T21:37:32.948 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/30 file after the snapshot 2023-11-28T21:37:32.953 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/31 file after the snapshot 2023-11-28T21:37:32.954 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/32 file after the snapshot 2023-11-28T21:37:32.954 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/33 file after the snapshot 2023-11-28T21:37:32.954 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/34 file after the snapshot 2023-11-28T21:37:32.954 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/35 file after the snapshot 2023-11-28T21:37:32.954 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/36 file after the snapshot 2023-11-28T21:37:32.954 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/37 file after the snapshot 2023-11-28T21:37:32.955 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/38 file after the snapshot 2023-11-28T21:37:32.955 INFO:tasks.workunit.client.0.smithi012.stdout: create 209/39 file after the snapshot 2023-11-28T21:37:32.955 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/.snap/snap-subdir-test 2023-11-28T21:37:33.023 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/30 file after the snapshot 2023-11-28T21:37:33.026 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/31 file after the snapshot 2023-11-28T21:37:33.026 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/32 file after the snapshot 2023-11-28T21:37:33.027 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/33 file after the snapshot 2023-11-28T21:37:33.027 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/34 file after the snapshot 2023-11-28T21:37:33.027 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/35 file after the snapshot 2023-11-28T21:37:33.027 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/36 file after the snapshot 2023-11-28T21:37:33.027 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/37 file after the snapshot 2023-11-28T21:37:33.027 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/38 file after the snapshot 2023-11-28T21:37:33.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 210/39 file after the snapshot 2023-11-28T21:37:33.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/.snap/snap-subdir-test 2023-11-28T21:37:33.080 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/30 file after the snapshot 2023-11-28T21:37:33.083 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/31 file after the snapshot 2023-11-28T21:37:33.083 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/32 file after the snapshot 2023-11-28T21:37:33.083 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/33 file after the snapshot 2023-11-28T21:37:33.083 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/34 file after the snapshot 2023-11-28T21:37:33.083 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/35 file after the snapshot 2023-11-28T21:37:33.083 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/36 file after the snapshot 2023-11-28T21:37:33.083 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/37 file after the snapshot 2023-11-28T21:37:33.083 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/38 file after the snapshot 2023-11-28T21:37:33.084 INFO:tasks.workunit.client.0.smithi012.stdout: create 211/39 file after the snapshot 2023-11-28T21:37:33.084 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/.snap/snap-subdir-test 2023-11-28T21:37:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.114+0000 7f5ed249e640 1 -- 172.21.15.12:0/3315627587 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ecc072c10 con 0x7f5ecc06e300 2023-11-28T21:37:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.114+0000 7f5ec8ff9640 1 -- 172.21.15.12:0/3315627587 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6531 (secure 0 0 0) 0x7f5ebc053b50 con 0x7f5ecc06e300 2023-11-28T21:37:33.121 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:33.121 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:03.325643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1,2],"up":{"mds_0":4261,"mds_1":4250,"mds_2":4247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"a","rank":2,"incarnation":12,"state":"up:stopping","state_seq":13,"addr":"172.21.15.12:6838/949980653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":949980653},{"type":"v1","addr":"172.21.15.12:6838","nonce":949980653}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:active","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 14 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 -- 172.21.15.12:0/3315627587 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5eb8066470 msgr2=0x7f5eb8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5eb8066470 0x7f5eb8068930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc145c00 tx=0x7f5ec0009040 comp rx=0 tx=0).stop 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 -- 172.21.15.12:0/3315627587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 msgr2=0x7f5ecc06d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 0x7f5ecc06d980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c4a0 tx=0x7f5ebc00c960 comp rx=0 tx=0).stop 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 -- 172.21.15.12:0/3315627587 shutdown_connections 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5eb8066470 0x7f5eb8068930 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc1496d0 0x7f5ecc146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ecc151af0 0x7f5ecc146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 --2- 172.21.15.12:0/3315627587 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ecc06e300 0x7f5ecc06d980 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 -- 172.21.15.12:0/3315627587 >> 172.21.15.12:0/3315627587 conn(0x7f5ecc066360 msgr2=0x7f5ecc14bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 -- 172.21.15.12:0/3315627587 shutdown_connections 2023-11-28T21:37:33.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:33.118+0000 7f5ed249e640 1 -- 172.21.15.12:0/3315627587 wait complete. 2023-11-28T21:37:33.130 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/30 file after the snapshot 2023-11-28T21:37:33.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/31 file after the snapshot 2023-11-28T21:37:33.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/32 file after the snapshot 2023-11-28T21:37:33.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/33 file after the snapshot 2023-11-28T21:37:33.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/34 file after the snapshot 2023-11-28T21:37:33.133 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/35 file after the snapshot 2023-11-28T21:37:33.134 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/36 file after the snapshot 2023-11-28T21:37:33.134 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/37 file after the snapshot 2023-11-28T21:37:33.134 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/38 file after the snapshot 2023-11-28T21:37:33.134 INFO:tasks.workunit.client.0.smithi012.stdout: create 212/39 file after the snapshot 2023-11-28T21:37:33.134 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/.snap/snap-subdir-test 2023-11-28T21:37:33.143 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:33.533 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-11-28T21:37:33.972 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/30 file after the snapshot 2023-11-28T21:37:33.975 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/31 file after the snapshot 2023-11-28T21:37:33.975 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/32 file after the snapshot 2023-11-28T21:37:33.975 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/33 file after the snapshot 2023-11-28T21:37:33.975 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/34 file after the snapshot 2023-11-28T21:37:33.975 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/35 file after the snapshot 2023-11-28T21:37:33.975 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/36 file after the snapshot 2023-11-28T21:37:33.975 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/37 file after the snapshot 2023-11-28T21:37:33.975 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/38 file after the snapshot 2023-11-28T21:37:33.976 INFO:tasks.workunit.client.0.smithi012.stdout: create 213/39 file after the snapshot 2023-11-28T21:37:33.976 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/.snap/snap-subdir-test 2023-11-28T21:37:35.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:35.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.294+0000 7f9979357640 1 Processor -- start 2023-11-28T21:37:35.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.294+0000 7f9979357640 1 -- start start 2023-11-28T21:37:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.294+0000 7f9979357640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f99740712b0 0x7f99740716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.294+0000 7f9979357640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997406f050 con 0x7f9974143070 2023-11-28T21:37:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.294+0000 7f9979357640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997406f580 con 0x7f997415a910 2023-11-28T21:37:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.294+0000 7f9979357640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997415e330 con 0x7f99740712b0 2023-11-28T21:37:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.294+0000 7f9972ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9974143070 0x7f997415b340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33600/0 (socket says 172.21.15.12:33600) 2023-11-28T21:37:35.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.294+0000 7f9972ffd640 1 -- 172.21.15.12:0/3010671874 learned_addr learned my addr 172.21.15.12:0/3010671874 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.294+0000 7f99737fe640 1 --2- 172.21.15.12:0/3010671874 >> v2:172.21.15.12:3300/0 conn(0x7f99740712b0 0x7f99740716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f99727fc640 1 -- 172.21.15.12:0/3010671874 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683086060 0 0) 0x7f997406f050 con 0x7f9974143070 2023-11-28T21:37:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f99737fe640 1 -- 172.21.15.12:0/3010671874 >> v1:172.21.15.144:6789/0 conn(0x7f997415a910 legacy=0x7f9974070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f99737fe640 1 -- 172.21.15.12:0/3010671874 >> v1:172.21.15.12:6789/0 conn(0x7f9974143070 legacy=0x7f997415b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:35.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f99737fe640 1 -- 172.21.15.12:0/3010671874 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997406f2a0 con 0x7f99740712b0 2023-11-28T21:37:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f99737fe640 1 --2- 172.21.15.12:0/3010671874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99740712b0 0x7f99740716c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f996c009d90 tx=0x7f996c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e09a5221e4bca88 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f99727fc640 1 -- 172.21.15.12:0/3010671874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0a59e0 con 0x7f99740712b0 2023-11-28T21:37:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f99727fc640 1 -- 172.21.15.12:0/3010671874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c0a5b40 con 0x7f99740712b0 2023-11-28T21:37:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f99727fc640 1 -- 172.21.15.12:0/3010671874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0ad440 con 0x7f99740712b0 2023-11-28T21:37:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f9979357640 1 -- 172.21.15.12:0/3010671874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99740712b0 msgr2=0x7f99740716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f9979357640 1 --2- 172.21.15.12:0/3010671874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99740712b0 0x7f99740716c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f996c009d90 tx=0x7f996c01c920 comp rx=0 tx=0).stop 2023-11-28T21:37:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f9979357640 1 -- 172.21.15.12:0/3010671874 shutdown_connections 2023-11-28T21:37:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f9979357640 1 --2- 172.21.15.12:0/3010671874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f99740712b0 0x7f99740716c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.298+0000 7f9979357640 1 -- 172.21.15.12:0/3010671874 >> 172.21.15.12:0/3010671874 conn(0x7f997406b6c0 msgr2=0x7f997406bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 -- 172.21.15.12:0/3010671874 shutdown_connections 2023-11-28T21:37:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 -- 172.21.15.12:0/3010671874 wait complete. 2023-11-28T21:37:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 Processor -- start 2023-11-28T21:37:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 -- start start 2023-11-28T21:37:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99740712b0 0x7f9974155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9974143070 0x7f9974155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f997415a910 0x7f9974140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99741555a0 con 0x7f9974143070 2023-11-28T21:37:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99741404b0 con 0x7f99740712b0 2023-11-28T21:37:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997413af10 con 0x7f997415a910 2023-11-28T21:37:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9974143070 0x7f9974155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9974143070 0x7f9974155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42388/0 (socket says 172.21.15.12:42388) 2023-11-28T21:37:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996bfff640 1 -- 172.21.15.12:0/3959235173 learned_addr learned my addr 172.21.15.12:0/3959235173 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f99737fe640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f997415a910 0x7f9974140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9972ffd640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99740712b0 0x7f9974155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996bfff640 1 -- 172.21.15.12:0/3959235173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f997415a910 msgr2=0x7f9974140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996bfff640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f997415a910 0x7f9974140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996bfff640 1 -- 172.21.15.12:0/3959235173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99740712b0 msgr2=0x7f9974155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996bfff640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99740712b0 0x7f9974155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996bfff640 1 -- 172.21.15.12:0/3959235173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c009a70 con 0x7f9974143070 2023-11-28T21:37:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996bfff640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9974143070 0x7f9974155e80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c530 tx=0x7f995c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996b7fe640 1 -- 172.21.15.12:0/3959235173 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c005220 con 0x7f9974143070 2023-11-28T21:37:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996b7fe640 1 -- 172.21.15.12:0/3959235173 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c005380 con 0x7f9974143070 2023-11-28T21:37:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f9979357640 1 -- 172.21.15.12:0/3959235173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997413a950 con 0x7f9974143070 2023-11-28T21:37:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.302+0000 7f996b7fe640 1 -- 172.21.15.12:0/3959235173 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c007270 con 0x7f9974143070 2023-11-28T21:37:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.306+0000 7f9979357640 1 -- 172.21.15.12:0/3959235173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997413fe60 con 0x7f9974143070 2023-11-28T21:37:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.306+0000 7f9979357640 1 -- 172.21.15.12:0/3959235173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9974074f60 con 0x7f9974143070 2023-11-28T21:37:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.306+0000 7f996b7fe640 1 -- 172.21.15.12:0/3959235173 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f995c0073d0 con 0x7f9974143070 2023-11-28T21:37:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.306+0000 7f996b7fe640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 0x7f9954068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.306+0000 7f996b7fe640 1 -- 172.21.15.12:0/3959235173 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f995c083b80 con 0x7f9974143070 2023-11-28T21:37:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.306+0000 7f9972ffd640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 0x7f9954068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:35.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.306+0000 7f9972ffd640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 0x7f9954068930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9964007fd0 tx=0x7f9964007ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:35.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.310+0000 7f996b7fe640 1 -- 172.21.15.12:0/3959235173 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f995c050c00 con 0x7f9974143070 2023-11-28T21:37:35.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 -- 172.21.15.12:0/3959235173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9974157510 con 0x7f9974143070 2023-11-28T21:37:35.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f996b7fe640 1 -- 172.21.15.12:0/3959235173 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f995c054220 con 0x7f9974143070 2023-11-28T21:37:35.507 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:35.507 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:35.507 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:35.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 -- 172.21.15.12:0/3959235173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 msgr2=0x7f9954068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:35.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 0x7f9954068930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9964007fd0 tx=0x7f9964007ba0 comp rx=0 tx=0).stop 2023-11-28T21:37:35.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 -- 172.21.15.12:0/3959235173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9974143070 msgr2=0x7f9974155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:35.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9974143070 0x7f9974155e80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c530 tx=0x7f995c00c9f0 comp rx=0 tx=0).stop 2023-11-28T21:37:35.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 -- 172.21.15.12:0/3959235173 shutdown_connections 2023-11-28T21:37:35.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9954066470 0x7f9954068930 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:35.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f997415a910 0x7f9974140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:35.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9974143070 0x7f9974155e80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:35.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 --2- 172.21.15.12:0/3959235173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99740712b0 0x7f9974155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:35.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 -- 172.21.15.12:0/3959235173 >> 172.21.15.12:0/3959235173 conn(0x7f997406b6c0 msgr2=0x7f9974157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:35.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 -- 172.21.15.12:0/3959235173 shutdown_connections 2023-11-28T21:37:35.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:35.502+0000 7f9979357640 1 -- 172.21.15.12:0/3959235173 wait complete. 2023-11-28T21:37:35.528 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:36.888 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:37:36.892 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:37:37.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.674+0000 7fbc6e479640 1 Processor -- start 2023-11-28T21:37:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.674+0000 7fbc6e479640 1 -- start start 2023-11-28T21:37:37.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.674+0000 7fbc6e479640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc68156af0 0x7fbc6806d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:37.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.674+0000 7fbc6e479640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc680c6ce0 con 0x7fbc681500d0 2023-11-28T21:37:37.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.674+0000 7fbc6e479640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc68150540 con 0x7fbc680c27c0 2023-11-28T21:37:37.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.674+0000 7fbc6e479640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc680c7170 con 0x7fbc68156af0 2023-11-28T21:37:37.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.674+0000 7fbc677fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbc681500d0 0x7fbc6806cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33608/0 (socket says 172.21.15.12:33608) 2023-11-28T21:37:37.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.674+0000 7fbc677fe640 1 -- 172.21.15.12:0/2554273348 learned_addr learned my addr 172.21.15.12:0/2554273348 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:37.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc6ca16640 1 --2- 172.21.15.12:0/2554273348 >> v2:172.21.15.144:3300/0 conn(0x7fbc68156af0 0x7fbc6806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824304094 0 0) 0x7fbc680c6ce0 con 0x7fbc681500d0 2023-11-28T21:37:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc50005740 con 0x7fbc681500d0 2023-11-28T21:37:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440104799 0 0) 0x7fbc68150540 con 0x7fbc680c27c0 2023-11-28T21:37:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc680c6ce0 con 0x7fbc680c27c0 2023-11-28T21:37:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3309018271 0 0) 0x7fbc50005740 con 0x7fbc681500d0 2023-11-28T21:37:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 >> v2:172.21.15.144:3300/0 conn(0x7fbc68156af0 msgr2=0x7fbc6806d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 --2- 172.21.15.12:0/2554273348 >> v2:172.21.15.144:3300/0 conn(0x7fbc68156af0 0x7fbc6806d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 >> v1:172.21.15.144:6789/0 conn(0x7fbc680c27c0 legacy=0x7fbc680c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc680c2ba0 con 0x7fbc681500d0 2023-11-28T21:37:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fbc5c001680 con 0x7fbc681500d0 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 >> v1:172.21.15.12:6789/0 conn(0x7fbc681500d0 legacy=0x7fbc6806cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc50005de0 0x7fbc500061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc500069e0 0x7fbc50006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc500073a0 0x7fbc5000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc680c7170 con 0x7fbc500069e0 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5000e340 con 0x7fbc50005de0 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5000e500 con 0x7fbc500073a0 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc677fe640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc500069e0 0x7fbc50006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc67fff640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc50005de0 0x7fbc500061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc6ca16640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc500073a0 0x7fbc5000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc677fe640 1 -- 172.21.15.12:0/2554273348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc500073a0 msgr2=0x7fbc5000dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:37.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc677fe640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc500073a0 0x7fbc5000dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc677fe640 1 -- 172.21.15.12:0/2554273348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc50005de0 msgr2=0x7fbc500061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:37.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc677fe640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc50005de0 0x7fbc500061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc677fe640 1 -- 172.21.15.12:0/2554273348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc50005740 con 0x7fbc500069e0 2023-11-28T21:37:37.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.678+0000 7fbc6ca16640 1 -- 172.21.15.12:0/2554273348 reap_dead start 2023-11-28T21:37:37.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc677fe640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc500069e0 0x7fbc50006e60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c007fb0 tx=0x7fbc5c01cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0fc11e6c612f39 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:37.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c005ae0 con 0x7fbc500069e0 2023-11-28T21:37:37.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c005c40 con 0x7fbc500069e0 2023-11-28T21:37:37.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc66ffd640 1 -- 172.21.15.12:0/2554273348 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0a5a00 con 0x7fbc500069e0 2023-11-28T21:37:37.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc6e479640 1 -- 172.21.15.12:0/2554273348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc500069e0 msgr2=0x7fbc50006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:37.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc6e479640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc500069e0 0x7fbc50006e60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c007fb0 tx=0x7fbc5c01cac0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc6e479640 1 -- 172.21.15.12:0/2554273348 shutdown_connections 2023-11-28T21:37:37.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc6e479640 1 --2- 172.21.15.12:0/2554273348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc500069e0 0x7fbc50006e60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc6e479640 1 -- 172.21.15.12:0/2554273348 >> 172.21.15.12:0/2554273348 conn(0x7fbc68066360 msgr2=0x7fbc68066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:37.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc6e479640 1 -- 172.21.15.12:0/2554273348 shutdown_connections 2023-11-28T21:37:37.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc6e479640 1 -- 172.21.15.12:0/2554273348 wait complete. 2023-11-28T21:37:37.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc6e479640 1 Processor -- start 2023-11-28T21:37:37.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.682+0000 7fbc6e479640 1 -- start start 2023-11-28T21:37:37.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc6e479640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc500069e0 0x7fbc6814cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:37.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc6e479640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc680c18f0 0x7fbc6814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:37.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc6e479640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc680c9f80 0x7fbc680ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:37.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc6e479640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6815a370 con 0x7fbc680c18f0 2023-11-28T21:37:37.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc6e479640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc681e06c0 con 0x7fbc500069e0 2023-11-28T21:37:37.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc6e479640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc680ca940 con 0x7fbc680c9f80 2023-11-28T21:37:37.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc680c18f0 0x7fbc6814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:37.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc680c18f0 0x7fbc6814d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42410/0 (socket says 172.21.15.12:42410) 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc677fe640 1 -- 172.21.15.12:0/3866520644 learned_addr learned my addr 172.21.15.12:0/3866520644 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc67fff640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc500069e0 0x7fbc6814cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc6ca16640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc680c9f80 0x7fbc680ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc677fe640 1 -- 172.21.15.12:0/3866520644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc680c9f80 msgr2=0x7fbc680ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc677fe640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc680c9f80 0x7fbc680ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc677fe640 1 -- 172.21.15.12:0/3866520644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc500069e0 msgr2=0x7fbc6814cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc677fe640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc500069e0 0x7fbc6814cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc677fe640 1 -- 172.21.15.12:0/3866520644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc681e0bf0 con 0x7fbc680c18f0 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc677fe640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc680c18f0 0x7fbc6814d310 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c007fb0 tx=0x7fbc50007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc64ff9640 1 -- 172.21.15.12:0/3866520644 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0b4070 con 0x7fbc680c18f0 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc64ff9640 1 -- 172.21.15.12:0/3866520644 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c0a5a90 con 0x7fbc680c18f0 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc6e479640 1 -- 172.21.15.12:0/3866520644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc681e0910 con 0x7fbc680c18f0 2023-11-28T21:37:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc64ff9640 1 -- 172.21.15.12:0/3866520644 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0a5e60 con 0x7fbc680c18f0 2023-11-28T21:37:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc6e479640 1 -- 172.21.15.12:0/3866520644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc681f4920 con 0x7fbc680c18f0 2023-11-28T21:37:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc6e479640 1 -- 172.21.15.12:0/3866520644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6806bb00 con 0x7fbc680c18f0 2023-11-28T21:37:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.686+0000 7fbc64ff9640 1 -- 172.21.15.12:0/3866520644 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fbc5c007070 con 0x7fbc680c18f0 2023-11-28T21:37:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.690+0000 7fbc64ff9640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc54066470 0x7fbc54068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.690+0000 7fbc64ff9640 1 -- 172.21.15.12:0/3866520644 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fbc58001d30 con 0x7fbc680c18f0 2023-11-28T21:37:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.690+0000 7fbc67fff640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc54066470 0x7fbc54068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.690+0000 7fbc67fff640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc54066470 0x7fbc54068930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbc580079b0 tx=0x7fbc58007940 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.694+0000 7fbc64ff9640 1 -- 172.21.15.12:0/3866520644 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc5c0eb4a0 con 0x7fbc680c18f0 2023-11-28T21:37:37.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 -- 172.21.15.12:0/3866520644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc680c4920 con 0x7fbc680c18f0 2023-11-28T21:37:37.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc64ff9640 1 -- 172.21.15.12:0/3866520644 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7fbc5c0eeac0 con 0x7fbc680c18f0 2023-11-28T21:37:37.887 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:37.888 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:37.888 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:37.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 -- 172.21.15.12:0/3866520644 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc54066470 msgr2=0x7fbc54068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:37.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc54066470 0x7fbc54068930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbc580079b0 tx=0x7fbc58007940 comp rx=0 tx=0).stop 2023-11-28T21:37:37.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 -- 172.21.15.12:0/3866520644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc680c18f0 msgr2=0x7fbc6814d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:37.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc680c18f0 0x7fbc6814d310 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c007fb0 tx=0x7fbc50007a10 comp rx=0 tx=0).stop 2023-11-28T21:37:37.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 -- 172.21.15.12:0/3866520644 shutdown_connections 2023-11-28T21:37:37.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc54066470 0x7fbc54068930 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc680c9f80 0x7fbc680ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc680c18f0 0x7fbc6814d310 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 --2- 172.21.15.12:0/3866520644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc500069e0 0x7fbc6814cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:37.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 -- 172.21.15.12:0/3866520644 >> 172.21.15.12:0/3866520644 conn(0x7fbc68066360 msgr2=0x7fbc680c4210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:37.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 -- 172.21.15.12:0/3866520644 shutdown_connections 2023-11-28T21:37:37.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:37.882+0000 7fbc6e479640 1 -- 172.21.15.12:0/3866520644 wait complete. 2023-11-28T21:37:37.909 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:38.496 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/30 file after the snapshot 2023-11-28T21:37:38.500 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/31 file after the snapshot 2023-11-28T21:37:38.500 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/32 file after the snapshot 2023-11-28T21:37:38.500 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/33 file after the snapshot 2023-11-28T21:37:38.500 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/34 file after the snapshot 2023-11-28T21:37:38.500 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/35 file after the snapshot 2023-11-28T21:37:38.501 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/36 file after the snapshot 2023-11-28T21:37:38.501 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/37 file after the snapshot 2023-11-28T21:37:38.501 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/38 file after the snapshot 2023-11-28T21:37:38.501 INFO:tasks.workunit.client.0.smithi012.stdout: create 214/39 file after the snapshot 2023-11-28T21:37:38.501 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/.snap/snap-subdir-test 2023-11-28T21:37:38.556 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/30 file after the snapshot 2023-11-28T21:37:38.808 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/31 file after the snapshot 2023-11-28T21:37:38.809 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/32 file after the snapshot 2023-11-28T21:37:38.809 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/33 file after the snapshot 2023-11-28T21:37:38.809 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/34 file after the snapshot 2023-11-28T21:37:38.809 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/35 file after the snapshot 2023-11-28T21:37:38.809 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/36 file after the snapshot 2023-11-28T21:37:38.809 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/37 file after the snapshot 2023-11-28T21:37:38.809 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/38 file after the snapshot 2023-11-28T21:37:38.809 INFO:tasks.workunit.client.0.smithi012.stdout: create 215/39 file after the snapshot 2023-11-28T21:37:38.809 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/.snap/snap-subdir-test 2023-11-28T21:37:39.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/30 file after the snapshot 2023-11-28T21:37:39.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/31 file after the snapshot 2023-11-28T21:37:39.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/32 file after the snapshot 2023-11-28T21:37:39.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/33 file after the snapshot 2023-11-28T21:37:39.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/34 file after the snapshot 2023-11-28T21:37:39.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/35 file after the snapshot 2023-11-28T21:37:39.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/36 file after the snapshot 2023-11-28T21:37:39.360 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/37 file after the snapshot 2023-11-28T21:37:39.361 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/38 file after the snapshot 2023-11-28T21:37:39.361 INFO:tasks.workunit.client.0.smithi012.stdout: create 216/39 file after the snapshot 2023-11-28T21:37:39.361 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/.snap/snap-subdir-test 2023-11-28T21:37:39.466 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/30 file after the snapshot 2023-11-28T21:37:39.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/31 file after the snapshot 2023-11-28T21:37:39.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/32 file after the snapshot 2023-11-28T21:37:39.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/33 file after the snapshot 2023-11-28T21:37:39.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/34 file after the snapshot 2023-11-28T21:37:39.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/35 file after the snapshot 2023-11-28T21:37:39.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/36 file after the snapshot 2023-11-28T21:37:39.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/37 file after the snapshot 2023-11-28T21:37:39.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/38 file after the snapshot 2023-11-28T21:37:39.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 217/39 file after the snapshot 2023-11-28T21:37:39.470 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/.snap/snap-subdir-test 2023-11-28T21:37:39.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/30 file after the snapshot 2023-11-28T21:37:39.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/31 file after the snapshot 2023-11-28T21:37:39.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/32 file after the snapshot 2023-11-28T21:37:39.757 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/33 file after the snapshot 2023-11-28T21:37:39.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/34 file after the snapshot 2023-11-28T21:37:39.758 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/35 file after the snapshot 2023-11-28T21:37:39.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/36 file after the snapshot 2023-11-28T21:37:39.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/37 file after the snapshot 2023-11-28T21:37:39.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/38 file after the snapshot 2023-11-28T21:37:39.759 INFO:tasks.workunit.client.0.smithi012.stdout: create 218/39 file after the snapshot 2023-11-28T21:37:39.760 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/.snap/snap-subdir-test 2023-11-28T21:37:39.812 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/30 file after the snapshot 2023-11-28T21:37:39.816 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/31 file after the snapshot 2023-11-28T21:37:39.816 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/32 file after the snapshot 2023-11-28T21:37:39.816 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/33 file after the snapshot 2023-11-28T21:37:39.817 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/34 file after the snapshot 2023-11-28T21:37:39.817 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/35 file after the snapshot 2023-11-28T21:37:39.817 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/36 file after the snapshot 2023-11-28T21:37:39.817 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/37 file after the snapshot 2023-11-28T21:37:39.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/38 file after the snapshot 2023-11-28T21:37:39.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 219/39 file after the snapshot 2023-11-28T21:37:39.818 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/.snap/snap-subdir-test 2023-11-28T21:37:39.910 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.058+0000 7f9b640fd640 1 Processor -- start 2023-11-28T21:37:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.058+0000 7f9b640fd640 1 -- start start 2023-11-28T21:37:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.058+0000 7f9b640fd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9b5c150c20 0x7f9b5c156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.058+0000 7f9b640fd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b5c06e300 0x7f9b5c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.058+0000 7f9b640fd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5c067410 con 0x7f9b5c06cf40 2023-11-28T21:37:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.058+0000 7f9b640fd640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5c06e090 con 0x7f9b5c150c20 2023-11-28T21:37:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.058+0000 7f9b640fd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c157f30 con 0x7f9b5c06e300 2023-11-28T21:37:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.058+0000 7f9b61e99640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9b5c150c20 0x7f9b5c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.058+0000 7f9b61e99640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9b5c150c20 0x7f9b5c156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42418/0 (socket says 172.21.15.12:42418) 2023-11-28T21:37:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.058+0000 7f9b61e99640 1 -- 172.21.15.12:0/3044912910 learned_addr learned my addr 172.21.15.12:0/3044912910 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b61698640 1 --2- 172.21.15.12:0/3044912910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b5c06e300 0x7f9b5c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b61e99640 1 -- 172.21.15.12:0/3044912910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b5c06e300 msgr2=0x7f9b5c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b61e99640 1 --2- 172.21.15.12:0/3044912910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b5c06e300 0x7f9b5c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b61e99640 1 -- 172.21.15.12:0/3044912910 >> v1:172.21.15.144:6789/0 conn(0x7f9b5c06cf40 legacy=0x7f9b5c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b61e99640 1 -- 172.21.15.12:0/3044912910 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c151000 con 0x7f9b5c150c20 2023-11-28T21:37:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b61e99640 1 --2- 172.21.15.12:0/3044912910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 0x7f9b5c156910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009870 tx=0x7f9b5801e750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbe104d1148be0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b60e97640 1 -- 172.21.15.12:0/3044912910 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b580ab630 con 0x7f9b5c150c20 2023-11-28T21:37:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b60e97640 1 -- 172.21.15.12:0/3044912910 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b580ab790 con 0x7f9b5c150c20 2023-11-28T21:37:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b60e97640 1 -- 172.21.15.12:0/3044912910 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b580abb20 con 0x7f9b5c150c20 2023-11-28T21:37:40.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b640fd640 1 -- 172.21.15.12:0/3044912910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 msgr2=0x7f9b5c156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:40.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.062+0000 7f9b640fd640 1 --2- 172.21.15.12:0/3044912910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 0x7f9b5c156910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009870 tx=0x7f9b5801e750 comp rx=0 tx=0).stop 2023-11-28T21:37:40.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 -- 172.21.15.12:0/3044912910 shutdown_connections 2023-11-28T21:37:40.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 --2- 172.21.15.12:0/3044912910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b5c06e300 0x7f9b5c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:40.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 --2- 172.21.15.12:0/3044912910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 0x7f9b5c156910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:40.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 -- 172.21.15.12:0/3044912910 >> 172.21.15.12:0/3044912910 conn(0x7f9b5c066360 msgr2=0x7f9b5c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:40.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 -- 172.21.15.12:0/3044912910 shutdown_connections 2023-11-28T21:37:40.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 -- 172.21.15.12:0/3044912910 wait complete. 2023-11-28T21:37:40.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 Processor -- start 2023-11-28T21:37:40.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 -- start start 2023-11-28T21:37:40.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b5c06cf40 0x7f9b5c0c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:40.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c06e300 0x7f9b5c0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 0x7f9b5c1f2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c13dd10 con 0x7f9b5c150c20 2023-11-28T21:37:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0c3390 con 0x7f9b5c06e300 2023-11-28T21:37:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b640fd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0cb0a0 con 0x7f9b5c06cf40 2023-11-28T21:37:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b6269a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 0x7f9b5c1f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b6269a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 0x7f9b5c1f2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42430/0 (socket says 172.21.15.12:42430) 2023-11-28T21:37:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b6269a640 1 -- 172.21.15.12:0/1799358728 learned_addr learned my addr 172.21.15.12:0/1799358728 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b61698640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c06e300 0x7f9b5c0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b61e99640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b5c06cf40 0x7f9b5c0c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:40.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b6269a640 1 -- 172.21.15.12:0/1799358728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b5c06cf40 msgr2=0x7f9b5c0c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:40.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b6269a640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b5c06cf40 0x7f9b5c0c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:40.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b6269a640 1 -- 172.21.15.12:0/1799358728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c06e300 msgr2=0x7f9b5c0c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:40.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b6269a640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c06e300 0x7f9b5c0c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b6269a640 1 -- 172.21.15.12:0/1799358728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b580094d0 con 0x7f9b5c150c20 2023-11-28T21:37:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b6269a640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 0x7f9b5c1f2050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c770 tx=0x7f9b5400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b527fc640 1 -- 172.21.15.12:0/1799358728 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54003b80 con 0x7f9b5c150c20 2023-11-28T21:37:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b527fc640 1 -- 172.21.15.12:0/1799358728 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b54003ce0 con 0x7f9b5c150c20 2023-11-28T21:37:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.066+0000 7f9b527fc640 1 -- 172.21.15.12:0/1799358728 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400aa90 con 0x7f9b5c150c20 2023-11-28T21:37:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.070+0000 7f9b640fd640 1 -- 172.21.15.12:0/1799358728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c1409c0 con 0x7f9b5c150c20 2023-11-28T21:37:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.070+0000 7f9b640fd640 1 -- 172.21.15.12:0/1799358728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c1486d0 con 0x7f9b5c150c20 2023-11-28T21:37:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.070+0000 7f9b640fd640 1 -- 172.21.15.12:0/1799358728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c074f60 con 0x7f9b5c150c20 2023-11-28T21:37:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.070+0000 7f9b527fc640 1 -- 172.21.15.12:0/1799358728 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9b5400abf0 con 0x7f9b5c150c20 2023-11-28T21:37:40.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.070+0000 7f9b527fc640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b40066470 0x7f9b40068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:40.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.070+0000 7f9b527fc640 1 -- 172.21.15.12:0/1799358728 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f9b54084140 con 0x7f9b5c150c20 2023-11-28T21:37:40.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.070+0000 7f9b61e99640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b40066470 0x7f9b40068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:40.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.070+0000 7f9b61e99640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b40066470 0x7f9b40068930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9b580b0000 tx=0x7f9b580b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.074+0000 7f9b527fc640 1 -- 172.21.15.12:0/1799358728 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9b540510a0 con 0x7f9b5c150c20 2023-11-28T21:37:40.228 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/30 file after the snapshot 2023-11-28T21:37:40.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/31 file after the snapshot 2023-11-28T21:37:40.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/32 file after the snapshot 2023-11-28T21:37:40.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/33 file after the snapshot 2023-11-28T21:37:40.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/34 file after the snapshot 2023-11-28T21:37:40.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/35 file after the snapshot 2023-11-28T21:37:40.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/36 file after the snapshot 2023-11-28T21:37:40.231 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/37 file after the snapshot 2023-11-28T21:37:40.232 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/38 file after the snapshot 2023-11-28T21:37:40.232 INFO:tasks.workunit.client.0.smithi012.stdout: create 220/39 file after the snapshot 2023-11-28T21:37:40.232 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/.snap/snap-subdir-test 2023-11-28T21:37:40.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.270+0000 7f9b640fd640 1 -- 172.21.15.12:0/1799358728 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b5c0c2c30 con 0x7f9b5c150c20 2023-11-28T21:37:40.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.270+0000 7f9b527fc640 1 -- 172.21.15.12:0/1799358728 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f9b540546c0 con 0x7f9b5c150c20 2023-11-28T21:37:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:40.276 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:40.277 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:40.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.270+0000 7f9b640fd640 1 -- 172.21.15.12:0/1799358728 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b40066470 msgr2=0x7f9b40068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:40.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.270+0000 7f9b640fd640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b40066470 0x7f9b40068930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9b580b0000 tx=0x7f9b580b5040 comp rx=0 tx=0).stop 2023-11-28T21:37:40.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.270+0000 7f9b640fd640 1 -- 172.21.15.12:0/1799358728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 msgr2=0x7f9b5c1f2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:40.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.270+0000 7f9b640fd640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 0x7f9b5c1f2050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400c770 tx=0x7f9b5400cc30 comp rx=0 tx=0).stop 2023-11-28T21:37:40.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.274+0000 7f9b640fd640 1 -- 172.21.15.12:0/1799358728 shutdown_connections 2023-11-28T21:37:40.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.274+0000 7f9b640fd640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9b40066470 0x7f9b40068930 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:40.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.274+0000 7f9b640fd640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b5c150c20 0x7f9b5c1f2050 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:40.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.274+0000 7f9b640fd640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b5c06e300 0x7f9b5c0c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:40.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.274+0000 7f9b640fd640 1 --2- 172.21.15.12:0/1799358728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b5c06cf40 0x7f9b5c0c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:40.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.274+0000 7f9b640fd640 1 -- 172.21.15.12:0/1799358728 >> 172.21.15.12:0/1799358728 conn(0x7f9b5c066360 msgr2=0x7f9b5c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:40.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.274+0000 7f9b640fd640 1 -- 172.21.15.12:0/1799358728 shutdown_connections 2023-11-28T21:37:40.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:40.274+0000 7f9b640fd640 1 -- 172.21.15.12:0/1799358728 wait complete. 2023-11-28T21:37:40.297 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:40.859 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/30 file after the snapshot 2023-11-28T21:37:40.863 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/31 file after the snapshot 2023-11-28T21:37:40.863 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/32 file after the snapshot 2023-11-28T21:37:40.864 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/33 file after the snapshot 2023-11-28T21:37:40.864 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/34 file after the snapshot 2023-11-28T21:37:40.864 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/35 file after the snapshot 2023-11-28T21:37:40.864 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/36 file after the snapshot 2023-11-28T21:37:40.864 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/37 file after the snapshot 2023-11-28T21:37:40.865 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/38 file after the snapshot 2023-11-28T21:37:40.865 INFO:tasks.workunit.client.0.smithi012.stdout: create 221/39 file after the snapshot 2023-11-28T21:37:40.865 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/.snap/snap-subdir-test 2023-11-28T21:37:40.932 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/30 file after the snapshot 2023-11-28T21:37:40.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/31 file after the snapshot 2023-11-28T21:37:40.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/32 file after the snapshot 2023-11-28T21:37:40.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/33 file after the snapshot 2023-11-28T21:37:40.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/34 file after the snapshot 2023-11-28T21:37:40.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/35 file after the snapshot 2023-11-28T21:37:40.936 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/36 file after the snapshot 2023-11-28T21:37:40.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/37 file after the snapshot 2023-11-28T21:37:40.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/38 file after the snapshot 2023-11-28T21:37:40.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 222/39 file after the snapshot 2023-11-28T21:37:40.937 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/.snap/snap-subdir-test 2023-11-28T21:37:41.001 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/30 file after the snapshot 2023-11-28T21:37:41.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/31 file after the snapshot 2023-11-28T21:37:41.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/32 file after the snapshot 2023-11-28T21:37:41.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/33 file after the snapshot 2023-11-28T21:37:41.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/34 file after the snapshot 2023-11-28T21:37:41.004 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/35 file after the snapshot 2023-11-28T21:37:41.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/36 file after the snapshot 2023-11-28T21:37:41.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/37 file after the snapshot 2023-11-28T21:37:41.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/38 file after the snapshot 2023-11-28T21:37:41.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 223/39 file after the snapshot 2023-11-28T21:37:41.005 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/.snap/snap-subdir-test 2023-11-28T21:37:41.356 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/30 file after the snapshot 2023-11-28T21:37:41.593 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/31 file after the snapshot 2023-11-28T21:37:41.593 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/32 file after the snapshot 2023-11-28T21:37:41.593 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/33 file after the snapshot 2023-11-28T21:37:41.594 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/34 file after the snapshot 2023-11-28T21:37:41.594 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/35 file after the snapshot 2023-11-28T21:37:41.594 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/36 file after the snapshot 2023-11-28T21:37:41.594 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/37 file after the snapshot 2023-11-28T21:37:41.594 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/38 file after the snapshot 2023-11-28T21:37:41.594 INFO:tasks.workunit.client.0.smithi012.stdout: create 224/39 file after the snapshot 2023-11-28T21:37:41.594 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/.snap/snap-subdir-test 2023-11-28T21:37:42.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/30 file after the snapshot 2023-11-28T21:37:42.147 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/31 file after the snapshot 2023-11-28T21:37:42.147 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/32 file after the snapshot 2023-11-28T21:37:42.147 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/33 file after the snapshot 2023-11-28T21:37:42.147 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/34 file after the snapshot 2023-11-28T21:37:42.147 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/35 file after the snapshot 2023-11-28T21:37:42.147 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/36 file after the snapshot 2023-11-28T21:37:42.147 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/37 file after the snapshot 2023-11-28T21:37:42.148 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/38 file after the snapshot 2023-11-28T21:37:42.148 INFO:tasks.workunit.client.0.smithi012.stdout: create 225/39 file after the snapshot 2023-11-28T21:37:42.148 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/.snap/snap-subdir-test 2023-11-28T21:37:42.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/30 file after the snapshot 2023-11-28T21:37:42.203 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/31 file after the snapshot 2023-11-28T21:37:42.203 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/32 file after the snapshot 2023-11-28T21:37:42.203 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/33 file after the snapshot 2023-11-28T21:37:42.203 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/34 file after the snapshot 2023-11-28T21:37:42.203 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/35 file after the snapshot 2023-11-28T21:37:42.204 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/36 file after the snapshot 2023-11-28T21:37:42.204 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/37 file after the snapshot 2023-11-28T21:37:42.204 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/38 file after the snapshot 2023-11-28T21:37:42.204 INFO:tasks.workunit.client.0.smithi012.stdout: create 226/39 file after the snapshot 2023-11-28T21:37:42.204 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/.snap/snap-subdir-test 2023-11-28T21:37:42.259 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/30 file after the snapshot 2023-11-28T21:37:42.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/31 file after the snapshot 2023-11-28T21:37:42.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/32 file after the snapshot 2023-11-28T21:37:42.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/33 file after the snapshot 2023-11-28T21:37:42.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/34 file after the snapshot 2023-11-28T21:37:42.262 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/35 file after the snapshot 2023-11-28T21:37:42.263 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/36 file after the snapshot 2023-11-28T21:37:42.263 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/37 file after the snapshot 2023-11-28T21:37:42.263 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/38 file after the snapshot 2023-11-28T21:37:42.263 INFO:tasks.workunit.client.0.smithi012.stdout: create 227/39 file after the snapshot 2023-11-28T21:37:42.263 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/.snap/snap-subdir-test 2023-11-28T21:37:42.298 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:42.337 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/30 file after the snapshot 2023-11-28T21:37:42.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/31 file after the snapshot 2023-11-28T21:37:42.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/32 file after the snapshot 2023-11-28T21:37:42.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/33 file after the snapshot 2023-11-28T21:37:42.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/34 file after the snapshot 2023-11-28T21:37:42.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/35 file after the snapshot 2023-11-28T21:37:42.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/36 file after the snapshot 2023-11-28T21:37:42.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/37 file after the snapshot 2023-11-28T21:37:42.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/38 file after the snapshot 2023-11-28T21:37:42.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 228/39 file after the snapshot 2023-11-28T21:37:42.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/.snap/snap-subdir-test 2023-11-28T21:37:42.414 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/30 file after the snapshot 2023-11-28T21:37:42.419 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/31 file after the snapshot 2023-11-28T21:37:42.419 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/32 file after the snapshot 2023-11-28T21:37:42.419 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/33 file after the snapshot 2023-11-28T21:37:42.419 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/34 file after the snapshot 2023-11-28T21:37:42.419 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/35 file after the snapshot 2023-11-28T21:37:42.419 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/36 file after the snapshot 2023-11-28T21:37:42.420 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/37 file after the snapshot 2023-11-28T21:37:42.420 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/38 file after the snapshot 2023-11-28T21:37:42.420 INFO:tasks.workunit.client.0.smithi012.stdout: create 229/39 file after the snapshot 2023-11-28T21:37:42.420 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/.snap/snap-subdir-test 2023-11-28T21:37:42.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.442+0000 7fadb7079640 1 Processor -- start 2023-11-28T21:37:42.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.442+0000 7fadb7079640 1 -- start start 2023-11-28T21:37:42.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadb7079640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fadb0151af0 0x7fadb00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:42.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadb7079640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb0157b70 con 0x7fadb006e300 2023-11-28T21:37:42.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadb7079640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb0157e50 con 0x7fadb0151ed0 2023-11-28T21:37:42.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadb7079640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00707f0 con 0x7fadb0151af0 2023-11-28T21:37:42.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadb5616640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fadb006e300 0x7fadb0072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33612/0 (socket says 172.21.15.12:33612) 2023-11-28T21:37:42.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadb5616640 1 -- 172.21.15.12:0/655600054 learned_addr learned my addr 172.21.15.12:0/655600054 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:42.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadb4e15640 1 --2- 172.21.15.12:0/655600054 >> v2:172.21.15.144:3300/0 conn(0x7fadb0151af0 0x7fadb00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:42.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadaf7fe640 1 -- 172.21.15.12:0/655600054 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621587548 0 0) 0x7fadb0157b70 con 0x7fadb006e300 2023-11-28T21:37:42.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadaf7fe640 1 -- 172.21.15.12:0/655600054 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fada0003530 con 0x7fadb006e300 2023-11-28T21:37:42.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadb4e15640 1 -- 172.21.15.12:0/655600054 >> v1:172.21.15.144:6789/0 conn(0x7fadb0151ed0 legacy=0x7fadb00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:42.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadb4e15640 1 -- 172.21.15.12:0/655600054 >> v1:172.21.15.12:6789/0 conn(0x7fadb006e300 legacy=0x7fadb0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:42.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadaf7fe640 1 -- 172.21.15.12:0/655600054 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3962421049 0 0) 0x7fada0003530 con 0x7fadb006e300 2023-11-28T21:37:42.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.446+0000 7fadb4e15640 1 -- 172.21.15.12:0/655600054 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb0158130 con 0x7fadb0151af0 2023-11-28T21:37:42.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadb4e15640 1 --2- 172.21.15.12:0/655600054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadb0151af0 0x7fadb00c7650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fada40097c0 tx=0x7fada401c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d1771e1080dbf97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:42.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadaf7fe640 1 -- 172.21.15.12:0/655600054 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada8001a00 con 0x7fadb0151af0 2023-11-28T21:37:42.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadaf7fe640 1 -- 172.21.15.12:0/655600054 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada40a7050 con 0x7fadb0151af0 2023-11-28T21:37:42.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadaf7fe640 1 -- 172.21.15.12:0/655600054 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada40ab600 con 0x7fadb0151af0 2023-11-28T21:37:42.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadb7079640 1 -- 172.21.15.12:0/655600054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadb0151af0 msgr2=0x7fadb00c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:42.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadb7079640 1 --2- 172.21.15.12:0/655600054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadb0151af0 0x7fadb00c7650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fada40097c0 tx=0x7fada401c3a0 comp rx=0 tx=0).stop 2023-11-28T21:37:42.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadb7079640 1 -- 172.21.15.12:0/655600054 shutdown_connections 2023-11-28T21:37:42.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadb7079640 1 --2- 172.21.15.12:0/655600054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadb0151af0 0x7fadb00c7650 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:42.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadb7079640 1 -- 172.21.15.12:0/655600054 >> 172.21.15.12:0/655600054 conn(0x7fadb0066360 msgr2=0x7fadb0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:42.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadb7079640 1 -- 172.21.15.12:0/655600054 shutdown_connections 2023-11-28T21:37:42.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadb7079640 1 -- 172.21.15.12:0/655600054 wait complete. 2023-11-28T21:37:42.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.450+0000 7fadb7079640 1 Processor -- start 2023-11-28T21:37:42.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb7079640 1 -- start start 2023-11-28T21:37:42.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb7079640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadb006e300 0x7fadb00c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:42.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb7079640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadb0151af0 0x7fadb0156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:42.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb7079640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadb01420c0 0x7fadb0142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:42.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb7079640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb0147440 con 0x7fadb0151af0 2023-11-28T21:37:42.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb7079640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb00c46e0 con 0x7fadb006e300 2023-11-28T21:37:42.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb7079640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb01468a0 con 0x7fadb01420c0 2023-11-28T21:37:42.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadaffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadb0151af0 0x7fadb0156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:42.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb4e15640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadb006e300 0x7fadb00c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:42.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadaffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadb0151af0 0x7fadb0156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42432/0 (socket says 172.21.15.12:42432) 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadaffff640 1 -- 172.21.15.12:0/3746315140 learned_addr learned my addr 172.21.15.12:0/3746315140 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb5616640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadb01420c0 0x7fadb0142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadaffff640 1 -- 172.21.15.12:0/3746315140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadb01420c0 msgr2=0x7fadb0142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadaffff640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadb01420c0 0x7fadb0142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadaffff640 1 -- 172.21.15.12:0/3746315140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadb006e300 msgr2=0x7fadb00c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadaffff640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadb006e300 0x7fadb00c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadaffff640 1 -- 172.21.15.12:0/3746315140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada4009420 con 0x7fadb0151af0 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadaffff640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadb0151af0 0x7fadb0156930 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00c310 tx=0x7fad9c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadad7fa640 1 -- 172.21.15.12:0/3746315140 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0052d0 con 0x7fadb0151af0 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadad7fa640 1 -- 172.21.15.12:0/3746315140 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9c005430 con 0x7fadb0151af0 2023-11-28T21:37:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadad7fa640 1 -- 172.21.15.12:0/3746315140 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c007260 con 0x7fadb0151af0 2023-11-28T21:37:42.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb7079640 1 -- 172.21.15.12:0/3746315140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb0146b10 con 0x7fadb0151af0 2023-11-28T21:37:42.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb7079640 1 -- 172.21.15.12:0/3746315140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb01470d0 con 0x7fadb0151af0 2023-11-28T21:37:42.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.454+0000 7fadb7079640 1 -- 172.21.15.12:0/3746315140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb0147290 con 0x7fadb0151af0 2023-11-28T21:37:42.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.458+0000 7fadad7fa640 1 -- 172.21.15.12:0/3746315140 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fad9c0073c0 con 0x7fadb0151af0 2023-11-28T21:37:42.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.458+0000 7fadad7fa640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fad94066470 0x7fad94068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:42.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.458+0000 7fadad7fa640 1 -- 172.21.15.12:0/3746315140 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fad9c083640 con 0x7fadb0151af0 2023-11-28T21:37:42.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.458+0000 7fadb4e15640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fad94066470 0x7fad94068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:42.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.458+0000 7fadb4e15640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fad94066470 0x7fad94068930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fada40ad000 tx=0x7fada40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:42.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.462+0000 7fadad7fa640 1 -- 172.21.15.12:0/3746315140 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fad9c050650 con 0x7fadb0151af0 2023-11-28T21:37:42.469 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/30 file after the snapshot 2023-11-28T21:37:42.472 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/31 file after the snapshot 2023-11-28T21:37:42.473 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/32 file after the snapshot 2023-11-28T21:37:42.473 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/33 file after the snapshot 2023-11-28T21:37:42.473 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/34 file after the snapshot 2023-11-28T21:37:42.473 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/35 file after the snapshot 2023-11-28T21:37:42.473 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/36 file after the snapshot 2023-11-28T21:37:42.473 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/37 file after the snapshot 2023-11-28T21:37:42.473 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/38 file after the snapshot 2023-11-28T21:37:42.473 INFO:tasks.workunit.client.0.smithi012.stdout: create 230/39 file after the snapshot 2023-11-28T21:37:42.473 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/.snap/snap-subdir-test 2023-11-28T21:37:42.536 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/30 file after the snapshot 2023-11-28T21:37:42.539 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/31 file after the snapshot 2023-11-28T21:37:42.539 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/32 file after the snapshot 2023-11-28T21:37:42.539 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/33 file after the snapshot 2023-11-28T21:37:42.539 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/34 file after the snapshot 2023-11-28T21:37:42.539 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/35 file after the snapshot 2023-11-28T21:37:42.540 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/36 file after the snapshot 2023-11-28T21:37:42.540 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/37 file after the snapshot 2023-11-28T21:37:42.540 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/38 file after the snapshot 2023-11-28T21:37:42.540 INFO:tasks.workunit.client.0.smithi012.stdout: create 231/39 file after the snapshot 2023-11-28T21:37:42.540 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/.snap/snap-subdir-test 2023-11-28T21:37:42.597 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/30 file after the snapshot 2023-11-28T21:37:42.600 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/31 file after the snapshot 2023-11-28T21:37:42.600 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/32 file after the snapshot 2023-11-28T21:37:42.600 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/33 file after the snapshot 2023-11-28T21:37:42.600 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/34 file after the snapshot 2023-11-28T21:37:42.600 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/35 file after the snapshot 2023-11-28T21:37:42.600 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/36 file after the snapshot 2023-11-28T21:37:42.600 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/37 file after the snapshot 2023-11-28T21:37:42.600 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/38 file after the snapshot 2023-11-28T21:37:42.600 INFO:tasks.workunit.client.0.smithi012.stdout: create 232/39 file after the snapshot 2023-11-28T21:37:42.600 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/.snap/snap-subdir-test 2023-11-28T21:37:42.655 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/30 file after the snapshot 2023-11-28T21:37:42.657 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/31 file after the snapshot 2023-11-28T21:37:42.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/32 file after the snapshot 2023-11-28T21:37:42.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/33 file after the snapshot 2023-11-28T21:37:42.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/34 file after the snapshot 2023-11-28T21:37:42.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/35 file after the snapshot 2023-11-28T21:37:42.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/36 file after the snapshot 2023-11-28T21:37:42.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/37 file after the snapshot 2023-11-28T21:37:42.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/38 file after the snapshot 2023-11-28T21:37:42.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 233/39 file after the snapshot 2023-11-28T21:37:42.658 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/.snap/snap-subdir-test 2023-11-28T21:37:42.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.658+0000 7fadb7079640 1 -- 172.21.15.12:0/3746315140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadb0074f60 con 0x7fadb0151af0 2023-11-28T21:37:42.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.658+0000 7fadad7fa640 1 -- 172.21.15.12:0/3746315140 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7fad9c053c70 con 0x7fadb0151af0 2023-11-28T21:37:42.665 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:42.665 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:42.665 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:42.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 -- 172.21.15.12:0/3746315140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fad94066470 msgr2=0x7fad94068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:42.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fad94066470 0x7fad94068930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fada40ad000 tx=0x7fada40a6000 comp rx=0 tx=0).stop 2023-11-28T21:37:42.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 -- 172.21.15.12:0/3746315140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadb0151af0 msgr2=0x7fadb0156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:42.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadb0151af0 0x7fadb0156930 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00c310 tx=0x7fad9c00c7d0 comp rx=0 tx=0).stop 2023-11-28T21:37:42.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 -- 172.21.15.12:0/3746315140 shutdown_connections 2023-11-28T21:37:42.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fad94066470 0x7fad94068930 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:42.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadb01420c0 0x7fadb0142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fadb0151af0 0x7fadb0156930 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 --2- 172.21.15.12:0/3746315140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadb006e300 0x7fadb00c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 -- 172.21.15.12:0/3746315140 >> 172.21.15.12:0/3746315140 conn(0x7fadb0066360 msgr2=0x7fadb014fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 -- 172.21.15.12:0/3746315140 shutdown_connections 2023-11-28T21:37:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:42.662+0000 7fadb7079640 1 -- 172.21.15.12:0/3746315140 wait complete. 2023-11-28T21:37:42.687 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-11-28T21:37:42.687 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 17, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 16, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_4250": { "addr": "172.21.15.12:6839/3027655738", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3027655738, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3027655738, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4250, "incarnation": 8, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:stopping", "state_seq": 11 }, "gid_4261": { "addr": "172.21.15.144:6833/3053317078", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 3053317078, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 3053317078, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4261, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:37:33.466684+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 4261, "mds_1": 4250 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6838/2321889805", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2321889805, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 2321889805, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 17, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5127, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:37:42.713 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/30 file after the snapshot 2023-11-28T21:37:42.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/31 file after the snapshot 2023-11-28T21:37:42.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/32 file after the snapshot 2023-11-28T21:37:42.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/33 file after the snapshot 2023-11-28T21:37:42.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/34 file after the snapshot 2023-11-28T21:37:42.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/35 file after the snapshot 2023-11-28T21:37:42.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/36 file after the snapshot 2023-11-28T21:37:42.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/37 file after the snapshot 2023-11-28T21:37:42.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/38 file after the snapshot 2023-11-28T21:37:42.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 234/39 file after the snapshot 2023-11-28T21:37:42.716 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/.snap/snap-subdir-test 2023-11-28T21:37:42.770 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/30 file after the snapshot 2023-11-28T21:37:42.773 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/31 file after the snapshot 2023-11-28T21:37:42.773 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/32 file after the snapshot 2023-11-28T21:37:42.773 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/33 file after the snapshot 2023-11-28T21:37:42.773 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/34 file after the snapshot 2023-11-28T21:37:42.773 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/35 file after the snapshot 2023-11-28T21:37:42.773 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/36 file after the snapshot 2023-11-28T21:37:42.773 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/37 file after the snapshot 2023-11-28T21:37:42.773 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/38 file after the snapshot 2023-11-28T21:37:42.773 INFO:tasks.workunit.client.0.smithi012.stdout: create 235/39 file after the snapshot 2023-11-28T21:37:42.773 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/.snap/snap-subdir-test 2023-11-28T21:37:42.827 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/30 file after the snapshot 2023-11-28T21:37:42.830 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/31 file after the snapshot 2023-11-28T21:37:42.830 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/32 file after the snapshot 2023-11-28T21:37:42.830 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/33 file after the snapshot 2023-11-28T21:37:42.830 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/34 file after the snapshot 2023-11-28T21:37:42.830 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/35 file after the snapshot 2023-11-28T21:37:42.830 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/36 file after the snapshot 2023-11-28T21:37:42.830 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/37 file after the snapshot 2023-11-28T21:37:42.830 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/38 file after the snapshot 2023-11-28T21:37:42.830 INFO:tasks.workunit.client.0.smithi012.stdout: create 236/39 file after the snapshot 2023-11-28T21:37:42.830 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/.snap/snap-subdir-test 2023-11-28T21:37:42.884 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/30 file after the snapshot 2023-11-28T21:37:42.887 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/31 file after the snapshot 2023-11-28T21:37:42.888 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/32 file after the snapshot 2023-11-28T21:37:42.888 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/33 file after the snapshot 2023-11-28T21:37:42.888 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/34 file after the snapshot 2023-11-28T21:37:42.888 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/35 file after the snapshot 2023-11-28T21:37:42.888 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/36 file after the snapshot 2023-11-28T21:37:42.888 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/37 file after the snapshot 2023-11-28T21:37:42.888 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/38 file after the snapshot 2023-11-28T21:37:42.888 INFO:tasks.workunit.client.0.smithi012.stdout: create 237/39 file after the snapshot 2023-11-28T21:37:42.888 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/.snap/snap-subdir-test 2023-11-28T21:37:42.943 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/30 file after the snapshot 2023-11-28T21:37:42.946 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/31 file after the snapshot 2023-11-28T21:37:42.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/32 file after the snapshot 2023-11-28T21:37:42.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/33 file after the snapshot 2023-11-28T21:37:42.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/34 file after the snapshot 2023-11-28T21:37:42.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/35 file after the snapshot 2023-11-28T21:37:42.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/36 file after the snapshot 2023-11-28T21:37:42.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/37 file after the snapshot 2023-11-28T21:37:42.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/38 file after the snapshot 2023-11-28T21:37:42.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 238/39 file after the snapshot 2023-11-28T21:37:42.947 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/.snap/snap-subdir-test 2023-11-28T21:37:43.243 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/30 file after the snapshot 2023-11-28T21:37:43.247 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/31 file after the snapshot 2023-11-28T21:37:43.247 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/32 file after the snapshot 2023-11-28T21:37:43.247 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/33 file after the snapshot 2023-11-28T21:37:43.247 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/34 file after the snapshot 2023-11-28T21:37:43.248 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/35 file after the snapshot 2023-11-28T21:37:43.248 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/36 file after the snapshot 2023-11-28T21:37:43.248 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/37 file after the snapshot 2023-11-28T21:37:43.248 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/38 file after the snapshot 2023-11-28T21:37:43.248 INFO:tasks.workunit.client.0.smithi012.stdout: create 239/39 file after the snapshot 2023-11-28T21:37:43.249 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/.snap/snap-subdir-test 2023-11-28T21:37:43.630 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/30 file after the snapshot 2023-11-28T21:37:43.632 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/31 file after the snapshot 2023-11-28T21:37:43.632 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/32 file after the snapshot 2023-11-28T21:37:43.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/33 file after the snapshot 2023-11-28T21:37:43.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/34 file after the snapshot 2023-11-28T21:37:43.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/35 file after the snapshot 2023-11-28T21:37:43.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/36 file after the snapshot 2023-11-28T21:37:43.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/37 file after the snapshot 2023-11-28T21:37:43.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/38 file after the snapshot 2023-11-28T21:37:43.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 240/39 file after the snapshot 2023-11-28T21:37:43.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/.snap/snap-subdir-test 2023-11-28T21:37:44.048 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/30 file after the snapshot 2023-11-28T21:37:44.051 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/31 file after the snapshot 2023-11-28T21:37:44.051 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/32 file after the snapshot 2023-11-28T21:37:44.051 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/33 file after the snapshot 2023-11-28T21:37:44.051 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/34 file after the snapshot 2023-11-28T21:37:44.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/35 file after the snapshot 2023-11-28T21:37:44.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/36 file after the snapshot 2023-11-28T21:37:44.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/37 file after the snapshot 2023-11-28T21:37:44.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/38 file after the snapshot 2023-11-28T21:37:44.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 241/39 file after the snapshot 2023-11-28T21:37:44.052 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/.snap/snap-subdir-test 2023-11-28T21:37:44.353 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/30 file after the snapshot 2023-11-28T21:37:44.356 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/31 file after the snapshot 2023-11-28T21:37:44.356 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/32 file after the snapshot 2023-11-28T21:37:44.356 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/33 file after the snapshot 2023-11-28T21:37:44.356 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/34 file after the snapshot 2023-11-28T21:37:44.356 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/35 file after the snapshot 2023-11-28T21:37:44.356 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/36 file after the snapshot 2023-11-28T21:37:44.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/37 file after the snapshot 2023-11-28T21:37:44.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/38 file after the snapshot 2023-11-28T21:37:44.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 242/39 file after the snapshot 2023-11-28T21:37:44.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/.snap/snap-subdir-test 2023-11-28T21:37:44.622 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/30 file after the snapshot 2023-11-28T21:37:44.624 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/31 file after the snapshot 2023-11-28T21:37:44.625 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/32 file after the snapshot 2023-11-28T21:37:44.625 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/33 file after the snapshot 2023-11-28T21:37:44.625 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/34 file after the snapshot 2023-11-28T21:37:44.626 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/35 file after the snapshot 2023-11-28T21:37:44.626 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/36 file after the snapshot 2023-11-28T21:37:44.626 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/37 file after the snapshot 2023-11-28T21:37:44.626 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/38 file after the snapshot 2023-11-28T21:37:44.626 INFO:tasks.workunit.client.0.smithi012.stdout: create 243/39 file after the snapshot 2023-11-28T21:37:44.626 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/.snap/snap-subdir-test 2023-11-28T21:37:44.681 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/30 file after the snapshot 2023-11-28T21:37:44.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/31 file after the snapshot 2023-11-28T21:37:44.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/32 file after the snapshot 2023-11-28T21:37:44.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/33 file after the snapshot 2023-11-28T21:37:44.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/34 file after the snapshot 2023-11-28T21:37:44.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/35 file after the snapshot 2023-11-28T21:37:44.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/36 file after the snapshot 2023-11-28T21:37:44.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/37 file after the snapshot 2023-11-28T21:37:44.685 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/38 file after the snapshot 2023-11-28T21:37:44.685 INFO:tasks.workunit.client.0.smithi012.stdout: create 244/39 file after the snapshot 2023-11-28T21:37:44.685 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/.snap/snap-subdir-test 2023-11-28T21:37:44.688 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:44.738 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/30 file after the snapshot 2023-11-28T21:37:44.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/31 file after the snapshot 2023-11-28T21:37:44.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/32 file after the snapshot 2023-11-28T21:37:44.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/33 file after the snapshot 2023-11-28T21:37:44.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/34 file after the snapshot 2023-11-28T21:37:44.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/35 file after the snapshot 2023-11-28T21:37:44.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/36 file after the snapshot 2023-11-28T21:37:44.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/37 file after the snapshot 2023-11-28T21:37:44.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/38 file after the snapshot 2023-11-28T21:37:44.741 INFO:tasks.workunit.client.0.smithi012.stdout: create 245/39 file after the snapshot 2023-11-28T21:37:44.742 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/.snap/snap-subdir-test 2023-11-28T21:37:44.796 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/30 file after the snapshot 2023-11-28T21:37:44.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/31 file after the snapshot 2023-11-28T21:37:44.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/32 file after the snapshot 2023-11-28T21:37:44.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/33 file after the snapshot 2023-11-28T21:37:44.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/34 file after the snapshot 2023-11-28T21:37:44.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/35 file after the snapshot 2023-11-28T21:37:44.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/36 file after the snapshot 2023-11-28T21:37:44.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/37 file after the snapshot 2023-11-28T21:37:44.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/38 file after the snapshot 2023-11-28T21:37:44.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 246/39 file after the snapshot 2023-11-28T21:37:44.799 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/.snap/snap-subdir-test 2023-11-28T21:37:44.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f5932c54640 1 Processor -- start 2023-11-28T21:37:44.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f5932c54640 1 -- start start 2023-11-28T21:37:44.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f5932c54640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f592c151af0 0x7f592c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:44.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f5932c54640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f592c06e300 0x7f592c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:44.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f5932c54640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c157b70 con 0x7f592c151ed0 2023-11-28T21:37:44.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f5932c54640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c0707f0 con 0x7f592c06e300 2023-11-28T21:37:44.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f5932c54640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c0c7b90 con 0x7f592c151af0 2023-11-28T21:37:44.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f59311f1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f592c06e300 0x7f592c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:44.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f59311f1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f592c06e300 0x7f592c072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47830/0 (socket says 172.21.15.12:47830) 2023-11-28T21:37:44.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f59311f1640 1 -- 172.21.15.12:0/3421979780 learned_addr learned my addr 172.21.15.12:0/3421979780 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:44.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f59309f0640 1 --2- 172.21.15.12:0/3421979780 >> v2:172.21.15.144:3300/0 conn(0x7f592c151af0 0x7f592c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:44.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f59311f1640 1 -- 172.21.15.12:0/3421979780 >> v2:172.21.15.144:3300/0 conn(0x7f592c151af0 msgr2=0x7f592c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f59311f1640 1 --2- 172.21.15.12:0/3421979780 >> v2:172.21.15.144:3300/0 conn(0x7f592c151af0 0x7f592c0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f59311f1640 1 -- 172.21.15.12:0/3421979780 >> v1:172.21.15.12:6789/0 conn(0x7f592c151ed0 legacy=0x7f592c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f59311f1640 1 -- 172.21.15.12:0/3421979780 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c157e50 con 0x7f592c06e300 2023-11-28T21:37:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f592b7fe640 1 -- 172.21.15.12:0/3421979780 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116447462 0 0) 0x7f592c157b70 con 0x7f592c151ed0 2023-11-28T21:37:44.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.838+0000 7f59309f0640 1 --2- 172.21.15.12:0/3421979780 >> v2:172.21.15.144:3300/0 conn(0x7f592c151af0 0x7f592c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:44.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f59311f1640 1 --2- 172.21.15.12:0/3421979780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 0x7f592c072070 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f592c0c7d20 tx=0x7f592401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7daaa2d124a60e35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:44.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f592b7fe640 1 -- 172.21.15.12:0/3421979780 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59240a5690 con 0x7f592c06e300 2023-11-28T21:37:44.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f592b7fe640 1 -- 172.21.15.12:0/3421979780 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59240a57f0 con 0x7f592c06e300 2023-11-28T21:37:44.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f592b7fe640 1 -- 172.21.15.12:0/3421979780 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59240a5b80 con 0x7f592c06e300 2023-11-28T21:37:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f5932c54640 1 -- 172.21.15.12:0/3421979780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 msgr2=0x7f592c072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f5932c54640 1 --2- 172.21.15.12:0/3421979780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 0x7f592c072070 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f592c0c7d20 tx=0x7f592401c9d0 comp rx=0 tx=0).stop 2023-11-28T21:37:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f5932c54640 1 -- 172.21.15.12:0/3421979780 shutdown_connections 2023-11-28T21:37:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f5932c54640 1 --2- 172.21.15.12:0/3421979780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 0x7f592c072070 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f5932c54640 1 --2- 172.21.15.12:0/3421979780 >> v2:172.21.15.144:3300/0 conn(0x7f592c151af0 0x7f592c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f5932c54640 1 -- 172.21.15.12:0/3421979780 >> 172.21.15.12:0/3421979780 conn(0x7f592c066360 msgr2=0x7f592c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f5932c54640 1 -- 172.21.15.12:0/3421979780 shutdown_connections 2023-11-28T21:37:44.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.842+0000 7f5932c54640 1 -- 172.21.15.12:0/3421979780 wait complete. 2023-11-28T21:37:44.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f5932c54640 1 Processor -- start 2023-11-28T21:37:44.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f5932c54640 1 -- start start 2023-11-28T21:37:44.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f5932c54640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 0x7f592c13d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:44.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f5932c54640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f592c151af0 0x7f592c13e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:44.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f5932c54640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c140c00 0x7f592c1f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:44.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f5932c54640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c13de10 con 0x7f592c06e300 2023-11-28T21:37:44.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f5932c54640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c13c540 con 0x7f592c140c00 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f5932c54640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c14de00 con 0x7f592c151af0 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59309f0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 0x7f592c13d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59309f0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 0x7f592c13d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47836/0 (socket says 172.21.15.12:47836) 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59309f0640 1 -- 172.21.15.12:0/2029313966 learned_addr learned my addr 172.21.15.12:0/2029313966 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f592bfff640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f592c151af0 0x7f592c13e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59311f1640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c140c00 0x7f592c1f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59309f0640 1 -- 172.21.15.12:0/2029313966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f592c151af0 msgr2=0x7f592c13e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59309f0640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f592c151af0 0x7f592c13e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59309f0640 1 -- 172.21.15.12:0/2029313966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c140c00 msgr2=0x7f592c1f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59309f0640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c140c00 0x7f592c1f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59309f0640 1 -- 172.21.15.12:0/2029313966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5924009b30 con 0x7f592c06e300 2023-11-28T21:37:44.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59311f1640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c140c00 0x7f592c1f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:44.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59309f0640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 0x7f592c13d8d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f592000e810 tx=0x7f592000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:44.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59297fa640 1 -- 172.21.15.12:0/2029313966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592000c8d0 con 0x7f592c06e300 2023-11-28T21:37:44.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59297fa640 1 -- 172.21.15.12:0/2029313966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592000ca30 con 0x7f592c06e300 2023-11-28T21:37:44.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f5932c54640 1 -- 172.21.15.12:0/2029313966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c1f69b0 con 0x7f592c06e300 2023-11-28T21:37:44.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f59297fa640 1 -- 172.21.15.12:0/2029313966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592000cdc0 con 0x7f592c06e300 2023-11-28T21:37:44.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.846+0000 7f5932c54640 1 -- 172.21.15.12:0/2029313966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c14db20 con 0x7f592c06e300 2023-11-28T21:37:44.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.850+0000 7f5932c54640 1 -- 172.21.15.12:0/2029313966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c074f60 con 0x7f592c06e300 2023-11-28T21:37:44.855 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/30 file after the snapshot 2023-11-28T21:37:44.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.850+0000 7f59297fa640 1 -- 172.21.15.12:0/2029313966 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5920013410 con 0x7f592c06e300 2023-11-28T21:37:44.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.850+0000 7f59297fa640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5910066470 0x7f5910068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:44.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.850+0000 7f59297fa640 1 -- 172.21.15.12:0/2029313966 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f59200840c0 con 0x7f592c06e300 2023-11-28T21:37:44.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.850+0000 7f592bfff640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5910066470 0x7f5910068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:44.857 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/31 file after the snapshot 2023-11-28T21:37:44.858 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/32 file after the snapshot 2023-11-28T21:37:44.858 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/33 file after the snapshot 2023-11-28T21:37:44.858 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/34 file after the snapshot 2023-11-28T21:37:44.858 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/35 file after the snapshot 2023-11-28T21:37:44.858 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/36 file after the snapshot 2023-11-28T21:37:44.858 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/37 file after the snapshot 2023-11-28T21:37:44.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.850+0000 7f592bfff640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5910066470 0x7f5910068930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f592c140b60 tx=0x7f5918009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:44.859 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/38 file after the snapshot 2023-11-28T21:37:44.859 INFO:tasks.workunit.client.0.smithi012.stdout: create 247/39 file after the snapshot 2023-11-28T21:37:44.859 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/.snap/snap-subdir-test 2023-11-28T21:37:44.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:44.854+0000 7f59297fa640 1 -- 172.21.15.12:0/2029313966 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5920051140 con 0x7f592c06e300 2023-11-28T21:37:44.912 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/30 file after the snapshot 2023-11-28T21:37:44.915 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/31 file after the snapshot 2023-11-28T21:37:44.915 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/32 file after the snapshot 2023-11-28T21:37:44.915 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/33 file after the snapshot 2023-11-28T21:37:44.915 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/34 file after the snapshot 2023-11-28T21:37:44.915 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/35 file after the snapshot 2023-11-28T21:37:44.915 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/36 file after the snapshot 2023-11-28T21:37:44.915 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/37 file after the snapshot 2023-11-28T21:37:44.915 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/38 file after the snapshot 2023-11-28T21:37:44.916 INFO:tasks.workunit.client.0.smithi012.stdout: create 248/39 file after the snapshot 2023-11-28T21:37:44.916 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/.snap/snap-subdir-test 2023-11-28T21:37:44.968 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/30 file after the snapshot 2023-11-28T21:37:44.971 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/31 file after the snapshot 2023-11-28T21:37:44.971 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/32 file after the snapshot 2023-11-28T21:37:44.971 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/33 file after the snapshot 2023-11-28T21:37:44.972 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/34 file after the snapshot 2023-11-28T21:37:44.972 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/35 file after the snapshot 2023-11-28T21:37:44.972 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/36 file after the snapshot 2023-11-28T21:37:44.972 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/37 file after the snapshot 2023-11-28T21:37:44.972 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/38 file after the snapshot 2023-11-28T21:37:44.972 INFO:tasks.workunit.client.0.smithi012.stdout: create 249/39 file after the snapshot 2023-11-28T21:37:44.972 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/.snap/snap-subdir-test 2023-11-28T21:37:45.026 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/30 file after the snapshot 2023-11-28T21:37:45.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/31 file after the snapshot 2023-11-28T21:37:45.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/32 file after the snapshot 2023-11-28T21:37:45.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/33 file after the snapshot 2023-11-28T21:37:45.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/34 file after the snapshot 2023-11-28T21:37:45.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/35 file after the snapshot 2023-11-28T21:37:45.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/36 file after the snapshot 2023-11-28T21:37:45.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/37 file after the snapshot 2023-11-28T21:37:45.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/38 file after the snapshot 2023-11-28T21:37:45.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 250/39 file after the snapshot 2023-11-28T21:37:45.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/.snap/snap-subdir-test 2023-11-28T21:37:45.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.046+0000 7f5932c54640 1 -- 172.21.15.12:0/2029313966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f592c072c10 con 0x7f592c06e300 2023-11-28T21:37:45.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f59297fa640 1 -- 172.21.15.12:0/2029313966 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f5920054760 con 0x7f592c06e300 2023-11-28T21:37:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:45.054 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:45.054 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:45.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 -- 172.21.15.12:0/2029313966 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5910066470 msgr2=0x7f5910068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5910066470 0x7f5910068930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f592c140b60 tx=0x7f5918009040 comp rx=0 tx=0).stop 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 -- 172.21.15.12:0/2029313966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 msgr2=0x7f592c13d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 0x7f592c13d8d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f592000e810 tx=0x7f592000ecd0 comp rx=0 tx=0).stop 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 -- 172.21.15.12:0/2029313966 shutdown_connections 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5910066470 0x7f5910068930 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c140c00 0x7f592c1f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f592c151af0 0x7f592c13e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 --2- 172.21.15.12:0/2029313966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f592c06e300 0x7f592c13d8d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 -- 172.21.15.12:0/2029313966 >> 172.21.15.12:0/2029313966 conn(0x7f592c066360 msgr2=0x7f592c14f950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 -- 172.21.15.12:0/2029313966 shutdown_connections 2023-11-28T21:37:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:45.050+0000 7f5932c54640 1 -- 172.21.15.12:0/2029313966 wait complete. 2023-11-28T21:37:45.075 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:45.784 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/30 file after the snapshot 2023-11-28T21:37:45.787 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/31 file after the snapshot 2023-11-28T21:37:45.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/32 file after the snapshot 2023-11-28T21:37:45.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/33 file after the snapshot 2023-11-28T21:37:45.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/34 file after the snapshot 2023-11-28T21:37:45.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/35 file after the snapshot 2023-11-28T21:37:45.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/36 file after the snapshot 2023-11-28T21:37:45.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/37 file after the snapshot 2023-11-28T21:37:45.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/38 file after the snapshot 2023-11-28T21:37:45.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 251/39 file after the snapshot 2023-11-28T21:37:45.788 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/.snap/snap-subdir-test 2023-11-28T21:37:45.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/30 file after the snapshot 2023-11-28T21:37:45.851 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/31 file after the snapshot 2023-11-28T21:37:45.851 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/32 file after the snapshot 2023-11-28T21:37:45.851 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/33 file after the snapshot 2023-11-28T21:37:45.852 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/34 file after the snapshot 2023-11-28T21:37:45.852 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/35 file after the snapshot 2023-11-28T21:37:45.852 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/36 file after the snapshot 2023-11-28T21:37:45.852 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/37 file after the snapshot 2023-11-28T21:37:45.852 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/38 file after the snapshot 2023-11-28T21:37:45.852 INFO:tasks.workunit.client.0.smithi012.stdout: create 252/39 file after the snapshot 2023-11-28T21:37:45.852 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/.snap/snap-subdir-test 2023-11-28T21:37:45.924 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/30 file after the snapshot 2023-11-28T21:37:45.927 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/31 file after the snapshot 2023-11-28T21:37:45.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/32 file after the snapshot 2023-11-28T21:37:45.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/33 file after the snapshot 2023-11-28T21:37:45.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/34 file after the snapshot 2023-11-28T21:37:45.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/35 file after the snapshot 2023-11-28T21:37:45.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/36 file after the snapshot 2023-11-28T21:37:45.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/37 file after the snapshot 2023-11-28T21:37:45.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/38 file after the snapshot 2023-11-28T21:37:45.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 253/39 file after the snapshot 2023-11-28T21:37:45.928 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/.snap/snap-subdir-test 2023-11-28T21:37:45.997 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/30 file after the snapshot 2023-11-28T21:37:46.000 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/31 file after the snapshot 2023-11-28T21:37:46.000 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/32 file after the snapshot 2023-11-28T21:37:46.000 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/33 file after the snapshot 2023-11-28T21:37:46.000 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/34 file after the snapshot 2023-11-28T21:37:46.001 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/35 file after the snapshot 2023-11-28T21:37:46.001 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/36 file after the snapshot 2023-11-28T21:37:46.001 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/37 file after the snapshot 2023-11-28T21:37:46.001 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/38 file after the snapshot 2023-11-28T21:37:46.001 INFO:tasks.workunit.client.0.smithi012.stdout: create 254/39 file after the snapshot 2023-11-28T21:37:46.001 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/.snap/snap-subdir-test 2023-11-28T21:37:46.070 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/30 file after the snapshot 2023-11-28T21:37:46.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/31 file after the snapshot 2023-11-28T21:37:46.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/32 file after the snapshot 2023-11-28T21:37:46.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/33 file after the snapshot 2023-11-28T21:37:46.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/34 file after the snapshot 2023-11-28T21:37:46.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/35 file after the snapshot 2023-11-28T21:37:46.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/36 file after the snapshot 2023-11-28T21:37:46.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/37 file after the snapshot 2023-11-28T21:37:46.073 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/38 file after the snapshot 2023-11-28T21:37:46.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 255/39 file after the snapshot 2023-11-28T21:37:46.074 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/.snap/snap-subdir-test 2023-11-28T21:37:46.338 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/30 file after the snapshot 2023-11-28T21:37:46.344 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/31 file after the snapshot 2023-11-28T21:37:46.345 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/32 file after the snapshot 2023-11-28T21:37:46.345 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/33 file after the snapshot 2023-11-28T21:37:46.346 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/34 file after the snapshot 2023-11-28T21:37:46.346 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/35 file after the snapshot 2023-11-28T21:37:46.346 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/36 file after the snapshot 2023-11-28T21:37:46.346 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/37 file after the snapshot 2023-11-28T21:37:46.347 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/38 file after the snapshot 2023-11-28T21:37:46.347 INFO:tasks.workunit.client.0.smithi012.stdout: create 256/39 file after the snapshot 2023-11-28T21:37:46.347 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/.snap/snap-subdir-test 2023-11-28T21:37:47.077 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:47.087 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/30 file after the snapshot 2023-11-28T21:37:47.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.218+0000 7f65d85df640 1 Processor -- start 2023-11-28T21:37:47.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.218+0000 7f65d85df640 1 -- start start 2023-11-28T21:37:47.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d85df640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f65d0150c20 0x7f65d0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d85df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65d006cf40 0x7f65d0158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d85df640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d0067410 con 0x7f65d006e300 2023-11-28T21:37:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d85df640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d006e090 con 0x7f65d0150c20 2023-11-28T21:37:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d85df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0157f30 con 0x7f65d006cf40 2023-11-28T21:37:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d5b7a640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f65d006e300 0x7f65d00c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:47628/0 (socket says 172.21.15.12:47628) 2023-11-28T21:37:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d5b7a640 1 -- 172.21.15.12:0/2047517585 learned_addr learned my addr 172.21.15.12:0/2047517585 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d637b640 1 --2- 172.21.15.12:0/2047517585 >> v2:172.21.15.144:3300/0 conn(0x7f65d0150c20 0x7f65d0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d6b7c640 1 --2- 172.21.15.12:0/2047517585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65d006cf40 0x7f65d0158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d5379640 1 -- 172.21.15.12:0/2047517585 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163882091 0 0) 0x7f65d0067410 con 0x7f65d006e300 2023-11-28T21:37:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d5379640 1 -- 172.21.15.12:0/2047517585 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65bc003530 con 0x7f65d006e300 2023-11-28T21:37:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d637b640 1 -- 172.21.15.12:0/2047517585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65d006cf40 msgr2=0x7f65d0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d637b640 1 --2- 172.21.15.12:0/2047517585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65d006cf40 0x7f65d0158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d637b640 1 -- 172.21.15.12:0/2047517585 >> v1:172.21.15.12:6789/0 conn(0x7f65d006e300 legacy=0x7f65d00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.222+0000 7f65d637b640 1 -- 172.21.15.12:0/2047517585 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d0151000 con 0x7f65d0150c20 2023-11-28T21:37:47.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d637b640 1 --2- 172.21.15.12:0/2047517585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65d0150c20 0x7f65d0156910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0097c0 tx=0x7f65cc01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a14bd0e26bd0b9f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:47.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d5379640 1 -- 172.21.15.12:0/2047517585 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0ad640 con 0x7f65d0150c20 2023-11-28T21:37:47.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d5379640 1 -- 172.21.15.12:0/2047517585 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc0ad7a0 con 0x7f65d0150c20 2023-11-28T21:37:47.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d5379640 1 -- 172.21.15.12:0/2047517585 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc0adb10 con 0x7f65d0150c20 2023-11-28T21:37:47.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d85df640 1 -- 172.21.15.12:0/2047517585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65d0150c20 msgr2=0x7f65d0156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:47.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d85df640 1 --2- 172.21.15.12:0/2047517585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65d0150c20 0x7f65d0156910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0097c0 tx=0x7f65cc01c480 comp rx=0 tx=0).stop 2023-11-28T21:37:47.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d85df640 1 -- 172.21.15.12:0/2047517585 shutdown_connections 2023-11-28T21:37:47.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d85df640 1 --2- 172.21.15.12:0/2047517585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65d006cf40 0x7f65d0158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:47.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d85df640 1 --2- 172.21.15.12:0/2047517585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65d0150c20 0x7f65d0156910 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:47.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d85df640 1 -- 172.21.15.12:0/2047517585 >> 172.21.15.12:0/2047517585 conn(0x7f65d0066360 msgr2=0x7f65d0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:47.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d85df640 1 -- 172.21.15.12:0/2047517585 shutdown_connections 2023-11-28T21:37:47.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d85df640 1 -- 172.21.15.12:0/2047517585 wait complete. 2023-11-28T21:37:47.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d85df640 1 Processor -- start 2023-11-28T21:37:47.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.226+0000 7f65d85df640 1 -- start start 2023-11-28T21:37:47.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d85df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65d006cf40 0x7f65d0153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:47.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d85df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65d006e300 0x7f65d00ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:47.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d85df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65d0150c20 0x7f65d014ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:47.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d85df640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0154170 con 0x7f65d0150c20 2023-11-28T21:37:47.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d85df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d0154390 con 0x7f65d006cf40 2023-11-28T21:37:47.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d85df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d01f2b00 con 0x7f65d006e300 2023-11-28T21:37:47.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d6b7c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65d0150c20 0x7f65d014ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:47.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d6b7c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65d0150c20 0x7f65d014ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47846/0 (socket says 172.21.15.12:47846) 2023-11-28T21:37:47.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d6b7c640 1 -- 172.21.15.12:0/4156340411 learned_addr learned my addr 172.21.15.12:0/4156340411 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:47.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d5b7a640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65d006e300 0x7f65d00ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:47.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d637b640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65d006cf40 0x7f65d0153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:47.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d6b7c640 1 -- 172.21.15.12:0/4156340411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65d006e300 msgr2=0x7f65d00ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:47.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d6b7c640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65d006e300 0x7f65d00ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:47.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d6b7c640 1 -- 172.21.15.12:0/4156340411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65d006cf40 msgr2=0x7f65d0153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:47.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d6b7c640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65d006cf40 0x7f65d0153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:47.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d6b7c640 1 -- 172.21.15.12:0/4156340411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc009420 con 0x7f65d0150c20 2023-11-28T21:37:47.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d6b7c640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65d0150c20 0x7f65d014ac90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f65c800efc0 tx=0x7f65c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:47.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65c6ffd640 1 -- 172.21.15.12:0/4156340411 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8014480 con 0x7f65d0150c20 2023-11-28T21:37:47.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65c6ffd640 1 -- 172.21.15.12:0/4156340411 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c800f040 con 0x7f65d0150c20 2023-11-28T21:37:47.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65c6ffd640 1 -- 172.21.15.12:0/4156340411 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8014820 con 0x7f65d0150c20 2023-11-28T21:37:47.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d85df640 1 -- 172.21.15.12:0/4156340411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d01f2de0 con 0x7f65d0150c20 2023-11-28T21:37:47.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d85df640 1 -- 172.21.15.12:0/4156340411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d01f33a0 con 0x7f65d0150c20 2023-11-28T21:37:47.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65d85df640 1 -- 172.21.15.12:0/4156340411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d0074f60 con 0x7f65d0150c20 2023-11-28T21:37:47.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.230+0000 7f65c6ffd640 1 -- 172.21.15.12:0/4156340411 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f65c8014980 con 0x7f65d0150c20 2023-11-28T21:37:47.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.234+0000 7f65c6ffd640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65b4066470 0x7f65b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:47.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.234+0000 7f65c6ffd640 1 -- 172.21.15.12:0/4156340411 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f65c8084f70 con 0x7f65d0150c20 2023-11-28T21:37:47.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.234+0000 7f65d637b640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65b4066470 0x7f65b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:47.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.234+0000 7f65d637b640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65b4066470 0x7f65b4068930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0af000 tx=0x7f65cc0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:47.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.238+0000 7f65c6ffd640 1 -- 172.21.15.12:0/4156340411 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f65c80b6280 con 0x7f65d0150c20 2023-11-28T21:37:47.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.430+0000 7f65d85df640 1 -- 172.21.15.12:0/4156340411 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65d00c2c30 con 0x7f65d0150c20 2023-11-28T21:37:47.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.430+0000 7f65c6ffd640 1 -- 172.21.15.12:0/4156340411 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f65c8018090 con 0x7f65d0150c20 2023-11-28T21:37:47.436 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:47.436 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:47.436 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:47.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 -- 172.21.15.12:0/4156340411 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65b4066470 msgr2=0x7f65b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:47.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65b4066470 0x7f65b4068930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0af000 tx=0x7f65cc0a7070 comp rx=0 tx=0).stop 2023-11-28T21:37:47.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 -- 172.21.15.12:0/4156340411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65d0150c20 msgr2=0x7f65d014ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:47.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65d0150c20 0x7f65d014ac90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f65c800efc0 tx=0x7f65c800c490 comp rx=0 tx=0).stop 2023-11-28T21:37:47.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 -- 172.21.15.12:0/4156340411 shutdown_connections 2023-11-28T21:37:47.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65b4066470 0x7f65b4068930 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:47.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65d0150c20 0x7f65d014ac90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:47.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65d006e300 0x7f65d00ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:47.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 --2- 172.21.15.12:0/4156340411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65d006cf40 0x7f65d0153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:47.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 -- 172.21.15.12:0/4156340411 >> 172.21.15.12:0/4156340411 conn(0x7f65d0066360 msgr2=0x7f65d00c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:47.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 -- 172.21.15.12:0/4156340411 shutdown_connections 2023-11-28T21:37:47.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:47.434+0000 7f65d85df640 1 -- 172.21.15.12:0/4156340411 wait complete. 2023-11-28T21:37:47.458 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:47.628 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/31 file after the snapshot 2023-11-28T21:37:47.629 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/32 file after the snapshot 2023-11-28T21:37:47.629 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/33 file after the snapshot 2023-11-28T21:37:47.629 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/34 file after the snapshot 2023-11-28T21:37:47.629 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/35 file after the snapshot 2023-11-28T21:37:47.629 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/36 file after the snapshot 2023-11-28T21:37:47.629 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/37 file after the snapshot 2023-11-28T21:37:47.629 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/38 file after the snapshot 2023-11-28T21:37:47.629 INFO:tasks.workunit.client.0.smithi012.stdout: create 257/39 file after the snapshot 2023-11-28T21:37:47.629 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/.snap/snap-subdir-test 2023-11-28T21:37:48.375 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/30 file after the snapshot 2023-11-28T21:37:48.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/31 file after the snapshot 2023-11-28T21:37:48.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/32 file after the snapshot 2023-11-28T21:37:48.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/33 file after the snapshot 2023-11-28T21:37:48.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/34 file after the snapshot 2023-11-28T21:37:48.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/35 file after the snapshot 2023-11-28T21:37:48.379 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/36 file after the snapshot 2023-11-28T21:37:48.380 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/37 file after the snapshot 2023-11-28T21:37:48.380 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/38 file after the snapshot 2023-11-28T21:37:48.380 INFO:tasks.workunit.client.0.smithi012.stdout: create 258/39 file after the snapshot 2023-11-28T21:37:48.380 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/.snap/snap-subdir-test 2023-11-28T21:37:48.448 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/30 file after the snapshot 2023-11-28T21:37:48.451 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/31 file after the snapshot 2023-11-28T21:37:48.451 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/32 file after the snapshot 2023-11-28T21:37:48.451 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/33 file after the snapshot 2023-11-28T21:37:48.451 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/34 file after the snapshot 2023-11-28T21:37:48.451 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/35 file after the snapshot 2023-11-28T21:37:48.451 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/36 file after the snapshot 2023-11-28T21:37:48.451 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/37 file after the snapshot 2023-11-28T21:37:48.451 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/38 file after the snapshot 2023-11-28T21:37:48.451 INFO:tasks.workunit.client.0.smithi012.stdout: create 259/39 file after the snapshot 2023-11-28T21:37:48.451 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/.snap/snap-subdir-test 2023-11-28T21:37:48.751 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/30 file after the snapshot 2023-11-28T21:37:48.753 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/31 file after the snapshot 2023-11-28T21:37:48.754 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/32 file after the snapshot 2023-11-28T21:37:48.754 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/33 file after the snapshot 2023-11-28T21:37:48.754 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/34 file after the snapshot 2023-11-28T21:37:48.754 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/35 file after the snapshot 2023-11-28T21:37:48.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/36 file after the snapshot 2023-11-28T21:37:48.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/37 file after the snapshot 2023-11-28T21:37:48.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/38 file after the snapshot 2023-11-28T21:37:48.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 260/39 file after the snapshot 2023-11-28T21:37:48.755 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/.snap/snap-subdir-test 2023-11-28T21:37:48.811 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/30 file after the snapshot 2023-11-28T21:37:48.814 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/31 file after the snapshot 2023-11-28T21:37:48.814 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/32 file after the snapshot 2023-11-28T21:37:48.814 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/33 file after the snapshot 2023-11-28T21:37:48.814 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/34 file after the snapshot 2023-11-28T21:37:48.815 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/35 file after the snapshot 2023-11-28T21:37:48.815 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/36 file after the snapshot 2023-11-28T21:37:48.815 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/37 file after the snapshot 2023-11-28T21:37:48.815 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/38 file after the snapshot 2023-11-28T21:37:48.815 INFO:tasks.workunit.client.0.smithi012.stdout: create 261/39 file after the snapshot 2023-11-28T21:37:48.815 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/.snap/snap-subdir-test 2023-11-28T21:37:48.866 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/30 file after the snapshot 2023-11-28T21:37:48.870 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/31 file after the snapshot 2023-11-28T21:37:48.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/32 file after the snapshot 2023-11-28T21:37:48.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/33 file after the snapshot 2023-11-28T21:37:48.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/34 file after the snapshot 2023-11-28T21:37:48.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/35 file after the snapshot 2023-11-28T21:37:48.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/36 file after the snapshot 2023-11-28T21:37:48.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/37 file after the snapshot 2023-11-28T21:37:48.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/38 file after the snapshot 2023-11-28T21:37:48.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 262/39 file after the snapshot 2023-11-28T21:37:48.871 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/.snap/snap-subdir-test 2023-11-28T21:37:48.938 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/30 file after the snapshot 2023-11-28T21:37:48.942 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/31 file after the snapshot 2023-11-28T21:37:48.942 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/32 file after the snapshot 2023-11-28T21:37:48.942 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/33 file after the snapshot 2023-11-28T21:37:48.942 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/34 file after the snapshot 2023-11-28T21:37:48.942 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/35 file after the snapshot 2023-11-28T21:37:48.942 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/36 file after the snapshot 2023-11-28T21:37:48.942 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/37 file after the snapshot 2023-11-28T21:37:48.942 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/38 file after the snapshot 2023-11-28T21:37:48.943 INFO:tasks.workunit.client.0.smithi012.stdout: create 263/39 file after the snapshot 2023-11-28T21:37:48.943 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/.snap/snap-subdir-test 2023-11-28T21:37:49.022 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/30 file after the snapshot 2023-11-28T21:37:49.025 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/31 file after the snapshot 2023-11-28T21:37:49.025 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/32 file after the snapshot 2023-11-28T21:37:49.025 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/33 file after the snapshot 2023-11-28T21:37:49.025 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/34 file after the snapshot 2023-11-28T21:37:49.025 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/35 file after the snapshot 2023-11-28T21:37:49.026 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/36 file after the snapshot 2023-11-28T21:37:49.026 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/37 file after the snapshot 2023-11-28T21:37:49.026 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/38 file after the snapshot 2023-11-28T21:37:49.026 INFO:tasks.workunit.client.0.smithi012.stdout: create 264/39 file after the snapshot 2023-11-28T21:37:49.026 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/.snap/snap-subdir-test 2023-11-28T21:37:49.459 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:49.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.610+0000 7faec2df5640 1 Processor -- start 2023-11-28T21:37:49.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.610+0000 7faec2df5640 1 -- start start 2023-11-28T21:37:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.610+0000 7faec2df5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cac40 0x7faebc06b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.610+0000 7faec2df5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faebc0cb020 0x7faebc159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.610+0000 7faec2df5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faebc0c4da0 con 0x7faebc0c4980 2023-11-28T21:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.610+0000 7faec2df5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faebc0cb4b0 con 0x7faebc0cb020 2023-11-28T21:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.610+0000 7faec2df5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15a5e0 con 0x7faebc0cac40 2023-11-28T21:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.610+0000 7faec1392640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faebc0cb020 0x7faebc159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faec1392640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faebc0cb020 0x7faebc159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47854/0 (socket says 172.21.15.12:47854) 2023-11-28T21:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faec0b91640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cac40 0x7faebc06b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faec1392640 1 -- 172.21.15.12:0/1354511248 learned_addr learned my addr 172.21.15.12:0/1354511248 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faec1392640 1 -- 172.21.15.12:0/1354511248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cac40 msgr2=0x7faebc06b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faec1392640 1 --2- 172.21.15.12:0/1354511248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cac40 0x7faebc06b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faec1392640 1 -- 172.21.15.12:0/1354511248 >> v1:172.21.15.12:6789/0 conn(0x7faebc0c4980 legacy=0x7faebc0ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:49.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faec1392640 1 -- 172.21.15.12:0/1354511248 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc15a8c0 con 0x7faebc0cb020 2023-11-28T21:37:49.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faec1392640 1 --2- 172.21.15.12:0/1354511248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0cb020 0x7faebc159d50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009e50 tx=0x7faeb401f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1709bf8ba5d65bc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:49.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faebb7fe640 1 -- 172.21.15.12:0/1354511248 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb40af400 con 0x7faebc0cb020 2023-11-28T21:37:49.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faebb7fe640 1 -- 172.21.15.12:0/1354511248 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb40ab050 con 0x7faebc0cb020 2023-11-28T21:37:49.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.614+0000 7faebb7fe640 1 -- 172.21.15.12:0/1354511248 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb40af780 con 0x7faebc0cb020 2023-11-28T21:37:49.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 -- 172.21.15.12:0/1354511248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0cb020 msgr2=0x7faebc159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:49.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 --2- 172.21.15.12:0/1354511248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0cb020 0x7faebc159d50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009e50 tx=0x7faeb401f1f0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 -- 172.21.15.12:0/1354511248 shutdown_connections 2023-11-28T21:37:49.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 --2- 172.21.15.12:0/1354511248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0cb020 0x7faebc159d50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 --2- 172.21.15.12:0/1354511248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cac40 0x7faebc06b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 -- 172.21.15.12:0/1354511248 >> 172.21.15.12:0/1354511248 conn(0x7faebc0720e0 msgr2=0x7faebc0724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:49.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 -- 172.21.15.12:0/1354511248 shutdown_connections 2023-11-28T21:37:49.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 -- 172.21.15.12:0/1354511248 wait complete. 2023-11-28T21:37:49.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 Processor -- start 2023-11-28T21:37:49.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 -- start start 2023-11-28T21:37:49.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0c4980 0x7faebc0b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:49.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faebc0cac40 0x7faebc1f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:49.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cb020 0x7faebc1f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:49.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15c060 con 0x7faebc0c4980 2023-11-28T21:37:49.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15baa0 con 0x7faebc0cac40 2023-11-28T21:37:49.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec2df5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15bd80 con 0x7faebc0cb020 2023-11-28T21:37:49.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec0b91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0c4980 0x7faebc0b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec0b91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0c4980 0x7faebc0b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47864/0 (socket says 172.21.15.12:47864) 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.618+0000 7faec0b91640 1 -- 172.21.15.12:0/2970747372 learned_addr learned my addr 172.21.15.12:0/2970747372 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec1392640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cb020 0x7faebc1f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faebbfff640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faebc0cac40 0x7faebc1f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec0b91640 1 -- 172.21.15.12:0/2970747372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cb020 msgr2=0x7faebc1f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec0b91640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cb020 0x7faebc1f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec0b91640 1 -- 172.21.15.12:0/2970747372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faebc0cac40 msgr2=0x7faebc1f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec0b91640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faebc0cac40 0x7faebc1f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec0b91640 1 -- 172.21.15.12:0/2970747372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4009b30 con 0x7faebc0c4980 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec1392640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cb020 0x7faebc1f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec0b91640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0c4980 0x7faebc0b4280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faeb000e930 tx=0x7faeb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faeb97fa640 1 -- 172.21.15.12:0/2970747372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb0009690 con 0x7faebc0c4980 2023-11-28T21:37:49.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faeb97fa640 1 -- 172.21.15.12:0/2970747372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb000ca00 con 0x7faebc0c4980 2023-11-28T21:37:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faeb97fa640 1 -- 172.21.15.12:0/2970747372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb000cd70 con 0x7faebc0c4980 2023-11-28T21:37:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec2df5640 1 -- 172.21.15.12:0/2970747372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faebc15b0b0 con 0x7faebc0c4980 2023-11-28T21:37:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec2df5640 1 -- 172.21.15.12:0/2970747372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faebc15b670 con 0x7faebc0c4980 2023-11-28T21:37:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faec2df5640 1 -- 172.21.15.12:0/2970747372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faebc067c70 con 0x7faebc0c4980 2023-11-28T21:37:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faeb97fa640 1 -- 172.21.15.12:0/2970747372 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faeb0013470 con 0x7faebc0c4980 2023-11-28T21:37:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faeb97fa640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea00664a0 0x7faea0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.622+0000 7faeb97fa640 1 -- 172.21.15.12:0/2970747372 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7faeb0084570 con 0x7faebc0c4980 2023-11-28T21:37:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.626+0000 7faebbfff640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea00664a0 0x7faea0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:49.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.626+0000 7faebbfff640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea00664a0 0x7faea0068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faebc154040 tx=0x7faea8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:49.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.630+0000 7faeb97fa640 1 -- 172.21.15.12:0/2970747372 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faeb0051590 con 0x7faebc0c4980 2023-11-28T21:37:49.636 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/30 file after the snapshot 2023-11-28T21:37:49.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.822+0000 7faec2df5640 1 -- 172.21.15.12:0/2970747372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faebc15af10 con 0x7faebc0c4980 2023-11-28T21:37:49.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.822+0000 7faeb97fa640 1 -- 172.21.15.12:0/2970747372 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7faebc15af10 con 0x7faebc0c4980 2023-11-28T21:37:49.827 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:49.828 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:49.828 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.822+0000 7faec2df5640 1 -- 172.21.15.12:0/2970747372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea00664a0 msgr2=0x7faea0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.822+0000 7faec2df5640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea00664a0 0x7faea0068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faebc154040 tx=0x7faea8009040 comp rx=0 tx=0).stop 2023-11-28T21:37:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.822+0000 7faec2df5640 1 -- 172.21.15.12:0/2970747372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0c4980 msgr2=0x7faebc0b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.822+0000 7faec2df5640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0c4980 0x7faebc0b4280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faeb000e930 tx=0x7faeb000edf0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.826+0000 7faec2df5640 1 -- 172.21.15.12:0/2970747372 shutdown_connections 2023-11-28T21:37:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.826+0000 7faec2df5640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea00664a0 0x7faea0068960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.826+0000 7faec2df5640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faebc0cb020 0x7faebc1f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.826+0000 7faec2df5640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faebc0cac40 0x7faebc1f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.826+0000 7faec2df5640 1 --2- 172.21.15.12:0/2970747372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faebc0c4980 0x7faebc0b4280 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.826+0000 7faec2df5640 1 -- 172.21.15.12:0/2970747372 >> 172.21.15.12:0/2970747372 conn(0x7faebc0720e0 msgr2=0x7faebc0cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.826+0000 7faec2df5640 1 -- 172.21.15.12:0/2970747372 shutdown_connections 2023-11-28T21:37:49.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:49.826+0000 7faec2df5640 1 -- 172.21.15.12:0/2970747372 wait complete. 2023-11-28T21:37:49.850 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:49.969 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/31 file after the snapshot 2023-11-28T21:37:49.969 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/32 file after the snapshot 2023-11-28T21:37:49.969 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/33 file after the snapshot 2023-11-28T21:37:49.969 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/34 file after the snapshot 2023-11-28T21:37:49.969 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/35 file after the snapshot 2023-11-28T21:37:49.969 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/36 file after the snapshot 2023-11-28T21:37:49.970 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/37 file after the snapshot 2023-11-28T21:37:49.970 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/38 file after the snapshot 2023-11-28T21:37:49.970 INFO:tasks.workunit.client.0.smithi012.stdout: create 265/39 file after the snapshot 2023-11-28T21:37:49.970 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/.snap/snap-subdir-test 2023-11-28T21:37:50.026 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/30 file after the snapshot 2023-11-28T21:37:50.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/31 file after the snapshot 2023-11-28T21:37:50.028 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/32 file after the snapshot 2023-11-28T21:37:50.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/33 file after the snapshot 2023-11-28T21:37:50.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/34 file after the snapshot 2023-11-28T21:37:50.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/35 file after the snapshot 2023-11-28T21:37:50.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/36 file after the snapshot 2023-11-28T21:37:50.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/37 file after the snapshot 2023-11-28T21:37:50.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/38 file after the snapshot 2023-11-28T21:37:50.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 266/39 file after the snapshot 2023-11-28T21:37:50.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/.snap/snap-subdir-test 2023-11-28T21:37:50.083 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/30 file after the snapshot 2023-11-28T21:37:50.085 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/31 file after the snapshot 2023-11-28T21:37:50.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/32 file after the snapshot 2023-11-28T21:37:50.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/33 file after the snapshot 2023-11-28T21:37:50.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/34 file after the snapshot 2023-11-28T21:37:50.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/35 file after the snapshot 2023-11-28T21:37:50.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/36 file after the snapshot 2023-11-28T21:37:50.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/37 file after the snapshot 2023-11-28T21:37:50.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/38 file after the snapshot 2023-11-28T21:37:50.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 267/39 file after the snapshot 2023-11-28T21:37:50.086 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/.snap/snap-subdir-test 2023-11-28T21:37:50.140 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/30 file after the snapshot 2023-11-28T21:37:50.143 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/31 file after the snapshot 2023-11-28T21:37:50.143 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/32 file after the snapshot 2023-11-28T21:37:50.143 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/33 file after the snapshot 2023-11-28T21:37:50.143 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/34 file after the snapshot 2023-11-28T21:37:50.143 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/35 file after the snapshot 2023-11-28T21:37:50.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/36 file after the snapshot 2023-11-28T21:37:50.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/37 file after the snapshot 2023-11-28T21:37:50.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/38 file after the snapshot 2023-11-28T21:37:50.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 268/39 file after the snapshot 2023-11-28T21:37:50.144 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/.snap/snap-subdir-test 2023-11-28T21:37:50.208 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/30 file after the snapshot 2023-11-28T21:37:50.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/31 file after the snapshot 2023-11-28T21:37:50.212 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/32 file after the snapshot 2023-11-28T21:37:50.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/33 file after the snapshot 2023-11-28T21:37:50.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/34 file after the snapshot 2023-11-28T21:37:50.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/35 file after the snapshot 2023-11-28T21:37:50.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/36 file after the snapshot 2023-11-28T21:37:50.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/37 file after the snapshot 2023-11-28T21:37:50.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/38 file after the snapshot 2023-11-28T21:37:50.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 269/39 file after the snapshot 2023-11-28T21:37:50.213 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/.snap/snap-subdir-test 2023-11-28T21:37:50.276 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/30 file after the snapshot 2023-11-28T21:37:50.279 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/31 file after the snapshot 2023-11-28T21:37:50.279 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/32 file after the snapshot 2023-11-28T21:37:50.279 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/33 file after the snapshot 2023-11-28T21:37:50.279 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/34 file after the snapshot 2023-11-28T21:37:50.279 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/35 file after the snapshot 2023-11-28T21:37:50.279 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/36 file after the snapshot 2023-11-28T21:37:50.279 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/37 file after the snapshot 2023-11-28T21:37:50.280 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/38 file after the snapshot 2023-11-28T21:37:50.280 INFO:tasks.workunit.client.0.smithi012.stdout: create 270/39 file after the snapshot 2023-11-28T21:37:50.280 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/.snap/snap-subdir-test 2023-11-28T21:37:50.638 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/30 file after the snapshot 2023-11-28T21:37:50.641 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/31 file after the snapshot 2023-11-28T21:37:50.641 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/32 file after the snapshot 2023-11-28T21:37:50.641 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/33 file after the snapshot 2023-11-28T21:37:50.641 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/34 file after the snapshot 2023-11-28T21:37:50.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/35 file after the snapshot 2023-11-28T21:37:50.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/36 file after the snapshot 2023-11-28T21:37:50.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/37 file after the snapshot 2023-11-28T21:37:50.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/38 file after the snapshot 2023-11-28T21:37:50.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 271/39 file after the snapshot 2023-11-28T21:37:50.642 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/.snap/snap-subdir-test 2023-11-28T21:37:50.694 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/30 file after the snapshot 2023-11-28T21:37:50.697 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/31 file after the snapshot 2023-11-28T21:37:50.697 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/32 file after the snapshot 2023-11-28T21:37:50.697 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/33 file after the snapshot 2023-11-28T21:37:50.697 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/34 file after the snapshot 2023-11-28T21:37:50.697 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/35 file after the snapshot 2023-11-28T21:37:50.697 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/36 file after the snapshot 2023-11-28T21:37:50.697 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/37 file after the snapshot 2023-11-28T21:37:50.697 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/38 file after the snapshot 2023-11-28T21:37:50.698 INFO:tasks.workunit.client.0.smithi012.stdout: create 272/39 file after the snapshot 2023-11-28T21:37:50.698 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/.snap/snap-subdir-test 2023-11-28T21:37:50.749 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/30 file after the snapshot 2023-11-28T21:37:50.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/31 file after the snapshot 2023-11-28T21:37:50.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/32 file after the snapshot 2023-11-28T21:37:50.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/33 file after the snapshot 2023-11-28T21:37:50.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/34 file after the snapshot 2023-11-28T21:37:50.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/35 file after the snapshot 2023-11-28T21:37:50.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/36 file after the snapshot 2023-11-28T21:37:50.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/37 file after the snapshot 2023-11-28T21:37:50.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/38 file after the snapshot 2023-11-28T21:37:50.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 273/39 file after the snapshot 2023-11-28T21:37:50.752 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/.snap/snap-subdir-test 2023-11-28T21:37:50.802 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/30 file after the snapshot 2023-11-28T21:37:50.805 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/31 file after the snapshot 2023-11-28T21:37:50.805 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/32 file after the snapshot 2023-11-28T21:37:50.805 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/33 file after the snapshot 2023-11-28T21:37:50.805 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/34 file after the snapshot 2023-11-28T21:37:50.805 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/35 file after the snapshot 2023-11-28T21:37:50.805 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/36 file after the snapshot 2023-11-28T21:37:50.805 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/37 file after the snapshot 2023-11-28T21:37:50.805 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/38 file after the snapshot 2023-11-28T21:37:50.805 INFO:tasks.workunit.client.0.smithi012.stdout: create 274/39 file after the snapshot 2023-11-28T21:37:50.806 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/.snap/snap-subdir-test 2023-11-28T21:37:51.852 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.990+0000 7f94d6259640 1 Processor -- start 2023-11-28T21:37:51.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.990+0000 7f94d6259640 1 -- start start 2023-11-28T21:37:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d6259640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f94d015a980 0x7f94d00b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d6259640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d00b44a0 con 0x7f94d0153fb0 2023-11-28T21:37:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d6259640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d015d390 con 0x7f94d00b4010 2023-11-28T21:37:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d6259640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d00b2b60 con 0x7f94d015a980 2023-11-28T21:37:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d4a56640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f94d015a980 0x7f94d00b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d5257640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f94d0153fb0 0x7f94d00ca4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:47640/0 (socket says 172.21.15.12:47640) 2023-11-28T21:37:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d4a56640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f94d015a980 0x7f94d00b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47866/0 (socket says 172.21.15.12:47866) 2023-11-28T21:37:51.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d5257640 1 -- 172.21.15.12:0/3012894930 learned_addr learned my addr 172.21.15.12:0/3012894930 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:51.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d4a56640 1 -- 172.21.15.12:0/3012894930 >> v1:172.21.15.144:6789/0 conn(0x7f94d00b4010 legacy=0x7f94d00cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:51.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d4a56640 1 -- 172.21.15.12:0/3012894930 >> v1:172.21.15.12:6789/0 conn(0x7f94d0153fb0 legacy=0x7f94d00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:51.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d4a56640 1 -- 172.21.15.12:0/3012894930 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d015d630 con 0x7f94d015a980 2023-11-28T21:37:52.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.994+0000 7f94d4a56640 1 --2- 172.21.15.12:0/3012894930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 0x7f94d00b3ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f94c00097c0 tx=0x7f94c001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e49406c9d3d137 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:52.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94c7fff640 1 -- 172.21.15.12:0/3012894930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c00a8930 con 0x7f94d015a980 2023-11-28T21:37:52.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94c7fff640 1 -- 172.21.15.12:0/3012894930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c00a8a90 con 0x7f94d015a980 2023-11-28T21:37:52.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94c7fff640 1 -- 172.21.15.12:0/3012894930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c00a8e20 con 0x7f94d015a980 2023-11-28T21:37:52.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 -- 172.21.15.12:0/3012894930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 msgr2=0x7f94d00b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:52.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 --2- 172.21.15.12:0/3012894930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 0x7f94d00b3ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f94c00097c0 tx=0x7f94c001ec30 comp rx=0 tx=0).stop 2023-11-28T21:37:52.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 -- 172.21.15.12:0/3012894930 shutdown_connections 2023-11-28T21:37:52.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 --2- 172.21.15.12:0/3012894930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 0x7f94d00b3ad0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:52.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 -- 172.21.15.12:0/3012894930 >> 172.21.15.12:0/3012894930 conn(0x7f94d0071ce0 msgr2=0x7f94d00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:52.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 -- 172.21.15.12:0/3012894930 shutdown_connections 2023-11-28T21:37:52.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 -- 172.21.15.12:0/3012894930 wait complete. 2023-11-28T21:37:52.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 Processor -- start 2023-11-28T21:37:52.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 -- start start 2023-11-28T21:37:52.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d00b4010 0x7f94d014e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:52.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94d0153fb0 0x7f94d01f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:52.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 0x7f94d00c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:52.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d015e1b0 con 0x7f94d015a980 2023-11-28T21:37:52.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d014fcc0 con 0x7f94d00b4010 2023-11-28T21:37:52.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d6259640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01f0b60 con 0x7f94d0153fb0 2023-11-28T21:37:52.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d5a58640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 0x7f94d00c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:52.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d5a58640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 0x7f94d00c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47874/0 (socket says 172.21.15.12:47874) 2023-11-28T21:37:52.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d5a58640 1 -- 172.21.15.12:0/2647303879 learned_addr learned my addr 172.21.15.12:0/2647303879 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:52.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d4a56640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94d0153fb0 0x7f94d01f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:52.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:51.998+0000 7f94d5257640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d00b4010 0x7f94d014e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:52.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d5a58640 1 -- 172.21.15.12:0/2647303879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94d0153fb0 msgr2=0x7f94d01f0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:52.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d5a58640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94d0153fb0 0x7f94d01f0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:52.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d5a58640 1 -- 172.21.15.12:0/2647303879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d00b4010 msgr2=0x7f94d014e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d5a58640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d00b4010 0x7f94d014e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d5a58640 1 -- 172.21.15.12:0/2647303879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c0009420 con 0x7f94d015a980 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d4a56640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94d0153fb0 0x7f94d01f0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d5257640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d00b4010 0x7f94d014e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d5a58640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 0x7f94d00c5820 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f94c800c780 tx=0x7f94c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94c5ffb640 1 -- 172.21.15.12:0/2647303879 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8004b70 con 0x7f94d015a980 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94c5ffb640 1 -- 172.21.15.12:0/2647303879 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c800ce90 con 0x7f94d015a980 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94c5ffb640 1 -- 172.21.15.12:0/2647303879 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c800a850 con 0x7f94d015a980 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d6259640 1 -- 172.21.15.12:0/2647303879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d01f4550 con 0x7f94d015a980 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d6259640 1 -- 172.21.15.12:0/2647303879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d00c8b20 con 0x7f94d015a980 2023-11-28T21:37:52.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94d6259640 1 -- 172.21.15.12:0/2647303879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d00c6600 con 0x7f94d015a980 2023-11-28T21:37:52.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94c5ffb640 1 -- 172.21.15.12:0/2647303879 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f94c800a9b0 con 0x7f94d015a980 2023-11-28T21:37:52.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.002+0000 7f94c5ffb640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94b40664a0 0x7f94b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:52.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.006+0000 7f94c5ffb640 1 -- 172.21.15.12:0/2647303879 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f94c8084430 con 0x7f94d015a980 2023-11-28T21:37:52.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.006+0000 7f94d5257640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94b40664a0 0x7f94b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:52.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.006+0000 7f94d5257640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94b40664a0 0x7f94b4068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94cc005890 tx=0x7f94cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:52.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.006+0000 7f94c5ffb640 1 -- 172.21.15.12:0/2647303879 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f94c8051420 con 0x7f94d015a980 2023-11-28T21:37:52.026 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/30 file after the snapshot 2023-11-28T21:37:52.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/31 file after the snapshot 2023-11-28T21:37:52.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/32 file after the snapshot 2023-11-28T21:37:52.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/33 file after the snapshot 2023-11-28T21:37:52.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/34 file after the snapshot 2023-11-28T21:37:52.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/35 file after the snapshot 2023-11-28T21:37:52.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/36 file after the snapshot 2023-11-28T21:37:52.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/37 file after the snapshot 2023-11-28T21:37:52.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/38 file after the snapshot 2023-11-28T21:37:52.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 275/39 file after the snapshot 2023-11-28T21:37:52.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/.snap/snap-subdir-test 2023-11-28T21:37:52.168 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/30 file after the snapshot 2023-11-28T21:37:52.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/31 file after the snapshot 2023-11-28T21:37:52.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/32 file after the snapshot 2023-11-28T21:37:52.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/33 file after the snapshot 2023-11-28T21:37:52.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/34 file after the snapshot 2023-11-28T21:37:52.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/35 file after the snapshot 2023-11-28T21:37:52.171 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/36 file after the snapshot 2023-11-28T21:37:52.172 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/37 file after the snapshot 2023-11-28T21:37:52.172 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/38 file after the snapshot 2023-11-28T21:37:52.172 INFO:tasks.workunit.client.0.smithi012.stdout: create 276/39 file after the snapshot 2023-11-28T21:37:52.172 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/.snap/snap-subdir-test 2023-11-28T21:37:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.202+0000 7f94d6259640 1 -- 172.21.15.12:0/2647303879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94d0074f60 con 0x7f94d015a980 2023-11-28T21:37:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.202+0000 7f94c5ffb640 1 -- 172.21.15.12:0/2647303879 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f94c8054a40 con 0x7f94d015a980 2023-11-28T21:37:52.210 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:52.210 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:52.210 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:52.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 -- 172.21.15.12:0/2647303879 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94b40664a0 msgr2=0x7f94b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94b40664a0 0x7f94b4068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94cc005890 tx=0x7f94cc009040 comp rx=0 tx=0).stop 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 -- 172.21.15.12:0/2647303879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 msgr2=0x7f94d00c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 0x7f94d00c5820 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f94c800c780 tx=0x7f94c800cc40 comp rx=0 tx=0).stop 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 -- 172.21.15.12:0/2647303879 shutdown_connections 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94b40664a0 0x7f94b4068960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94d015a980 0x7f94d00c5820 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94d0153fb0 0x7f94d01f0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 --2- 172.21.15.12:0/2647303879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94d00b4010 0x7f94d014e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 -- 172.21.15.12:0/2647303879 >> 172.21.15.12:0/2647303879 conn(0x7f94d0071ce0 msgr2=0x7f94d00720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 -- 172.21.15.12:0/2647303879 shutdown_connections 2023-11-28T21:37:52.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:52.206+0000 7f94d6259640 1 -- 172.21.15.12:0/2647303879 wait complete. 2023-11-28T21:37:52.217 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/30 file after the snapshot 2023-11-28T21:37:52.220 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/31 file after the snapshot 2023-11-28T21:37:52.220 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/32 file after the snapshot 2023-11-28T21:37:52.220 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/33 file after the snapshot 2023-11-28T21:37:52.220 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/34 file after the snapshot 2023-11-28T21:37:52.220 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/35 file after the snapshot 2023-11-28T21:37:52.221 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/36 file after the snapshot 2023-11-28T21:37:52.221 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/37 file after the snapshot 2023-11-28T21:37:52.221 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/38 file after the snapshot 2023-11-28T21:37:52.221 INFO:tasks.workunit.client.0.smithi012.stdout: create 277/39 file after the snapshot 2023-11-28T21:37:52.221 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/.snap/snap-subdir-test 2023-11-28T21:37:52.231 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:52.269 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/30 file after the snapshot 2023-11-28T21:37:52.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/31 file after the snapshot 2023-11-28T21:37:52.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/32 file after the snapshot 2023-11-28T21:37:52.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/33 file after the snapshot 2023-11-28T21:37:52.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/34 file after the snapshot 2023-11-28T21:37:52.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/35 file after the snapshot 2023-11-28T21:37:52.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/36 file after the snapshot 2023-11-28T21:37:52.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/37 file after the snapshot 2023-11-28T21:37:52.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/38 file after the snapshot 2023-11-28T21:37:52.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 278/39 file after the snapshot 2023-11-28T21:37:52.273 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/.snap/snap-subdir-test 2023-11-28T21:37:52.454 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/30 file after the snapshot 2023-11-28T21:37:52.458 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/31 file after the snapshot 2023-11-28T21:37:52.458 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/32 file after the snapshot 2023-11-28T21:37:52.459 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/33 file after the snapshot 2023-11-28T21:37:52.459 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/34 file after the snapshot 2023-11-28T21:37:52.459 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/35 file after the snapshot 2023-11-28T21:37:52.459 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/36 file after the snapshot 2023-11-28T21:37:52.459 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/37 file after the snapshot 2023-11-28T21:37:52.459 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/38 file after the snapshot 2023-11-28T21:37:52.459 INFO:tasks.workunit.client.0.smithi012.stdout: create 279/39 file after the snapshot 2023-11-28T21:37:52.459 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/.snap/snap-subdir-test 2023-11-28T21:37:52.523 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/30 file after the snapshot 2023-11-28T21:37:52.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/31 file after the snapshot 2023-11-28T21:37:52.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/32 file after the snapshot 2023-11-28T21:37:52.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/33 file after the snapshot 2023-11-28T21:37:52.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/34 file after the snapshot 2023-11-28T21:37:52.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/35 file after the snapshot 2023-11-28T21:37:52.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/36 file after the snapshot 2023-11-28T21:37:52.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/37 file after the snapshot 2023-11-28T21:37:52.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/38 file after the snapshot 2023-11-28T21:37:52.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 280/39 file after the snapshot 2023-11-28T21:37:52.526 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/.snap/snap-subdir-test 2023-11-28T21:37:52.581 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/30 file after the snapshot 2023-11-28T21:37:52.583 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/31 file after the snapshot 2023-11-28T21:37:52.583 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/32 file after the snapshot 2023-11-28T21:37:52.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/33 file after the snapshot 2023-11-28T21:37:52.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/34 file after the snapshot 2023-11-28T21:37:52.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/35 file after the snapshot 2023-11-28T21:37:52.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/36 file after the snapshot 2023-11-28T21:37:52.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/37 file after the snapshot 2023-11-28T21:37:52.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/38 file after the snapshot 2023-11-28T21:37:52.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 281/39 file after the snapshot 2023-11-28T21:37:52.584 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/.snap/snap-subdir-test 2023-11-28T21:37:52.630 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/30 file after the snapshot 2023-11-28T21:37:52.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/31 file after the snapshot 2023-11-28T21:37:52.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/32 file after the snapshot 2023-11-28T21:37:52.633 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/33 file after the snapshot 2023-11-28T21:37:52.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/34 file after the snapshot 2023-11-28T21:37:52.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/35 file after the snapshot 2023-11-28T21:37:52.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/36 file after the snapshot 2023-11-28T21:37:52.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/37 file after the snapshot 2023-11-28T21:37:52.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/38 file after the snapshot 2023-11-28T21:37:52.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 282/39 file after the snapshot 2023-11-28T21:37:52.634 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/.snap/snap-subdir-test 2023-11-28T21:37:52.684 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/30 file after the snapshot 2023-11-28T21:37:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/31 file after the snapshot 2023-11-28T21:37:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/32 file after the snapshot 2023-11-28T21:37:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/33 file after the snapshot 2023-11-28T21:37:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/34 file after the snapshot 2023-11-28T21:37:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/35 file after the snapshot 2023-11-28T21:37:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/36 file after the snapshot 2023-11-28T21:37:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/37 file after the snapshot 2023-11-28T21:37:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/38 file after the snapshot 2023-11-28T21:37:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: create 283/39 file after the snapshot 2023-11-28T21:37:52.688 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/.snap/snap-subdir-test 2023-11-28T21:37:52.740 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/30 file after the snapshot 2023-11-28T21:37:52.743 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/31 file after the snapshot 2023-11-28T21:37:52.743 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/32 file after the snapshot 2023-11-28T21:37:52.743 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/33 file after the snapshot 2023-11-28T21:37:52.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/34 file after the snapshot 2023-11-28T21:37:52.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/35 file after the snapshot 2023-11-28T21:37:52.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/36 file after the snapshot 2023-11-28T21:37:52.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/37 file after the snapshot 2023-11-28T21:37:52.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/38 file after the snapshot 2023-11-28T21:37:52.744 INFO:tasks.workunit.client.0.smithi012.stdout: create 284/39 file after the snapshot 2023-11-28T21:37:52.745 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/.snap/snap-subdir-test 2023-11-28T21:37:52.795 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/30 file after the snapshot 2023-11-28T21:37:52.797 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/31 file after the snapshot 2023-11-28T21:37:52.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/32 file after the snapshot 2023-11-28T21:37:52.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/33 file after the snapshot 2023-11-28T21:37:52.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/34 file after the snapshot 2023-11-28T21:37:52.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/35 file after the snapshot 2023-11-28T21:37:52.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/36 file after the snapshot 2023-11-28T21:37:52.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/37 file after the snapshot 2023-11-28T21:37:52.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/38 file after the snapshot 2023-11-28T21:37:52.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 285/39 file after the snapshot 2023-11-28T21:37:52.798 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/.snap/snap-subdir-test 2023-11-28T21:37:52.847 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/30 file after the snapshot 2023-11-28T21:37:52.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/31 file after the snapshot 2023-11-28T21:37:52.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/32 file after the snapshot 2023-11-28T21:37:52.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/33 file after the snapshot 2023-11-28T21:37:52.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/34 file after the snapshot 2023-11-28T21:37:52.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/35 file after the snapshot 2023-11-28T21:37:52.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/36 file after the snapshot 2023-11-28T21:37:52.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/37 file after the snapshot 2023-11-28T21:37:52.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/38 file after the snapshot 2023-11-28T21:37:52.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 286/39 file after the snapshot 2023-11-28T21:37:52.850 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/.snap/snap-subdir-test 2023-11-28T21:37:52.899 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/30 file after the snapshot 2023-11-28T21:37:52.902 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/31 file after the snapshot 2023-11-28T21:37:52.903 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/32 file after the snapshot 2023-11-28T21:37:52.903 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/33 file after the snapshot 2023-11-28T21:37:52.903 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/34 file after the snapshot 2023-11-28T21:37:52.903 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/35 file after the snapshot 2023-11-28T21:37:52.903 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/36 file after the snapshot 2023-11-28T21:37:52.903 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/37 file after the snapshot 2023-11-28T21:37:52.903 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/38 file after the snapshot 2023-11-28T21:37:52.903 INFO:tasks.workunit.client.0.smithi012.stdout: create 287/39 file after the snapshot 2023-11-28T21:37:52.903 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/.snap/snap-subdir-test 2023-11-28T21:37:53.196 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/30 file after the snapshot 2023-11-28T21:37:53.199 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/31 file after the snapshot 2023-11-28T21:37:53.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/32 file after the snapshot 2023-11-28T21:37:53.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/33 file after the snapshot 2023-11-28T21:37:53.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/34 file after the snapshot 2023-11-28T21:37:53.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/35 file after the snapshot 2023-11-28T21:37:53.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/36 file after the snapshot 2023-11-28T21:37:53.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/37 file after the snapshot 2023-11-28T21:37:53.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/38 file after the snapshot 2023-11-28T21:37:53.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 288/39 file after the snapshot 2023-11-28T21:37:53.200 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/.snap/snap-subdir-test 2023-11-28T21:37:53.249 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/30 file after the snapshot 2023-11-28T21:37:53.251 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/31 file after the snapshot 2023-11-28T21:37:53.252 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/32 file after the snapshot 2023-11-28T21:37:53.252 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/33 file after the snapshot 2023-11-28T21:37:53.252 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/34 file after the snapshot 2023-11-28T21:37:53.252 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/35 file after the snapshot 2023-11-28T21:37:53.252 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/36 file after the snapshot 2023-11-28T21:37:53.252 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/37 file after the snapshot 2023-11-28T21:37:53.252 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/38 file after the snapshot 2023-11-28T21:37:53.252 INFO:tasks.workunit.client.0.smithi012.stdout: create 289/39 file after the snapshot 2023-11-28T21:37:53.252 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/.snap/snap-subdir-test 2023-11-28T21:37:53.301 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/30 file after the snapshot 2023-11-28T21:37:53.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/31 file after the snapshot 2023-11-28T21:37:53.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/32 file after the snapshot 2023-11-28T21:37:53.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/33 file after the snapshot 2023-11-28T21:37:53.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/34 file after the snapshot 2023-11-28T21:37:53.304 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/35 file after the snapshot 2023-11-28T21:37:53.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/36 file after the snapshot 2023-11-28T21:37:53.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/37 file after the snapshot 2023-11-28T21:37:53.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/38 file after the snapshot 2023-11-28T21:37:53.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 290/39 file after the snapshot 2023-11-28T21:37:53.305 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/.snap/snap-subdir-test 2023-11-28T21:37:53.355 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/30 file after the snapshot 2023-11-28T21:37:53.357 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/31 file after the snapshot 2023-11-28T21:37:53.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/32 file after the snapshot 2023-11-28T21:37:53.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/33 file after the snapshot 2023-11-28T21:37:53.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/34 file after the snapshot 2023-11-28T21:37:53.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/35 file after the snapshot 2023-11-28T21:37:53.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/36 file after the snapshot 2023-11-28T21:37:53.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/37 file after the snapshot 2023-11-28T21:37:53.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/38 file after the snapshot 2023-11-28T21:37:53.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 291/39 file after the snapshot 2023-11-28T21:37:53.358 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/.snap/snap-subdir-test 2023-11-28T21:37:54.232 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:54.340 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/30 file after the snapshot 2023-11-28T21:37:54.342 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/31 file after the snapshot 2023-11-28T21:37:54.343 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/32 file after the snapshot 2023-11-28T21:37:54.343 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/33 file after the snapshot 2023-11-28T21:37:54.343 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/34 file after the snapshot 2023-11-28T21:37:54.343 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/35 file after the snapshot 2023-11-28T21:37:54.343 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/36 file after the snapshot 2023-11-28T21:37:54.343 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/37 file after the snapshot 2023-11-28T21:37:54.343 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/38 file after the snapshot 2023-11-28T21:37:54.343 INFO:tasks.workunit.client.0.smithi012.stdout: create 292/39 file after the snapshot 2023-11-28T21:37:54.343 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/.snap/snap-subdir-test 2023-11-28T21:37:54.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.386+0000 7f5429172640 1 Processor -- start 2023-11-28T21:37:54.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.386+0000 7f5429172640 1 -- start start 2023-11-28T21:37:54.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.386+0000 7f5429172640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542406e300 0x7f5424156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:54.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.386+0000 7f5429172640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5424067410 con 0x7f542406cf40 2023-11-28T21:37:54.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.386+0000 7f5429172640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542406e090 con 0x7f5424150c20 2023-11-28T21:37:54.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.386+0000 7f5429172640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5424157f30 con 0x7f542406e300 2023-11-28T21:37:54.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.386+0000 7f542359e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f542406cf40 0x7f5424158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:47656/0 (socket says 172.21.15.12:47656) 2023-11-28T21:37:54.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.390+0000 7f542359e640 1 -- 172.21.15.12:0/2172605640 learned_addr learned my addr 172.21.15.12:0/2172605640 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:54.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.390+0000 7f542259c640 1 --2- 172.21.15.12:0/2172605640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542406e300 0x7f5424156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:54.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.390+0000 7f5421d9b640 1 -- 172.21.15.12:0/2172605640 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971095870 0 0) 0x7f5424067410 con 0x7f542406cf40 2023-11-28T21:37:54.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.390+0000 7f5421d9b640 1 -- 172.21.15.12:0/2172605640 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f540c005740 con 0x7f542406cf40 2023-11-28T21:37:54.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.390+0000 7f5421d9b640 1 -- 172.21.15.12:0/2172605640 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584589346 0 0) 0x7f542406e090 con 0x7f5424150c20 2023-11-28T21:37:54.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.390+0000 7f5421d9b640 1 -- 172.21.15.12:0/2172605640 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5424067410 con 0x7f5424150c20 2023-11-28T21:37:54.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.390+0000 7f542259c640 1 -- 172.21.15.12:0/2172605640 >> v1:172.21.15.144:6789/0 conn(0x7f5424150c20 legacy=0x7f54240c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:54.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.390+0000 7f542259c640 1 -- 172.21.15.12:0/2172605640 >> v1:172.21.15.12:6789/0 conn(0x7f542406cf40 legacy=0x7f5424158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:54.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.390+0000 7f542259c640 1 -- 172.21.15.12:0/2172605640 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424151000 con 0x7f542406e300 2023-11-28T21:37:54.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.390+0000 7f542259c640 1 --2- 172.21.15.12:0/2172605640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542406e300 0x7f5424156910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54180097f0 tx=0x7f541801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0016e2efbd1287 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:54.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5421d9b640 1 -- 172.21.15.12:0/2172605640 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180ab600 con 0x7f542406e300 2023-11-28T21:37:54.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5421d9b640 1 -- 172.21.15.12:0/2172605640 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54180a7050 con 0x7f542406e300 2023-11-28T21:37:54.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5421d9b640 1 -- 172.21.15.12:0/2172605640 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180ab960 con 0x7f542406e300 2023-11-28T21:37:54.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5429172640 1 -- 172.21.15.12:0/2172605640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542406e300 msgr2=0x7f5424156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:54.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5429172640 1 --2- 172.21.15.12:0/2172605640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542406e300 0x7f5424156910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54180097f0 tx=0x7f541801c4c0 comp rx=0 tx=0).stop 2023-11-28T21:37:54.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5429172640 1 -- 172.21.15.12:0/2172605640 shutdown_connections 2023-11-28T21:37:54.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5429172640 1 --2- 172.21.15.12:0/2172605640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f542406e300 0x7f5424156910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:54.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5429172640 1 -- 172.21.15.12:0/2172605640 >> 172.21.15.12:0/2172605640 conn(0x7f5424066360 msgr2=0x7f5424066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:54.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5429172640 1 -- 172.21.15.12:0/2172605640 shutdown_connections 2023-11-28T21:37:54.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5429172640 1 -- 172.21.15.12:0/2172605640 wait complete. 2023-11-28T21:37:54.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5429172640 1 Processor -- start 2023-11-28T21:37:54.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.394+0000 7f5429172640 1 -- start start 2023-11-28T21:37:54.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f5429172640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f542406cf40 0x7f542406ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:54.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f5429172640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f542406e300 0x7f54241f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:54.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f5429172640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5424150c20 0x7f54241f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:54.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f5429172640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5424152fe0 con 0x7f542406e300 2023-11-28T21:37:54.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f5429172640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5424153230 con 0x7f542406cf40 2023-11-28T21:37:54.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f5429172640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5424153510 con 0x7f5424150c20 2023-11-28T21:37:54.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f542259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f542406e300 0x7f54241f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f542259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f542406e300 0x7f54241f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47890/0 (socket says 172.21.15.12:47890) 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f542259c640 1 -- 172.21.15.12:0/174621435 learned_addr learned my addr 172.21.15.12:0/174621435 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f5422d9d640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f542406cf40 0x7f542406ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f542359e640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5424150c20 0x7f54241f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f542259c640 1 -- 172.21.15.12:0/174621435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5424150c20 msgr2=0x7f54241f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f542259c640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5424150c20 0x7f54241f3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f542259c640 1 -- 172.21.15.12:0/174621435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f542406cf40 msgr2=0x7f542406ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f542259c640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f542406cf40 0x7f542406ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f542259c640 1 -- 172.21.15.12:0/174621435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418009420 con 0x7f542406e300 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f542259c640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f542406e300 0x7f54241f2d40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5418000c10 tx=0x7f5418019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:54.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f540b7fe640 1 -- 172.21.15.12:0/174621435 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180b9070 con 0x7f542406e300 2023-11-28T21:37:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f540b7fe640 1 -- 172.21.15.12:0/174621435 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54180038b0 con 0x7f542406e300 2023-11-28T21:37:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f540b7fe640 1 -- 172.21.15.12:0/174621435 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418003c20 con 0x7f542406e300 2023-11-28T21:37:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f5429172640 1 -- 172.21.15.12:0/174621435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54241537f0 con 0x7f542406e300 2023-11-28T21:37:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f5429172640 1 -- 172.21.15.12:0/174621435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5424153db0 con 0x7f542406e300 2023-11-28T21:37:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f5429172640 1 -- 172.21.15.12:0/174621435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54241f2ab0 con 0x7f542406e300 2023-11-28T21:37:54.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.398+0000 7f540b7fe640 1 -- 172.21.15.12:0/174621435 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f54180b2040 con 0x7f542406e300 2023-11-28T21:37:54.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.402+0000 7f540b7fe640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5410066470 0x7f5410068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:54.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.402+0000 7f540b7fe640 1 -- 172.21.15.12:0/174621435 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f541811d8d0 con 0x7f542406e300 2023-11-28T21:37:54.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.402+0000 7f5422d9d640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5410066470 0x7f5410068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:54.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.402+0000 7f5422d9d640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5410066470 0x7f5410068930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f54140011f0 tx=0x7f5414009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:54.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.406+0000 7f540b7fe640 1 -- 172.21.15.12:0/174621435 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f54180ea8c0 con 0x7f542406e300 2023-11-28T21:37:54.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.594+0000 7f5429172640 1 -- 172.21.15.12:0/174621435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5424074f60 con 0x7f542406e300 2023-11-28T21:37:54.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f540b7fe640 1 -- 172.21.15.12:0/174621435 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f54180b0090 con 0x7f542406e300 2023-11-28T21:37:54.603 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:54.603 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:54.605 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:54.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 -- 172.21.15.12:0/174621435 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5410066470 msgr2=0x7f5410068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:54.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5410066470 0x7f5410068930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f54140011f0 tx=0x7f5414009040 comp rx=0 tx=0).stop 2023-11-28T21:37:54.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 -- 172.21.15.12:0/174621435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f542406e300 msgr2=0x7f54241f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:54.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f542406e300 0x7f54241f2d40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5418000c10 tx=0x7f5418019260 comp rx=0 tx=0).stop 2023-11-28T21:37:54.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 -- 172.21.15.12:0/174621435 shutdown_connections 2023-11-28T21:37:54.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5410066470 0x7f5410068930 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:54.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5424150c20 0x7f54241f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:54.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f542406e300 0x7f54241f2d40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 --2- 172.21.15.12:0/174621435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f542406cf40 0x7f542406ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 -- 172.21.15.12:0/174621435 >> 172.21.15.12:0/174621435 conn(0x7f5424066360 msgr2=0x7f5424156910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 -- 172.21.15.12:0/174621435 shutdown_connections 2023-11-28T21:37:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:54.598+0000 7f5429172640 1 -- 172.21.15.12:0/174621435 wait complete. 2023-11-28T21:37:54.623 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-11-28T21:37:54.625 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 17, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 16, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_4250": { "addr": "172.21.15.12:6839/3027655738", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3027655738, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3027655738, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4250, "incarnation": 8, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:stopping", "state_seq": 11 }, "gid_4261": { "addr": "172.21.15.144:6833/3053317078", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 3053317078, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 3053317078, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4261, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:37:33.466684+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 4261, "mds_1": 4250 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6838/2321889805", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2321889805, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 2321889805, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 17, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5127, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:37:54.782 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/30 file after the snapshot 2023-11-28T21:37:54.786 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/31 file after the snapshot 2023-11-28T21:37:54.786 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/32 file after the snapshot 2023-11-28T21:37:54.786 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/33 file after the snapshot 2023-11-28T21:37:54.786 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/34 file after the snapshot 2023-11-28T21:37:54.832 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/35 file after the snapshot 2023-11-28T21:37:54.832 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/36 file after the snapshot 2023-11-28T21:37:54.833 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/37 file after the snapshot 2023-11-28T21:37:54.833 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/38 file after the snapshot 2023-11-28T21:37:54.833 INFO:tasks.workunit.client.0.smithi012.stdout: create 293/39 file after the snapshot 2023-11-28T21:37:54.833 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/.snap/snap-subdir-test 2023-11-28T21:37:54.833 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/30 file after the snapshot 2023-11-28T21:37:54.836 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/31 file after the snapshot 2023-11-28T21:37:54.836 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/32 file after the snapshot 2023-11-28T21:37:54.836 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/33 file after the snapshot 2023-11-28T21:37:54.836 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/34 file after the snapshot 2023-11-28T21:37:54.836 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/35 file after the snapshot 2023-11-28T21:37:54.836 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/36 file after the snapshot 2023-11-28T21:37:54.837 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/37 file after the snapshot 2023-11-28T21:37:54.837 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/38 file after the snapshot 2023-11-28T21:37:54.837 INFO:tasks.workunit.client.0.smithi012.stdout: create 294/39 file after the snapshot 2023-11-28T21:37:54.837 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/.snap/snap-subdir-test 2023-11-28T21:37:54.886 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/30 file after the snapshot 2023-11-28T21:37:54.890 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/31 file after the snapshot 2023-11-28T21:37:54.890 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/32 file after the snapshot 2023-11-28T21:37:54.891 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/33 file after the snapshot 2023-11-28T21:37:54.891 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/34 file after the snapshot 2023-11-28T21:37:54.891 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/35 file after the snapshot 2023-11-28T21:37:54.891 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/36 file after the snapshot 2023-11-28T21:37:54.891 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/37 file after the snapshot 2023-11-28T21:37:54.891 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/38 file after the snapshot 2023-11-28T21:37:54.891 INFO:tasks.workunit.client.0.smithi012.stdout: create 295/39 file after the snapshot 2023-11-28T21:37:54.891 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/.snap/snap-subdir-test 2023-11-28T21:37:54.954 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/30 file after the snapshot 2023-11-28T21:37:54.958 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/31 file after the snapshot 2023-11-28T21:37:54.958 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/32 file after the snapshot 2023-11-28T21:37:54.958 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/33 file after the snapshot 2023-11-28T21:37:54.959 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/34 file after the snapshot 2023-11-28T21:37:54.959 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/35 file after the snapshot 2023-11-28T21:37:54.959 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/36 file after the snapshot 2023-11-28T21:37:54.959 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/37 file after the snapshot 2023-11-28T21:37:54.959 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/38 file after the snapshot 2023-11-28T21:37:54.959 INFO:tasks.workunit.client.0.smithi012.stdout: create 296/39 file after the snapshot 2023-11-28T21:37:54.959 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/.snap/snap-subdir-test 2023-11-28T21:37:55.027 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/30 file after the snapshot 2023-11-28T21:37:55.029 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/31 file after the snapshot 2023-11-28T21:37:55.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/32 file after the snapshot 2023-11-28T21:37:55.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/33 file after the snapshot 2023-11-28T21:37:55.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/34 file after the snapshot 2023-11-28T21:37:55.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/35 file after the snapshot 2023-11-28T21:37:55.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/36 file after the snapshot 2023-11-28T21:37:55.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/37 file after the snapshot 2023-11-28T21:37:55.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/38 file after the snapshot 2023-11-28T21:37:55.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 297/39 file after the snapshot 2023-11-28T21:37:55.030 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/.snap/snap-subdir-test 2023-11-28T21:37:55.089 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/30 file after the snapshot 2023-11-28T21:37:55.092 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/31 file after the snapshot 2023-11-28T21:37:55.092 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/32 file after the snapshot 2023-11-28T21:37:55.092 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/33 file after the snapshot 2023-11-28T21:37:55.092 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/34 file after the snapshot 2023-11-28T21:37:55.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/35 file after the snapshot 2023-11-28T21:37:55.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/36 file after the snapshot 2023-11-28T21:37:55.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/37 file after the snapshot 2023-11-28T21:37:55.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/38 file after the snapshot 2023-11-28T21:37:55.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 298/39 file after the snapshot 2023-11-28T21:37:55.093 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/.snap/snap-subdir-test 2023-11-28T21:37:55.149 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/30 file after the snapshot 2023-11-28T21:37:55.152 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/31 file after the snapshot 2023-11-28T21:37:55.152 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/32 file after the snapshot 2023-11-28T21:37:55.153 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/33 file after the snapshot 2023-11-28T21:37:55.153 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/34 file after the snapshot 2023-11-28T21:37:55.153 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/35 file after the snapshot 2023-11-28T21:37:55.153 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/36 file after the snapshot 2023-11-28T21:37:55.153 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/37 file after the snapshot 2023-11-28T21:37:55.153 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/38 file after the snapshot 2023-11-28T21:37:55.153 INFO:tasks.workunit.client.0.smithi012.stdout: create 299/39 file after the snapshot 2023-11-28T21:37:55.154 INFO:tasks.workunit.client.0.smithi012.stdout:Take second snapshot .snap/test2 2023-11-28T21:37:55.743 INFO:tasks.workunit.client.0.smithi012.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2023-11-28T21:37:56.626 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:56.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedd8f2a640 1 Processor -- start 2023-11-28T21:37:56.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedd8f2a640 1 -- start start 2023-11-28T21:37:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedd8f2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd4150c20 0x7fedd4156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedd8f2a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fedd406e300 0x7fedd40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedd8f2a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd4067410 con 0x7fedd406cf40 2023-11-28T21:37:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedd8f2a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd406e090 con 0x7fedd406e300 2023-11-28T21:37:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedd8f2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd4157f30 con 0x7fedd4150c20 2023-11-28T21:37:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedcbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fedd406e300 0x7fedd40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:56.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedcbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fedd406e300 0x7fedd40c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54466/0 (socket says 172.21.15.12:54466) 2023-11-28T21:37:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedcbfff640 1 -- 172.21.15.12:0/3402547093 learned_addr learned my addr 172.21.15.12:0/3402547093 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedd259c640 1 --2- 172.21.15.12:0/3402547093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd4150c20 0x7fedd4156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedcbfff640 1 -- 172.21.15.12:0/3402547093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd4150c20 msgr2=0x7fedd4156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedcbfff640 1 --2- 172.21.15.12:0/3402547093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd4150c20 0x7fedd4156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedcbfff640 1 -- 172.21.15.12:0/3402547093 >> v1:172.21.15.144:6789/0 conn(0x7fedd406cf40 legacy=0x7fedd4158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:56.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.798+0000 7fedcbfff640 1 -- 172.21.15.12:0/3402547093 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd4151000 con 0x7fedd406e300 2023-11-28T21:37:56.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedcbfff640 1 --2- 172.21.15.12:0/3402547093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406e300 0x7fedd40c6dd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009870 tx=0x7fedbc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a50909d357857b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:56.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd1d9b640 1 -- 172.21.15.12:0/3402547093 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0ab650 con 0x7fedd406e300 2023-11-28T21:37:56.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd1d9b640 1 -- 172.21.15.12:0/3402547093 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc0ab7b0 con 0x7fedd406e300 2023-11-28T21:37:56.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd1d9b640 1 -- 172.21.15.12:0/3402547093 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0abb40 con 0x7fedd406e300 2023-11-28T21:37:56.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd8f2a640 1 -- 172.21.15.12:0/3402547093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406e300 msgr2=0x7fedd40c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:56.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd8f2a640 1 --2- 172.21.15.12:0/3402547093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406e300 0x7fedd40c6dd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009870 tx=0x7fedbc01c540 comp rx=0 tx=0).stop 2023-11-28T21:37:56.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd8f2a640 1 -- 172.21.15.12:0/3402547093 shutdown_connections 2023-11-28T21:37:56.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd8f2a640 1 --2- 172.21.15.12:0/3402547093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406e300 0x7fedd40c6dd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd8f2a640 1 --2- 172.21.15.12:0/3402547093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd4150c20 0x7fedd4156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd8f2a640 1 -- 172.21.15.12:0/3402547093 >> 172.21.15.12:0/3402547093 conn(0x7fedd4066360 msgr2=0x7fedd4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd8f2a640 1 -- 172.21.15.12:0/3402547093 shutdown_connections 2023-11-28T21:37:56.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.802+0000 7fedd8f2a640 1 -- 172.21.15.12:0/3402547093 wait complete. 2023-11-28T21:37:56.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 Processor -- start 2023-11-28T21:37:56.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 -- start start 2023-11-28T21:37:56.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406cf40 0x7fedd406a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:56.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd406e300 0x7fedd4162650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:56.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd4150c20 0x7fedd4162b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:56.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd40c3420 con 0x7fedd406cf40 2023-11-28T21:37:56.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd40c4580 con 0x7fedd406e300 2023-11-28T21:37:56.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd41f4290 con 0x7fedd4150c20 2023-11-28T21:37:56.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406cf40 0x7fedd406a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:56.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406cf40 0x7fedd406a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54470/0 (socket says 172.21.15.12:54470) 2023-11-28T21:37:56.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd259c640 1 -- 172.21.15.12:0/383068452 learned_addr learned my addr 172.21.15.12:0/383068452 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:56.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd2d9d640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd4150c20 0x7fedd4162b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:56.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedcbfff640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd406e300 0x7fedd4162650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:56.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd259c640 1 -- 172.21.15.12:0/383068452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd4150c20 msgr2=0x7fedd4162b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:56.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd259c640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd4150c20 0x7fedd4162b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:56.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd259c640 1 -- 172.21.15.12:0/383068452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd406e300 msgr2=0x7fedd4162650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:56.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd259c640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd406e300 0x7fedd4162650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:56.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd259c640 1 -- 172.21.15.12:0/383068452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc0094d0 con 0x7fedd406cf40 2023-11-28T21:37:56.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd259c640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406cf40 0x7fedd406a810 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fedc400e8a0 tx=0x7fedc400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:56.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedcaffd640 1 -- 172.21.15.12:0/383068452 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4009510 con 0x7fedd406cf40 2023-11-28T21:37:56.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedcaffd640 1 -- 172.21.15.12:0/383068452 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc400c940 con 0x7fedd406cf40 2023-11-28T21:37:56.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedcaffd640 1 -- 172.21.15.12:0/383068452 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400ccb0 con 0x7fedd406cf40 2023-11-28T21:37:56.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 -- 172.21.15.12:0/383068452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd40c47f0 con 0x7fedd406cf40 2023-11-28T21:37:56.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 -- 172.21.15.12:0/383068452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd40c4db0 con 0x7fedd406cf40 2023-11-28T21:37:56.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.806+0000 7fedd8f2a640 1 -- 172.21.15.12:0/383068452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd4074f60 con 0x7fedd406cf40 2023-11-28T21:37:56.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.810+0000 7fedcaffd640 1 -- 172.21.15.12:0/383068452 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fedc4013470 con 0x7fedd406cf40 2023-11-28T21:37:56.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.810+0000 7fedcaffd640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fedb4066470 0x7fedb4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:56.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.810+0000 7fedcaffd640 1 -- 172.21.15.12:0/383068452 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fedc40844c0 con 0x7fedd406cf40 2023-11-28T21:37:56.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.810+0000 7fedcbfff640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fedb4066470 0x7fedb4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:56.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.810+0000 7fedcbfff640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fedb4066470 0x7fedb4068930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0af000 tx=0x7fedbc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:56.814+0000 7fedcaffd640 1 -- 172.21.15.12:0/383068452 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fedc4051420 con 0x7fedd406cf40 2023-11-28T21:37:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 -- 172.21.15.12:0/383068452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedd406bae0 con 0x7fedd406cf40 2023-11-28T21:37:57.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedcaffd640 1 -- 172.21.15.12:0/383068452 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7fedc4054a40 con 0x7fedd406cf40 2023-11-28T21:37:57.011 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:57.012 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:57.012 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 -- 172.21.15.12:0/383068452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fedb4066470 msgr2=0x7fedb4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fedb4066470 0x7fedb4068930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0af000 tx=0x7fedbc0a7000 comp rx=0 tx=0).stop 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 -- 172.21.15.12:0/383068452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406cf40 msgr2=0x7fedd406a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406cf40 0x7fedd406a810 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fedc400e8a0 tx=0x7fedc400eca0 comp rx=0 tx=0).stop 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 -- 172.21.15.12:0/383068452 shutdown_connections 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fedb4066470 0x7fedb4068930 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedd4150c20 0x7fedd4162b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedd406e300 0x7fedd4162650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 --2- 172.21.15.12:0/383068452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedd406cf40 0x7fedd406a810 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.006+0000 7fedd8f2a640 1 -- 172.21.15.12:0/383068452 >> 172.21.15.12:0/383068452 conn(0x7fedd4066360 msgr2=0x7fedd40724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.010+0000 7fedd8f2a640 1 -- 172.21.15.12:0/383068452 shutdown_connections 2023-11-28T21:37:57.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:57.010+0000 7fedd8f2a640 1 -- 172.21.15.12:0/383068452 wait complete. 2023-11-28T21:37:57.032 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:37:59.033 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:37:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.182+0000 7f7201c9e640 1 Processor -- start 2023-11-28T21:37:59.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.182+0000 7f7201c9e640 1 -- start start 2023-11-28T21:37:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.182+0000 7f7201c9e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71fc0c4000 0x7f71fc06d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.182+0000 7f7201c9e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f71fc161600 0x7f71fc161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.182+0000 7f7201c9e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc0c3460 con 0x7f71fc158420 2023-11-28T21:37:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.182+0000 7f7201c9e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc158800 con 0x7f71fc161600 2023-11-28T21:37:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.182+0000 7f7201c9e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc0c52c0 con 0x7f71fc0c4000 2023-11-28T21:37:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f71fc161600 0x7f71fc161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fb7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f71fc158420 0x7f71fc0c38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34128/0 (socket says 172.21.15.12:34128) 2023-11-28T21:37:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fb7fe640 1 -- 172.21.15.12:0/2724502144 learned_addr learned my addr 172.21.15.12:0/2724502144 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:37:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71faffd640 1 --2- 172.21.15.12:0/2724502144 >> v2:172.21.15.144:3300/0 conn(0x7f71fc0c4000 0x7f71fc06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fbfff640 1 -- 172.21.15.12:0/2724502144 >> v2:172.21.15.144:3300/0 conn(0x7f71fc0c4000 msgr2=0x7f71fc06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:37:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fbfff640 1 --2- 172.21.15.12:0/2724502144 >> v2:172.21.15.144:3300/0 conn(0x7f71fc0c4000 0x7f71fc06d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fbfff640 1 -- 172.21.15.12:0/2724502144 >> v1:172.21.15.12:6789/0 conn(0x7f71fc158420 legacy=0x7f71fc0c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fbfff640 1 -- 172.21.15.12:0/2724502144 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc0c2b70 con 0x7f71fc161600 2023-11-28T21:37:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fa7fc640 1 -- 172.21.15.12:0/2724502144 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426060734 0 0) 0x7f71fc0c3460 con 0x7f71fc158420 2023-11-28T21:37:59.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71faffd640 1 --2- 172.21.15.12:0/2724502144 >> v2:172.21.15.144:3300/0 conn(0x7f71fc0c4000 0x7f71fc06d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fbfff640 1 --2- 172.21.15.12:0/2724502144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 0x7f71fc161a10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f71fc06b930 tx=0x7f71f401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3405e3870d2292af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fa7fc640 1 -- 172.21.15.12:0/2724502144 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40a57d0 con 0x7f71fc161600 2023-11-28T21:37:59.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fa7fc640 1 -- 172.21.15.12:0/2724502144 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f40a5930 con 0x7f71fc161600 2023-11-28T21:37:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.186+0000 7f71fa7fc640 1 -- 172.21.15.12:0/2724502144 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40a5cc0 con 0x7f71fc161600 2023-11-28T21:37:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 -- 172.21.15.12:0/2724502144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 msgr2=0x7f71fc161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 --2- 172.21.15.12:0/2724502144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 0x7f71fc161a10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f71fc06b930 tx=0x7f71f401c910 comp rx=0 tx=0).stop 2023-11-28T21:37:59.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 -- 172.21.15.12:0/2724502144 shutdown_connections 2023-11-28T21:37:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 --2- 172.21.15.12:0/2724502144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 0x7f71fc161a10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 --2- 172.21.15.12:0/2724502144 >> v2:172.21.15.144:3300/0 conn(0x7f71fc0c4000 0x7f71fc06d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 -- 172.21.15.12:0/2724502144 >> 172.21.15.12:0/2724502144 conn(0x7f71fc066360 msgr2=0x7f71fc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 -- 172.21.15.12:0/2724502144 shutdown_connections 2023-11-28T21:37:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 -- 172.21.15.12:0/2724502144 wait complete. 2023-11-28T21:37:59.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 Processor -- start 2023-11-28T21:37:59.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 -- start start 2023-11-28T21:37:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71fc0c4000 0x7f71fc070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71fc158420 0x7f71fc15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 0x7f71fc14a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc1573a0 con 0x7f71fc161600 2023-11-28T21:37:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc15b990 con 0x7f71fc158420 2023-11-28T21:37:59.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f7201c9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc1570e0 con 0x7f71fc0c4000 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f71fbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 0x7f71fc14a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f71fbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 0x7f71fc14a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54486/0 (socket says 172.21.15.12:54486) 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f71fbfff640 1 -- 172.21.15.12:0/828517732 learned_addr learned my addr 172.21.15.12:0/828517732 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f71faffd640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71fc158420 0x7f71fc15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.190+0000 7f71fb7fe640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71fc0c4000 0x7f71fc070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71fbfff640 1 -- 172.21.15.12:0/828517732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71fc0c4000 msgr2=0x7f71fc070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71fbfff640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71fc0c4000 0x7f71fc070b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71fbfff640 1 -- 172.21.15.12:0/828517732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71fc158420 msgr2=0x7f71fc15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71fbfff640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71fc158420 0x7f71fc15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71fbfff640 1 -- 172.21.15.12:0/828517732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f4009a70 con 0x7f71fc161600 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71fb7fe640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71fc0c4000 0x7f71fc070b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:37:59.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71fbfff640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 0x7f71fc14a630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f71f4019520 tx=0x7f71f40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71e3fff640 1 -- 172.21.15.12:0/828517732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40b9650 con 0x7f71fc161600 2023-11-28T21:37:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71e3fff640 1 -- 172.21.15.12:0/828517732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f40b97b0 con 0x7f71fc161600 2023-11-28T21:37:59.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71e3fff640 1 -- 172.21.15.12:0/828517732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f40b9b60 con 0x7f71fc161600 2023-11-28T21:37:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f7201c9e640 1 -- 172.21.15.12:0/828517732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc14dd50 con 0x7f71fc161600 2023-11-28T21:37:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f7201c9e640 1 -- 172.21.15.12:0/828517732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc15b620 con 0x7f71fc161600 2023-11-28T21:37:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f7201c9e640 1 -- 172.21.15.12:0/828517732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71fc074f60 con 0x7f71fc161600 2023-11-28T21:37:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71e3fff640 1 -- 172.21.15.12:0/828517732 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f71f40b9cc0 con 0x7f71fc161600 2023-11-28T21:37:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71e3fff640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71e80664c0 0x7f71e8068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:37:59.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.194+0000 7f71e3fff640 1 -- 172.21.15.12:0/828517732 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f71f411f3f0 con 0x7f71fc161600 2023-11-28T21:37:59.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.198+0000 7f71fb7fe640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71e80664c0 0x7f71e8068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:37:59.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.198+0000 7f71fb7fe640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71e80664c0 0x7f71e8068980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f71ec007fd0 tx=0x7f71ec007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:37:59.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.198+0000 7f71e3fff640 1 -- 172.21.15.12:0/828517732 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f71f40ec4f0 con 0x7f71fc161600 2023-11-28T21:37:59.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.390+0000 7f7201c9e640 1 -- 172.21.15.12:0/828517732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71fc0727f0 con 0x7f71fc161600 2023-11-28T21:37:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f71e3fff640 1 -- 172.21.15.12:0/828517732 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f71f40efb10 con 0x7f71fc161600 2023-11-28T21:37:59.398 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:37:59.398 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:37:59.398 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:37:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 -- 172.21.15.12:0/828517732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71e80664c0 msgr2=0x7f71e8068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71e80664c0 0x7f71e8068980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f71ec007fd0 tx=0x7f71ec007c20 comp rx=0 tx=0).stop 2023-11-28T21:37:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 -- 172.21.15.12:0/828517732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 msgr2=0x7f71fc14a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:37:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 0x7f71fc14a630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f71f4019520 tx=0x7f71f40196b0 comp rx=0 tx=0).stop 2023-11-28T21:37:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 -- 172.21.15.12:0/828517732 shutdown_connections 2023-11-28T21:37:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f71e80664c0 0x7f71e8068980 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71fc161600 0x7f71fc14a630 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71fc158420 0x7f71fc15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:59.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 --2- 172.21.15.12:0/828517732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71fc0c4000 0x7f71fc070b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:37:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 -- 172.21.15.12:0/828517732 >> 172.21.15.12:0/828517732 conn(0x7f71fc066360 msgr2=0x7f71fc0c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:37:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 -- 172.21.15.12:0/828517732 shutdown_connections 2023-11-28T21:37:59.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:37:59.394+0000 7f7201c9e640 1 -- 172.21.15.12:0/828517732 wait complete. 2023-11-28T21:37:59.419 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:38:01.420 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:01.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.570+0000 7fe05e25f640 1 Processor -- start 2023-11-28T21:38:01.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.570+0000 7fe05e25f640 1 -- start start 2023-11-28T21:38:01.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe05e25f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe05806e300 0x7fe058156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:01.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe05e25f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe05806cf40 0x7fe058158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:01.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe05e25f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe058067410 con 0x7fe058150c20 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe05e25f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05806e090 con 0x7fe05806cf40 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe05e25f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe058157f30 con 0x7fe05806e300 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe0577fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe058150c20 0x7fe0580c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34132/0 (socket says 172.21.15.12:34132) 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe0577fe640 1 -- 172.21.15.12:0/2924599471 learned_addr learned my addr 172.21.15.12:0/2924599471 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe057fff640 1 --2- 172.21.15.12:0/2924599471 >> v2:172.21.15.12:3300/0 conn(0x7fe05806cf40 0x7fe058158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe056ffd640 1 --2- 172.21.15.12:0/2924599471 >> v2:172.21.15.144:3300/0 conn(0x7fe05806e300 0x7fe058156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe0567fc640 1 -- 172.21.15.12:0/2924599471 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563261543 0 0) 0x7fe058067410 con 0x7fe058150c20 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe057fff640 1 -- 172.21.15.12:0/2924599471 >> v2:172.21.15.144:3300/0 conn(0x7fe05806e300 msgr2=0x7fe058156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe057fff640 1 --2- 172.21.15.12:0/2924599471 >> v2:172.21.15.144:3300/0 conn(0x7fe05806e300 0x7fe058156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe057fff640 1 -- 172.21.15.12:0/2924599471 >> v1:172.21.15.12:6789/0 conn(0x7fe058150c20 legacy=0x7fe0580c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:01.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe057fff640 1 -- 172.21.15.12:0/2924599471 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe058151000 con 0x7fe05806cf40 2023-11-28T21:38:01.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.574+0000 7fe057fff640 1 --2- 172.21.15.12:0/2924599471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 0x7fe058158410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe0580c8e60 tx=0x7fe05001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f24ce48164e6ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:01.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe0567fc640 1 -- 172.21.15.12:0/2924599471 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0500a58c0 con 0x7fe05806cf40 2023-11-28T21:38:01.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe0567fc640 1 -- 172.21.15.12:0/2924599471 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0500a5a20 con 0x7fe05806cf40 2023-11-28T21:38:01.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe0567fc640 1 -- 172.21.15.12:0/2924599471 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0500a5db0 con 0x7fe05806cf40 2023-11-28T21:38:01.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 -- 172.21.15.12:0/2924599471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 msgr2=0x7fe058158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:01.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 --2- 172.21.15.12:0/2924599471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 0x7fe058158410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe0580c8e60 tx=0x7fe05001c910 comp rx=0 tx=0).stop 2023-11-28T21:38:01.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 -- 172.21.15.12:0/2924599471 shutdown_connections 2023-11-28T21:38:01.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 --2- 172.21.15.12:0/2924599471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 0x7fe058158410 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:01.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 --2- 172.21.15.12:0/2924599471 >> v2:172.21.15.144:3300/0 conn(0x7fe05806e300 0x7fe058156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:01.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 -- 172.21.15.12:0/2924599471 >> 172.21.15.12:0/2924599471 conn(0x7fe058066360 msgr2=0x7fe058066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:01.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 -- 172.21.15.12:0/2924599471 shutdown_connections 2023-11-28T21:38:01.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 -- 172.21.15.12:0/2924599471 wait complete. 2023-11-28T21:38:01.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 Processor -- start 2023-11-28T21:38:01.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 -- start start 2023-11-28T21:38:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 0x7fe0581f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe05806e300 0x7fe0581f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe058150c20 0x7fe05814b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0581517a0 con 0x7fe05806cf40 2023-11-28T21:38:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe058072c20 con 0x7fe05806e300 2023-11-28T21:38:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe05e25f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0581f7370 con 0x7fe058150c20 2023-11-28T21:38:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe0577fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 0x7fe0581f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe0577fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 0x7fe0581f64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54512/0 (socket says 172.21.15.12:54512) 2023-11-28T21:38:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.578+0000 7fe0577fe640 1 -- 172.21.15.12:0/520332943 learned_addr learned my addr 172.21.15.12:0/520332943 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe056ffd640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe05806e300 0x7fe0581f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe057fff640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe058150c20 0x7fe05814b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe0577fe640 1 -- 172.21.15.12:0/520332943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe058150c20 msgr2=0x7fe05814b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe0577fe640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe058150c20 0x7fe05814b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe0577fe640 1 -- 172.21.15.12:0/520332943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe05806e300 msgr2=0x7fe0581f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe0577fe640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe05806e300 0x7fe0581f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe0577fe640 1 -- 172.21.15.12:0/520332943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe050009a70 con 0x7fe05806cf40 2023-11-28T21:38:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe0577fe640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 0x7fe0581f64b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe0480011f0 tx=0x7fe04800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe03ffff640 1 -- 172.21.15.12:0/520332943 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0480074e0 con 0x7fe05806cf40 2023-11-28T21:38:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe03ffff640 1 -- 172.21.15.12:0/520332943 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe048007640 con 0x7fe05806cf40 2023-11-28T21:38:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe05e25f640 1 -- 172.21.15.12:0/520332943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0581f6ad0 con 0x7fe05806cf40 2023-11-28T21:38:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe03ffff640 1 -- 172.21.15.12:0/520332943 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048004a10 con 0x7fe05806cf40 2023-11-28T21:38:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe05e25f640 1 -- 172.21.15.12:0/520332943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0581f7090 con 0x7fe05806cf40 2023-11-28T21:38:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe05e25f640 1 -- 172.21.15.12:0/520332943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe058074f60 con 0x7fe05806cf40 2023-11-28T21:38:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe03ffff640 1 -- 172.21.15.12:0/520332943 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe048004b70 con 0x7fe05806cf40 2023-11-28T21:38:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe03ffff640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe044066470 0x7fe044068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.582+0000 7fe03ffff640 1 -- 172.21.15.12:0/520332943 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fe0480834b0 con 0x7fe05806cf40 2023-11-28T21:38:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.586+0000 7fe056ffd640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe044066470 0x7fe044068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:01.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.586+0000 7fe056ffd640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe044066470 0x7fe044068930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe0581f0b70 tx=0x7fe04c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:01.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.586+0000 7fe03ffff640 1 -- 172.21.15.12:0/520332943 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe0480505e0 con 0x7fe05806cf40 2023-11-28T21:38:01.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.870+0000 7fe05e25f640 1 -- 172.21.15.12:0/520332943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0580c9cc0 con 0x7fe05806cf40 2023-11-28T21:38:01.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.870+0000 7fe03ffff640 1 -- 172.21.15.12:0/520332943 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7fe048018090 con 0x7fe05806cf40 2023-11-28T21:38:01.877 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:01.877 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:01.878 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:38:01.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 -- 172.21.15.12:0/520332943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe044066470 msgr2=0x7fe044068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:01.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe044066470 0x7fe044068930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe0581f0b70 tx=0x7fe04c009190 comp rx=0 tx=0).stop 2023-11-28T21:38:01.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 -- 172.21.15.12:0/520332943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 msgr2=0x7fe0581f64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 0x7fe0581f64b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe0480011f0 tx=0x7fe04800c9b0 comp rx=0 tx=0).stop 2023-11-28T21:38:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 -- 172.21.15.12:0/520332943 shutdown_connections 2023-11-28T21:38:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe044066470 0x7fe044068930 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe058150c20 0x7fe05814b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe05806e300 0x7fe0581f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 --2- 172.21.15.12:0/520332943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05806cf40 0x7fe0581f64b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 -- 172.21.15.12:0/520332943 >> 172.21.15.12:0/520332943 conn(0x7fe058066360 msgr2=0x7fe0580ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 -- 172.21.15.12:0/520332943 shutdown_connections 2023-11-28T21:38:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:01.874+0000 7fe05e25f640 1 -- 172.21.15.12:0/520332943 wait complete. 2023-11-28T21:38:01.898 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:38:03.899 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:04.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.050+0000 7f193b11d640 1 Processor -- start 2023-11-28T21:38:04.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.050+0000 7f193b11d640 1 -- start start 2023-11-28T21:38:04.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.050+0000 7f193b11d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1934169ed0 0x7f193408c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:04.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.050+0000 7f193b11d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1934072db0 0x7f19340c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:04.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.054+0000 7f193b11d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19340c3f90 con 0x7f19340c3b00 2023-11-28T21:38:04.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.054+0000 7f193b11d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193406e390 con 0x7f1934072db0 2023-11-28T21:38:04.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.054+0000 7f193b11d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193406dfb0 con 0x7f1934169ed0 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.054+0000 7f1933fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1934072db0 0x7f19340c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.054+0000 7f19396ba640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f19340c3b00 0x7f193416a2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34144/0 (socket says 172.21.15.12:34144) 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.054+0000 7f19396ba640 1 -- 172.21.15.12:0/1384459922 learned_addr learned my addr 172.21.15.12:0/1384459922 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.054+0000 7f1938eb9640 1 --2- 172.21.15.12:0/1384459922 >> v2:172.21.15.144:3300/0 conn(0x7f1934169ed0 0x7f193408c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.054+0000 7f19337fe640 1 -- 172.21.15.12:0/1384459922 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884714494 0 0) 0x7f19340c3f90 con 0x7f19340c3b00 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.118+0000 7f19337fe640 1 -- 172.21.15.12:0/1384459922 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1924003430 con 0x7f19340c3b00 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.118+0000 7f1933fff640 1 -- 172.21.15.12:0/1384459922 >> v2:172.21.15.144:3300/0 conn(0x7f1934169ed0 msgr2=0x7f193408c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.118+0000 7f1933fff640 1 --2- 172.21.15.12:0/1384459922 >> v2:172.21.15.144:3300/0 conn(0x7f1934169ed0 0x7f193408c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.118+0000 7f1933fff640 1 -- 172.21.15.12:0/1384459922 >> v1:172.21.15.12:6789/0 conn(0x7f19340c3b00 legacy=0x7f193416a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.118+0000 7f19337fe640 1 -- 172.21.15.12:0/1384459922 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1504858965 0 0) 0x7f1924003430 con 0x7f19340c3b00 2023-11-28T21:38:04.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.118+0000 7f1933fff640 1 -- 172.21.15.12:0/1384459922 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193406d9f0 con 0x7f1934072db0 2023-11-28T21:38:04.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f1933fff640 1 --2- 172.21.15.12:0/1384459922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934072db0 0x7f19340c35c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f19340c7d90 tx=0x7f1920027410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e498a48230f945 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:04.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f19337fe640 1 -- 172.21.15.12:0/1384459922 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c001a00 con 0x7f1934072db0 2023-11-28T21:38:04.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f19337fe640 1 -- 172.21.15.12:0/1384459922 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19200b3050 con 0x7f1934072db0 2023-11-28T21:38:04.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f19337fe640 1 -- 172.21.15.12:0/1384459922 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19200b7620 con 0x7f1934072db0 2023-11-28T21:38:04.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f193b11d640 1 -- 172.21.15.12:0/1384459922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934072db0 msgr2=0x7f19340c35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f193b11d640 1 --2- 172.21.15.12:0/1384459922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934072db0 0x7f19340c35c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f19340c7d90 tx=0x7f1920027410 comp rx=0 tx=0).stop 2023-11-28T21:38:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f193b11d640 1 -- 172.21.15.12:0/1384459922 shutdown_connections 2023-11-28T21:38:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f193b11d640 1 --2- 172.21.15.12:0/1384459922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934072db0 0x7f19340c35c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f193b11d640 1 --2- 172.21.15.12:0/1384459922 >> v2:172.21.15.144:3300/0 conn(0x7f1934169ed0 0x7f193408c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f193b11d640 1 -- 172.21.15.12:0/1384459922 >> 172.21.15.12:0/1384459922 conn(0x7f1934066360 msgr2=0x7f1934066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f193b11d640 1 -- 172.21.15.12:0/1384459922 shutdown_connections 2023-11-28T21:38:04.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f193b11d640 1 -- 172.21.15.12:0/1384459922 wait complete. 2023-11-28T21:38:04.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.122+0000 7f193b11d640 1 Processor -- start 2023-11-28T21:38:04.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f193b11d640 1 -- start start 2023-11-28T21:38:04.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f193b11d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1934072db0 0x7f19341fd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:04.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f193b11d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19340c3b00 0x7f1934164b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:04.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f193b11d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934169ed0 0x7f1934168560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:04.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f193b11d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1934165d30 con 0x7f1934169ed0 2023-11-28T21:38:04.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f193b11d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19341670d0 con 0x7f19340c3b00 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f193b11d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19341654d0 con 0x7f1934072db0 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19396ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934169ed0 0x7f1934168560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19396ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934169ed0 0x7f1934168560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54524/0 (socket says 172.21.15.12:54524) 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19396ba640 1 -- 172.21.15.12:0/4021952011 learned_addr learned my addr 172.21.15.12:0/4021952011 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f1933fff640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19340c3b00 0x7f1934164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f1938eb9640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1934072db0 0x7f19341fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19396ba640 1 -- 172.21.15.12:0/4021952011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1934072db0 msgr2=0x7f19341fd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19396ba640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1934072db0 0x7f19341fd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19396ba640 1 -- 172.21.15.12:0/4021952011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19340c3b00 msgr2=0x7f1934164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19396ba640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19340c3b00 0x7f1934164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19396ba640 1 -- 172.21.15.12:0/4021952011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1920013970 con 0x7f1934169ed0 2023-11-28T21:38:04.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f1933fff640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19340c3b00 0x7f1934164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:38:04.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19396ba640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934169ed0 0x7f1934168560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f192c001f80 tx=0x7f192c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:04.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19317fa640 1 -- 172.21.15.12:0/4021952011 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c012400 con 0x7f1934169ed0 2023-11-28T21:38:04.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19317fa640 1 -- 172.21.15.12:0/4021952011 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c012560 con 0x7f1934169ed0 2023-11-28T21:38:04.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f193b11d640 1 -- 172.21.15.12:0/4021952011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1934165f80 con 0x7f1934169ed0 2023-11-28T21:38:04.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f19317fa640 1 -- 172.21.15.12:0/4021952011 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c012910 con 0x7f1934169ed0 2023-11-28T21:38:04.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f193b11d640 1 -- 172.21.15.12:0/4021952011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1934167650 con 0x7f1934169ed0 2023-11-28T21:38:04.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.126+0000 7f193b11d640 1 -- 172.21.15.12:0/4021952011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1934074f60 con 0x7f1934169ed0 2023-11-28T21:38:04.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.130+0000 7f19317fa640 1 -- 172.21.15.12:0/4021952011 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f192c012a70 con 0x7f1934169ed0 2023-11-28T21:38:04.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.130+0000 7f19317fa640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19180664a0 0x7f1918068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:04.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.130+0000 7f19317fa640 1 -- 172.21.15.12:0/4021952011 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f192c084970 con 0x7f1934169ed0 2023-11-28T21:38:04.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.130+0000 7f1938eb9640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19180664a0 0x7f1918068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:04.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.130+0000 7f1938eb9640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19180664a0 0x7f1918068960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1928009ac0 tx=0x7f1928009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:04.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.134+0000 7f19317fa640 1 -- 172.21.15.12:0/4021952011 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f192c0519e0 con 0x7f1934169ed0 2023-11-28T21:38:04.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.330+0000 7f193b11d640 1 -- 172.21.15.12:0/4021952011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19340c2ab0 con 0x7f1934169ed0 2023-11-28T21:38:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.330+0000 7f19317fa640 1 -- 172.21.15.12:0/4021952011 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+6520 (secure 0 0 0) 0x7f192c055000 con 0x7f1934169ed0 2023-11-28T21:38:04.337 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:04.337 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:37:33.466684+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0,1],"up":{"mds_0":4261,"mds_1":4250},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_4250":{"gid":4250,"name":"e","rank":1,"incarnation":8,"state":"up:stopping","state_seq":11,"addr":"172.21.15.12:6839/3027655738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3027655738},{"type":"v1","addr":"172.21.15.12:6839","nonce":3027655738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:04.337 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 17 2023-11-28T21:38:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 -- 172.21.15.12:0/4021952011 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19180664a0 msgr2=0x7f1918068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19180664a0 0x7f1918068960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1928009ac0 tx=0x7f1928009190 comp rx=0 tx=0).stop 2023-11-28T21:38:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 -- 172.21.15.12:0/4021952011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934169ed0 msgr2=0x7f1934168560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934169ed0 0x7f1934168560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f192c001f80 tx=0x7f192c00a430 comp rx=0 tx=0).stop 2023-11-28T21:38:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 -- 172.21.15.12:0/4021952011 shutdown_connections 2023-11-28T21:38:04.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19180664a0 0x7f1918068960 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1934169ed0 0x7f1934168560 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19340c3b00 0x7f1934164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 --2- 172.21.15.12:0/4021952011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1934072db0 0x7f19341fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 -- 172.21.15.12:0/4021952011 >> 172.21.15.12:0/4021952011 conn(0x7f1934066360 msgr2=0x7f1934162410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 -- 172.21.15.12:0/4021952011 shutdown_connections 2023-11-28T21:38:04.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:04.334+0000 7f193b11d640 1 -- 172.21.15.12:0/4021952011 wait complete. 2023-11-28T21:38:04.367 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:38:04.420 INFO:tasks.ceph.mds.e.smithi012.stdout:starting mds.e at 2023-11-28T21:38:06.225 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2023-11-28T21:38:06.225 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2023-11-28T21:38:06.225 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2023-11-28T21:38:06.225 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2023-11-28T21:38:06.225 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2023-11-28T21:38:06.225 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2023-11-28T21:38:06.225 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2023-11-28T21:38:06.225 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2023-11-28T21:38:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2023-11-28T21:38:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2023-11-28T21:38:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2023-11-28T21:38:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2023-11-28T21:38:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2023-11-28T21:38:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2023-11-28T21:38:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2023-11-28T21:38:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2023-11-28T21:38:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2023-11-28T21:38:06.226 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2023-11-28T21:38:06.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2023-11-28T21:38:06.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2023-11-28T21:38:06.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2023-11-28T21:38:06.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2023-11-28T21:38:06.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2023-11-28T21:38:06.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2023-11-28T21:38:06.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2023-11-28T21:38:06.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2023-11-28T21:38:06.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2023-11-28T21:38:06.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2023-11-28T21:38:06.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2023-11-28T21:38:06.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2023-11-28T21:38:06.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2023-11-28T21:38:06.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2023-11-28T21:38:06.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2023-11-28T21:38:06.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2023-11-28T21:38:06.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2023-11-28T21:38:06.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2023-11-28T21:38:06.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:06.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0ec12d6640 1 Processor -- start 2023-11-28T21:38:06.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0ec12d6640 1 -- start start 2023-11-28T21:38:06.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0ec12d6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0ebc151ed0 0x7f0ebc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0ec12d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ebc157b70 con 0x7f0ebc06e300 2023-11-28T21:38:06.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0ec12d6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ebc157e50 con 0x7f0ebc151af0 2023-11-28T21:38:06.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0ec12d6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ebc0707f0 con 0x7f0ebc151ed0 2023-11-28T21:38:06.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0ebb7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0ebc06e300 0x7f0ebc072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38090/0 (socket says 172.21.15.12:38090) 2023-11-28T21:38:06.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0ebb7fe640 1 -- 172.21.15.12:0/2036498204 learned_addr learned my addr 172.21.15.12:0/2036498204 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:38:06.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0eb3fff640 1 --2- 172.21.15.12:0/2036498204 >> v2:172.21.15.144:3300/0 conn(0x7f0ebc151ed0 0x7f0ebc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0eba7fc640 1 -- 172.21.15.12:0/2036498204 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860805867 0 0) 0x7f0ebc157b70 con 0x7f0ebc06e300 2023-11-28T21:38:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.510+0000 7f0eba7fc640 1 -- 172.21.15.12:0/2036498204 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea8003530 con 0x7f0ebc06e300 2023-11-28T21:38:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.514+0000 7f0eb3fff640 1 -- 172.21.15.12:0/2036498204 >> v1:172.21.15.144:6789/0 conn(0x7f0ebc151af0 legacy=0x7f0ebc0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.514+0000 7f0eba7fc640 1 -- 172.21.15.12:0/2036498204 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3238943429 0 0) 0x7f0ea8003530 con 0x7f0ebc06e300 2023-11-28T21:38:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.514+0000 7f0eb3fff640 1 -- 172.21.15.12:0/2036498204 >> v1:172.21.15.12:6789/0 conn(0x7f0ebc06e300 legacy=0x7f0ebc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:06.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.514+0000 7f0eb3fff640 1 -- 172.21.15.12:0/2036498204 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc158130 con 0x7f0ebc151ed0 2023-11-28T21:38:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.514+0000 7f0eb3fff640 1 --2- 172.21.15.12:0/2036498204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ebc151ed0 0x7f0ebc0c7650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40097c0 tx=0x7f0ea401c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1e04c0091bcece1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.514+0000 7f0eba7fc640 1 -- 172.21.15.12:0/2036498204 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4001a00 con 0x7f0ebc151ed0 2023-11-28T21:38:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.514+0000 7f0eba7fc640 1 -- 172.21.15.12:0/2036498204 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea40a7050 con 0x7f0ebc151ed0 2023-11-28T21:38:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.514+0000 7f0eba7fc640 1 -- 172.21.15.12:0/2036498204 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40ab600 con 0x7f0ebc151ed0 2023-11-28T21:38:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2036498204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ebc151ed0 msgr2=0x7f0ebc0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:06.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 --2- 172.21.15.12:0/2036498204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ebc151ed0 0x7f0ebc0c7650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40097c0 tx=0x7f0ea401c3a0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2036498204 shutdown_connections 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 --2- 172.21.15.12:0/2036498204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ebc151ed0 0x7f0ebc0c7650 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2036498204 >> 172.21.15.12:0/2036498204 conn(0x7f0ebc066360 msgr2=0x7f0ebc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2036498204 shutdown_connections 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2036498204 wait complete. 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 Processor -- start 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 -- start start 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ebc06e300 0x7f0ebc157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ebc151af0 0x7f0ebc0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ebc155690 0x7f0ebc144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc155fe0 con 0x7f0ebc151af0 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc142a60 con 0x7f0ebc155690 2023-11-28T21:38:06.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ec12d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc157730 con 0x7f0ebc06e300 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0eb3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ebc151af0 0x7f0ebc0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0eb3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ebc151af0 0x7f0ebc0c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35416/0 (socket says 172.21.15.12:35416) 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0eb3fff640 1 -- 172.21.15.12:0/2157258987 learned_addr learned my addr 172.21.15.12:0/2157258987 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ebaffd640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ebc06e300 0x7f0ebc157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0ebb7fe640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ebc155690 0x7f0ebc144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0eb3fff640 1 -- 172.21.15.12:0/2157258987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ebc06e300 msgr2=0x7f0ebc157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0eb3fff640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ebc06e300 0x7f0ebc157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0eb3fff640 1 -- 172.21.15.12:0/2157258987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ebc155690 msgr2=0x7f0ebc144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0eb3fff640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ebc155690 0x7f0ebc144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0eb3fff640 1 -- 172.21.15.12:0/2157258987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4009420 con 0x7f0ebc151af0 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0eb3fff640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ebc151af0 0x7f0ebc0c6050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000c10 tx=0x7f0ea4019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.518+0000 7f0eb37fe640 1 -- 172.21.15.12:0/2157258987 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40b9070 con 0x7f0ebc151af0 2023-11-28T21:38:06.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.522+0000 7f0eb37fe640 1 -- 172.21.15.12:0/2157258987 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea40038b0 con 0x7f0ebc151af0 2023-11-28T21:38:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.522+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2157258987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ebc142cb0 con 0x7f0ebc151af0 2023-11-28T21:38:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.522+0000 7f0eb37fe640 1 -- 172.21.15.12:0/2157258987 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4003c20 con 0x7f0ebc151af0 2023-11-28T21:38:06.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.522+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2157258987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ebc143270 con 0x7f0ebc151af0 2023-11-28T21:38:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.522+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2157258987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ebc074f60 con 0x7f0ebc151af0 2023-11-28T21:38:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.522+0000 7f0eb37fe640 1 -- 172.21.15.12:0/2157258987 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0ea40b2040 con 0x7f0ebc151af0 2023-11-28T21:38:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.522+0000 7f0eb37fe640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e9c0664a0 0x7f0e9c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.522+0000 7f0eb37fe640 1 -- 172.21.15.12:0/2157258987 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f0ea411da60 con 0x7f0ebc151af0 2023-11-28T21:38:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.522+0000 7f0ebaffd640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e9c0664a0 0x7f0e9c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.522+0000 7f0ebaffd640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e9c0664a0 0x7f0e9c068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0eac001650 tx=0x7f0eac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:06.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.526+0000 7f0eb37fe640 1 -- 172.21.15.12:0/2157258987 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0ea40eaa50 con 0x7f0ebc151af0 2023-11-28T21:38:06.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.726+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2157258987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ebc072c10 con 0x7f0ebc151af0 2023-11-28T21:38:06.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.726+0000 7f0eb37fe640 1 -- 172.21.15.12:0/2157258987 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f0ea40b0090 con 0x7f0ebc151af0 2023-11-28T21:38:06.732 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:06.733 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:06.733 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-11-28T21:38:06.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2157258987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e9c0664a0 msgr2=0x7f0e9c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:06.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e9c0664a0 0x7f0e9c068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0eac001650 tx=0x7f0eac009040 comp rx=0 tx=0).stop 2023-11-28T21:38:06.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2157258987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ebc151af0 msgr2=0x7f0ebc0c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ebc151af0 0x7f0ebc0c6050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000c10 tx=0x7f0ea4019140 comp rx=0 tx=0).stop 2023-11-28T21:38:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2157258987 shutdown_connections 2023-11-28T21:38:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0e9c0664a0 0x7f0e9c068960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ebc155690 0x7f0ebc144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0ebc151af0 0x7f0ebc0c6050 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 --2- 172.21.15.12:0/2157258987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ebc06e300 0x7f0ebc157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2157258987 >> 172.21.15.12:0/2157258987 conn(0x7f0ebc066360 msgr2=0x7f0ebc06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2157258987 shutdown_connections 2023-11-28T21:38:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.730+0000 7f0ec12d6640 1 -- 172.21.15.12:0/2157258987 wait complete. 2023-11-28T21:38:06.753 INFO:tasks.mds_thrash.fs.[cephfs]:mds cluster has 1 alive and active, now stable! 2023-11-28T21:38:06.754 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=2) 2023-11-28T21:38:06.754 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T21:38:06.754 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:38:06.777 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T21:38:06.778 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:06.959 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:38:06.963 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:38:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.970+0000 7fae48772640 1 Processor -- start 2023-11-28T21:38:06.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.970+0000 7fae48772640 1 -- start start 2023-11-28T21:38:06.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.970+0000 7fae48772640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae40154120 0x7fae400ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.970+0000 7fae48772640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fae400cade0 0x7fae400cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.970+0000 7fae48772640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae400cbba0 0x7fae400cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.970+0000 7fae48772640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae400b3040 con 0x7fae400cade0 2023-11-28T21:38:06.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.970+0000 7fae48772640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4015d370 con 0x7fae400cbba0 2023-11-28T21:38:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.970+0000 7fae48772640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae40154500 con 0x7fae40154120 2023-11-28T21:38:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae45d0d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fae400cade0 0x7fae400cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae45d0d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fae400cade0 0x7fae400cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35432/0 (socket says 172.21.15.12:35432) 2023-11-28T21:38:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae45d0d640 1 -- 172.21.15.12:0/1002756311 learned_addr learned my addr 172.21.15.12:0/1002756311 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae4650e640 1 --2- 172.21.15.12:0/1002756311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae40154120 0x7fae400ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae46d0f640 1 --2- 172.21.15.12:0/1002756311 >> v2:172.21.15.144:3300/0 conn(0x7fae400cbba0 0x7fae400cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae45d0d640 1 -- 172.21.15.12:0/1002756311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae40154120 msgr2=0x7fae400ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae45d0d640 1 --2- 172.21.15.12:0/1002756311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae40154120 0x7fae400ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae45d0d640 1 -- 172.21.15.12:0/1002756311 >> v2:172.21.15.144:3300/0 conn(0x7fae400cbba0 msgr2=0x7fae400cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:06.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae45d0d640 1 --2- 172.21.15.12:0/1002756311 >> v2:172.21.15.144:3300/0 conn(0x7fae400cbba0 0x7fae400cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae45d0d640 1 -- 172.21.15.12:0/1002756311 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae40159d80 con 0x7fae400cade0 2023-11-28T21:38:06.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae45d0d640 1 --2- 172.21.15.12:0/1002756311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cade0 0x7fae400cc0c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fae40072c40 tx=0x7fae3001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b34ab4250ddece0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:06.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae4550c640 1 -- 172.21.15.12:0/1002756311 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300aa5f0 con 0x7fae400cade0 2023-11-28T21:38:06.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae4550c640 1 -- 172.21.15.12:0/1002756311 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae300aa750 con 0x7fae400cade0 2023-11-28T21:38:06.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.974+0000 7fae4550c640 1 -- 172.21.15.12:0/1002756311 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300aaae0 con 0x7fae400cade0 2023-11-28T21:38:06.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 -- 172.21.15.12:0/1002756311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cade0 msgr2=0x7fae400cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 --2- 172.21.15.12:0/1002756311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cade0 0x7fae400cc0c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fae40072c40 tx=0x7fae3001c4d0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 -- 172.21.15.12:0/1002756311 shutdown_connections 2023-11-28T21:38:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 --2- 172.21.15.12:0/1002756311 >> v2:172.21.15.144:3300/0 conn(0x7fae400cbba0 0x7fae400cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 --2- 172.21.15.12:0/1002756311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cade0 0x7fae400cc0c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 --2- 172.21.15.12:0/1002756311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae40154120 0x7fae400ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 -- 172.21.15.12:0/1002756311 >> 172.21.15.12:0/1002756311 conn(0x7fae40071ce0 msgr2=0x7fae400720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 -- 172.21.15.12:0/1002756311 shutdown_connections 2023-11-28T21:38:06.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 -- 172.21.15.12:0/1002756311 wait complete. 2023-11-28T21:38:06.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 Processor -- start 2023-11-28T21:38:06.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 -- start start 2023-11-28T21:38:06.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae400cade0 0x7fae400c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cbba0 0x7fae400b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae40154120 0x7fae401f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400c72e0 con 0x7fae400cbba0 2023-11-28T21:38:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae401f5a30 con 0x7fae400cade0 2023-11-28T21:38:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae48772640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae400ce7c0 con 0x7fae40154120 2023-11-28T21:38:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae45d0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cbba0 0x7fae400b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae45d0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cbba0 0x7fae400b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35448/0 (socket says 172.21.15.12:35448) 2023-11-28T21:38:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae45d0d640 1 -- 172.21.15.12:0/310524847 learned_addr learned my addr 172.21.15.12:0/310524847 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:06.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae4650e640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae400cade0 0x7fae400c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.978+0000 7fae46d0f640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae40154120 0x7fae401f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae45d0d640 1 -- 172.21.15.12:0/310524847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae40154120 msgr2=0x7fae401f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:06.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae45d0d640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae40154120 0x7fae401f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae45d0d640 1 -- 172.21.15.12:0/310524847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae400cade0 msgr2=0x7fae400c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:06.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae45d0d640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae400cade0 0x7fae400c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:06.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae45d0d640 1 -- 172.21.15.12:0/310524847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae300094d0 con 0x7fae400cbba0 2023-11-28T21:38:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae45d0d640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cbba0 0x7fae400b4b40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae3001c9b0 tx=0x7fae3001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae36ffd640 1 -- 172.21.15.12:0/310524847 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300bd5f0 con 0x7fae400cbba0 2023-11-28T21:38:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae36ffd640 1 -- 172.21.15.12:0/310524847 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae300bd770 con 0x7fae400cbba0 2023-11-28T21:38:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae36ffd640 1 -- 172.21.15.12:0/310524847 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300bdb20 con 0x7fae400cbba0 2023-11-28T21:38:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae48772640 1 -- 172.21.15.12:0/310524847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae400ceaa0 con 0x7fae400cbba0 2023-11-28T21:38:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae48772640 1 -- 172.21.15.12:0/310524847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae400cf060 con 0x7fae400cbba0 2023-11-28T21:38:06.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae48772640 1 -- 172.21.15.12:0/310524847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae40074f60 con 0x7fae400cbba0 2023-11-28T21:38:06.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae36ffd640 1 -- 172.21.15.12:0/310524847 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fae300b33f0 con 0x7fae400cbba0 2023-11-28T21:38:06.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae36ffd640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae24066470 0x7fae24068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:06.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.982+0000 7fae36ffd640 1 -- 172.21.15.12:0/310524847 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fae300a7070 con 0x7fae400cbba0 2023-11-28T21:38:06.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.986+0000 7fae4650e640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae24066470 0x7fae24068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:06.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.986+0000 7fae4650e640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae24066470 0x7fae24068930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009ac0 tx=0x7fae3c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:06.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:06.986+0000 7fae36ffd640 1 -- 172.21.15.12:0/310524847 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae300ec050 con 0x7fae400cbba0 2023-11-28T21:38:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.178+0000 7fae48772640 1 -- 172.21.15.12:0/310524847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae400c4650 con 0x7fae400cbba0 2023-11-28T21:38:07.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.178+0000 7fae36ffd640 1 -- 172.21.15.12:0/310524847 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fae300c3090 con 0x7fae400cbba0 2023-11-28T21:38:07.184 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:07.185 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:07.185 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-11-28T21:38:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 -- 172.21.15.12:0/310524847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae24066470 msgr2=0x7fae24068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:07.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae24066470 0x7fae24068930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009ac0 tx=0x7fae3c009190 comp rx=0 tx=0).stop 2023-11-28T21:38:07.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 -- 172.21.15.12:0/310524847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cbba0 msgr2=0x7fae400b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:07.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cbba0 0x7fae400b4b40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae3001c9b0 tx=0x7fae3001ad10 comp rx=0 tx=0).stop 2023-11-28T21:38:07.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 -- 172.21.15.12:0/310524847 shutdown_connections 2023-11-28T21:38:07.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae24066470 0x7fae24068930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:07.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae40154120 0x7fae401f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:07.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae400cbba0 0x7fae400b4b40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:07.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 --2- 172.21.15.12:0/310524847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae400cade0 0x7fae400c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:07.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 -- 172.21.15.12:0/310524847 >> 172.21.15.12:0/310524847 conn(0x7fae40071ce0 msgr2=0x7fae40149d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:07.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 -- 172.21.15.12:0/310524847 shutdown_connections 2023-11-28T21:38:07.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:07.182+0000 7fae48772640 1 -- 172.21.15.12:0/310524847 wait complete. 2023-11-28T21:38:07.206 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:38:08.639 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2023-11-28T21:38:08.639 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2023-11-28T21:38:08.639 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2023-11-28T21:38:08.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2023-11-28T21:38:08.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2023-11-28T21:38:08.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2023-11-28T21:38:08.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2023-11-28T21:38:08.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2023-11-28T21:38:08.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2023-11-28T21:38:09.206 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:09.241 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2023-11-28T21:38:09.241 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2023-11-28T21:38:09.242 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2023-11-28T21:38:09.242 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2023-11-28T21:38:09.243 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2023-11-28T21:38:09.243 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2023-11-28T21:38:09.243 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2023-11-28T21:38:09.243 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2023-11-28T21:38:09.243 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2023-11-28T21:38:09.244 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2023-11-28T21:38:09.245 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2023-11-28T21:38:09.246 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2023-11-28T21:38:09.247 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2023-11-28T21:38:09.248 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2023-11-28T21:38:09.249 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2023-11-28T21:38:09.249 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2023-11-28T21:38:09.249 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2023-11-28T21:38:09.249 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2023-11-28T21:38:09.249 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2023-11-28T21:38:09.249 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2023-11-28T21:38:09.249 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2023-11-28T21:38:09.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb36d62640 1 Processor -- start 2023-11-28T21:38:09.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb36d62640 1 -- start start 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb36d62640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb30153fb0 0x7fdb300ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb36d62640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb3015a980 0x7fdb300caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb36d62640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb300b3ad0 0x7fdb300b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb36d62640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb300b2b60 con 0x7fdb30153fb0 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb36d62640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3015e1b0 con 0x7fdb3015a980 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb36d62640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300b44a0 con 0x7fdb300b3ad0 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb35d60640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb30153fb0 0x7fdb300ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb35d60640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdb30153fb0 0x7fdb300ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35452/0 (socket says 172.21.15.12:35452) 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb35d60640 1 -- 172.21.15.12:0/3362275804 learned_addr learned my addr 172.21.15.12:0/3362275804 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb36561640 1 --2- 172.21.15.12:0/3362275804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb300b3ad0 0x7fdb300b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb3555f640 1 --2- 172.21.15.12:0/3362275804 >> v2:172.21.15.144:3300/0 conn(0x7fdb3015a980 0x7fdb300caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:09.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb35d60640 1 -- 172.21.15.12:0/3362275804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb300b3ad0 msgr2=0x7fdb300b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:09.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb35d60640 1 --2- 172.21.15.12:0/3362275804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb300b3ad0 0x7fdb300b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb35d60640 1 -- 172.21.15.12:0/3362275804 >> v2:172.21.15.144:3300/0 conn(0x7fdb3015a980 msgr2=0x7fdb300caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:09.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb35d60640 1 --2- 172.21.15.12:0/3362275804 >> v2:172.21.15.144:3300/0 conn(0x7fdb3015a980 0x7fdb300caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.354+0000 7fdb35d60640 1 -- 172.21.15.12:0/3362275804 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3015d390 con 0x7fdb30153fb0 2023-11-28T21:38:09.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb35d60640 1 --2- 172.21.15.12:0/3362275804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb30153fb0 0x7fdb300ca4f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdb300c9fa0 tx=0x7fdb2c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ed576120528e37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb34d5e640 1 -- 172.21.15.12:0/3362275804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0aa5f0 con 0x7fdb30153fb0 2023-11-28T21:38:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb34d5e640 1 -- 172.21.15.12:0/3362275804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c0aa750 con 0x7fdb30153fb0 2023-11-28T21:38:09.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb34d5e640 1 -- 172.21.15.12:0/3362275804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0aaae0 con 0x7fdb30153fb0 2023-11-28T21:38:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb36d62640 1 -- 172.21.15.12:0/3362275804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb30153fb0 msgr2=0x7fdb300ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb36d62640 1 --2- 172.21.15.12:0/3362275804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb30153fb0 0x7fdb300ca4f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdb300c9fa0 tx=0x7fdb2c01c4d0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb36d62640 1 -- 172.21.15.12:0/3362275804 shutdown_connections 2023-11-28T21:38:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb36d62640 1 --2- 172.21.15.12:0/3362275804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb300b3ad0 0x7fdb300b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb36d62640 1 --2- 172.21.15.12:0/3362275804 >> v2:172.21.15.144:3300/0 conn(0x7fdb3015a980 0x7fdb300caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb36d62640 1 --2- 172.21.15.12:0/3362275804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb30153fb0 0x7fdb300ca4f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb36d62640 1 -- 172.21.15.12:0/3362275804 >> 172.21.15.12:0/3362275804 conn(0x7fdb30071ce0 msgr2=0x7fdb300720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb36d62640 1 -- 172.21.15.12:0/3362275804 shutdown_connections 2023-11-28T21:38:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb36d62640 1 -- 172.21.15.12:0/3362275804 wait complete. 2023-11-28T21:38:09.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.358+0000 7fdb36d62640 1 Processor -- start 2023-11-28T21:38:09.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36d62640 1 -- start start 2023-11-28T21:38:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36d62640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb300b3ad0 0x7fdb301f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36d62640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30153fb0 0x7fdb3014e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36d62640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3015a980 0x7fdb300c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36d62640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3014f740 con 0x7fdb3015a980 2023-11-28T21:38:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36d62640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300c4d90 con 0x7fdb300b3ad0 2023-11-28T21:38:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36d62640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3014fea0 con 0x7fdb30153fb0 2023-11-28T21:38:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36561640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3015a980 0x7fdb300c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36561640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3015a980 0x7fdb300c5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35460/0 (socket says 172.21.15.12:35460) 2023-11-28T21:38:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36561640 1 -- 172.21.15.12:0/1838041043 learned_addr learned my addr 172.21.15.12:0/1838041043 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb35d60640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb300b3ad0 0x7fdb301f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb3555f640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30153fb0 0x7fdb3014e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36561640 1 -- 172.21.15.12:0/1838041043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30153fb0 msgr2=0x7fdb3014e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36561640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30153fb0 0x7fdb3014e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36561640 1 -- 172.21.15.12:0/1838041043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb300b3ad0 msgr2=0x7fdb301f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36561640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb300b3ad0 0x7fdb301f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36561640 1 -- 172.21.15.12:0/1838041043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c0094d0 con 0x7fdb3015a980 2023-11-28T21:38:09.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb35d60640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb300b3ad0 0x7fdb301f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:38:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb3555f640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30153fb0 0x7fdb3014e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:38:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36561640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3015a980 0x7fdb300c5af0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdb28007e60 tx=0x7fdb2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb267fc640 1 -- 172.21.15.12:0/1838041043 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28014620 con 0x7fdb3015a980 2023-11-28T21:38:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb267fc640 1 -- 172.21.15.12:0/1838041043 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2800ee90 con 0x7fdb3015a980 2023-11-28T21:38:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb267fc640 1 -- 172.21.15.12:0/1838041043 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb280149e0 con 0x7fdb3015a980 2023-11-28T21:38:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36d62640 1 -- 172.21.15.12:0/1838041043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb30150180 con 0x7fdb3015a980 2023-11-28T21:38:09.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36d62640 1 -- 172.21.15.12:0/1838041043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb300c52f0 con 0x7fdb3015a980 2023-11-28T21:38:09.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.362+0000 7fdb36d62640 1 -- 172.21.15.12:0/1838041043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb30074f60 con 0x7fdb3015a980 2023-11-28T21:38:09.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.366+0000 7fdb267fc640 1 -- 172.21.15.12:0/1838041043 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdb28014b40 con 0x7fdb3015a980 2023-11-28T21:38:09.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.366+0000 7fdb267fc640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb140664a0 0x7fdb14068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:09.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.366+0000 7fdb267fc640 1 -- 172.21.15.12:0/1838041043 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fdb28084c20 con 0x7fdb3015a980 2023-11-28T21:38:09.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.366+0000 7fdb35d60640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb140664a0 0x7fdb14068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:09.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.366+0000 7fdb35d60640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb140664a0 0x7fdb14068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0ae000 tx=0x7fdb2c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:09.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.370+0000 7fdb267fc640 1 -- 172.21.15.12:0/1838041043 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb28051c10 con 0x7fdb3015a980 2023-11-28T21:38:09.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.562+0000 7fdb36d62640 1 -- 172.21.15.12:0/1838041043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb300b5720 con 0x7fdb3015a980 2023-11-28T21:38:09.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.562+0000 7fdb267fc640 1 -- 172.21.15.12:0/1838041043 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fdb28055230 con 0x7fdb3015a980 2023-11-28T21:38:09.569 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:09.569 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:09.569 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-11-28T21:38:09.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 -- 172.21.15.12:0/1838041043 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb140664a0 msgr2=0x7fdb14068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:09.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb140664a0 0x7fdb14068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0ae000 tx=0x7fdb2c0a7000 comp rx=0 tx=0).stop 2023-11-28T21:38:09.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 -- 172.21.15.12:0/1838041043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3015a980 msgr2=0x7fdb300c5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:09.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3015a980 0x7fdb300c5af0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdb28007e60 tx=0x7fdb2800e4b0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 -- 172.21.15.12:0/1838041043 shutdown_connections 2023-11-28T21:38:09.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb140664a0 0x7fdb14068960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb3015a980 0x7fdb300c5af0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30153fb0 0x7fdb3014e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 --2- 172.21.15.12:0/1838041043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb300b3ad0 0x7fdb301f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:09.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 -- 172.21.15.12:0/1838041043 >> 172.21.15.12:0/1838041043 conn(0x7fdb30071ce0 msgr2=0x7fdb3014a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:09.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 -- 172.21.15.12:0/1838041043 shutdown_connections 2023-11-28T21:38:09.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:09.566+0000 7fdb36d62640 1 -- 172.21.15.12:0/1838041043 wait complete. 2023-11-28T21:38:09.590 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:38:10.764 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2023-11-28T21:38:10.764 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2023-11-28T21:38:10.764 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2023-11-28T21:38:10.764 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2023-11-28T21:38:10.764 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2023-11-28T21:38:10.764 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2023-11-28T21:38:10.764 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2023-11-28T21:38:10.764 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2023-11-28T21:38:10.764 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2023-11-28T21:38:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2023-11-28T21:38:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2023-11-28T21:38:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2023-11-28T21:38:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2023-11-28T21:38:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2023-11-28T21:38:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2023-11-28T21:38:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2023-11-28T21:38:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2023-11-28T21:38:10.765 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2023-11-28T21:38:10.766 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2023-11-28T21:38:10.767 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2023-11-28T21:38:10.768 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2023-11-28T21:38:10.769 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2023-11-28T21:38:10.770 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2023-11-28T21:38:10.770 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2023-11-28T21:38:10.770 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2023-11-28T21:38:10.770 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2023-11-28T21:38:10.770 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2023-11-28T21:38:10.770 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2023-11-28T21:38:10.771 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2023-11-28T21:38:10.771 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2023-11-28T21:38:10.771 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2023-11-28T21:38:10.771 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2023-11-28T21:38:11.590 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:11.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dec17640 1 Processor -- start 2023-11-28T21:38:11.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dec17640 1 -- start start 2023-11-28T21:38:11.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dec17640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe8d815a980 0x7fe8d80b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dec17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d80b44a0 con 0x7fe8d80b4010 2023-11-28T21:38:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dec17640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d815d390 con 0x7fe8d8153fb0 2023-11-28T21:38:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dec17640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d80b2b60 con 0x7fe8d815a980 2023-11-28T21:38:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dd414640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe8d815a980 0x7fe8d80b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dd414640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe8d815a980 0x7fe8d80b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35468/0 (socket says 172.21.15.12:35468) 2023-11-28T21:38:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dd414640 1 -- 172.21.15.12:0/1044361323 learned_addr learned my addr 172.21.15.12:0/1044361323 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:11.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dd414640 1 -- 172.21.15.12:0/1044361323 >> v1:172.21.15.144:6789/0 conn(0x7fe8d8153fb0 legacy=0x7fe8d80ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:11.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dd414640 1 -- 172.21.15.12:0/1044361323 >> v1:172.21.15.12:6789/0 conn(0x7fe8d80b4010 legacy=0x7fe8d80cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:11.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.742+0000 7fe8dd414640 1 -- 172.21.15.12:0/1044361323 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d815d630 con 0x7fe8d815a980 2023-11-28T21:38:11.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dd414640 1 --2- 172.21.15.12:0/1044361323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 0x7fe8d80b3ad0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009870 tx=0x7fe8c801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a184c835a682c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:11.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dcc13640 1 -- 172.21.15.12:0/1044361323 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80a8a30 con 0x7fe8d815a980 2023-11-28T21:38:11.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dcc13640 1 -- 172.21.15.12:0/1044361323 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c80a8b90 con 0x7fe8d815a980 2023-11-28T21:38:11.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dcc13640 1 -- 172.21.15.12:0/1044361323 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80b0440 con 0x7fe8d815a980 2023-11-28T21:38:11.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dec17640 1 -- 172.21.15.12:0/1044361323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 msgr2=0x7fe8d80b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:11.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dec17640 1 --2- 172.21.15.12:0/1044361323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 0x7fe8d80b3ad0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009870 tx=0x7fe8c801ed30 comp rx=0 tx=0).stop 2023-11-28T21:38:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dec17640 1 -- 172.21.15.12:0/1044361323 shutdown_connections 2023-11-28T21:38:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dec17640 1 --2- 172.21.15.12:0/1044361323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 0x7fe8d80b3ad0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dec17640 1 -- 172.21.15.12:0/1044361323 >> 172.21.15.12:0/1044361323 conn(0x7fe8d8071ce0 msgr2=0x7fe8d80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dec17640 1 -- 172.21.15.12:0/1044361323 shutdown_connections 2023-11-28T21:38:11.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.746+0000 7fe8dec17640 1 -- 172.21.15.12:0/1044361323 wait complete. 2023-11-28T21:38:11.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dec17640 1 Processor -- start 2023-11-28T21:38:11.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dec17640 1 -- start start 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dec17640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8d80b4010 0x7fe8d814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dec17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d8153fb0 0x7fe8d815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dec17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 0x7fe8d815c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dec17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d815e1b0 con 0x7fe8d815a980 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dec17640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8148690 con 0x7fe8d80b4010 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dec17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80c6160 con 0x7fe8d8153fb0 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8de416640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 0x7fe8d815c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8de416640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 0x7fe8d815c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35480/0 (socket says 172.21.15.12:35480) 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8de416640 1 -- 172.21.15.12:0/1542375845 learned_addr learned my addr 172.21.15.12:0/1542375845 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8ddc15640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8d80b4010 0x7fe8d814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:11.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dd414640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d8153fb0 0x7fe8d815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8de416640 1 -- 172.21.15.12:0/1542375845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d8153fb0 msgr2=0x7fe8d815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8de416640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d8153fb0 0x7fe8d815bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8de416640 1 -- 172.21.15.12:0/1542375845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8d80b4010 msgr2=0x7fe8d814f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8de416640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8d80b4010 0x7fe8d814f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8de416640 1 -- 172.21.15.12:0/1542375845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c80094d0 con 0x7fe8d815a980 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dd414640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d8153fb0 0x7fe8d815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8de416640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 0x7fe8d815c460 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c7a0 tx=0x7fe8d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8ce7fc640 1 -- 172.21.15.12:0/1542375845 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0003b80 con 0x7fe8d815a980 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8ce7fc640 1 -- 172.21.15.12:0/1542375845 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d0003ce0 con 0x7fe8d815a980 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dec17640 1 -- 172.21.15.12:0/1542375845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d8147a70 con 0x7fe8d815a980 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8ce7fc640 1 -- 172.21.15.12:0/1542375845 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000aa90 con 0x7fe8d815a980 2023-11-28T21:38:11.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.750+0000 7fe8dec17640 1 -- 172.21.15.12:0/1542375845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d81480d0 con 0x7fe8d815a980 2023-11-28T21:38:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.754+0000 7fe8dec17640 1 -- 172.21.15.12:0/1542375845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d80c6600 con 0x7fe8d815a980 2023-11-28T21:38:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.754+0000 7fe8ce7fc640 1 -- 172.21.15.12:0/1542375845 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe8d000abf0 con 0x7fe8d815a980 2023-11-28T21:38:11.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.754+0000 7fe8ce7fc640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8bc0664a0 0x7fe8bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:11.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.754+0000 7fe8ce7fc640 1 -- 172.21.15.12:0/1542375845 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fe8d0084150 con 0x7fe8d815a980 2023-11-28T21:38:11.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.754+0000 7fe8ddc15640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8bc0664a0 0x7fe8bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:11.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.754+0000 7fe8ddc15640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8bc0664a0 0x7fe8bc068960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4007fd0 tx=0x7fe8d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.758+0000 7fe8ce7fc640 1 -- 172.21.15.12:0/1542375845 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe8d00510b0 con 0x7fe8d815a980 2023-11-28T21:38:11.789 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2023-11-28T21:38:11.789 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2023-11-28T21:38:11.789 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2023-11-28T21:38:11.789 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2023-11-28T21:38:11.789 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2023-11-28T21:38:11.789 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2023-11-28T21:38:11.789 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2023-11-28T21:38:11.790 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2023-11-28T21:38:11.790 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2023-11-28T21:38:11.790 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2023-11-28T21:38:11.790 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2023-11-28T21:38:11.790 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2023-11-28T21:38:11.790 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2023-11-28T21:38:11.790 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2023-11-28T21:38:11.791 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2023-11-28T21:38:11.791 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2023-11-28T21:38:11.791 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2023-11-28T21:38:11.791 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2023-11-28T21:38:11.791 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2023-11-28T21:38:11.791 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2023-11-28T21:38:11.791 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2023-11-28T21:38:11.791 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2023-11-28T21:38:11.791 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2023-11-28T21:38:11.791 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2023-11-28T21:38:11.792 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2023-11-28T21:38:11.793 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2023-11-28T21:38:11.794 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2023-11-28T21:38:11.795 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2023-11-28T21:38:11.796 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2023-11-28T21:38:11.796 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2023-11-28T21:38:11.796 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2023-11-28T21:38:11.796 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2023-11-28T21:38:11.796 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2023-11-28T21:38:11.796 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2023-11-28T21:38:11.796 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2023-11-28T21:38:11.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.990+0000 7fe8dec17640 1 -- 172.21.15.12:0/1542375845 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8d815bc40 con 0x7fe8d815a980 2023-11-28T21:38:11.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.990+0000 7fe8ce7fc640 1 -- 172.21.15.12:0/1542375845 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fe8d815bc40 con 0x7fe8d815a980 2023-11-28T21:38:11.996 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:11.997 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:11.997 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-11-28T21:38:11.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 -- 172.21.15.12:0/1542375845 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8bc0664a0 msgr2=0x7fe8bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:11.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8bc0664a0 0x7fe8bc068960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4007fd0 tx=0x7fe8d4009040 comp rx=0 tx=0).stop 2023-11-28T21:38:11.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 -- 172.21.15.12:0/1542375845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 msgr2=0x7fe8d815c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:11.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 0x7fe8d815c460 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c7a0 tx=0x7fe8d000cc60 comp rx=0 tx=0).stop 2023-11-28T21:38:11.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 -- 172.21.15.12:0/1542375845 shutdown_connections 2023-11-28T21:38:12.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8bc0664a0 0x7fe8bc068960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:12.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d815a980 0x7fe8d815c460 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:12.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d8153fb0 0x7fe8d815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:12.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 --2- 172.21.15.12:0/1542375845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8d80b4010 0x7fe8d814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:12.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 -- 172.21.15.12:0/1542375845 >> 172.21.15.12:0/1542375845 conn(0x7fe8d8071ce0 msgr2=0x7fe8d80cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:12.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 -- 172.21.15.12:0/1542375845 shutdown_connections 2023-11-28T21:38:12.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:11.994+0000 7fe8dec17640 1 -- 172.21.15.12:0/1542375845 wait complete. 2023-11-28T21:38:12.018 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:38:13.164 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2023-11-28T21:38:13.165 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2023-11-28T21:38:13.166 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2023-11-28T21:38:13.167 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2023-11-28T21:38:13.168 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2023-11-28T21:38:13.169 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2023-11-28T21:38:13.170 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2023-11-28T21:38:13.171 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2023-11-28T21:38:13.171 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2023-11-28T21:38:13.171 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2023-11-28T21:38:13.171 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2023-11-28T21:38:14.020 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2faa29c640 1 Processor -- start 2023-11-28T21:38:14.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2faa29c640 1 -- start start 2023-11-28T21:38:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2faa29c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fa4153e40 0x7f2fa40ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2faa29c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2fa415dc80 0x7f2fa40cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2faa29c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa415acc0 con 0x7f2fa40cfba0 2023-11-28T21:38:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2faa29c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa4159cb0 con 0x7f2fa4153e40 2023-11-28T21:38:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2faa29c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa40cffa0 con 0x7f2fa415dc80 2023-11-28T21:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2fa3fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fa4153e40 0x7f2fa40ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2fa3fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fa4153e40 0x7f2fa40ca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35490/0 (socket says 172.21.15.12:35490) 2023-11-28T21:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2fa3fff640 1 -- 172.21.15.12:0/1580385255 learned_addr learned my addr 172.21.15.12:0/1580385255 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2fa37fe640 1 --2- 172.21.15.12:0/1580385255 >> v2:172.21.15.144:3300/0 conn(0x7f2fa415dc80 0x7f2fa40cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2fa3fff640 1 -- 172.21.15.12:0/1580385255 >> v2:172.21.15.144:3300/0 conn(0x7f2fa415dc80 msgr2=0x7f2fa40cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2fa3fff640 1 --2- 172.21.15.12:0/1580385255 >> v2:172.21.15.144:3300/0 conn(0x7f2fa415dc80 0x7f2fa40cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2fa3fff640 1 -- 172.21.15.12:0/1580385255 >> v1:172.21.15.144:6789/0 conn(0x7f2fa40cfba0 legacy=0x7f2fa40cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:14.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.170+0000 7f2fa3fff640 1 -- 172.21.15.12:0/1580385255 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa4159ea0 con 0x7f2fa4153e40 2023-11-28T21:38:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2fa3fff640 1 --2- 172.21.15.12:0/1580385255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa4153e40 0x7f2fa40ca110 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40d0320 tx=0x7f2f9401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33073a59732e9ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2fa2ffd640 1 -- 172.21.15.12:0/1580385255 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f940aa5f0 con 0x7f2fa4153e40 2023-11-28T21:38:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2fa2ffd640 1 -- 172.21.15.12:0/1580385255 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f940aa750 con 0x7f2fa4153e40 2023-11-28T21:38:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2fa2ffd640 1 -- 172.21.15.12:0/1580385255 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f940aaae0 con 0x7f2fa4153e40 2023-11-28T21:38:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2faa29c640 1 -- 172.21.15.12:0/1580385255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa4153e40 msgr2=0x7f2fa40ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2faa29c640 1 --2- 172.21.15.12:0/1580385255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa4153e40 0x7f2fa40ca110 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40d0320 tx=0x7f2f9401c490 comp rx=0 tx=0).stop 2023-11-28T21:38:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2faa29c640 1 -- 172.21.15.12:0/1580385255 shutdown_connections 2023-11-28T21:38:14.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2faa29c640 1 --2- 172.21.15.12:0/1580385255 >> v2:172.21.15.144:3300/0 conn(0x7f2fa415dc80 0x7f2fa40cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:14.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2faa29c640 1 --2- 172.21.15.12:0/1580385255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa4153e40 0x7f2fa40ca110 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:14.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2faa29c640 1 -- 172.21.15.12:0/1580385255 >> 172.21.15.12:0/1580385255 conn(0x7f2fa406b7b0 msgr2=0x7f2fa4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:14.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2faa29c640 1 -- 172.21.15.12:0/1580385255 shutdown_connections 2023-11-28T21:38:14.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.174+0000 7f2faa29c640 1 -- 172.21.15.12:0/1580385255 wait complete. 2023-11-28T21:38:14.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2faa29c640 1 Processor -- start 2023-11-28T21:38:14.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2faa29c640 1 -- start start 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2faa29c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa40cfba0 0x7f2fa40c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2faa29c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa4153e40 0x7f2fa41517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2faa29c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa415dc80 0x7f2fa4151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2faa29c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa4150c90 con 0x7f2fa40cfba0 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2faa29c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa41f2fa0 con 0x7f2fa415dc80 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2faa29c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa41506d0 con 0x7f2fa4153e40 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa40cfba0 0x7f2fa40c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa40cfba0 0x7f2fa40c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35494/0 (socket says 172.21.15.12:35494) 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa3fff640 1 -- 172.21.15.12:0/2090973820 learned_addr learned my addr 172.21.15.12:0/2090973820 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa37fe640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa4153e40 0x7f2fa41517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa8839640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa415dc80 0x7f2fa4151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:14.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa3fff640 1 -- 172.21.15.12:0/2090973820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa4153e40 msgr2=0x7f2fa41517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa3fff640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa4153e40 0x7f2fa41517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa3fff640 1 -- 172.21.15.12:0/2090973820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa415dc80 msgr2=0x7f2fa4151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa3fff640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa415dc80 0x7f2fa4151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa3fff640 1 -- 172.21.15.12:0/2090973820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f940094d0 con 0x7f2fa40cfba0 2023-11-28T21:38:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa3fff640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa40cfba0 0x7f2fa40c9360 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2f9401a0c0 tx=0x7f2f9401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa0ff9640 1 -- 172.21.15.12:0/2090973820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f940b3800 con 0x7f2fa40cfba0 2023-11-28T21:38:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa0ff9640 1 -- 172.21.15.12:0/2090973820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f940b3960 con 0x7f2fa40cfba0 2023-11-28T21:38:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2fa0ff9640 1 -- 172.21.15.12:0/2090973820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f940b3cd0 con 0x7f2fa40cfba0 2023-11-28T21:38:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.178+0000 7f2faa29c640 1 -- 172.21.15.12:0/2090973820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa41509b0 con 0x7f2fa40cfba0 2023-11-28T21:38:14.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.182+0000 7f2faa29c640 1 -- 172.21.15.12:0/2090973820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa41512c0 con 0x7f2fa40cfba0 2023-11-28T21:38:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.182+0000 7f2faa29c640 1 -- 172.21.15.12:0/2090973820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa4072af0 con 0x7f2fa40cfba0 2023-11-28T21:38:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.182+0000 7f2fa0ff9640 1 -- 172.21.15.12:0/2090973820 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2f940c73a0 con 0x7f2fa40cfba0 2023-11-28T21:38:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.182+0000 7f2fa0ff9640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f900664a0 0x7f2f90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.182+0000 7f2fa0ff9640 1 -- 172.21.15.12:0/2090973820 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f2f940a7070 con 0x7f2fa40cfba0 2023-11-28T21:38:14.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.182+0000 7f2fa37fe640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f900664a0 0x7f2f90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:14.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.182+0000 7f2fa37fe640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f900664a0 0x7f2f90068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40c8ec0 tx=0x7f2f98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:14.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.186+0000 7f2fa0ff9640 1 -- 172.21.15.12:0/2090973820 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2f940ebda0 con 0x7f2fa40cfba0 2023-11-28T21:38:14.384 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2023-11-28T21:38:14.384 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2023-11-28T21:38:14.384 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2023-11-28T21:38:14.384 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2023-11-28T21:38:14.384 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2023-11-28T21:38:14.384 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2023-11-28T21:38:14.385 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2023-11-28T21:38:14.386 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2023-11-28T21:38:14.387 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2023-11-28T21:38:14.388 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2023-11-28T21:38:14.389 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2023-11-28T21:38:14.390 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2023-11-28T21:38:14.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.382+0000 7f2faa29c640 1 -- 172.21.15.12:0/2090973820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fa4074f60 con 0x7f2fa40cfba0 2023-11-28T21:38:14.391 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:14.391 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:14.392 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-11-28T21:38:14.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.382+0000 7f2fa0ff9640 1 -- 172.21.15.12:0/2090973820 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f2f940c3090 con 0x7f2fa40cfba0 2023-11-28T21:38:14.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 -- 172.21.15.12:0/2090973820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f900664a0 msgr2=0x7f2f90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f900664a0 0x7f2f90068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40c8ec0 tx=0x7f2f98008040 comp rx=0 tx=0).stop 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 -- 172.21.15.12:0/2090973820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa40cfba0 msgr2=0x7f2fa40c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa40cfba0 0x7f2fa40c9360 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2f9401a0c0 tx=0x7f2f9401acd0 comp rx=0 tx=0).stop 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 -- 172.21.15.12:0/2090973820 shutdown_connections 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f900664a0 0x7f2f90068960 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fa415dc80 0x7f2fa4151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fa4153e40 0x7f2fa41517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 --2- 172.21.15.12:0/2090973820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fa40cfba0 0x7f2fa40c9360 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 -- 172.21.15.12:0/2090973820 >> 172.21.15.12:0/2090973820 conn(0x7f2fa406b7b0 msgr2=0x7f2fa4071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 -- 172.21.15.12:0/2090973820 shutdown_connections 2023-11-28T21:38:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:14.386+0000 7f2faa29c640 1 -- 172.21.15.12:0/2090973820 wait complete. 2023-11-28T21:38:14.411 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:38:14.689 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2023-11-28T21:38:14.689 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2023-11-28T21:38:14.690 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2023-11-28T21:38:14.690 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2023-11-28T21:38:14.690 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2023-11-28T21:38:14.690 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2023-11-28T21:38:14.691 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2023-11-28T21:38:14.691 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2023-11-28T21:38:14.691 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2023-11-28T21:38:14.691 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2023-11-28T21:38:14.691 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2023-11-28T21:38:14.692 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2023-11-28T21:38:14.692 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2023-11-28T21:38:14.692 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2023-11-28T21:38:14.692 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2023-11-28T21:38:14.692 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2023-11-28T21:38:14.692 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2023-11-28T21:38:14.693 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2023-11-28T21:38:14.693 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2023-11-28T21:38:14.693 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2023-11-28T21:38:14.693 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2023-11-28T21:38:14.693 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2023-11-28T21:38:14.693 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2023-11-28T21:38:14.693 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2023-11-28T21:38:14.693 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2023-11-28T21:38:14.694 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2023-11-28T21:38:14.694 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2023-11-28T21:38:14.694 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2023-11-28T21:38:14.694 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2023-11-28T21:38:14.694 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2023-11-28T21:38:14.694 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2023-11-28T21:38:14.694 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2023-11-28T21:38:14.694 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2023-11-28T21:38:14.694 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2023-11-28T21:38:14.695 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2023-11-28T21:38:14.695 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2023-11-28T21:38:14.695 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2023-11-28T21:38:14.695 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2023-11-28T21:38:14.695 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2023-11-28T21:38:14.695 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2023-11-28T21:38:14.695 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2023-11-28T21:38:14.695 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2023-11-28T21:38:14.696 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2023-11-28T21:38:14.696 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2023-11-28T21:38:14.696 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2023-11-28T21:38:14.696 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2023-11-28T21:38:14.696 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2023-11-28T21:38:14.696 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2023-11-28T21:38:14.696 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2023-11-28T21:38:14.697 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2023-11-28T21:38:14.698 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2023-11-28T21:38:14.699 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2023-11-28T21:38:14.699 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2023-11-28T21:38:14.699 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2023-11-28T21:38:14.699 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2023-11-28T21:38:14.699 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2023-11-28T21:38:14.699 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2023-11-28T21:38:15.510 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2023-11-28T21:38:15.510 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2023-11-28T21:38:15.510 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2023-11-28T21:38:15.510 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2023-11-28T21:38:15.510 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2023-11-28T21:38:15.510 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2023-11-28T21:38:15.510 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2023-11-28T21:38:15.510 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2023-11-28T21:38:15.511 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2023-11-28T21:38:15.512 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2023-11-28T21:38:15.513 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2023-11-28T21:38:15.514 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2023-11-28T21:38:15.515 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2023-11-28T21:38:15.516 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2023-11-28T21:38:15.516 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2023-11-28T21:38:15.516 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2023-11-28T21:38:15.516 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2023-11-28T21:38:15.516 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2023-11-28T21:38:15.516 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2023-11-28T21:38:15.516 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2023-11-28T21:38:15.516 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2023-11-28T21:38:16.412 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:16.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0fa79640 1 Processor -- start 2023-11-28T21:38:16.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0fa79640 1 -- start start 2023-11-28T21:38:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0fa79640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2e080c48e0 0x7f2e080c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0fa79640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0806fc10 con 0x7f2e080ca240 2023-11-28T21:38:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0fa79640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e08157cc0 con 0x7f2e08151ae0 2023-11-28T21:38:16.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0fa79640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e08158620 con 0x7f2e080c48e0 2023-11-28T21:38:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0d014640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2e080ca240 0x7f2e0815b1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33036/0 (socket says 172.21.15.12:33036) 2023-11-28T21:38:16.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0d014640 1 -- 172.21.15.12:0/4093599415 learned_addr learned my addr 172.21.15.12:0/4093599415 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:38:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0e016640 1 --2- 172.21.15.12:0/4093599415 >> v2:172.21.15.144:3300/0 conn(0x7f2e080c48e0 0x7f2e080c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158423876 0 0) 0x7f2e0806fc10 con 0x7f2e080ca240 2023-11-28T21:38:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df4003530 con 0x7f2e080ca240 2023-11-28T21:38:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 517434197 0 0) 0x7f2df4003530 con 0x7f2e080ca240 2023-11-28T21:38:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 >> v2:172.21.15.144:3300/0 conn(0x7f2e080c48e0 msgr2=0x7f2e080c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 --2- 172.21.15.12:0/4093599415 >> v2:172.21.15.144:3300/0 conn(0x7f2e080c48e0 0x7f2e080c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 >> v1:172.21.15.144:6789/0 conn(0x7f2e08151ae0 legacy=0x7f2e080c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e080ca880 con 0x7f2e080ca240 2023-11-28T21:38:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f2df8001680 con 0x7f2e080ca240 2023-11-28T21:38:16.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 >> v1:172.21.15.12:6789/0 conn(0x7f2e080ca240 legacy=0x7f2e0815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2df4003a40 0x7f2df4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.554+0000 7f2e0c813640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2df40047d0 0x7f2df4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0c813640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2df40051e0 0x7f2df400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e08158620 con 0x7f2df40047d0 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df400c0f0 con 0x7f2df40051e0 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df400c2b0 con 0x7f2df4003a40 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0d014640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2df40047d0 0x7f2df4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0d815640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2df4003a40 0x7f2df4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0e016640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2df40051e0 0x7f2df400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0d014640 1 -- 172.21.15.12:0/4093599415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2df4003a40 msgr2=0x7f2df4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0d014640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2df4003a40 0x7f2df4003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:16.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0d014640 1 -- 172.21.15.12:0/4093599415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2df40051e0 msgr2=0x7f2df400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:16.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0d014640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2df40051e0 0x7f2df400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:16.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0d014640 1 -- 172.21.15.12:0/4093599415 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df4003530 con 0x7f2df40047d0 2023-11-28T21:38:16.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0e016640 1 -- 172.21.15.12:0/4093599415 reap_dead start 2023-11-28T21:38:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0d014640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2df40047d0 0x7f2df4004c50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2df800c0e0 tx=0x7f2df801cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ce2c76f05a1457 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8005ae0 con 0x7f2df40047d0 2023-11-28T21:38:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df8005c40 con 0x7f2df40047d0 2023-11-28T21:38:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.558+0000 7f2e0c813640 1 -- 172.21.15.12:0/4093599415 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80a5a60 con 0x7f2df40047d0 2023-11-28T21:38:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 -- 172.21.15.12:0/4093599415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2df40047d0 msgr2=0x7f2df4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2df40047d0 0x7f2df4004c50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2df800c0e0 tx=0x7f2df801cb20 comp rx=0 tx=0).stop 2023-11-28T21:38:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 -- 172.21.15.12:0/4093599415 shutdown_connections 2023-11-28T21:38:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 --2- 172.21.15.12:0/4093599415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2df40047d0 0x7f2df4004c50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:16.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 -- 172.21.15.12:0/4093599415 >> 172.21.15.12:0/4093599415 conn(0x7f2e08066530 msgr2=0x7f2e08066940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 -- 172.21.15.12:0/4093599415 shutdown_connections 2023-11-28T21:38:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 -- 172.21.15.12:0/4093599415 wait complete. 2023-11-28T21:38:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 Processor -- start 2023-11-28T21:38:16.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 -- start start 2023-11-28T21:38:16.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2df40047d0 0x7f2e0806c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:16.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0806b7c0 0x7f2e080cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:16.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e080cc8d0 0x7f2e0813de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:16.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e080cbf70 con 0x7f2e0806b7c0 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e080cb660 con 0x7f2df40047d0 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0fa79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e080ccd50 con 0x7f2e080cc8d0 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0d014640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0806b7c0 0x7f2e080cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0d014640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0806b7c0 0x7f2e080cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35924/0 (socket says 172.21.15.12:35924) 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0d014640 1 -- 172.21.15.12:0/1281652594 learned_addr learned my addr 172.21.15.12:0/1281652594 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0d815640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2df40047d0 0x7f2e0806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0e016640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e080cc8d0 0x7f2e0813de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0d014640 1 -- 172.21.15.12:0/1281652594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e080cc8d0 msgr2=0x7f2e0813de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0d014640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e080cc8d0 0x7f2e0813de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0d014640 1 -- 172.21.15.12:0/1281652594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2df40047d0 msgr2=0x7f2e0806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0d014640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2df40047d0 0x7f2e0806c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:16.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0d014640 1 -- 172.21.15.12:0/1281652594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0806ccd0 con 0x7f2e0806b7c0 2023-11-28T21:38:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.562+0000 7f2e0d014640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0806b7c0 0x7f2e080cc390 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2df801c590 tx=0x7f2df4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:16.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2dfe7fc640 1 -- 172.21.15.12:0/1281652594 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80b4070 con 0x7f2e0806b7c0 2023-11-28T21:38:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2dfe7fc640 1 -- 172.21.15.12:0/1281652594 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df80a5820 con 0x7f2e0806b7c0 2023-11-28T21:38:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2dfe7fc640 1 -- 172.21.15.12:0/1281652594 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80a5bd0 con 0x7f2e0806b7c0 2023-11-28T21:38:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2e0fa79640 1 -- 172.21.15.12:0/1281652594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0806cfb0 con 0x7f2e0806b7c0 2023-11-28T21:38:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2e0fa79640 1 -- 172.21.15.12:0/1281652594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e080cbb90 con 0x7f2e0806b7c0 2023-11-28T21:38:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2e0fa79640 1 -- 172.21.15.12:0/1281652594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e08074f60 con 0x7f2e0806b7c0 2023-11-28T21:38:16.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2dfe7fc640 1 -- 172.21.15.12:0/1281652594 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2df80b7070 con 0x7f2e0806b7c0 2023-11-28T21:38:16.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2dfe7fc640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2dec066460 0x7f2dec068920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:16.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2dfe7fc640 1 -- 172.21.15.12:0/1281652594 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f2df811dae0 con 0x7f2e0806b7c0 2023-11-28T21:38:16.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2e0d815640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2dec066460 0x7f2dec068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:16.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.566+0000 7f2e0d815640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2dec066460 0x7f2dec068920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2e04000c10 tx=0x7f2e04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:16.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.570+0000 7f2dfe7fc640 1 -- 172.21.15.12:0/1281652594 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2df80eab60 con 0x7f2e0806b7c0 2023-11-28T21:38:16.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 -- 172.21.15.12:0/1281652594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e080c6fa0 con 0x7f2e0806b7c0 2023-11-28T21:38:16.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2dfe7fc640 1 -- 172.21.15.12:0/1281652594 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f2df80ee180 con 0x7f2e0806b7c0 2023-11-28T21:38:16.767 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:16.767 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:16.767 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-11-28T21:38:16.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 -- 172.21.15.12:0/1281652594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2dec066460 msgr2=0x7f2dec068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2dec066460 0x7f2dec068920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2e04000c10 tx=0x7f2e04009040 comp rx=0 tx=0).stop 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 -- 172.21.15.12:0/1281652594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0806b7c0 msgr2=0x7f2e080cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0806b7c0 0x7f2e080cc390 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2df801c590 tx=0x7f2df4005800 comp rx=0 tx=0).stop 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 -- 172.21.15.12:0/1281652594 shutdown_connections 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2dec066460 0x7f2dec068920 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e080cc8d0 0x7f2e0813de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e0806b7c0 0x7f2e080cc390 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 --2- 172.21.15.12:0/1281652594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2df40047d0 0x7f2e0806c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 -- 172.21.15.12:0/1281652594 >> 172.21.15.12:0/1281652594 conn(0x7f2e08066530 msgr2=0x7f2e0806f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 -- 172.21.15.12:0/1281652594 shutdown_connections 2023-11-28T21:38:16.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:16.762+0000 7f2e0fa79640 1 -- 172.21.15.12:0/1281652594 wait complete. 2023-11-28T21:38:16.788 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:38:18.789 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.950+0000 7f9afb59e640 1 Processor -- start 2023-11-28T21:38:18.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.950+0000 7f9afb59e640 1 -- start start 2023-11-28T21:38:18.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9afb59e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9afc15a980 0x7f9afc0b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9afb59e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9afc0b4010 0x7f9afc0cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9afb59e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc0b44a0 con 0x7f9afc153fb0 2023-11-28T21:38:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9afb59e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc0b2b60 con 0x7f9afc15a980 2023-11-28T21:38:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9afb59e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc15e1b0 con 0x7f9afc0b4010 2023-11-28T21:38:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9af9d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9afc15a980 0x7f9afc0b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9af9d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9afc15a980 0x7f9afc0b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35926/0 (socket says 172.21.15.12:35926) 2023-11-28T21:38:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9af9d9b640 1 -- 172.21.15.12:0/1554412778 learned_addr learned my addr 172.21.15.12:0/1554412778 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:18.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9afad9d640 1 --2- 172.21.15.12:0/1554412778 >> v2:172.21.15.144:3300/0 conn(0x7f9afc0b4010 0x7f9afc0cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:18.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9af9d9b640 1 -- 172.21.15.12:0/1554412778 >> v2:172.21.15.144:3300/0 conn(0x7f9afc0b4010 msgr2=0x7f9afc0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:18.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9af9d9b640 1 --2- 172.21.15.12:0/1554412778 >> v2:172.21.15.144:3300/0 conn(0x7f9afc0b4010 0x7f9afc0cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:18.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9af9d9b640 1 -- 172.21.15.12:0/1554412778 >> v1:172.21.15.144:6789/0 conn(0x7f9afc153fb0 legacy=0x7f9afc0ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:18.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.954+0000 7f9af9d9b640 1 -- 172.21.15.12:0/1554412778 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc15d390 con 0x7f9afc15a980 2023-11-28T21:38:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9af9d9b640 1 --2- 172.21.15.12:0/1554412778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 0x7f9afc0b3ad0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9afc159cb0 tx=0x7f9af001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13f5db189459c66 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9af959a640 1 -- 172.21.15.12:0/1554412778 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00a89f0 con 0x7f9afc15a980 2023-11-28T21:38:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9af959a640 1 -- 172.21.15.12:0/1554412778 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af00a8b50 con 0x7f9afc15a980 2023-11-28T21:38:18.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9af959a640 1 -- 172.21.15.12:0/1554412778 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00b0440 con 0x7f9afc15a980 2023-11-28T21:38:18.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9afb59e640 1 -- 172.21.15.12:0/1554412778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 msgr2=0x7f9afc0b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:18.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9afb59e640 1 --2- 172.21.15.12:0/1554412778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 0x7f9afc0b3ad0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9afc159cb0 tx=0x7f9af001ecf0 comp rx=0 tx=0).stop 2023-11-28T21:38:18.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9afb59e640 1 -- 172.21.15.12:0/1554412778 shutdown_connections 2023-11-28T21:38:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9afb59e640 1 --2- 172.21.15.12:0/1554412778 >> v2:172.21.15.144:3300/0 conn(0x7f9afc0b4010 0x7f9afc0cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9afb59e640 1 --2- 172.21.15.12:0/1554412778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 0x7f9afc0b3ad0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9afb59e640 1 -- 172.21.15.12:0/1554412778 >> 172.21.15.12:0/1554412778 conn(0x7f9afc071ce0 msgr2=0x7f9afc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9afb59e640 1 -- 172.21.15.12:0/1554412778 shutdown_connections 2023-11-28T21:38:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9afb59e640 1 -- 172.21.15.12:0/1554412778 wait complete. 2023-11-28T21:38:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9afb59e640 1 Processor -- start 2023-11-28T21:38:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.958+0000 7f9afb59e640 1 -- start start 2023-11-28T21:38:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9afc0b4010 0x7f9afc1f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9afc153fb0 0x7f9afc158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afb59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 0x7f9afc15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afb59e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1f0dd0 con 0x7f9afc15a980 2023-11-28T21:38:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc15c4c0 con 0x7f9afc0b4010 2023-11-28T21:38:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc15c760 con 0x7f9afc153fb0 2023-11-28T21:38:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 0x7f9afc15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 0x7f9afc15b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35928/0 (socket says 172.21.15.12:35928) 2023-11-28T21:38:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afad9d640 1 -- 172.21.15.12:0/931054451 learned_addr learned my addr 172.21.15.12:0/931054451 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afa59c640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9afc0b4010 0x7f9afc1f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9af9d9b640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9afc153fb0 0x7f9afc158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afad9d640 1 -- 172.21.15.12:0/931054451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9afc153fb0 msgr2=0x7f9afc158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afad9d640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9afc153fb0 0x7f9afc158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afad9d640 1 -- 172.21.15.12:0/931054451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9afc0b4010 msgr2=0x7f9afc1f0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afad9d640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9afc0b4010 0x7f9afc1f0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afad9d640 1 -- 172.21.15.12:0/931054451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af00094d0 con 0x7f9afc15a980 2023-11-28T21:38:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afa59c640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9afc0b4010 0x7f9afc1f0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:38:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afad9d640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 0x7f9afc15b650 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9af400efc0 tx=0x7f9af400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9aeaffd640 1 -- 172.21.15.12:0/931054451 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4014480 con 0x7f9afc15a980 2023-11-28T21:38:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9aeaffd640 1 -- 172.21.15.12:0/931054451 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af400f040 con 0x7f9afc15a980 2023-11-28T21:38:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afb59e640 1 -- 172.21.15.12:0/931054451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc15ca40 con 0x7f9afc15a980 2023-11-28T21:38:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9aeaffd640 1 -- 172.21.15.12:0/931054451 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4014820 con 0x7f9afc15a980 2023-11-28T21:38:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afb59e640 1 -- 172.21.15.12:0/931054451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc15d000 con 0x7f9afc15a980 2023-11-28T21:38:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.962+0000 7f9afb59e640 1 -- 172.21.15.12:0/931054451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc1e3950 con 0x7f9afc15a980 2023-11-28T21:38:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.966+0000 7f9aeaffd640 1 -- 172.21.15.12:0/931054451 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9af4014980 con 0x7f9afc15a980 2023-11-28T21:38:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.966+0000 7f9aeaffd640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9adc066470 0x7f9adc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.966+0000 7f9aeaffd640 1 -- 172.21.15.12:0/931054451 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f9af4084f30 con 0x7f9afc15a980 2023-11-28T21:38:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.966+0000 7f9afa59c640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9adc066470 0x7f9adc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:18.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.966+0000 7f9afa59c640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9adc066470 0x7f9adc068930 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9aec000f70 tx=0x7f9aec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:18.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:18.970+0000 7f9aeaffd640 1 -- 172.21.15.12:0/931054451 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9afc1e3950 con 0x7f9afc15a980 2023-11-28T21:38:19.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.158+0000 7f9afb59e640 1 -- 172.21.15.12:0/931054451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9afc1e3950 con 0x7f9afc15a980 2023-11-28T21:38:19.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.158+0000 7f9aeaffd640 1 -- 172.21.15.12:0/931054451 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f9afc1e3950 con 0x7f9afc15a980 2023-11-28T21:38:19.165 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:19.165 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:19.165 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-11-28T21:38:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 -- 172.21.15.12:0/931054451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9adc066470 msgr2=0x7f9adc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9adc066470 0x7f9adc068930 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9aec000f70 tx=0x7f9aec009040 comp rx=0 tx=0).stop 2023-11-28T21:38:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 -- 172.21.15.12:0/931054451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 msgr2=0x7f9afc15b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:19.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 0x7f9afc15b650 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9af400efc0 tx=0x7f9af400c490 comp rx=0 tx=0).stop 2023-11-28T21:38:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 -- 172.21.15.12:0/931054451 shutdown_connections 2023-11-28T21:38:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9adc066470 0x7f9adc068930 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9afc15a980 0x7f9afc15b650 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9afc153fb0 0x7f9afc158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 --2- 172.21.15.12:0/931054451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9afc0b4010 0x7f9afc1f0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 -- 172.21.15.12:0/931054451 >> 172.21.15.12:0/931054451 conn(0x7f9afc071ce0 msgr2=0x7f9afc0ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 -- 172.21.15.12:0/931054451 shutdown_connections 2023-11-28T21:38:19.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:19.162+0000 7f9afb59e640 1 -- 172.21.15.12:0/931054451 wait complete. 2023-11-28T21:38:19.186 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:38:20.361 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2023-11-28T21:38:20.361 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2023-11-28T21:38:20.361 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2023-11-28T21:38:20.362 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2023-11-28T21:38:20.363 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2023-11-28T21:38:20.364 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2023-11-28T21:38:20.365 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2023-11-28T21:38:20.366 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2023-11-28T21:38:20.367 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2023-11-28T21:38:20.368 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2023-11-28T21:38:20.368 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2023-11-28T21:38:20.368 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2023-11-28T21:38:20.368 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2023-11-28T21:38:20.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2023-11-28T21:38:20.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2023-11-28T21:38:20.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2023-11-28T21:38:20.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2023-11-28T21:38:20.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2023-11-28T21:38:20.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2023-11-28T21:38:20.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2023-11-28T21:38:20.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2023-11-28T21:38:20.935 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2023-11-28T21:38:20.936 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2023-11-28T21:38:20.937 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2023-11-28T21:38:20.938 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2023-11-28T21:38:20.939 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2023-11-28T21:38:20.940 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2023-11-28T21:38:20.941 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2023-11-28T21:38:20.941 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2023-11-28T21:38:20.941 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2023-11-28T21:38:20.941 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2023-11-28T21:38:20.941 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2023-11-28T21:38:20.941 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2023-11-28T21:38:20.941 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2023-11-28T21:38:20.941 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2023-11-28T21:38:20.941 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2023-11-28T21:38:20.942 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2023-11-28T21:38:20.942 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2023-11-28T21:38:21.187 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:21.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.334+0000 7f13f689c640 1 Processor -- start 2023-11-28T21:38:21.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.334+0000 7f13f689c640 1 -- start start 2023-11-28T21:38:21.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f689c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f13f015a980 0x7f13f00b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:21.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f689c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00b4010 0x7f13f00cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:21.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f689c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f00b44a0 con 0x7f13f0153fb0 2023-11-28T21:38:21.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f689c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f00b2b60 con 0x7f13f015a980 2023-11-28T21:38:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f689c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f015e1b0 con 0x7f13f00b4010 2023-11-28T21:38:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f589a640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f13f0153fb0 0x7f13f00ca4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33046/0 (socket says 172.21.15.12:33046) 2023-11-28T21:38:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f589a640 1 -- 172.21.15.12:0/3911778677 learned_addr learned my addr 172.21.15.12:0/3911778677 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:38:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f5099640 1 --2- 172.21.15.12:0/3911778677 >> v2:172.21.15.144:3300/0 conn(0x7f13f015a980 0x7f13f00b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:21.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f609b640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00b4010 0x7f13f00cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905102766 0 0) 0x7f13f00b44a0 con 0x7f13f0153fb0 2023-11-28T21:38:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13dc003530 con 0x7f13f0153fb0 2023-11-28T21:38:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2675335586 0 0) 0x7f13dc003530 con 0x7f13f0153fb0 2023-11-28T21:38:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00b4010 msgr2=0x7f13f00cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00b4010 0x7f13f00cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 >> v2:172.21.15.144:3300/0 conn(0x7f13f015a980 msgr2=0x7f13f00b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 --2- 172.21.15.12:0/3911778677 >> v2:172.21.15.144:3300/0 conn(0x7f13f015a980 0x7f13f00b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f015d390 con 0x7f13f0153fb0 2023-11-28T21:38:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f13ec001680 con 0x7f13f0153fb0 2023-11-28T21:38:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 >> v1:172.21.15.12:6789/0 conn(0x7f13f0153fb0 legacy=0x7f13f00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00b4010 0x7f13dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13dc003f30 0x7f13dc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13dc004d10 0x7f13dc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f00b2b60 con 0x7f13dc003f30 2023-11-28T21:38:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f015e1b0 con 0x7f13dc004d10 2023-11-28T21:38:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc00b990 con 0x7f13f00b4010 2023-11-28T21:38:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f5099640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13dc003f30 0x7f13dc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f609b640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13dc004d10 0x7f13dc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f589a640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00b4010 0x7f13dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f5099640 1 -- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00b4010 msgr2=0x7f13dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f5099640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f00b4010 0x7f13dc0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f5099640 1 -- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13dc004d10 msgr2=0x7f13dc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f5099640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13dc004d10 0x7f13dc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.338+0000 7f13f5099640 1 -- 172.21.15.12:0/3911778677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc003530 con 0x7f13dc003f30 2023-11-28T21:38:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f5099640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13dc003f30 0x7f13dc0047d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f13f015ad90 tx=0x7f13e001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2d60337b5b3c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e00a5620 con 0x7f13dc003f30 2023-11-28T21:38:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e00a5780 con 0x7f13dc003f30 2023-11-28T21:38:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f4898640 1 -- 172.21.15.12:0/3911778677 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e00a5b30 con 0x7f13dc003f30 2023-11-28T21:38:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f689c640 1 -- 172.21.15.12:0/3911778677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13dc003f30 msgr2=0x7f13dc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f689c640 1 --2- 172.21.15.12:0/3911778677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13dc003f30 0x7f13dc0047d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f13f015ad90 tx=0x7f13e001ca30 comp rx=0 tx=0).stop 2023-11-28T21:38:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f609b640 1 -- 172.21.15.12:0/3911778677 reap_dead start 2023-11-28T21:38:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f689c640 1 -- 172.21.15.12:0/3911778677 shutdown_connections 2023-11-28T21:38:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f689c640 1 -- 172.21.15.12:0/3911778677 >> 172.21.15.12:0/3911778677 conn(0x7f13f0071ce0 msgr2=0x7f13f00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f689c640 1 -- 172.21.15.12:0/3911778677 shutdown_connections 2023-11-28T21:38:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.342+0000 7f13f689c640 1 -- 172.21.15.12:0/3911778677 wait complete. 2023-11-28T21:38:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 Processor -- start 2023-11-28T21:38:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 -- start start 2023-11-28T21:38:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13dc003f30 0x7f13f01e4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:21.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13f00cdcd0 0x7f13f01f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f01e64a0 0x7f13f0149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f01e5c80 con 0x7f13f00cdcd0 2023-11-28T21:38:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f00cd4a0 con 0x7f13dc003f30 2023-11-28T21:38:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f00cd780 con 0x7f13f01e64a0 2023-11-28T21:38:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f5099640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13f00cdcd0 0x7f13f01f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f5099640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13f00cdcd0 0x7f13f01f0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35956/0 (socket says 172.21.15.12:35956) 2023-11-28T21:38:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f5099640 1 -- 172.21.15.12:0/646410962 learned_addr learned my addr 172.21.15.12:0/646410962 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f589a640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13dc003f30 0x7f13f01e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f609b640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f01e64a0 0x7f13f0149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f5099640 1 -- 172.21.15.12:0/646410962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f01e64a0 msgr2=0x7f13f0149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f5099640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f01e64a0 0x7f13f0149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f5099640 1 -- 172.21.15.12:0/646410962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13dc003f30 msgr2=0x7f13f01e4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f5099640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13dc003f30 0x7f13f01e4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f5099640 1 -- 172.21.15.12:0/646410962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f01f0240 con 0x7f13f00cdcd0 2023-11-28T21:38:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f5099640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13f00cdcd0 0x7f13f01f0a40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f13e00b0040 tx=0x7f13dc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13e67fc640 1 -- 172.21.15.12:0/646410962 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e00a5620 con 0x7f13f00cdcd0 2023-11-28T21:38:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13e67fc640 1 -- 172.21.15.12:0/646410962 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e0004260 con 0x7f13f00cdcd0 2023-11-28T21:38:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 -- 172.21.15.12:0/646410962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f01f0520 con 0x7f13f00cdcd0 2023-11-28T21:38:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13e67fc640 1 -- 172.21.15.12:0/646410962 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0003620 con 0x7f13f00cdcd0 2023-11-28T21:38:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 -- 172.21.15.12:0/646410962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f01e5940 con 0x7f13f00cdcd0 2023-11-28T21:38:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.346+0000 7f13f689c640 1 -- 172.21.15.12:0/646410962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f01f06e0 con 0x7f13f00cdcd0 2023-11-28T21:38:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.350+0000 7f13e67fc640 1 -- 172.21.15.12:0/646410962 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f13e00b4040 con 0x7f13f00cdcd0 2023-11-28T21:38:21.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.350+0000 7f13e67fc640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13d40664f0 0x7f13d40689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.350+0000 7f13e67fc640 1 -- 172.21.15.12:0/646410962 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f13e011dbd0 con 0x7f13f00cdcd0 2023-11-28T21:38:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.350+0000 7f13f589a640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13d40664f0 0x7f13d40689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.350+0000 7f13f589a640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13d40664f0 0x7f13d40689b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0017b0 tx=0x7f13ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:21.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.354+0000 7f13e67fc640 1 -- 172.21.15.12:0/646410962 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f13e00eacf0 con 0x7f13f00cdcd0 2023-11-28T21:38:21.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.542+0000 7f13f689c640 1 -- 172.21.15.12:0/646410962 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13f00cd8f0 con 0x7f13f00cdcd0 2023-11-28T21:38:21.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.542+0000 7f13e67fc640 1 -- 172.21.15.12:0/646410962 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f13e00ee310 con 0x7f13f00cdcd0 2023-11-28T21:38:21.548 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:21.548 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2321889805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2321889805},{"type":"v1","addr":"172.21.15.12:6838","nonce":2321889805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:21.549 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 19 2023-11-28T21:38:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 -- 172.21.15.12:0/646410962 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13d40664f0 msgr2=0x7f13d40689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13d40664f0 0x7f13d40689b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0017b0 tx=0x7f13ec009040 comp rx=0 tx=0).stop 2023-11-28T21:38:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 -- 172.21.15.12:0/646410962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13f00cdcd0 msgr2=0x7f13f01f0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13f00cdcd0 0x7f13f01f0a40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f13e00b0040 tx=0x7f13dc005800 comp rx=0 tx=0).stop 2023-11-28T21:38:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 -- 172.21.15.12:0/646410962 shutdown_connections 2023-11-28T21:38:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f13d40664f0 0x7f13d40689b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:21.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13f01e64a0 0x7f13f0149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:21.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f13f00cdcd0 0x7f13f01f0a40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:21.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 --2- 172.21.15.12:0/646410962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13dc003f30 0x7f13f01e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:21.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 -- 172.21.15.12:0/646410962 >> 172.21.15.12:0/646410962 conn(0x7f13f0071ce0 msgr2=0x7f13f00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:21.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 -- 172.21.15.12:0/646410962 shutdown_connections 2023-11-28T21:38:21.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:21.546+0000 7f13f689c640 1 -- 172.21.15.12:0/646410962 wait complete. 2023-11-28T21:38:21.569 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:38:22.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2023-11-28T21:38:22.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2023-11-28T21:38:22.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2023-11-28T21:38:22.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2023-11-28T21:38:22.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2023-11-28T21:38:22.640 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2023-11-28T21:38:22.641 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2023-11-28T21:38:22.642 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2023-11-28T21:38:22.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2023-11-28T21:38:22.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2023-11-28T21:38:22.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2023-11-28T21:38:22.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2023-11-28T21:38:22.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2023-11-28T21:38:22.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2023-11-28T21:38:22.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2023-11-28T21:38:22.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2023-11-28T21:38:22.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2023-11-28T21:38:22.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2023-11-28T21:38:22.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2023-11-28T21:38:22.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2023-11-28T21:38:22.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2023-11-28T21:38:22.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2023-11-28T21:38:22.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2023-11-28T21:38:22.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2023-11-28T21:38:22.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2023-11-28T21:38:22.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2023-11-28T21:38:23.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:23.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2023-11-28T21:38:23.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2023-11-28T21:38:23.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2023-11-28T21:38:23.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2023-11-28T21:38:23.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2023-11-28T21:38:23.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2023-11-28T21:38:23.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2023-11-28T21:38:23.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2023-11-28T21:38:23.643 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2023-11-28T21:38:23.644 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2023-11-28T21:38:23.645 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2023-11-28T21:38:23.646 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2023-11-28T21:38:23.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2023-11-28T21:38:23.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2023-11-28T21:38:23.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2023-11-28T21:38:23.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2023-11-28T21:38:23.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2023-11-28T21:38:23.647 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2023-11-28T21:38:23.648 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2023-11-28T21:38:23.649 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2023-11-28T21:38:23.649 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2023-11-28T21:38:23.649 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2023-11-28T21:38:23.649 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2023-11-28T21:38:23.649 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2023-11-28T21:38:23.649 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2023-11-28T21:38:23.649 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2023-11-28T21:38:23.649 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2023-11-28T21:38:23.650 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2023-11-28T21:38:23.650 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2023-11-28T21:38:23.650 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2023-11-28T21:38:23.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98e53a7640 1 Processor -- start 2023-11-28T21:38:23.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98e53a7640 1 -- start start 2023-11-28T21:38:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98e53a7640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f98e0153fb0 0x7f98e00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98e53a7640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e015d390 con 0x7f98e00b41e0 2023-11-28T21:38:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98e53a7640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e015d630 con 0x7f98e015a980 2023-11-28T21:38:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98e53a7640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e00b2b60 con 0x7f98e0153fb0 2023-11-28T21:38:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98e4ba6640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f98e00b41e0 0x7f98e00cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33058/0 (socket says 172.21.15.12:33058) 2023-11-28T21:38:23.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98e4ba6640 1 -- 172.21.15.12:0/515311358 learned_addr learned my addr 172.21.15.12:0/515311358 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:38:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98dffff640 1 --2- 172.21.15.12:0/515311358 >> v2:172.21.15.144:3300/0 conn(0x7f98e0153fb0 0x7f98e00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499937445 0 0) 0x7f98e015d390 con 0x7f98e00b41e0 2023-11-28T21:38:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c8003530 con 0x7f98e00b41e0 2023-11-28T21:38:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2729852683 0 0) 0x7f98c8003530 con 0x7f98e00b41e0 2023-11-28T21:38:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 >> v2:172.21.15.144:3300/0 conn(0x7f98e0153fb0 msgr2=0x7f98e00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 --2- 172.21.15.12:0/515311358 >> v2:172.21.15.144:3300/0 conn(0x7f98e0153fb0 0x7f98e00ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 >> v1:172.21.15.144:6789/0 conn(0x7f98e015a980 legacy=0x7f98e00b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e015d910 con 0x7f98e00b41e0 2023-11-28T21:38:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f98d8001a00 con 0x7f98e00b41e0 2023-11-28T21:38:23.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 >> v1:172.21.15.12:6789/0 conn(0x7f98e00b41e0 legacy=0x7f98e00cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c80039d0 0x7f98c8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c80047d0 0x7f98c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c8005190 0x7f98c800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00b2b60 con 0x7f98c8005190 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c800c070 con 0x7f98c80039d0 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c800c230 con 0x7f98c80047d0 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98dffff640 1 --2- 172.21.15.12:0/515311358 >> v2:172.21.15.144:3300/0 conn(0x7f98e0153fb0 0x7f98e00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98e4ba6640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c8005190 0x7f98c800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98df7fe640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c80047d0 0x7f98c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.730+0000 7f98dffff640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c80039d0 0x7f98c8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e4ba6640 1 -- 172.21.15.12:0/515311358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c80047d0 msgr2=0x7f98c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e4ba6640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c80047d0 0x7f98c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e4ba6640 1 -- 172.21.15.12:0/515311358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c80039d0 msgr2=0x7f98c8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:23.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e4ba6640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c80039d0 0x7f98c8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e4ba6640 1 -- 172.21.15.12:0/515311358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c8003530 con 0x7f98c8005190 2023-11-28T21:38:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e4ba6640 1 -- 172.21.15.12:0/515311358 reap_dead start 2023-11-28T21:38:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e4ba6640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c8005190 0x7f98c800b990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f98d800a760 tx=0x7f98d801d260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9efc6bed2087c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:23.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80ad630 con 0x7f98c8005190 2023-11-28T21:38:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80a8070 con 0x7f98c8005190 2023-11-28T21:38:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98deffd640 1 -- 172.21.15.12:0/515311358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80ada10 con 0x7f98c8005190 2023-11-28T21:38:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e53a7640 1 -- 172.21.15.12:0/515311358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c8005190 msgr2=0x7f98c800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e53a7640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c8005190 0x7f98c800b990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f98d800a760 tx=0x7f98d801d260 comp rx=0 tx=0).stop 2023-11-28T21:38:23.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e53a7640 1 -- 172.21.15.12:0/515311358 shutdown_connections 2023-11-28T21:38:23.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e53a7640 1 --2- 172.21.15.12:0/515311358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98c8005190 0x7f98c800b990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:23.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e53a7640 1 -- 172.21.15.12:0/515311358 >> 172.21.15.12:0/515311358 conn(0x7f98e0071ce0 msgr2=0x7f98e00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:23.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e53a7640 1 -- 172.21.15.12:0/515311358 shutdown_connections 2023-11-28T21:38:23.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.734+0000 7f98e53a7640 1 -- 172.21.15.12:0/515311358 wait complete. 2023-11-28T21:38:23.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e53a7640 1 Processor -- start 2023-11-28T21:38:23.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e53a7640 1 -- start start 2023-11-28T21:38:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e53a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c8005190 0x7f98e015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e53a7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e00c5ad0 0x7f98e0149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e53a7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e014a180 0x7f98e01f3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e53a7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e015e1b0 con 0x7f98e014a180 2023-11-28T21:38:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e53a7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00cf780 con 0x7f98c8005190 2023-11-28T21:38:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e53a7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00c62a0 con 0x7f98e00c5ad0 2023-11-28T21:38:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e4ba6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e014a180 0x7f98e01f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e4ba6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e014a180 0x7f98e01f3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35974/0 (socket says 172.21.15.12:35974) 2023-11-28T21:38:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e4ba6640 1 -- 172.21.15.12:0/292695346 learned_addr learned my addr 172.21.15.12:0/292695346 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:23.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98df7fe640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e00c5ad0 0x7f98e0149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98dffff640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c8005190 0x7f98e015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e4ba6640 1 -- 172.21.15.12:0/292695346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e00c5ad0 msgr2=0x7f98e0149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e4ba6640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e00c5ad0 0x7f98e0149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e4ba6640 1 -- 172.21.15.12:0/292695346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c8005190 msgr2=0x7f98e015ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e4ba6640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c8005190 0x7f98e015ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e4ba6640 1 -- 172.21.15.12:0/292695346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e00c5fc0 con 0x7f98e014a180 2023-11-28T21:38:23.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e4ba6640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e014a180 0x7f98e01f3810 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f98d801db00 tx=0x7f98c8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:23.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98dcff9640 1 -- 172.21.15.12:0/292695346 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8001a00 con 0x7f98e014a180 2023-11-28T21:38:23.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98dcff9640 1 -- 172.21.15.12:0/292695346 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80ad6c0 con 0x7f98e014a180 2023-11-28T21:38:23.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98dcff9640 1 -- 172.21.15.12:0/292695346 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8001b60 con 0x7f98e014a180 2023-11-28T21:38:23.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.738+0000 7f98e53a7640 1 -- 172.21.15.12:0/292695346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e00cee50 con 0x7f98e014a180 2023-11-28T21:38:23.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.742+0000 7f98e53a7640 1 -- 172.21.15.12:0/292695346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e00cf410 con 0x7f98e014a180 2023-11-28T21:38:23.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.742+0000 7f98e53a7640 1 -- 172.21.15.12:0/292695346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e00cf8f0 con 0x7f98e014a180 2023-11-28T21:38:23.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.742+0000 7f98dcff9640 1 -- 172.21.15.12:0/292695346 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f98d80aa030 con 0x7f98e014a180 2023-11-28T21:38:23.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.742+0000 7f98dcff9640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f98cc0664a0 0x7f98cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:23.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.742+0000 7f98dcff9640 1 -- 172.21.15.12:0/292695346 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f98d80a5e40 con 0x7f98e014a180 2023-11-28T21:38:23.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.742+0000 7f98dffff640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f98cc0664a0 0x7f98cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:23.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.746+0000 7f98dffff640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f98cc0664a0 0x7f98cc068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009b40 tx=0x7f98d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:23.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.746+0000 7f98dcff9640 1 -- 172.21.15.12:0/292695346 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98d80eaf60 con 0x7f98e014a180 2023-11-28T21:38:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 -- 172.21.15.12:0/292695346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98e00cf8f0 con 0x7f98e014a180 2023-11-28T21:38:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98dcff9640 1 -- 172.21.15.12:0/292695346 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f98d80ee580 con 0x7f98e014a180 2023-11-28T21:38:23.947 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:23.948 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2023-11-28T21:38:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 -- 172.21.15.12:0/292695346 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f98cc0664a0 msgr2=0x7f98cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f98cc0664a0 0x7f98cc068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009b40 tx=0x7f98d0008040 comp rx=0 tx=0).stop 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 -- 172.21.15.12:0/292695346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e014a180 msgr2=0x7f98e01f3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e014a180 0x7f98e01f3810 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f98d801db00 tx=0x7f98c8005800 comp rx=0 tx=0).stop 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 -- 172.21.15.12:0/292695346 shutdown_connections 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f98cc0664a0 0x7f98cc068960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98e014a180 0x7f98e01f3810 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98e00c5ad0 0x7f98e0149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 --2- 172.21.15.12:0/292695346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98c8005190 0x7f98e015ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.942+0000 7f98e53a7640 1 -- 172.21.15.12:0/292695346 >> 172.21.15.12:0/292695346 conn(0x7f98e0071ce0 msgr2=0x7f98e00cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.946+0000 7f98e53a7640 1 -- 172.21.15.12:0/292695346 shutdown_connections 2023-11-28T21:38:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:23.946+0000 7f98e53a7640 1 -- 172.21.15.12:0/292695346 wait complete. 2023-11-28T21:38:23.969 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2023-11-28T21:38:23.969 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:38:23.970 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:24.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.106+0000 7f039c70f640 1 Processor -- start 2023-11-28T21:38:24.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.106+0000 7f039c70f640 1 -- start start 2023-11-28T21:38:24.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039c70f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0394151af0 0x7f03940c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039c70f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f039406e300 0x7f0394072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039c70f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0394157b70 con 0x7f0394151ed0 2023-11-28T21:38:24.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039c70f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03940707f0 con 0x7f0394151af0 2023-11-28T21:38:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039c70f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03940c7b90 con 0x7f039406e300 2023-11-28T21:38:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039a4ab640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0394151af0 0x7f03940c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:24.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f0399caa640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0394151ed0 0x7f03940c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33066/0 (socket says 172.21.15.12:33066) 2023-11-28T21:38:24.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f0399caa640 1 -- 172.21.15.12:0/2209075040 learned_addr learned my addr 172.21.15.12:0/2209075040 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:38:24.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039acac640 1 --2- 172.21.15.12:0/2209075040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f039406e300 0x7f0394072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:24.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039a4ab640 1 -- 172.21.15.12:0/2209075040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f039406e300 msgr2=0x7f0394072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039a4ab640 1 --2- 172.21.15.12:0/2209075040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f039406e300 0x7f0394072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039a4ab640 1 -- 172.21.15.12:0/2209075040 >> v1:172.21.15.12:6789/0 conn(0x7f0394151ed0 legacy=0x7f03940c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:24.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039a4ab640 1 -- 172.21.15.12:0/2209075040 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394157e50 con 0x7f0394151af0 2023-11-28T21:38:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.110+0000 7f039a4ab640 1 --2- 172.21.15.12:0/2209075040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 0x7f03940c7650 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03900097c0 tx=0x7f039001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50089cea8acab3fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:24.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f03994a9640 1 -- 172.21.15.12:0/2209075040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03900ab600 con 0x7f0394151af0 2023-11-28T21:38:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f03994a9640 1 -- 172.21.15.12:0/2209075040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03900a7050 con 0x7f0394151af0 2023-11-28T21:38:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f03994a9640 1 -- 172.21.15.12:0/2209075040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03900ab980 con 0x7f0394151af0 2023-11-28T21:38:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 -- 172.21.15.12:0/2209075040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 msgr2=0x7f03940c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 --2- 172.21.15.12:0/2209075040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 0x7f03940c7650 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03900097c0 tx=0x7f039001c480 comp rx=0 tx=0).stop 2023-11-28T21:38:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 -- 172.21.15.12:0/2209075040 shutdown_connections 2023-11-28T21:38:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 --2- 172.21.15.12:0/2209075040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f039406e300 0x7f0394072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 --2- 172.21.15.12:0/2209075040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 0x7f03940c7650 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 -- 172.21.15.12:0/2209075040 >> 172.21.15.12:0/2209075040 conn(0x7f0394066360 msgr2=0x7f0394066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 -- 172.21.15.12:0/2209075040 shutdown_connections 2023-11-28T21:38:24.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 -- 172.21.15.12:0/2209075040 wait complete. 2023-11-28T21:38:24.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 Processor -- start 2023-11-28T21:38:24.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 -- start start 2023-11-28T21:38:24.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f039406e300 0x7f03940c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:24.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 0x7f03941f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:24.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f039413f700 0x7f039413e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:24.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03940c7030 con 0x7f0394151af0 2023-11-28T21:38:24.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03940c9160 con 0x7f039406e300 2023-11-28T21:38:24.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f039c70f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03940c9440 con 0x7f039413f700 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f0399caa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 0x7f03941f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f0399caa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 0x7f03941f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35994/0 (socket says 172.21.15.12:35994) 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.114+0000 7f0399caa640 1 -- 172.21.15.12:0/655988957 learned_addr learned my addr 172.21.15.12:0/655988957 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f039acac640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f039413f700 0x7f039413e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f039a4ab640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f039406e300 0x7f03940c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f0399caa640 1 -- 172.21.15.12:0/655988957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f039413f700 msgr2=0x7f039413e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f0399caa640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f039413f700 0x7f039413e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f0399caa640 1 -- 172.21.15.12:0/655988957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f039406e300 msgr2=0x7f03940c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f0399caa640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f039406e300 0x7f03940c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f0399caa640 1 -- 172.21.15.12:0/655988957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390009420 con 0x7f0394151af0 2023-11-28T21:38:24.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f0399caa640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 0x7f03941f6d40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f038400c3a0 tx=0x7f038400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:24.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f038affd640 1 -- 172.21.15.12:0/655988957 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384005cf0 con 0x7f0394151af0 2023-11-28T21:38:24.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f038affd640 1 -- 172.21.15.12:0/655988957 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0384005e50 con 0x7f0394151af0 2023-11-28T21:38:24.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f038affd640 1 -- 172.21.15.12:0/655988957 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03840072d0 con 0x7f0394151af0 2023-11-28T21:38:24.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f039c70f640 1 -- 172.21.15.12:0/655988957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03940c9720 con 0x7f0394151af0 2023-11-28T21:38:24.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f039c70f640 1 -- 172.21.15.12:0/655988957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03940c6c50 con 0x7f0394151af0 2023-11-28T21:38:24.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f039c70f640 1 -- 172.21.15.12:0/655988957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03940c7170 con 0x7f0394151af0 2023-11-28T21:38:24.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f038affd640 1 -- 172.21.15.12:0/655988957 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0384007430 con 0x7f0394151af0 2023-11-28T21:38:24.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f038affd640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0378066470 0x7f0378068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:24.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.118+0000 7f038affd640 1 -- 172.21.15.12:0/655988957 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f03840835d0 con 0x7f0394151af0 2023-11-28T21:38:24.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.122+0000 7f039a4ab640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0378066470 0x7f0378068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:24.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.122+0000 7f039a4ab640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0378066470 0x7f0378068930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f03900ad000 tx=0x7f0390008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:24.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.122+0000 7f038affd640 1 -- 172.21.15.12:0/655988957 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03940c7170 con 0x7f0394151af0 2023-11-28T21:38:24.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.314+0000 7f039c70f640 1 -- 172.21.15.12:0/655988957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03940c7170 con 0x7f0394151af0 2023-11-28T21:38:24.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.314+0000 7f038affd640 1 -- 172.21.15.12:0/655988957 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f03840507a0 con 0x7f0394151af0 2023-11-28T21:38:24.320 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:24.320 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:24.320 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2023-11-28T21:38:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.314+0000 7f039c70f640 1 -- 172.21.15.12:0/655988957 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0378066470 msgr2=0x7f0378068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.314+0000 7f039c70f640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0378066470 0x7f0378068930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f03900ad000 tx=0x7f0390008040 comp rx=0 tx=0).stop 2023-11-28T21:38:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.314+0000 7f039c70f640 1 -- 172.21.15.12:0/655988957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 msgr2=0x7f03941f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.314+0000 7f039c70f640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 0x7f03941f6d40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f038400c3a0 tx=0x7f038400c860 comp rx=0 tx=0).stop 2023-11-28T21:38:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.318+0000 7f039c70f640 1 -- 172.21.15.12:0/655988957 shutdown_connections 2023-11-28T21:38:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.318+0000 7f039c70f640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0378066470 0x7f0378068930 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.318+0000 7f039c70f640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f039413f700 0x7f039413e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.318+0000 7f039c70f640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0394151af0 0x7f03941f6d40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:24.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.318+0000 7f039c70f640 1 --2- 172.21.15.12:0/655988957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f039406e300 0x7f03940c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:24.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.318+0000 7f039c70f640 1 -- 172.21.15.12:0/655988957 >> 172.21.15.12:0/655988957 conn(0x7f0394066360 msgr2=0x7f039406faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:24.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.318+0000 7f039c70f640 1 -- 172.21.15.12:0/655988957 shutdown_connections 2023-11-28T21:38:24.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:24.318+0000 7f039c70f640 1 -- 172.21.15.12:0/655988957 wait complete. 2023-11-28T21:38:24.342 INFO:tasks.mds_thrash.fs.[cephfs]:cluster has 1 actives (max_mds is 1), no MDS can replace rank 2 2023-11-28T21:38:24.342 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.a 2023-11-28T21:38:25.198 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2023-11-28T21:38:25.199 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2023-11-28T21:38:25.200 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2023-11-28T21:38:25.201 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2023-11-28T21:38:25.202 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2023-11-28T21:38:25.203 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2023-11-28T21:38:25.204 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2023-11-28T21:38:25.205 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2023-11-28T21:38:25.205 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2023-11-28T21:38:25.205 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2023-11-28T21:38:26.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2023-11-28T21:38:26.227 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2023-11-28T21:38:26.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2023-11-28T21:38:26.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2023-11-28T21:38:26.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2023-11-28T21:38:26.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2023-11-28T21:38:26.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2023-11-28T21:38:26.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2023-11-28T21:38:26.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2023-11-28T21:38:26.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2023-11-28T21:38:26.228 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2023-11-28T21:38:26.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2023-11-28T21:38:26.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2023-11-28T21:38:26.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2023-11-28T21:38:26.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2023-11-28T21:38:26.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2023-11-28T21:38:26.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2023-11-28T21:38:26.229 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2023-11-28T21:38:26.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2023-11-28T21:38:26.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2023-11-28T21:38:26.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2023-11-28T21:38:26.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2023-11-28T21:38:26.230 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2023-11-28T21:38:26.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2023-11-28T21:38:26.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2023-11-28T21:38:26.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2023-11-28T21:38:26.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2023-11-28T21:38:26.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2023-11-28T21:38:26.231 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2023-11-28T21:38:26.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2023-11-28T21:38:26.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2023-11-28T21:38:26.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2023-11-28T21:38:26.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2023-11-28T21:38:26.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2023-11-28T21:38:26.232 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2023-11-28T21:38:26.233 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2023-11-28T21:38:26.233 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2023-11-28T21:38:26.233 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2023-11-28T21:38:26.233 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2023-11-28T21:38:26.233 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2023-11-28T21:38:26.233 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2023-11-28T21:38:26.234 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2023-11-28T21:38:26.234 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2023-11-28T21:38:26.234 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2023-11-28T21:38:26.234 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2023-11-28T21:38:26.234 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2023-11-28T21:38:26.235 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2023-11-28T21:38:26.235 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2023-11-28T21:38:26.235 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2023-11-28T21:38:26.235 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2023-11-28T21:38:26.235 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2023-11-28T21:38:26.235 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2023-11-28T21:38:26.235 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2023-11-28T21:38:26.236 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2023-11-28T21:38:26.236 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2023-11-28T21:38:26.236 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2023-11-28T21:38:26.236 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2023-11-28T21:38:26.236 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2023-11-28T21:38:26.236 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2023-11-28T21:38:26.236 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2023-11-28T21:38:26.237 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2023-11-28T21:38:26.237 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2023-11-28T21:38:26.237 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2023-11-28T21:38:26.237 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2023-11-28T21:38:26.237 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2023-11-28T21:38:26.237 INFO:tasks.workunit.client.0.smithi012.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2023-11-28T21:38:26.237 INFO:tasks.workunit.client.0.smithi012.stdout:Take third snapshot .snap/test3 2023-11-28T21:38:26.256 INFO:tasks.workunit.client.0.smithi012.stdout:Delete the snapshots... 2023-11-28T21:38:26.827 INFO:tasks.workunit.client.0.smithi012.stdout:.//.snap/test1 2023-11-28T21:38:26.827 INFO:tasks.workunit.client.0.smithi012.stdout:.//.snap/test2 2023-11-28T21:38:26.827 INFO:tasks.workunit.client.0.smithi012.stdout:.//.snap/test3 2023-11-28T21:38:26.837 INFO:tasks.workunit.client.0.smithi012.stdout:./274/.snap/snap-subdir-test 2023-11-28T21:38:26.846 INFO:tasks.workunit.client.0.smithi012.stdout:./166/.snap/snap-subdir-test 2023-11-28T21:38:26.855 INFO:tasks.workunit.client.0.smithi012.stdout:./256/.snap/snap-subdir-test 2023-11-28T21:38:27.628 INFO:tasks.workunit.client.0.smithi012.stdout:./247/.snap/snap-subdir-test 2023-11-28T21:38:27.967 INFO:tasks.workunit.client.0.smithi012.stdout:./248/.snap/snap-subdir-test 2023-11-28T21:38:27.981 INFO:tasks.workunit.client.0.smithi012.stdout:./147/.snap/snap-subdir-test 2023-11-28T21:38:27.993 INFO:tasks.workunit.client.0.smithi012.stdout:./151/.snap/snap-subdir-test 2023-11-28T21:38:28.005 INFO:tasks.workunit.client.0.smithi012.stdout:./170/.snap/snap-subdir-test 2023-11-28T21:38:28.014 INFO:tasks.workunit.client.0.smithi012.stdout:./167/.snap/snap-subdir-test 2023-11-28T21:38:28.023 INFO:tasks.workunit.client.0.smithi012.stdout:./140/.snap/snap-subdir-test 2023-11-28T21:38:28.033 INFO:tasks.workunit.client.0.smithi012.stdout:./142/.snap/snap-subdir-test 2023-11-28T21:38:28.042 INFO:tasks.workunit.client.0.smithi012.stdout:./150/.snap/snap-subdir-test 2023-11-28T21:38:28.053 INFO:tasks.workunit.client.0.smithi012.stdout:./273/.snap/snap-subdir-test 2023-11-28T21:38:28.063 INFO:tasks.workunit.client.0.smithi012.stdout:./269/.snap/snap-subdir-test 2023-11-28T21:38:28.072 INFO:tasks.workunit.client.0.smithi012.stdout:./105/.snap/snap-subdir-test 2023-11-28T21:38:28.082 INFO:tasks.workunit.client.0.smithi012.stdout:./210/.snap/snap-subdir-test 2023-11-28T21:38:28.091 INFO:tasks.workunit.client.0.smithi012.stdout:./265/.snap/snap-subdir-test 2023-11-28T21:38:28.355 INFO:tasks.workunit.client.0.smithi012.stdout:./218/.snap/snap-subdir-test 2023-11-28T21:38:28.617 INFO:tasks.workunit.client.0.smithi012.stdout:./290/.snap/snap-subdir-test 2023-11-28T21:38:29.465 INFO:tasks.workunit.client.0.smithi012.stdout:./149/.snap/snap-subdir-test 2023-11-28T21:38:29.476 INFO:tasks.workunit.client.0.smithi012.stdout:./109/.snap/snap-subdir-test 2023-11-28T21:38:29.486 INFO:tasks.workunit.client.0.smithi012.stdout:./292/.snap/snap-subdir-test 2023-11-28T21:38:29.497 INFO:tasks.workunit.client.0.smithi012.stdout:./242/.snap/snap-subdir-test 2023-11-28T21:38:29.506 INFO:tasks.workunit.client.0.smithi012.stdout:./158/.snap/snap-subdir-test 2023-11-28T21:38:29.515 INFO:tasks.workunit.client.0.smithi012.stdout:./243/.snap/snap-subdir-test 2023-11-28T21:38:29.525 INFO:tasks.workunit.client.0.smithi012.stdout:./106/.snap/snap-subdir-test 2023-11-28T21:38:29.535 INFO:tasks.workunit.client.0.smithi012.stdout:./250/.snap/snap-subdir-test 2023-11-28T21:38:29.545 INFO:tasks.workunit.client.0.smithi012.stdout:./221/.snap/snap-subdir-test 2023-11-28T21:38:29.554 INFO:tasks.workunit.client.0.smithi012.stdout:./244/.snap/snap-subdir-test 2023-11-28T21:38:29.564 INFO:tasks.workunit.client.0.smithi012.stdout:./258/.snap/snap-subdir-test 2023-11-28T21:38:29.574 INFO:tasks.workunit.client.0.smithi012.stdout:./268/.snap/snap-subdir-test 2023-11-28T21:38:29.583 INFO:tasks.workunit.client.0.smithi012.stdout:./189/.snap/snap-subdir-test 2023-11-28T21:38:29.927 INFO:tasks.workunit.client.0.smithi012.stdout:./285/.snap/snap-subdir-test 2023-11-28T21:38:29.938 INFO:tasks.workunit.client.0.smithi012.stdout:./208/.snap/snap-subdir-test 2023-11-28T21:38:30.201 INFO:tasks.workunit.client.0.smithi012.stdout:./288/.snap/snap-subdir-test 2023-11-28T21:38:30.343 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2023-11-28T21:38:30.344 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T21:38:30.344 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T21:38:30.348 INFO:tasks.ceph.mds.a:Started 2023-11-28T21:38:30.348 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:30.445 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-11-28T21:38:30.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.498+0000 7fe3129b8640 1 Processor -- start 2023-11-28T21:38:30.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.498+0000 7fe3129b8640 1 -- start start 2023-11-28T21:38:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.498+0000 7fe3129b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe30c153e40 0x7fe30c0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.498+0000 7fe3129b8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe30c15dc80 0x7fe30c0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.498+0000 7fe3129b8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c15acc0 con 0x7fe30c0cfba0 2023-11-28T21:38:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.498+0000 7fe3129b8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c159cb0 con 0x7fe30c15dc80 2023-11-28T21:38:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.498+0000 7fe3129b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c159ea0 con 0x7fe30c153e40 2023-11-28T21:38:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe30b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe30c15dc80 0x7fe30c0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe310f55640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe30c0cfba0 0x7fe30c0cb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35604/0 (socket says 172.21.15.12:35604) 2023-11-28T21:38:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe310f55640 1 -- 172.21.15.12:0/2696439535 learned_addr learned my addr 172.21.15.12:0/2696439535 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:38:30.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe30bfff640 1 --2- 172.21.15.12:0/2696439535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe30c153e40 0x7fe30c0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:30.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe30affd640 1 -- 172.21.15.12:0/2696439535 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494885496 0 0) 0x7fe30c15acc0 con 0x7fe30c0cfba0 2023-11-28T21:38:30.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe30affd640 1 -- 172.21.15.12:0/2696439535 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2f4003530 con 0x7fe30c0cfba0 2023-11-28T21:38:30.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe30b7fe640 1 -- 172.21.15.12:0/2696439535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe30c153e40 msgr2=0x7fe30c0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:30.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe30b7fe640 1 --2- 172.21.15.12:0/2696439535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe30c153e40 0x7fe30c0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:30.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe30b7fe640 1 -- 172.21.15.12:0/2696439535 >> v1:172.21.15.12:6789/0 conn(0x7fe30c0cfba0 legacy=0x7fe30c0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:30.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe30b7fe640 1 -- 172.21.15.12:0/2696439535 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c15a140 con 0x7fe30c15dc80 2023-11-28T21:38:30.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.502+0000 7fe30b7fe640 1 --2- 172.21.15.12:0/2696439535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 0x7fe30c0cf660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe3000097c0 tx=0x7fe30001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba4cbe451fc4f57 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:30.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe30affd640 1 -- 172.21.15.12:0/2696439535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000ab680 con 0x7fe30c15dc80 2023-11-28T21:38:30.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe30affd640 1 -- 172.21.15.12:0/2696439535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3000ab7e0 con 0x7fe30c15dc80 2023-11-28T21:38:30.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe30affd640 1 -- 172.21.15.12:0/2696439535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000abb70 con 0x7fe30c15dc80 2023-11-28T21:38:30.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 -- 172.21.15.12:0/2696439535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 msgr2=0x7fe30c0cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:30.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 --2- 172.21.15.12:0/2696439535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 0x7fe30c0cf660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe3000097c0 tx=0x7fe30001c480 comp rx=0 tx=0).stop 2023-11-28T21:38:30.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 -- 172.21.15.12:0/2696439535 shutdown_connections 2023-11-28T21:38:30.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 --2- 172.21.15.12:0/2696439535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 0x7fe30c0cf660 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:30.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 --2- 172.21.15.12:0/2696439535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe30c153e40 0x7fe30c0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:30.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 -- 172.21.15.12:0/2696439535 >> 172.21.15.12:0/2696439535 conn(0x7fe30c06b7b0 msgr2=0x7fe30c066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:30.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 -- 172.21.15.12:0/2696439535 shutdown_connections 2023-11-28T21:38:30.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 -- 172.21.15.12:0/2696439535 wait complete. 2023-11-28T21:38:30.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 Processor -- start 2023-11-28T21:38:30.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 -- start start 2023-11-28T21:38:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe30c0cfba0 0x7fe30c15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe30c153e40 0x7fe30c0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 0x7fe30c1f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c1f2570 con 0x7fe30c15dc80 2023-11-28T21:38:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0c6b10 con 0x7fe30c153e40 2023-11-28T21:38:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.506+0000 7fe3129b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0c7040 con 0x7fe30c0cfba0 2023-11-28T21:38:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe310f55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 0x7fe30c1f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:30.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe310f55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 0x7fe30c1f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46864/0 (socket says 172.21.15.12:46864) 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe310f55640 1 -- 172.21.15.12:0/971012338 learned_addr learned my addr 172.21.15.12:0/971012338 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe30bfff640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe30c0cfba0 0x7fe30c15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe30b7fe640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe30c153e40 0x7fe30c0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe310f55640 1 -- 172.21.15.12:0/971012338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe30c0cfba0 msgr2=0x7fe30c15cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe310f55640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe30c0cfba0 0x7fe30c15cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe310f55640 1 -- 172.21.15.12:0/971012338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe30c153e40 msgr2=0x7fe30c0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe310f55640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe30c153e40 0x7fe30c0c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe310f55640 1 -- 172.21.15.12:0/971012338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe300009420 con 0x7fe30c15dc80 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe310f55640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 0x7fe30c1f2990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe30400cb00 tx=0x7fe30400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe308ff9640 1 -- 172.21.15.12:0/971012338 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30400aca0 con 0x7fe30c15dc80 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe308ff9640 1 -- 172.21.15.12:0/971012338 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30400ae00 con 0x7fe30c15dc80 2023-11-28T21:38:30.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe308ff9640 1 -- 172.21.15.12:0/971012338 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304012620 con 0x7fe30c15dc80 2023-11-28T21:38:30.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe3129b8640 1 -- 172.21.15.12:0/971012338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c1f1ed0 con 0x7fe30c15dc80 2023-11-28T21:38:30.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe3129b8640 1 -- 172.21.15.12:0/971012338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c0c6d60 con 0x7fe30c15dc80 2023-11-28T21:38:30.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe3129b8640 1 -- 172.21.15.12:0/971012338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c1f26b0 con 0x7fe30c15dc80 2023-11-28T21:38:30.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.510+0000 7fe308ff9640 1 -- 172.21.15.12:0/971012338 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe304007070 con 0x7fe30c15dc80 2023-11-28T21:38:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.514+0000 7fe308ff9640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe2f8066470 0x7fe2f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.514+0000 7fe308ff9640 1 -- 172.21.15.12:0/971012338 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fe304084170 con 0x7fe30c15dc80 2023-11-28T21:38:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.514+0000 7fe30bfff640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe2f8066470 0x7fe2f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:30.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.514+0000 7fe30bfff640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe2f8066470 0x7fe2f8068930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009b60 tx=0x7fe2fc009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:30.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.518+0000 7fe308ff9640 1 -- 172.21.15.12:0/971012338 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe30c1f26b0 con 0x7fe30c15dc80 2023-11-28T21:38:30.638 INFO:tasks.workunit.client.0.smithi012.stdout:./209/.snap/snap-subdir-test 2023-11-28T21:38:30.690 INFO:tasks.workunit.client.0.smithi012.stdout:./253/.snap/snap-subdir-test 2023-11-28T21:38:30.698 INFO:tasks.workunit.client.0.smithi012.stdout:./249/.snap/snap-subdir-test 2023-11-28T21:38:30.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.706+0000 7fe3129b8640 1 -- 172.21.15.12:0/971012338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe30c1f26b0 con 0x7fe30c15dc80 2023-11-28T21:38:30.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe308ff9640 1 -- 172.21.15.12:0/971012338 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fe30c1f26b0 con 0x7fe30c15dc80 2023-11-28T21:38:30.714 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:30.714 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:30.714 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 20 2023-11-28T21:38:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 -- 172.21.15.12:0/971012338 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe2f8066470 msgr2=0x7fe2f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe2f8066470 0x7fe2f8068930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009b60 tx=0x7fe2fc009190 comp rx=0 tx=0).stop 2023-11-28T21:38:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 -- 172.21.15.12:0/971012338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 msgr2=0x7fe30c1f2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 0x7fe30c1f2990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe30400cb00 tx=0x7fe30400a060 comp rx=0 tx=0).stop 2023-11-28T21:38:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 -- 172.21.15.12:0/971012338 shutdown_connections 2023-11-28T21:38:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe2f8066470 0x7fe2f8068930 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe30c15dc80 0x7fe30c1f2990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:30.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe30c153e40 0x7fe30c0c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 --2- 172.21.15.12:0/971012338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe30c0cfba0 0x7fe30c15cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 -- 172.21.15.12:0/971012338 >> 172.21.15.12:0/971012338 conn(0x7fe30c06b7b0 msgr2=0x7fe30c0ca330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 -- 172.21.15.12:0/971012338 shutdown_connections 2023-11-28T21:38:30.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:30.710+0000 7fe3129b8640 1 -- 172.21.15.12:0/971012338 wait complete. 2023-11-28T21:38:30.735 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2023-11-28T21:38:30.866 INFO:tasks.workunit.client.0.smithi012.stdout:./133/.snap/snap-subdir-test 2023-11-28T21:38:30.875 INFO:tasks.workunit.client.0.smithi012.stdout:./291/.snap/snap-subdir-test 2023-11-28T21:38:30.884 INFO:tasks.workunit.client.0.smithi012.stdout:./226/.snap/snap-subdir-test 2023-11-28T21:38:30.894 INFO:tasks.workunit.client.0.smithi012.stdout:./173/.snap/snap-subdir-test 2023-11-28T21:38:30.903 INFO:tasks.workunit.client.0.smithi012.stdout:./174/.snap/snap-subdir-test 2023-11-28T21:38:30.911 INFO:tasks.workunit.client.0.smithi012.stdout:./299/.snap/snap-subdir-test 2023-11-28T21:38:30.920 INFO:tasks.workunit.client.0.smithi012.stdout:./262/.snap/snap-subdir-test 2023-11-28T21:38:30.929 INFO:tasks.workunit.client.0.smithi012.stdout:./231/.snap/snap-subdir-test 2023-11-28T21:38:30.939 INFO:tasks.workunit.client.0.smithi012.stdout:./214/.snap/snap-subdir-test 2023-11-28T21:38:30.947 INFO:tasks.workunit.client.0.smithi012.stdout:./281/.snap/snap-subdir-test 2023-11-28T21:38:30.957 INFO:tasks.workunit.client.0.smithi012.stdout:./289/.snap/snap-subdir-test 2023-11-28T21:38:30.966 INFO:tasks.workunit.client.0.smithi012.stdout:./283/.snap/snap-subdir-test 2023-11-28T21:38:30.975 INFO:tasks.workunit.client.0.smithi012.stdout:./232/.snap/snap-subdir-test 2023-11-28T21:38:30.984 INFO:tasks.workunit.client.0.smithi012.stdout:./298/.snap/snap-subdir-test 2023-11-28T21:38:30.994 INFO:tasks.workunit.client.0.smithi012.stdout:./233/.snap/snap-subdir-test 2023-11-28T21:38:31.002 INFO:tasks.workunit.client.0.smithi012.stdout:./155/.snap/snap-subdir-test 2023-11-28T21:38:31.011 INFO:tasks.workunit.client.0.smithi012.stdout:./114/.snap/snap-subdir-test 2023-11-28T21:38:31.020 INFO:tasks.workunit.client.0.smithi012.stdout:./159/.snap/snap-subdir-test 2023-11-28T21:38:31.029 INFO:tasks.workunit.client.0.smithi012.stdout:./187/.snap/snap-subdir-test 2023-11-28T21:38:31.038 INFO:tasks.workunit.client.0.smithi012.stdout:./185/.snap/snap-subdir-test 2023-11-28T21:38:31.046 INFO:tasks.workunit.client.0.smithi012.stdout:./293/.snap/snap-subdir-test 2023-11-28T21:38:31.056 INFO:tasks.workunit.client.0.smithi012.stdout:./228/.snap/snap-subdir-test 2023-11-28T21:38:31.066 INFO:tasks.workunit.client.0.smithi012.stdout:./120/.snap/snap-subdir-test 2023-11-28T21:38:31.075 INFO:tasks.workunit.client.0.smithi012.stdout:./110/.snap/snap-subdir-test 2023-11-28T21:38:31.084 INFO:tasks.workunit.client.0.smithi012.stdout:./267/.snap/snap-subdir-test 2023-11-28T21:38:31.094 INFO:tasks.workunit.client.0.smithi012.stdout:./202/.snap/snap-subdir-test 2023-11-28T21:38:31.103 INFO:tasks.workunit.client.0.smithi012.stdout:./137/.snap/snap-subdir-test 2023-11-28T21:38:31.112 INFO:tasks.workunit.client.0.smithi012.stdout:./246/.snap/snap-subdir-test 2023-11-28T21:38:31.121 INFO:tasks.workunit.client.0.smithi012.stdout:./219/.snap/snap-subdir-test 2023-11-28T21:38:31.131 INFO:tasks.workunit.client.0.smithi012.stdout:./198/.snap/snap-subdir-test 2023-11-28T21:38:31.140 INFO:tasks.workunit.client.0.smithi012.stdout:./136/.snap/snap-subdir-test 2023-11-28T21:38:31.149 INFO:tasks.workunit.client.0.smithi012.stdout:./270/.snap/snap-subdir-test 2023-11-28T21:38:31.158 INFO:tasks.workunit.client.0.smithi012.stdout:./125/.snap/snap-subdir-test 2023-11-28T21:38:31.167 INFO:tasks.workunit.client.0.smithi012.stdout:./194/.snap/snap-subdir-test 2023-11-28T21:38:31.176 INFO:tasks.workunit.client.0.smithi012.stdout:./119/.snap/snap-subdir-test 2023-11-28T21:38:31.185 INFO:tasks.workunit.client.0.smithi012.stdout:./297/.snap/snap-subdir-test 2023-11-28T21:38:31.194 INFO:tasks.workunit.client.0.smithi012.stdout:./230/.snap/snap-subdir-test 2023-11-28T21:38:31.203 INFO:tasks.workunit.client.0.smithi012.stdout:./260/.snap/snap-subdir-test 2023-11-28T21:38:31.212 INFO:tasks.workunit.client.0.smithi012.stdout:./157/.snap/snap-subdir-test 2023-11-28T21:38:31.221 INFO:tasks.workunit.client.0.smithi012.stdout:./172/.snap/snap-subdir-test 2023-11-28T21:38:31.231 INFO:tasks.workunit.client.0.smithi012.stdout:./277/.snap/snap-subdir-test 2023-11-28T21:38:31.241 INFO:tasks.workunit.client.0.smithi012.stdout:./186/.snap/snap-subdir-test 2023-11-28T21:38:31.250 INFO:tasks.workunit.client.0.smithi012.stdout:./237/.snap/snap-subdir-test 2023-11-28T21:38:31.259 INFO:tasks.workunit.client.0.smithi012.stdout:./190/.snap/snap-subdir-test 2023-11-28T21:38:31.269 INFO:tasks.workunit.client.0.smithi012.stdout:./213/.snap/snap-subdir-test 2023-11-28T21:38:31.277 INFO:tasks.workunit.client.0.smithi012.stdout:./104/.snap/snap-subdir-test 2023-11-28T21:38:31.286 INFO:tasks.workunit.client.0.smithi012.stdout:./164/.snap/snap-subdir-test 2023-11-28T21:38:31.294 INFO:tasks.workunit.client.0.smithi012.stdout:./112/.snap/snap-subdir-test 2023-11-28T21:38:31.304 INFO:tasks.workunit.client.0.smithi012.stdout:./295/.snap/snap-subdir-test 2023-11-28T21:38:31.313 INFO:tasks.workunit.client.0.smithi012.stdout:./272/.snap/snap-subdir-test 2023-11-28T21:38:31.322 INFO:tasks.workunit.client.0.smithi012.stdout:./100/.snap/snap-subdir-test 2023-11-28T21:38:32.651 INFO:tasks.workunit.client.0.smithi012.stdout:./134/.snap/snap-subdir-test 2023-11-28T21:38:32.661 INFO:tasks.workunit.client.0.smithi012.stdout:./200/.snap/snap-subdir-test 2023-11-28T21:38:32.672 INFO:tasks.workunit.client.0.smithi012.stdout:./115/.snap/snap-subdir-test 2023-11-28T21:38:32.682 INFO:tasks.workunit.client.0.smithi012.stdout:./284/.snap/snap-subdir-test 2023-11-28T21:38:32.691 INFO:tasks.workunit.client.0.smithi012.stdout:./280/.snap/snap-subdir-test 2023-11-28T21:38:32.700 INFO:tasks.workunit.client.0.smithi012.stdout:./116/.snap/snap-subdir-test 2023-11-28T21:38:32.710 INFO:tasks.workunit.client.0.smithi012.stdout:./245/.snap/snap-subdir-test 2023-11-28T21:38:32.720 INFO:tasks.workunit.client.0.smithi012.stdout:./195/.snap/snap-subdir-test 2023-11-28T21:38:32.730 INFO:tasks.workunit.client.0.smithi012.stdout:./138/.snap/snap-subdir-test 2023-11-28T21:38:32.736 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:38:32.740 INFO:tasks.workunit.client.0.smithi012.stdout:./135/.snap/snap-subdir-test 2023-11-28T21:38:32.748 INFO:tasks.workunit.client.0.smithi012.stdout:./117/.snap/snap-subdir-test 2023-11-28T21:38:32.756 INFO:tasks.workunit.client.0.smithi012.stdout:./241/.snap/snap-subdir-test 2023-11-28T21:38:32.764 INFO:tasks.workunit.client.0.smithi012.stdout:./143/.snap/snap-subdir-test 2023-11-28T21:38:32.772 INFO:tasks.workunit.client.0.smithi012.stdout:./180/.snap/snap-subdir-test 2023-11-28T21:38:32.783 INFO:tasks.workunit.client.0.smithi012.stdout:./126/.snap/snap-subdir-test 2023-11-28T21:38:32.794 INFO:tasks.workunit.client.0.smithi012.stdout:./161/.snap/snap-subdir-test 2023-11-28T21:38:32.805 INFO:tasks.workunit.client.0.smithi012.stdout:./204/.snap/snap-subdir-test 2023-11-28T21:38:32.811 INFO:tasks.workunit.client.0.smithi012.stdout:./176/.snap/snap-subdir-test 2023-11-28T21:38:32.820 INFO:tasks.workunit.client.0.smithi012.stdout:./255/.snap/snap-subdir-test 2023-11-28T21:38:32.829 INFO:tasks.workunit.client.0.smithi012.stdout:./215/.snap/snap-subdir-test 2023-11-28T21:38:32.837 INFO:tasks.workunit.client.0.smithi012.stdout:./111/.snap/snap-subdir-test 2023-11-28T21:38:32.845 INFO:tasks.workunit.client.0.smithi012.stdout:./238/.snap/snap-subdir-test 2023-11-28T21:38:32.854 INFO:tasks.workunit.client.0.smithi012.stdout:./191/.snap/snap-subdir-test 2023-11-28T21:38:32.862 INFO:tasks.workunit.client.0.smithi012.stdout:./184/.snap/snap-subdir-test 2023-11-28T21:38:32.870 INFO:tasks.workunit.client.0.smithi012.stdout:./192/.snap/snap-subdir-test 2023-11-28T21:38:32.878 INFO:tasks.workunit.client.0.smithi012.stdout:./103/.snap/snap-subdir-test 2023-11-28T21:38:32.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b971f8640 1 Processor -- start 2023-11-28T21:38:32.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b971f8640 1 -- start start 2023-11-28T21:38:32.891 INFO:tasks.workunit.client.0.smithi012.stdout:./128/.snap/snap-subdir-test 2023-11-28T21:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b971f8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b900cbba0 0x7f7b900cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b971f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b900cc600 0x7f7b9015a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:32.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b971f8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9015d370 con 0x7f7b90154120 2023-11-28T21:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b971f8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b900b3040 con 0x7f7b900cbba0 2023-11-28T21:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b971f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b90154500 con 0x7f7b900cc600 2023-11-28T21:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b8ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b900cbba0 0x7f7b900cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:32.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b8ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7b900cbba0 0x7f7b900cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46874/0 (socket says 172.21.15.12:46874) 2023-11-28T21:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b8ffff640 1 -- 172.21.15.12:0/3759793740 learned_addr learned my addr 172.21.15.12:0/3759793740 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b95795640 1 --2- 172.21.15.12:0/3759793740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b900cc600 0x7f7b9015a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b8ffff640 1 -- 172.21.15.12:0/3759793740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b900cc600 msgr2=0x7f7b9015a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b8ffff640 1 --2- 172.21.15.12:0/3759793740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b900cc600 0x7f7b9015a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b8ffff640 1 -- 172.21.15.12:0/3759793740 >> v1:172.21.15.144:6789/0 conn(0x7f7b90154120 legacy=0x7f7b900ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:32.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.886+0000 7f7b8ffff640 1 -- 172.21.15.12:0/3759793740 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90159d80 con 0x7f7b900cbba0 2023-11-28T21:38:32.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b8ffff640 1 --2- 172.21.15.12:0/3759793740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 0x7f7b900cc0c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009870 tx=0x7f7b7c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d9626d3a363b3b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:32.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b8f7fe640 1 -- 172.21.15.12:0/3759793740 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0ab620 con 0x7f7b900cbba0 2023-11-28T21:38:32.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b8f7fe640 1 -- 172.21.15.12:0/3759793740 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c0ab780 con 0x7f7b900cbba0 2023-11-28T21:38:32.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b8f7fe640 1 -- 172.21.15.12:0/3759793740 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0abb10 con 0x7f7b900cbba0 2023-11-28T21:38:32.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b971f8640 1 -- 172.21.15.12:0/3759793740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 msgr2=0x7f7b900cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:32.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b971f8640 1 --2- 172.21.15.12:0/3759793740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 0x7f7b900cc0c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009870 tx=0x7f7b7c01c510 comp rx=0 tx=0).stop 2023-11-28T21:38:32.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b971f8640 1 -- 172.21.15.12:0/3759793740 shutdown_connections 2023-11-28T21:38:32.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b971f8640 1 --2- 172.21.15.12:0/3759793740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b900cc600 0x7f7b9015a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:32.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b971f8640 1 --2- 172.21.15.12:0/3759793740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 0x7f7b900cc0c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:32.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b971f8640 1 -- 172.21.15.12:0/3759793740 >> 172.21.15.12:0/3759793740 conn(0x7f7b90071ce0 msgr2=0x7f7b900720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:32.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b971f8640 1 -- 172.21.15.12:0/3759793740 shutdown_connections 2023-11-28T21:38:32.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.890+0000 7f7b971f8640 1 -- 172.21.15.12:0/3759793740 wait complete. 2023-11-28T21:38:32.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 Processor -- start 2023-11-28T21:38:32.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 -- start start 2023-11-28T21:38:32.898 INFO:tasks.workunit.client.0.smithi012.stdout:./229/.snap/snap-subdir-test 2023-11-28T21:38:32.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 0x7f7b900b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:32.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b900cc600 0x7f7b901f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:32.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b90154120 0x7f7b9015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:32.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b901f63b0 con 0x7f7b900cbba0 2023-11-28T21:38:32.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b90150200 con 0x7f7b900cc600 2023-11-28T21:38:32.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9014f820 con 0x7f7b90154120 2023-11-28T21:38:32.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b94f94640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 0x7f7b900b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b94f94640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 0x7f7b900b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46888/0 (socket says 172.21.15.12:46888) 2023-11-28T21:38:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b94f94640 1 -- 172.21.15.12:0/595170159 learned_addr learned my addr 172.21.15.12:0/595170159 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:38:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b8ffff640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b900cc600 0x7f7b901f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b95795640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b90154120 0x7f7b9015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b94f94640 1 -- 172.21.15.12:0/595170159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b90154120 msgr2=0x7f7b9015c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b94f94640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b90154120 0x7f7b9015c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b94f94640 1 -- 172.21.15.12:0/595170159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b900cc600 msgr2=0x7f7b901f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b94f94640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b900cc600 0x7f7b901f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b94f94640 1 -- 172.21.15.12:0/595170159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c0094d0 con 0x7f7b900cbba0 2023-11-28T21:38:32.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b95795640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b90154120 0x7f7b9015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:38:32.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b94f94640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 0x7f7b900b5330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c530 tx=0x7f7b8400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:32.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b8d7fa640 1 -- 172.21.15.12:0/595170159 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84005220 con 0x7f7b900cbba0 2023-11-28T21:38:32.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b8d7fa640 1 -- 172.21.15.12:0/595170159 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b84005380 con 0x7f7b900cbba0 2023-11-28T21:38:32.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b8d7fa640 1 -- 172.21.15.12:0/595170159 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84007270 con 0x7f7b900cbba0 2023-11-28T21:38:32.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 -- 172.21.15.12:0/595170159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9014ff60 con 0x7f7b900cbba0 2023-11-28T21:38:32.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 -- 172.21.15.12:0/595170159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b900b5870 con 0x7f7b900cbba0 2023-11-28T21:38:32.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.894+0000 7f7b971f8640 1 -- 172.21.15.12:0/595170159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b90074f60 con 0x7f7b900cbba0 2023-11-28T21:38:32.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.898+0000 7f7b8d7fa640 1 -- 172.21.15.12:0/595170159 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7b840073d0 con 0x7f7b900cbba0 2023-11-28T21:38:32.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.898+0000 7f7b8d7fa640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b74066470 0x7f7b74068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:38:32.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.898+0000 7f7b8d7fa640 1 -- 172.21.15.12:0/595170159 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f7b84083b00 con 0x7f7b900cbba0 2023-11-28T21:38:32.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.898+0000 7f7b8ffff640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b74066470 0x7f7b74068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:38:32.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.898+0000 7f7b8ffff640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b74066470 0x7f7b74068930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0af000 tx=0x7f7b7c0b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:38:32.907 INFO:tasks.workunit.client.0.smithi012.stdout:./144/.snap/snap-subdir-test 2023-11-28T21:38:32.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:32.902+0000 7f7b8d7fa640 1 -- 172.21.15.12:0/595170159 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7b84085e60 con 0x7f7b900cbba0 2023-11-28T21:38:32.915 INFO:tasks.workunit.client.0.smithi012.stdout:./146/.snap/snap-subdir-test 2023-11-28T21:38:32.922 INFO:tasks.workunit.client.0.smithi012.stdout:./227/.snap/snap-subdir-test 2023-11-28T21:38:32.930 INFO:tasks.workunit.client.0.smithi012.stdout:./278/.snap/snap-subdir-test 2023-11-28T21:38:33.026 INFO:tasks.workunit.client.0.smithi012.stdout:./141/.snap/snap-subdir-test 2023-11-28T21:38:33.034 INFO:tasks.workunit.client.0.smithi012.stdout:./234/.snap/snap-subdir-test 2023-11-28T21:38:33.042 INFO:tasks.workunit.client.0.smithi012.stdout:./225/.snap/snap-subdir-test 2023-11-28T21:38:33.049 INFO:tasks.workunit.client.0.smithi012.stdout:./182/.snap/snap-subdir-test 2023-11-28T21:38:33.057 INFO:tasks.workunit.client.0.smithi012.stdout:./107/.snap/snap-subdir-test 2023-11-28T21:38:33.067 INFO:tasks.workunit.client.0.smithi012.stdout:./178/.snap/snap-subdir-test 2023-11-28T21:38:33.075 INFO:tasks.workunit.client.0.smithi012.stdout:./266/.snap/snap-subdir-test 2023-11-28T21:38:33.083 INFO:tasks.workunit.client.0.smithi012.stdout:./240/.snap/snap-subdir-test 2023-11-28T21:38:33.091 INFO:tasks.workunit.client.0.smithi012.stdout:./156/.snap/snap-subdir-test 2023-11-28T21:38:33.099 INFO:tasks.workunit.client.0.smithi012.stdout:./131/.snap/snap-subdir-test 2023-11-28T21:38:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.098+0000 7f7b971f8640 1 -- 172.21.15.12:0/595170159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b900c4650 con 0x7f7b900cbba0 2023-11-28T21:38:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.098+0000 7f7b8d7fa640 1 -- 172.21.15.12:0/595170159 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f7b84050b80 con 0x7f7b900cbba0 2023-11-28T21:38:33.104 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:38:33.104 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5307,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:38:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2023-11-28T21:38:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.098+0000 7f7b971f8640 1 -- 172.21.15.12:0/595170159 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b74066470 msgr2=0x7f7b74068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.098+0000 7f7b971f8640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b74066470 0x7f7b74068930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0af000 tx=0x7f7b7c0b6040 comp rx=0 tx=0).stop 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.098+0000 7f7b971f8640 1 -- 172.21.15.12:0/595170159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 msgr2=0x7f7b900b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.098+0000 7f7b971f8640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 0x7f7b900b5330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400c530 tx=0x7f7b8400c9f0 comp rx=0 tx=0).stop 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.102+0000 7f7b971f8640 1 -- 172.21.15.12:0/595170159 shutdown_connections 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.102+0000 7f7b971f8640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7b74066470 0x7f7b74068930 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.102+0000 7f7b971f8640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b90154120 0x7f7b9015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.102+0000 7f7b971f8640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b900cc600 0x7f7b901f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.102+0000 7f7b971f8640 1 --2- 172.21.15.12:0/595170159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b900cbba0 0x7f7b900b5330 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.102+0000 7f7b971f8640 1 -- 172.21.15.12:0/595170159 >> 172.21.15.12:0/595170159 conn(0x7f7b90071ce0 msgr2=0x7f7b90147500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.102+0000 7f7b971f8640 1 -- 172.21.15.12:0/595170159 shutdown_connections 2023-11-28T21:38:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:38:33.102+0000 7f7b971f8640 1 -- 172.21.15.12:0/595170159 wait complete. 2023-11-28T21:38:33.107 INFO:tasks.workunit.client.0.smithi012.stdout:./123/.snap/snap-subdir-test 2023-11-28T21:38:33.125 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2023-11-28T21:38:33.125 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 69 secs before thrashing 2023-11-28T21:38:33.611 INFO:tasks.workunit.client.0.smithi012.stdout:./160/.snap/snap-subdir-test 2023-11-28T21:38:33.965 INFO:tasks.workunit.client.0.smithi012.stdout:./165/.snap/snap-subdir-test 2023-11-28T21:38:33.975 INFO:tasks.workunit.client.0.smithi012.stdout:./101/.snap/snap-subdir-test 2023-11-28T21:38:33.984 INFO:tasks.workunit.client.0.smithi012.stdout:./148/.snap/snap-subdir-test 2023-11-28T21:38:33.993 INFO:tasks.workunit.client.0.smithi012.stdout:./168/.snap/snap-subdir-test 2023-11-28T21:38:34.003 INFO:tasks.workunit.client.0.smithi012.stdout:./179/.snap/snap-subdir-test 2023-11-28T21:38:34.012 INFO:tasks.workunit.client.0.smithi012.stdout:./257/.snap/snap-subdir-test 2023-11-28T21:38:34.021 INFO:tasks.workunit.client.0.smithi012.stdout:./252/.snap/snap-subdir-test 2023-11-28T21:38:34.029 INFO:tasks.workunit.client.0.smithi012.stdout:./132/.snap/snap-subdir-test 2023-11-28T21:38:34.038 INFO:tasks.workunit.client.0.smithi012.stdout:./236/.snap/snap-subdir-test 2023-11-28T21:38:34.048 INFO:tasks.workunit.client.0.smithi012.stdout:./211/.snap/snap-subdir-test 2023-11-28T21:38:34.057 INFO:tasks.workunit.client.0.smithi012.stdout:./199/.snap/snap-subdir-test 2023-11-28T21:38:34.068 INFO:tasks.workunit.client.0.smithi012.stdout:./205/.snap/snap-subdir-test 2023-11-28T21:38:34.076 INFO:tasks.workunit.client.0.smithi012.stdout:./163/.snap/snap-subdir-test 2023-11-28T21:38:34.085 INFO:tasks.workunit.client.0.smithi012.stdout:./261/.snap/snap-subdir-test 2023-11-28T21:38:34.095 INFO:tasks.workunit.client.0.smithi012.stdout:./201/.snap/snap-subdir-test 2023-11-28T21:38:34.105 INFO:tasks.workunit.client.0.smithi012.stdout:./282/.snap/snap-subdir-test 2023-11-28T21:38:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:./296/.snap/snap-subdir-test 2023-11-28T21:38:35.076 INFO:tasks.workunit.client.0.smithi012.stdout:./217/.snap/snap-subdir-test 2023-11-28T21:38:35.084 INFO:tasks.workunit.client.0.smithi012.stdout:./153/.snap/snap-subdir-test 2023-11-28T21:38:35.095 INFO:tasks.workunit.client.0.smithi012.stdout:./206/.snap/snap-subdir-test 2023-11-28T21:38:35.104 INFO:tasks.workunit.client.0.smithi012.stdout:./122/.snap/snap-subdir-test 2023-11-28T21:38:35.113 INFO:tasks.workunit.client.0.smithi012.stdout:./162/.snap/snap-subdir-test 2023-11-28T21:38:35.124 INFO:tasks.workunit.client.0.smithi012.stdout:./197/.snap/snap-subdir-test 2023-11-28T21:38:35.133 INFO:tasks.workunit.client.0.smithi012.stdout:./216/.snap/snap-subdir-test 2023-11-28T21:38:37.013 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:38:37.017 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:38:37.353 INFO:tasks.workunit.client.0.smithi012.stdout:./254/.snap/snap-subdir-test 2023-11-28T21:38:37.362 INFO:tasks.workunit.client.0.smithi012.stdout:./113/.snap/snap-subdir-test 2023-11-28T21:38:37.371 INFO:tasks.workunit.client.0.smithi012.stdout:./152/.snap/snap-subdir-test 2023-11-28T21:38:37.380 INFO:tasks.workunit.client.0.smithi012.stdout:./121/.snap/snap-subdir-test 2023-11-28T21:38:37.389 INFO:tasks.workunit.client.0.smithi012.stdout:./118/.snap/snap-subdir-test 2023-11-28T21:38:37.401 INFO:tasks.workunit.client.0.smithi012.stdout:./196/.snap/snap-subdir-test 2023-11-28T21:38:37.410 INFO:tasks.workunit.client.0.smithi012.stdout:./124/.snap/snap-subdir-test 2023-11-28T21:38:37.422 INFO:tasks.workunit.client.0.smithi012.stdout:./239/.snap/snap-subdir-test 2023-11-28T21:38:37.434 INFO:tasks.workunit.client.0.smithi012.stdout:./188/.snap/snap-subdir-test 2023-11-28T21:38:37.444 INFO:tasks.workunit.client.0.smithi012.stdout:./287/.snap/snap-subdir-test 2023-11-28T21:38:37.454 INFO:tasks.workunit.client.0.smithi012.stdout:./275/.snap/snap-subdir-test 2023-11-28T21:38:37.716 INFO:tasks.workunit.client.0.smithi012.stdout:./259/.snap/snap-subdir-test 2023-11-28T21:38:37.725 INFO:tasks.workunit.client.0.smithi012.stdout:./224/.snap/snap-subdir-test 2023-11-28T21:38:37.735 INFO:tasks.workunit.client.0.smithi012.stdout:./264/.snap/snap-subdir-test 2023-11-28T21:38:37.745 INFO:tasks.workunit.client.0.smithi012.stdout:./145/.snap/snap-subdir-test 2023-11-28T21:38:37.756 INFO:tasks.workunit.client.0.smithi012.stdout:./279/.snap/snap-subdir-test 2023-11-28T21:38:37.766 INFO:tasks.workunit.client.0.smithi012.stdout:./263/.snap/snap-subdir-test 2023-11-28T21:38:37.775 INFO:tasks.workunit.client.0.smithi012.stdout:./130/.snap/snap-subdir-test 2023-11-28T21:38:37.785 INFO:tasks.workunit.client.0.smithi012.stdout:./193/.snap/snap-subdir-test 2023-11-28T21:38:37.796 INFO:tasks.workunit.client.0.smithi012.stdout:./223/.snap/snap-subdir-test 2023-11-28T21:38:37.806 INFO:tasks.workunit.client.0.smithi012.stdout:./207/.snap/snap-subdir-test 2023-11-28T21:38:37.815 INFO:tasks.workunit.client.0.smithi012.stdout:./276/.snap/snap-subdir-test 2023-11-28T21:38:37.824 INFO:tasks.workunit.client.0.smithi012.stdout:./271/.snap/snap-subdir-test 2023-11-28T21:38:37.833 INFO:tasks.workunit.client.0.smithi012.stdout:./251/.snap/snap-subdir-test 2023-11-28T21:38:37.842 INFO:tasks.workunit.client.0.smithi012.stdout:./175/.snap/snap-subdir-test 2023-11-28T21:38:37.852 INFO:tasks.workunit.client.0.smithi012.stdout:./222/.snap/snap-subdir-test 2023-11-28T21:38:37.861 INFO:tasks.workunit.client.0.smithi012.stdout:./220/.snap/snap-subdir-test 2023-11-28T21:38:37.871 INFO:tasks.workunit.client.0.smithi012.stdout:./203/.snap/snap-subdir-test 2023-11-28T21:38:37.880 INFO:tasks.workunit.client.0.smithi012.stdout:./181/.snap/snap-subdir-test 2023-11-28T21:38:37.890 INFO:tasks.workunit.client.0.smithi012.stdout:./286/.snap/snap-subdir-test 2023-11-28T21:38:37.900 INFO:tasks.workunit.client.0.smithi012.stdout:./127/.snap/snap-subdir-test 2023-11-28T21:38:37.910 INFO:tasks.workunit.client.0.smithi012.stdout:./108/.snap/snap-subdir-test 2023-11-28T21:38:37.919 INFO:tasks.workunit.client.0.smithi012.stdout:./183/.snap/snap-subdir-test 2023-11-28T21:38:37.930 INFO:tasks.workunit.client.0.smithi012.stdout:./294/.snap/snap-subdir-test 2023-11-28T21:38:37.939 INFO:tasks.workunit.client.0.smithi012.stdout:./235/.snap/snap-subdir-test 2023-11-28T21:38:38.203 INFO:tasks.workunit.client.0.smithi012.stdout:./177/.snap/snap-subdir-test 2023-11-28T21:38:38.213 INFO:tasks.workunit.client.0.smithi012.stdout:./169/.snap/snap-subdir-test 2023-11-28T21:38:38.474 INFO:tasks.workunit.client.0.smithi012.stdout:./212/.snap/snap-subdir-test 2023-11-28T21:38:38.737 INFO:tasks.workunit.client.0.smithi012.stdout:./154/.snap/snap-subdir-test 2023-11-28T21:38:38.747 INFO:tasks.workunit.client.0.smithi012.stdout:./171/.snap/snap-subdir-test 2023-11-28T21:38:38.757 INFO:tasks.workunit.client.0.smithi012.stdout:./139/.snap/snap-subdir-test 2023-11-28T21:38:38.766 INFO:tasks.workunit.client.0.smithi012.stdout:./102/.snap/snap-subdir-test 2023-11-28T21:38:38.777 INFO:tasks.workunit.client.0.smithi012.stdout:./129/.snap/snap-subdir-test 2023-11-28T21:38:46.313 INFO:tasks.workunit.client.0.smithi012.stdout:Delete all the files and directories ... 2023-11-28T21:38:47.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/19' 2023-11-28T21:38:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/15' 2023-11-28T21:38:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/31' 2023-11-28T21:38:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/36' 2023-11-28T21:38:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/20' 2023-11-28T21:38:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/12' 2023-11-28T21:38:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/39' 2023-11-28T21:38:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/10' 2023-11-28T21:38:47.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/34' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/38' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/11' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/16' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/32' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/30' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/14' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/18' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/13' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/33' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/17' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/35' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './100/37' 2023-11-28T21:38:47.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './100' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/19' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/15' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/31' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/36' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/20' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/12' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/39' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/10' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/34' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/38' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/11' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/16' 2023-11-28T21:38:47.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/32' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/30' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/14' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/18' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/13' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/33' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/17' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/35' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './101/37' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './101' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/19' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/15' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/31' 2023-11-28T21:38:47.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/36' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/20' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/12' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/39' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/10' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/34' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/38' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/11' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/16' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/32' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/30' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/14' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/18' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/13' 2023-11-28T21:38:47.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/33' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/17' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/35' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './102/37' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './102' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/19' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/15' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/31' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/36' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/20' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/12' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/39' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/10' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/34' 2023-11-28T21:38:47.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/38' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/11' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/16' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/32' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/30' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/14' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/18' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/13' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/33' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/17' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/35' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './103/37' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './103' 2023-11-28T21:38:47.614 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/19' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/15' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/31' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/36' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/20' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/12' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/39' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/10' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/34' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/38' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/11' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/16' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/32' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/30' 2023-11-28T21:38:47.615 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/14' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/18' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/13' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/33' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/17' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/35' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './104/37' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './104' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/19' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/15' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/31' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/36' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/20' 2023-11-28T21:38:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/12' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/39' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/10' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/34' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/38' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/11' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/16' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/32' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/30' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/14' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/18' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/13' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/33' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/17' 2023-11-28T21:38:47.617 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/35' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './105/37' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './105' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/19' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/15' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/31' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/36' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/20' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/12' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/39' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/10' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/34' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/38' 2023-11-28T21:38:47.618 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/11' 2023-11-28T21:38:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/16' 2023-11-28T21:38:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/32' 2023-11-28T21:38:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/30' 2023-11-28T21:38:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/14' 2023-11-28T21:38:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/18' 2023-11-28T21:38:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/13' 2023-11-28T21:38:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/33' 2023-11-28T21:38:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/17' 2023-11-28T21:38:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/35' 2023-11-28T21:38:47.619 INFO:tasks.workunit.client.0.smithi012.stdout:removed './106/37' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './106' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/19' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/15' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/31' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/36' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/20' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/12' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/39' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/10' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/34' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/38' 2023-11-28T21:38:47.620 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/11' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/16' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/32' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/30' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/14' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/18' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/13' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/33' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/17' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/35' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './107/37' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './107' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/19' 2023-11-28T21:38:47.621 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/15' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/31' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/36' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/20' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/12' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/39' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/10' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/34' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/38' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/11' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/16' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/32' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/30' 2023-11-28T21:38:47.622 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/14' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/18' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/13' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/33' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/17' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/35' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './108/37' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './108' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/19' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/15' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/31' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/36' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/20' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/12' 2023-11-28T21:38:47.623 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/39' 2023-11-28T21:38:47.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/10' 2023-11-28T21:38:47.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/34' 2023-11-28T21:38:47.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/38' 2023-11-28T21:38:47.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/11' 2023-11-28T21:38:47.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/16' 2023-11-28T21:38:47.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/32' 2023-11-28T21:38:47.624 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/30' 2023-11-28T21:38:49.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/14' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/18' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/13' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/33' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/17' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/35' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './109/37' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './109' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/19' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/15' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/31' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/36' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/20' 2023-11-28T21:38:49.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/12' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/39' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/10' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/34' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/38' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/11' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/16' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/32' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/30' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/14' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/18' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/13' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/33' 2023-11-28T21:38:49.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/17' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/35' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './110/37' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './110' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/19' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/15' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/31' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/36' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/20' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/12' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/39' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/10' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/34' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/38' 2023-11-28T21:38:49.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/11' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/16' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/32' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/30' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/14' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/18' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/13' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/33' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/17' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/35' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './111/37' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './111' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/19' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/15' 2023-11-28T21:38:49.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/31' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/36' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/20' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/12' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/39' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/10' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/34' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/38' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/11' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/16' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/32' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/30' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/14' 2023-11-28T21:38:49.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/18' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/13' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/33' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/17' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/35' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './112/37' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './112' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/19' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/15' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/31' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/36' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/20' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/12' 2023-11-28T21:38:49.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/39' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/10' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/34' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/38' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/11' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/16' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/32' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/30' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/14' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/18' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/13' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/33' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/17' 2023-11-28T21:38:49.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/35' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './113/37' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './113' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/19' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/15' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/31' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/36' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/20' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/12' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/39' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/10' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/34' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/38' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/11' 2023-11-28T21:38:49.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/16' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/32' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/30' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/14' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/18' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/13' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/33' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/17' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/35' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './114/37' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './114' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/19' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/15' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/31' 2023-11-28T21:38:49.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/36' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/20' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/12' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/39' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/10' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/34' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/38' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/11' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/16' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/32' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/30' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/14' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/18' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/13' 2023-11-28T21:38:49.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/33' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/17' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/35' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './115/37' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './115' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/19' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/15' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/31' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/36' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/20' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/12' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/39' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/10' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/34' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/38' 2023-11-28T21:38:49.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/11' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/16' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/32' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/30' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/14' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/18' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/13' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/33' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/17' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/35' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './116/37' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './116' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/19' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/15' 2023-11-28T21:38:49.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/31' 2023-11-28T21:38:49.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/36' 2023-11-28T21:38:49.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/20' 2023-11-28T21:38:49.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/12' 2023-11-28T21:38:49.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/39' 2023-11-28T21:38:49.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/10' 2023-11-28T21:38:49.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/34' 2023-11-28T21:38:49.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/38' 2023-11-28T21:38:49.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/11' 2023-11-28T21:38:49.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/16' 2023-11-28T21:38:49.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/32' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/30' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/14' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/18' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/13' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/33' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/17' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/35' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './117/37' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './117' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/19' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/15' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/31' 2023-11-28T21:38:49.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/36' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/20' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/12' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/39' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/10' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/34' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/38' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/11' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/16' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/32' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/30' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/14' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/18' 2023-11-28T21:38:49.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/13' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/33' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/17' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/35' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './118/37' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './118' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/19' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/15' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/31' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/36' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/20' 2023-11-28T21:38:49.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/12' 2023-11-28T21:38:51.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/39' 2023-11-28T21:38:51.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/10' 2023-11-28T21:38:51.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/34' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/38' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/11' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/16' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/32' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/30' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/14' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/18' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/13' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/33' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/17' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/35' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './119/37' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './119' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/19' 2023-11-28T21:38:51.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/15' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/31' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/36' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/20' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/12' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/39' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/10' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/34' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/38' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/11' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/16' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/32' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/30' 2023-11-28T21:38:51.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/14' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/18' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/13' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/33' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/17' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/35' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './120/37' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './120' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/19' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/15' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/31' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/36' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/20' 2023-11-28T21:38:51.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/12' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/39' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/10' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/34' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/38' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/11' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/16' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/32' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/30' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/14' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/18' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/13' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/33' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/17' 2023-11-28T21:38:52.000 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/35' 2023-11-28T21:38:52.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './121/37' 2023-11-28T21:38:52.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './121' 2023-11-28T21:38:52.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/19' 2023-11-28T21:38:52.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/15' 2023-11-28T21:38:52.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/31' 2023-11-28T21:38:52.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/36' 2023-11-28T21:38:52.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/20' 2023-11-28T21:38:52.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/12' 2023-11-28T21:38:52.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/39' 2023-11-28T21:38:52.001 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/10' 2023-11-28T21:38:52.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/34' 2023-11-28T21:38:52.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/38' 2023-11-28T21:38:52.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/11' 2023-11-28T21:38:52.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/16' 2023-11-28T21:38:52.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/32' 2023-11-28T21:38:52.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/30' 2023-11-28T21:38:52.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/14' 2023-11-28T21:38:52.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/18' 2023-11-28T21:38:52.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/13' 2023-11-28T21:38:52.002 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/33' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/17' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/35' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './122/37' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './122' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/19' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/15' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/31' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/36' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/20' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/12' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/39' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/10' 2023-11-28T21:38:52.003 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/34' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/38' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/11' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/16' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/32' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/30' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/14' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/18' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/13' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/33' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/17' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/35' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './123/37' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './123' 2023-11-28T21:38:52.004 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/19' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/15' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/31' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/36' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/20' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/12' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/39' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/10' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/34' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/38' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/11' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/16' 2023-11-28T21:38:52.005 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/32' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/30' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/14' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/18' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/13' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/33' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/17' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/35' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './124/37' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './124' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/19' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/15' 2023-11-28T21:38:52.006 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/31' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/36' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/20' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/12' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/39' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/10' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/34' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/38' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/11' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/16' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/32' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/30' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/14' 2023-11-28T21:38:52.007 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/18' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/13' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/33' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/17' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/35' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './125/37' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './125' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/19' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/15' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/31' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/36' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/20' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/12' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/39' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/10' 2023-11-28T21:38:52.008 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/34' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/38' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/11' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/16' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/32' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/30' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/14' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/18' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/13' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/33' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/17' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/35' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './126/37' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './126' 2023-11-28T21:38:52.009 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/19' 2023-11-28T21:38:52.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/15' 2023-11-28T21:38:52.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/31' 2023-11-28T21:38:52.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/36' 2023-11-28T21:38:52.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/20' 2023-11-28T21:38:52.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/12' 2023-11-28T21:38:52.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/39' 2023-11-28T21:38:52.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/10' 2023-11-28T21:38:52.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/34' 2023-11-28T21:38:52.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/38' 2023-11-28T21:38:52.010 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/11' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/16' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/32' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/30' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/14' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/18' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/13' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/33' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/17' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/35' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './127/37' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './127' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/19' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/15' 2023-11-28T21:38:52.011 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/31' 2023-11-28T21:38:52.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/36' 2023-11-28T21:38:52.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/20' 2023-11-28T21:38:52.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/12' 2023-11-28T21:38:52.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/39' 2023-11-28T21:38:52.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/10' 2023-11-28T21:38:52.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/34' 2023-11-28T21:38:52.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/38' 2023-11-28T21:38:52.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/11' 2023-11-28T21:38:52.012 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/16' 2023-11-28T21:38:52.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/32' 2023-11-28T21:38:52.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/30' 2023-11-28T21:38:52.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/14' 2023-11-28T21:38:52.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/18' 2023-11-28T21:38:52.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/13' 2023-11-28T21:38:52.013 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/33' 2023-11-28T21:38:52.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/17' 2023-11-28T21:38:52.014 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/35' 2023-11-28T21:38:53.036 INFO:tasks.workunit.client.0.smithi012.stdout:removed './128/37' 2023-11-28T21:38:53.037 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './128' 2023-11-28T21:38:53.037 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/19' 2023-11-28T21:38:53.037 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/15' 2023-11-28T21:38:53.037 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/31' 2023-11-28T21:38:53.037 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/36' 2023-11-28T21:38:53.037 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/20' 2023-11-28T21:38:53.038 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/12' 2023-11-28T21:38:53.038 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/39' 2023-11-28T21:38:53.038 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/10' 2023-11-28T21:38:53.038 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/34' 2023-11-28T21:38:53.038 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/38' 2023-11-28T21:38:53.038 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/11' 2023-11-28T21:38:53.038 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/16' 2023-11-28T21:38:53.039 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/32' 2023-11-28T21:38:53.039 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/30' 2023-11-28T21:38:53.039 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/14' 2023-11-28T21:38:53.039 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/18' 2023-11-28T21:38:53.039 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/13' 2023-11-28T21:38:53.039 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/33' 2023-11-28T21:38:53.040 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/17' 2023-11-28T21:38:53.040 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/35' 2023-11-28T21:38:53.040 INFO:tasks.workunit.client.0.smithi012.stdout:removed './129/37' 2023-11-28T21:38:53.040 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './129' 2023-11-28T21:38:53.040 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/19' 2023-11-28T21:38:53.040 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/15' 2023-11-28T21:38:53.041 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/31' 2023-11-28T21:38:53.041 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/36' 2023-11-28T21:38:53.041 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/20' 2023-11-28T21:38:53.041 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/12' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/39' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/10' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/34' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/38' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/11' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/16' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/32' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/30' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/14' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/18' 2023-11-28T21:38:53.042 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/13' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/33' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/17' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/35' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './130/37' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './130' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/19' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/15' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/31' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/36' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/20' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/12' 2023-11-28T21:38:53.043 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/39' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/10' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/34' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/38' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/11' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/16' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/32' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/30' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/14' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/18' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/13' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/33' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/17' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/35' 2023-11-28T21:38:53.044 INFO:tasks.workunit.client.0.smithi012.stdout:removed './131/37' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './131' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/19' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/15' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/31' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/36' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/20' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/12' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/39' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/10' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/34' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/38' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/11' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/16' 2023-11-28T21:38:53.045 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/32' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/30' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/14' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/18' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/13' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/33' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/17' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/35' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './132/37' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './132' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/19' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/15' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/31' 2023-11-28T21:38:53.046 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/36' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/20' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/12' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/39' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/10' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/34' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/38' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/11' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/16' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/32' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/30' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/14' 2023-11-28T21:38:53.047 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/18' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/13' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/33' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/17' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/35' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './133/37' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './133' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/19' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/15' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/31' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/36' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/20' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/12' 2023-11-28T21:38:53.048 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/39' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/10' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/34' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/38' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/11' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/16' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/32' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/30' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/14' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/18' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/13' 2023-11-28T21:38:53.049 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/33' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/17' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/35' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './134/37' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './134' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/19' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/15' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/31' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/36' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/20' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/12' 2023-11-28T21:38:53.050 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/39' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/10' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/34' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/38' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/11' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/16' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/32' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/30' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/14' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/18' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/13' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/33' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/17' 2023-11-28T21:38:53.051 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/35' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './135/37' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './135' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/19' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/15' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/31' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/36' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/20' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/12' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/39' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/10' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/34' 2023-11-28T21:38:53.052 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/38' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/11' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/16' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/32' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/30' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/14' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/18' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/13' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/33' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/17' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/35' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './136/37' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './136' 2023-11-28T21:38:53.053 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/19' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/15' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/31' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/36' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/20' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/12' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/39' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/10' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/34' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/38' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/11' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/16' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/32' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/30' 2023-11-28T21:38:53.054 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/14' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/18' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/13' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/33' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/17' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/35' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './137/37' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './137' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/19' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/15' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/31' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/36' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/20' 2023-11-28T21:38:53.055 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/12' 2023-11-28T21:38:53.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/39' 2023-11-28T21:38:53.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/10' 2023-11-28T21:38:53.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/34' 2023-11-28T21:38:53.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/38' 2023-11-28T21:38:53.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/11' 2023-11-28T21:38:53.056 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/16' 2023-11-28T21:38:55.066 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/32' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/30' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/14' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/18' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/13' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/33' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/17' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/35' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './138/37' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './138' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/19' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/15' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/31' 2023-11-28T21:38:55.067 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/36' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/20' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/12' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/39' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/10' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/34' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/38' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/11' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/16' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/32' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/30' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/14' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/18' 2023-11-28T21:38:55.068 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/13' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/33' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/17' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/35' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './139/37' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './139' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/19' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/15' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/31' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/36' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/20' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/12' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/39' 2023-11-28T21:38:55.069 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/10' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/34' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/38' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/11' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/16' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/32' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/30' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/14' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/18' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/13' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/33' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/17' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/35' 2023-11-28T21:38:55.070 INFO:tasks.workunit.client.0.smithi012.stdout:removed './140/37' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './140' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/19' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/15' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/31' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/36' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/20' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/12' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/39' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/10' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/34' 2023-11-28T21:38:55.071 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/38' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/11' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/16' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/32' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/30' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/14' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/18' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/13' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/33' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/17' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/35' 2023-11-28T21:38:55.072 INFO:tasks.workunit.client.0.smithi012.stdout:removed './141/37' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './141' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/19' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/15' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/31' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/36' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/20' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/12' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/39' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/10' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/34' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/38' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/11' 2023-11-28T21:38:55.073 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/16' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/32' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/30' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/14' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/18' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/13' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/33' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/17' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/35' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './142/37' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './142' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/19' 2023-11-28T21:38:55.074 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/15' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/31' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/36' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/20' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/12' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/39' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/10' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/34' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/38' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/11' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/16' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/32' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/30' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/14' 2023-11-28T21:38:55.075 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/18' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/13' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/33' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/17' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/35' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './143/37' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './143' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/19' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/15' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/31' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/36' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/20' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/12' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/39' 2023-11-28T21:38:55.076 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/10' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/34' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/38' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/11' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/16' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/32' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/30' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/14' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/18' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/13' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/33' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/17' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/35' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed './144/37' 2023-11-28T21:38:55.077 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './144' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/19' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/15' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/31' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/36' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/20' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/12' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/39' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/10' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/34' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/38' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/11' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/16' 2023-11-28T21:38:55.078 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/32' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/30' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/14' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/18' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/13' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/33' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/17' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/35' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './145/37' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './145' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/19' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/15' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/31' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/36' 2023-11-28T21:38:55.079 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/20' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/12' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/39' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/10' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/34' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/38' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/11' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/16' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/32' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/30' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/14' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/18' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/13' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/33' 2023-11-28T21:38:55.080 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/17' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/35' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './146/37' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './146' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/19' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/15' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/31' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/36' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/20' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/12' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/39' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/10' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/34' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/38' 2023-11-28T21:38:55.081 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/11' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/16' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/32' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/30' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/14' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/18' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/13' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/33' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/17' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/35' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './147/37' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './147' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/19' 2023-11-28T21:38:55.082 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/15' 2023-11-28T21:38:55.083 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/31' 2023-11-28T21:38:55.083 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/36' 2023-11-28T21:38:56.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/20' 2023-11-28T21:38:56.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/12' 2023-11-28T21:38:56.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/39' 2023-11-28T21:38:56.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/10' 2023-11-28T21:38:56.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/34' 2023-11-28T21:38:56.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/38' 2023-11-28T21:38:56.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/11' 2023-11-28T21:38:56.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/16' 2023-11-28T21:38:56.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/32' 2023-11-28T21:38:56.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/30' 2023-11-28T21:38:56.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/14' 2023-11-28T21:38:56.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/18' 2023-11-28T21:38:56.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/13' 2023-11-28T21:38:56.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/33' 2023-11-28T21:38:56.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/17' 2023-11-28T21:38:56.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/35' 2023-11-28T21:38:56.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './148/37' 2023-11-28T21:38:56.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './148' 2023-11-28T21:38:56.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/19' 2023-11-28T21:38:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/15' 2023-11-28T21:38:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/31' 2023-11-28T21:38:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/36' 2023-11-28T21:38:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/20' 2023-11-28T21:38:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/12' 2023-11-28T21:38:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/39' 2023-11-28T21:38:56.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/10' 2023-11-28T21:38:56.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/34' 2023-11-28T21:38:56.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/38' 2023-11-28T21:38:56.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/11' 2023-11-28T21:38:56.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/16' 2023-11-28T21:38:56.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/32' 2023-11-28T21:38:56.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/30' 2023-11-28T21:38:56.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/14' 2023-11-28T21:38:56.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/18' 2023-11-28T21:38:56.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/13' 2023-11-28T21:38:56.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/33' 2023-11-28T21:38:56.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/17' 2023-11-28T21:38:56.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/35' 2023-11-28T21:38:56.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './149/37' 2023-11-28T21:38:56.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './149' 2023-11-28T21:38:56.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/19' 2023-11-28T21:38:56.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/15' 2023-11-28T21:38:56.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/31' 2023-11-28T21:38:56.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/36' 2023-11-28T21:38:56.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/20' 2023-11-28T21:38:56.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/12' 2023-11-28T21:38:56.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/39' 2023-11-28T21:38:56.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/10' 2023-11-28T21:38:56.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/34' 2023-11-28T21:38:56.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/38' 2023-11-28T21:38:56.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/11' 2023-11-28T21:38:56.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/16' 2023-11-28T21:38:56.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/32' 2023-11-28T21:38:56.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/30' 2023-11-28T21:38:56.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/14' 2023-11-28T21:38:56.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/18' 2023-11-28T21:38:56.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/13' 2023-11-28T21:38:56.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/33' 2023-11-28T21:38:56.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/17' 2023-11-28T21:38:56.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/35' 2023-11-28T21:38:56.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './150/37' 2023-11-28T21:38:56.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './150' 2023-11-28T21:38:56.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/19' 2023-11-28T21:38:56.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/15' 2023-11-28T21:38:56.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/31' 2023-11-28T21:38:56.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/36' 2023-11-28T21:38:56.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/20' 2023-11-28T21:38:56.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/12' 2023-11-28T21:38:56.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/39' 2023-11-28T21:38:56.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/10' 2023-11-28T21:38:56.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/34' 2023-11-28T21:38:56.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/38' 2023-11-28T21:38:56.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/11' 2023-11-28T21:38:56.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/16' 2023-11-28T21:38:56.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/32' 2023-11-28T21:38:56.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/30' 2023-11-28T21:38:56.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/14' 2023-11-28T21:38:56.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/18' 2023-11-28T21:38:56.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/13' 2023-11-28T21:38:56.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/33' 2023-11-28T21:38:56.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/17' 2023-11-28T21:38:56.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/35' 2023-11-28T21:38:56.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './151/37' 2023-11-28T21:38:56.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './151' 2023-11-28T21:38:56.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/19' 2023-11-28T21:38:56.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/15' 2023-11-28T21:38:56.951 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/31' 2023-11-28T21:38:56.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/36' 2023-11-28T21:38:56.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/20' 2023-11-28T21:38:56.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/12' 2023-11-28T21:38:56.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/39' 2023-11-28T21:38:56.952 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/10' 2023-11-28T21:38:56.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/34' 2023-11-28T21:38:56.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/38' 2023-11-28T21:38:56.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/11' 2023-11-28T21:38:56.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/16' 2023-11-28T21:38:56.953 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/32' 2023-11-28T21:38:56.954 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/30' 2023-11-28T21:38:56.954 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/14' 2023-11-28T21:38:56.954 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/18' 2023-11-28T21:38:56.954 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/13' 2023-11-28T21:38:56.954 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/33' 2023-11-28T21:38:56.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/17' 2023-11-28T21:38:56.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/35' 2023-11-28T21:38:56.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './152/37' 2023-11-28T21:38:56.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './152' 2023-11-28T21:38:56.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/19' 2023-11-28T21:38:56.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/15' 2023-11-28T21:38:56.955 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/31' 2023-11-28T21:38:56.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/36' 2023-11-28T21:38:56.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/20' 2023-11-28T21:38:56.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/12' 2023-11-28T21:38:56.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/39' 2023-11-28T21:38:56.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/10' 2023-11-28T21:38:56.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/34' 2023-11-28T21:38:56.956 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/38' 2023-11-28T21:38:56.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/11' 2023-11-28T21:38:56.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/16' 2023-11-28T21:38:56.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/32' 2023-11-28T21:38:56.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/30' 2023-11-28T21:38:56.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/14' 2023-11-28T21:38:56.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/18' 2023-11-28T21:38:56.957 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/13' 2023-11-28T21:38:56.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/33' 2023-11-28T21:38:56.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/17' 2023-11-28T21:38:56.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/35' 2023-11-28T21:38:56.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed './153/37' 2023-11-28T21:38:56.958 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './153' 2023-11-28T21:38:56.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/19' 2023-11-28T21:38:56.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/15' 2023-11-28T21:38:56.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/31' 2023-11-28T21:38:56.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/36' 2023-11-28T21:38:56.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/20' 2023-11-28T21:38:56.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/12' 2023-11-28T21:38:56.959 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/39' 2023-11-28T21:38:56.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/10' 2023-11-28T21:38:56.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/34' 2023-11-28T21:38:56.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/38' 2023-11-28T21:38:56.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/11' 2023-11-28T21:38:56.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/16' 2023-11-28T21:38:56.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/32' 2023-11-28T21:38:56.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/30' 2023-11-28T21:38:56.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/14' 2023-11-28T21:38:56.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/18' 2023-11-28T21:38:56.960 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/13' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/33' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/17' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/35' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './154/37' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './154' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/19' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/15' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/31' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/36' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/20' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/12' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/39' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/10' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/34' 2023-11-28T21:38:56.961 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/38' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/11' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/16' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/32' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/30' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/14' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/18' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/13' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/33' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/17' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/35' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './155/37' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './155' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/19' 2023-11-28T21:38:56.962 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/15' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/31' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/36' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/20' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/12' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/39' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/10' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/34' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/38' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/11' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/16' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/32' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/30' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/14' 2023-11-28T21:38:56.963 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/18' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/13' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/33' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/17' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/35' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './156/37' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './156' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/19' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/15' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/31' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/36' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/20' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/12' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/39' 2023-11-28T21:38:56.964 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/10' 2023-11-28T21:38:56.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/34' 2023-11-28T21:38:56.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/38' 2023-11-28T21:38:56.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/11' 2023-11-28T21:38:56.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/16' 2023-11-28T21:38:56.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/32' 2023-11-28T21:38:56.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/30' 2023-11-28T21:38:56.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/14' 2023-11-28T21:38:56.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/18' 2023-11-28T21:38:56.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/13' 2023-11-28T21:38:56.965 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/33' 2023-11-28T21:38:59.280 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/17' 2023-11-28T21:38:59.280 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/35' 2023-11-28T21:38:59.280 INFO:tasks.workunit.client.0.smithi012.stdout:removed './157/37' 2023-11-28T21:38:59.280 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './157' 2023-11-28T21:38:59.280 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/19' 2023-11-28T21:38:59.280 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/15' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/31' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/36' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/20' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/12' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/39' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/10' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/34' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/38' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/11' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/16' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/32' 2023-11-28T21:38:59.281 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/30' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/14' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/18' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/13' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/33' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/17' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/35' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './158/37' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './158' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/19' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/15' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/31' 2023-11-28T21:38:59.282 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/36' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/20' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/12' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/39' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/10' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/34' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/38' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/11' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/16' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/32' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/30' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/14' 2023-11-28T21:38:59.283 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/18' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/13' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/33' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/17' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/35' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed './159/37' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './159' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/19' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/15' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/31' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/36' 2023-11-28T21:38:59.284 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/20' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/12' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/39' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/10' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/34' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/38' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/11' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/16' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/32' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/30' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/14' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/18' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/13' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/33' 2023-11-28T21:38:59.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/17' 2023-11-28T21:38:59.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/35' 2023-11-28T21:38:59.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './160/37' 2023-11-28T21:38:59.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './160' 2023-11-28T21:38:59.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/19' 2023-11-28T21:38:59.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/15' 2023-11-28T21:38:59.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/31' 2023-11-28T21:38:59.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/36' 2023-11-28T21:38:59.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/20' 2023-11-28T21:38:59.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/12' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/39' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/10' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/34' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/38' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/11' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/16' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/32' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/30' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/14' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/18' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/13' 2023-11-28T21:38:59.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/33' 2023-11-28T21:38:59.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/17' 2023-11-28T21:38:59.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/35' 2023-11-28T21:38:59.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './161/37' 2023-11-28T21:38:59.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './161' 2023-11-28T21:38:59.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/19' 2023-11-28T21:38:59.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/15' 2023-11-28T21:38:59.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/31' 2023-11-28T21:38:59.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/36' 2023-11-28T21:38:59.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/20' 2023-11-28T21:38:59.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/12' 2023-11-28T21:38:59.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/39' 2023-11-28T21:38:59.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/10' 2023-11-28T21:38:59.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/34' 2023-11-28T21:38:59.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/38' 2023-11-28T21:38:59.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/11' 2023-11-28T21:38:59.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/16' 2023-11-28T21:38:59.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/32' 2023-11-28T21:38:59.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/30' 2023-11-28T21:38:59.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/14' 2023-11-28T21:38:59.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/18' 2023-11-28T21:38:59.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/13' 2023-11-28T21:38:59.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/33' 2023-11-28T21:38:59.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/17' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/35' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './162/37' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './162' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/19' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/15' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/31' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/36' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/20' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/12' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/39' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/10' 2023-11-28T21:38:59.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/34' 2023-11-28T21:38:59.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/38' 2023-11-28T21:38:59.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/11' 2023-11-28T21:38:59.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/16' 2023-11-28T21:38:59.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/32' 2023-11-28T21:38:59.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/30' 2023-11-28T21:38:59.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/14' 2023-11-28T21:38:59.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/18' 2023-11-28T21:38:59.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/13' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/33' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/17' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/35' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './163/37' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './163' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/19' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/15' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/31' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/36' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/20' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/12' 2023-11-28T21:38:59.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/39' 2023-11-28T21:38:59.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/10' 2023-11-28T21:38:59.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/34' 2023-11-28T21:38:59.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/38' 2023-11-28T21:38:59.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/11' 2023-11-28T21:38:59.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/16' 2023-11-28T21:38:59.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/32' 2023-11-28T21:38:59.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/30' 2023-11-28T21:38:59.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/14' 2023-11-28T21:38:59.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/18' 2023-11-28T21:38:59.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/13' 2023-11-28T21:38:59.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/33' 2023-11-28T21:38:59.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/17' 2023-11-28T21:38:59.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/35' 2023-11-28T21:38:59.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './164/37' 2023-11-28T21:38:59.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './164' 2023-11-28T21:38:59.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/19' 2023-11-28T21:38:59.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/15' 2023-11-28T21:38:59.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/31' 2023-11-28T21:38:59.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/36' 2023-11-28T21:38:59.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/20' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/12' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/39' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/10' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/34' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/38' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/11' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/16' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/32' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/30' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/14' 2023-11-28T21:38:59.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/18' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/13' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/33' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/17' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/35' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './165/37' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './165' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/19' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/15' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/31' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/36' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/20' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/12' 2023-11-28T21:38:59.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/39' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/10' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/34' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/38' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/11' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/16' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/32' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/30' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/14' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/18' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/13' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/33' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/17' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/35' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './166/37' 2023-11-28T21:38:59.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './166' 2023-11-28T21:38:59.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/19' 2023-11-28T21:38:59.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/15' 2023-11-28T21:38:59.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/31' 2023-11-28T21:38:59.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/36' 2023-11-28T21:38:59.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/20' 2023-11-28T21:38:59.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/12' 2023-11-28T21:38:59.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/39' 2023-11-28T21:38:59.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/10' 2023-11-28T21:38:59.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/34' 2023-11-28T21:38:59.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/38' 2023-11-28T21:39:01.934 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/11' 2023-11-28T21:39:01.934 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/16' 2023-11-28T21:39:01.934 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/32' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/30' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/14' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/18' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/13' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/33' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/17' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/35' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './167/37' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './167' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/19' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/15' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/31' 2023-11-28T21:39:01.935 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/36' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/20' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/12' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/39' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/10' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/34' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/38' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/11' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/16' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/32' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/30' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/14' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/18' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/13' 2023-11-28T21:39:01.936 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/33' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/17' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/35' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './168/37' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './168' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/19' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/15' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/31' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/36' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/20' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/12' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/39' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/10' 2023-11-28T21:39:01.937 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/34' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/38' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/11' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/16' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/32' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/30' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/14' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/18' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/13' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/33' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/17' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/35' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './169/37' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './169' 2023-11-28T21:39:01.938 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/19' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/15' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/31' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/36' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/20' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/12' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/39' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/10' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/34' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/38' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/11' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/16' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/32' 2023-11-28T21:39:01.939 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/30' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/14' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/18' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/13' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/33' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/17' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/35' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './170/37' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './170' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/19' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/15' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/31' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/36' 2023-11-28T21:39:01.940 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/20' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/12' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/39' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/10' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/34' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/38' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/11' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/16' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/32' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/30' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/14' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/18' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/13' 2023-11-28T21:39:01.941 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/33' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/17' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/35' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './171/37' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './171' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/19' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/15' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/31' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/36' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/20' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/12' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/39' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/10' 2023-11-28T21:39:01.942 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/34' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/38' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/11' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/16' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/32' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/30' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/14' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/18' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/13' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/33' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/17' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/35' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed './172/37' 2023-11-28T21:39:01.943 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './172' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/19' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/15' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/31' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/36' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/20' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/12' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/39' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/10' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/34' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/38' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/11' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/16' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/32' 2023-11-28T21:39:01.944 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/30' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/14' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/18' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/13' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/33' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/17' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/35' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './173/37' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './173' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/19' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/15' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/31' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/36' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/20' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/12' 2023-11-28T21:39:01.945 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/39' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/10' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/34' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/38' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/11' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/16' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/32' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/30' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/14' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/18' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/13' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/33' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/17' 2023-11-28T21:39:01.946 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/35' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './174/37' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './174' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/19' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/15' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/31' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/36' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/20' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/12' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/39' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/10' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/34' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/38' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/11' 2023-11-28T21:39:01.947 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/16' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/32' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/30' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/14' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/18' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/13' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/33' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/17' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/35' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './175/37' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './175' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/19' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/15' 2023-11-28T21:39:01.948 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/31' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/36' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/20' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/12' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/39' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/10' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/34' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/38' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/11' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/16' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/32' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/30' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/14' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/18' 2023-11-28T21:39:01.949 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/13' 2023-11-28T21:39:01.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/33' 2023-11-28T21:39:01.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/17' 2023-11-28T21:39:01.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/35' 2023-11-28T21:39:01.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './176/37' 2023-11-28T21:39:01.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './176' 2023-11-28T21:39:01.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/19' 2023-11-28T21:39:01.950 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/15' 2023-11-28T21:39:04.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/31' 2023-11-28T21:39:04.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/36' 2023-11-28T21:39:04.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/20' 2023-11-28T21:39:04.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/12' 2023-11-28T21:39:04.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/39' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/10' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/34' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/38' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/11' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/16' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/32' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/30' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/14' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/18' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/13' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/33' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/17' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/35' 2023-11-28T21:39:04.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './177/37' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './177' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/19' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/15' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/31' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/36' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/20' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/12' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/39' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/10' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/34' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/38' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/11' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/16' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/32' 2023-11-28T21:39:04.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/30' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/14' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/18' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/13' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/33' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/17' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/35' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './178/37' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './178' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/19' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/15' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/31' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/36' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/20' 2023-11-28T21:39:04.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/12' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/39' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/10' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/34' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/38' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/11' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/16' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/32' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/30' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/14' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/18' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/13' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/33' 2023-11-28T21:39:04.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/17' 2023-11-28T21:39:04.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/35' 2023-11-28T21:39:04.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './179/37' 2023-11-28T21:39:04.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './179' 2023-11-28T21:39:04.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/19' 2023-11-28T21:39:04.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/15' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/31' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/36' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/20' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/12' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/39' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/10' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/34' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/38' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/11' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/16' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/32' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/30' 2023-11-28T21:39:04.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/14' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/18' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/13' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/33' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/17' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/35' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './180/37' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './180' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/19' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/15' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/31' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/36' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/20' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/12' 2023-11-28T21:39:04.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/39' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/10' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/34' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/38' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/11' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/16' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/32' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/30' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/14' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/18' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/13' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/33' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/17' 2023-11-28T21:39:04.266 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/35' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './181/37' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './181' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/19' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/15' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/31' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/36' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/20' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/12' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/39' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/10' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/34' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/38' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/11' 2023-11-28T21:39:04.267 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/16' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/32' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/30' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/14' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/18' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/13' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/33' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/17' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/35' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './182/37' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './182' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/19' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/15' 2023-11-28T21:39:04.268 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/31' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/36' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/20' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/12' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/39' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/10' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/34' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/38' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/11' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/16' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/32' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/30' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/14' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/18' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/13' 2023-11-28T21:39:04.269 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/33' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/17' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/35' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './183/37' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './183' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/19' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/15' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/31' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/36' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/20' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/12' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/39' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/10' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/34' 2023-11-28T21:39:04.270 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/38' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/11' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/16' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/32' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/30' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/14' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/18' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/13' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/33' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/17' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/35' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './184/37' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './184' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/19' 2023-11-28T21:39:04.271 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/15' 2023-11-28T21:39:04.272 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/31' 2023-11-28T21:39:04.272 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/36' 2023-11-28T21:39:04.272 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/20' 2023-11-28T21:39:04.272 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/12' 2023-11-28T21:39:04.272 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/39' 2023-11-28T21:39:04.272 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/10' 2023-11-28T21:39:04.272 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/34' 2023-11-28T21:39:04.272 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/38' 2023-11-28T21:39:04.272 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/11' 2023-11-28T21:39:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/16' 2023-11-28T21:39:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/32' 2023-11-28T21:39:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/30' 2023-11-28T21:39:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/14' 2023-11-28T21:39:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/18' 2023-11-28T21:39:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/13' 2023-11-28T21:39:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/33' 2023-11-28T21:39:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/17' 2023-11-28T21:39:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/35' 2023-11-28T21:39:04.273 INFO:tasks.workunit.client.0.smithi012.stdout:removed './185/37' 2023-11-28T21:39:04.274 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './185' 2023-11-28T21:39:04.274 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/19' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/15' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/31' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/36' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/20' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/12' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/39' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/10' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/34' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/38' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/11' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/16' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/32' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/30' 2023-11-28T21:39:04.275 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/14' 2023-11-28T21:39:04.276 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/18' 2023-11-28T21:39:07.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:39:07.073 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:39:07.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/13' 2023-11-28T21:39:07.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/33' 2023-11-28T21:39:07.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/17' 2023-11-28T21:39:07.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/35' 2023-11-28T21:39:07.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './186/37' 2023-11-28T21:39:07.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './186' 2023-11-28T21:39:07.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/19' 2023-11-28T21:39:07.755 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/15' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/31' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/36' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/20' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/12' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/39' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/10' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/34' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/38' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/11' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/16' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/32' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/30' 2023-11-28T21:39:07.756 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/14' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/18' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/13' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/33' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/17' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/35' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './187/37' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './187' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/19' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/15' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/31' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/36' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/20' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/12' 2023-11-28T21:39:07.757 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/39' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/10' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/34' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/38' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/11' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/16' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/32' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/30' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/14' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/18' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/13' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/33' 2023-11-28T21:39:07.758 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/17' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/35' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './188/37' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './188' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/19' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/15' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/31' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/36' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/20' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/12' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/39' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/10' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/34' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/38' 2023-11-28T21:39:07.759 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/11' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/16' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/32' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/30' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/14' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/18' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/13' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/33' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/17' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/35' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './189/37' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './189' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/19' 2023-11-28T21:39:07.760 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/15' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/31' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/36' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/20' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/12' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/39' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/10' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/34' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/38' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/11' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/16' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/32' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/30' 2023-11-28T21:39:07.761 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/14' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/18' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/13' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/33' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/17' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/35' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './190/37' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './190' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/19' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/15' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/31' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/36' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/20' 2023-11-28T21:39:07.762 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/12' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/39' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/10' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/34' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/38' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/11' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/16' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/32' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/30' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/14' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/18' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/13' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/33' 2023-11-28T21:39:07.763 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/17' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/35' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './191/37' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './191' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/19' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/15' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/31' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/36' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/20' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/12' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/39' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/10' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/34' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/38' 2023-11-28T21:39:07.764 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/11' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/16' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/32' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/30' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/14' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/18' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/13' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/33' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/17' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/35' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './192/37' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './192' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/19' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/15' 2023-11-28T21:39:07.765 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/31' 2023-11-28T21:39:07.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/36' 2023-11-28T21:39:07.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/20' 2023-11-28T21:39:07.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/12' 2023-11-28T21:39:07.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/39' 2023-11-28T21:39:07.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/10' 2023-11-28T21:39:07.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/34' 2023-11-28T21:39:07.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/38' 2023-11-28T21:39:07.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/11' 2023-11-28T21:39:07.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/16' 2023-11-28T21:39:07.766 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/32' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/30' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/14' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/18' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/13' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/33' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/17' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/35' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './193/37' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './193' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/19' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/15' 2023-11-28T21:39:07.767 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/31' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/36' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/20' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/12' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/39' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/10' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/34' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/38' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/11' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/16' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/32' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/30' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/14' 2023-11-28T21:39:07.768 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/18' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/13' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/33' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/17' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/35' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './194/37' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './194' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/19' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/15' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/31' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/36' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/20' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/12' 2023-11-28T21:39:07.769 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/39' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/10' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/34' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/38' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/11' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/16' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/32' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/30' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/14' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/18' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/13' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/33' 2023-11-28T21:39:07.770 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/17' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/35' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './195/37' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './195' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/19' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/15' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/31' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/36' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/20' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/12' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/39' 2023-11-28T21:39:07.771 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/10' 2023-11-28T21:39:11.117 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/34' 2023-11-28T21:39:11.118 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/38' 2023-11-28T21:39:11.118 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/11' 2023-11-28T21:39:11.118 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/16' 2023-11-28T21:39:11.118 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/32' 2023-11-28T21:39:11.118 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/30' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/14' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/18' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/13' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/33' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/17' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/35' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed './196/37' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './196' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/19' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/15' 2023-11-28T21:39:11.119 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/31' 2023-11-28T21:39:11.120 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/36' 2023-11-28T21:39:11.120 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/20' 2023-11-28T21:39:11.120 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/12' 2023-11-28T21:39:11.120 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/39' 2023-11-28T21:39:11.120 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/10' 2023-11-28T21:39:11.120 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/34' 2023-11-28T21:39:11.120 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/38' 2023-11-28T21:39:11.120 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/11' 2023-11-28T21:39:11.120 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/16' 2023-11-28T21:39:11.120 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/32' 2023-11-28T21:39:11.121 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/30' 2023-11-28T21:39:11.121 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/14' 2023-11-28T21:39:11.121 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/18' 2023-11-28T21:39:11.121 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/13' 2023-11-28T21:39:11.121 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/33' 2023-11-28T21:39:11.121 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/17' 2023-11-28T21:39:11.121 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/35' 2023-11-28T21:39:11.121 INFO:tasks.workunit.client.0.smithi012.stdout:removed './197/37' 2023-11-28T21:39:11.121 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './197' 2023-11-28T21:39:11.121 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/19' 2023-11-28T21:39:11.122 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/15' 2023-11-28T21:39:11.122 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/31' 2023-11-28T21:39:11.122 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/36' 2023-11-28T21:39:11.122 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/20' 2023-11-28T21:39:11.122 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/12' 2023-11-28T21:39:11.122 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/39' 2023-11-28T21:39:11.122 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/10' 2023-11-28T21:39:11.122 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/34' 2023-11-28T21:39:11.122 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/38' 2023-11-28T21:39:11.123 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/11' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/16' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/32' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/30' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/14' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/18' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/13' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/33' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/17' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/35' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './198/37' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './198' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/19' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/15' 2023-11-28T21:39:11.124 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/31' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/36' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/20' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/12' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/39' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/10' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/34' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/38' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/11' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/16' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/32' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/30' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/14' 2023-11-28T21:39:11.125 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/18' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/13' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/33' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/17' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/35' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './199/37' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './199' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/21' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/27' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/31' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/36' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/20' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/39' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/34' 2023-11-28T21:39:11.126 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/38' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/28' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/24' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/29' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/32' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/22' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/25' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/30' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/33' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/23' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/35' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/26' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './200/37' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './200' 2023-11-28T21:39:11.127 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/21' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/27' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/31' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/36' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/20' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/39' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/34' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/38' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/28' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/24' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/29' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/32' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/22' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/25' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/30' 2023-11-28T21:39:11.128 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/33' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/23' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/35' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/26' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './201/37' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './201' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/21' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/27' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/31' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/36' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/20' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/39' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/34' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/38' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/28' 2023-11-28T21:39:11.129 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/24' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/29' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/32' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/22' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/25' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/30' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/33' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/23' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/35' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/26' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './202/37' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './202' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/21' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/27' 2023-11-28T21:39:11.130 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/31' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/36' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/20' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/39' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/34' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/38' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/28' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/24' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/29' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/32' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/22' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/25' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/30' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/33' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/23' 2023-11-28T21:39:11.131 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/35' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/26' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './203/37' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './203' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/21' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/27' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/31' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/36' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/20' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/39' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/34' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/38' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/28' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/24' 2023-11-28T21:39:11.132 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/29' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/32' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/22' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/25' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/30' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/33' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/23' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/35' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/26' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './204/37' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './204' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/21' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/27' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/31' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/36' 2023-11-28T21:39:11.133 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/20' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/39' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/34' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/38' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/28' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/24' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/29' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/32' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/22' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/25' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/30' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/33' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/23' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/35' 2023-11-28T21:39:11.134 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/26' 2023-11-28T21:39:11.135 INFO:tasks.workunit.client.0.smithi012.stdout:removed './205/37' 2023-11-28T21:39:11.135 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './205' 2023-11-28T21:39:11.135 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/21' 2023-11-28T21:39:11.135 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/27' 2023-11-28T21:39:11.135 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/31' 2023-11-28T21:39:11.135 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/36' 2023-11-28T21:39:11.135 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/20' 2023-11-28T21:39:11.135 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/39' 2023-11-28T21:39:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/34' 2023-11-28T21:39:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/38' 2023-11-28T21:39:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/28' 2023-11-28T21:39:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/24' 2023-11-28T21:39:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/29' 2023-11-28T21:39:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/32' 2023-11-28T21:39:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/22' 2023-11-28T21:39:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/25' 2023-11-28T21:39:12.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/30' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/33' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/23' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/35' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/26' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './206/37' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './206' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/21' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/27' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/31' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/36' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/20' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/39' 2023-11-28T21:39:12.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/34' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/38' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/28' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/24' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/29' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/32' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/22' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/25' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/30' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/33' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/23' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/35' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/26' 2023-11-28T21:39:12.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './207/37' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './207' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/21' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/27' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/31' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/36' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/20' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/39' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/34' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/38' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/28' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/24' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/29' 2023-11-28T21:39:12.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/32' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/22' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/25' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/30' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/33' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/23' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/35' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/26' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './208/37' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './208' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/21' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/27' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/31' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/36' 2023-11-28T21:39:12.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/20' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/39' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/34' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/38' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/28' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/24' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/29' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/32' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/22' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/25' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/30' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/33' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/23' 2023-11-28T21:39:12.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/35' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/26' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './209/37' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './209' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/21' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/27' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/31' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/36' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/20' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/39' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/34' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/38' 2023-11-28T21:39:12.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/28' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/24' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/29' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/32' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/22' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/25' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/30' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/33' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/23' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/35' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/26' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './210/37' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './210' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/21' 2023-11-28T21:39:12.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/27' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/31' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/36' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/20' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/39' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/34' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/38' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/28' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/24' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/29' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/32' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/22' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/25' 2023-11-28T21:39:12.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/30' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/33' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/23' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/35' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/26' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './211/37' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './211' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/21' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/27' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/31' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/36' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/20' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/39' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/34' 2023-11-28T21:39:12.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/38' 2023-11-28T21:39:12.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/28' 2023-11-28T21:39:12.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/24' 2023-11-28T21:39:12.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/29' 2023-11-28T21:39:12.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/32' 2023-11-28T21:39:12.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/22' 2023-11-28T21:39:12.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/25' 2023-11-28T21:39:12.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/30' 2023-11-28T21:39:12.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/33' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/23' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/35' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/26' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './212/37' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './212' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/21' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/27' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/31' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/36' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/20' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/39' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/34' 2023-11-28T21:39:12.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/38' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/28' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/24' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/29' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/32' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/22' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/25' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/30' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/33' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/23' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/35' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/26' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed './213/37' 2023-11-28T21:39:12.169 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './213' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/21' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/27' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/31' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/36' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/20' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/39' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/34' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/38' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/28' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/24' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/29' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/32' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/22' 2023-11-28T21:39:12.170 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/25' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/30' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/33' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/23' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/35' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/26' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './214/37' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './214' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/21' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/27' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/31' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/36' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/20' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/39' 2023-11-28T21:39:12.171 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/34' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/38' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/28' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/24' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/29' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/32' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/22' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/25' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/30' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/33' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/23' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/35' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/26' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed './215/37' 2023-11-28T21:39:12.172 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './215' 2023-11-28T21:39:12.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/21' 2023-11-28T21:39:12.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/27' 2023-11-28T21:39:12.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/31' 2023-11-28T21:39:12.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/36' 2023-11-28T21:39:12.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/20' 2023-11-28T21:39:12.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/39' 2023-11-28T21:39:12.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/34' 2023-11-28T21:39:12.173 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/38' 2023-11-28T21:39:14.868 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/28' 2023-11-28T21:39:14.868 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/24' 2023-11-28T21:39:14.868 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/29' 2023-11-28T21:39:14.868 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/32' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/22' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/25' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/30' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/33' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/23' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/35' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/26' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './216/37' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './216' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/21' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/27' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/31' 2023-11-28T21:39:14.869 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/36' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/20' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/39' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/34' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/38' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/28' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/24' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/29' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/32' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/22' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/25' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/30' 2023-11-28T21:39:14.870 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/33' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/23' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/35' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/26' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './217/37' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './217' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/21' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/27' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/31' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/36' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/20' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/39' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/34' 2023-11-28T21:39:14.871 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/38' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/28' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/24' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/29' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/32' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/22' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/25' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/30' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/33' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/23' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/35' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/26' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed './218/37' 2023-11-28T21:39:14.872 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './218' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/21' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/27' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/31' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/36' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/20' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/39' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/34' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/38' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/28' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/24' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/29' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/32' 2023-11-28T21:39:14.873 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/22' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/25' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/30' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/33' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/23' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/35' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/26' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed './219/37' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './219' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/21' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/27' 2023-11-28T21:39:14.874 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/31' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/36' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/20' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/39' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/34' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/38' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/28' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/24' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/29' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/32' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/22' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/25' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/30' 2023-11-28T21:39:14.875 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/33' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/23' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/35' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/26' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './220/37' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './220' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/21' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/27' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/31' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/36' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/20' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/39' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/34' 2023-11-28T21:39:14.876 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/38' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/28' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/24' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/29' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/32' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/22' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/25' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/30' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/33' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/23' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/35' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/26' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './221/37' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './221' 2023-11-28T21:39:14.877 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/21' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/27' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/31' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/36' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/20' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/39' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/34' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/38' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/28' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/24' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/29' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/32' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/22' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/25' 2023-11-28T21:39:14.878 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/30' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/33' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/23' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/35' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/26' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './222/37' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './222' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/21' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/27' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/31' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/36' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/20' 2023-11-28T21:39:14.879 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/39' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/34' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/38' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/28' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/24' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/29' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/32' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/22' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/25' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/30' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/33' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/23' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/35' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/26' 2023-11-28T21:39:14.880 INFO:tasks.workunit.client.0.smithi012.stdout:removed './223/37' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './223' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/21' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/27' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/31' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/36' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/20' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/39' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/34' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/38' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/28' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/24' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/29' 2023-11-28T21:39:14.881 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/32' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/22' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/25' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/30' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/33' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/23' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/35' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/26' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './224/37' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './224' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/21' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/27' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/31' 2023-11-28T21:39:14.882 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/36' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/20' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/39' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/34' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/38' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/28' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/24' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/29' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/32' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/22' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/25' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/30' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/33' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/23' 2023-11-28T21:39:14.883 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/35' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/26' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './225/37' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './225' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/21' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/27' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/31' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/36' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/20' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/39' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/34' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/38' 2023-11-28T21:39:14.884 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/28' 2023-11-28T21:39:14.885 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/24' 2023-11-28T21:39:18.595 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/29' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/32' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/22' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/25' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/30' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/33' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/23' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/35' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/26' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './226/37' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './226' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/21' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/27' 2023-11-28T21:39:18.596 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/31' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/36' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/20' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/39' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/34' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/38' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/28' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/24' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/29' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/32' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/22' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/25' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/30' 2023-11-28T21:39:18.597 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/33' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/23' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/35' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/26' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './227/37' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './227' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/21' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/27' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/31' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/36' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/20' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/39' 2023-11-28T21:39:18.598 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/34' 2023-11-28T21:39:18.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/38' 2023-11-28T21:39:18.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/28' 2023-11-28T21:39:18.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/24' 2023-11-28T21:39:18.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/29' 2023-11-28T21:39:18.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/32' 2023-11-28T21:39:18.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/22' 2023-11-28T21:39:18.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/25' 2023-11-28T21:39:18.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/30' 2023-11-28T21:39:18.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/33' 2023-11-28T21:39:18.599 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/23' 2023-11-28T21:39:18.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/35' 2023-11-28T21:39:18.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/26' 2023-11-28T21:39:18.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed './228/37' 2023-11-28T21:39:18.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './228' 2023-11-28T21:39:18.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/21' 2023-11-28T21:39:18.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/27' 2023-11-28T21:39:18.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/31' 2023-11-28T21:39:18.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/36' 2023-11-28T21:39:18.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/20' 2023-11-28T21:39:18.600 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/39' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/34' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/38' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/28' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/24' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/29' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/32' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/22' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/25' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/30' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/33' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/23' 2023-11-28T21:39:18.601 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/35' 2023-11-28T21:39:18.602 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/26' 2023-11-28T21:39:18.602 INFO:tasks.workunit.client.0.smithi012.stdout:removed './229/37' 2023-11-28T21:39:18.602 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './229' 2023-11-28T21:39:18.602 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/21' 2023-11-28T21:39:18.602 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/27' 2023-11-28T21:39:18.602 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/31' 2023-11-28T21:39:18.602 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/36' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/20' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/39' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/34' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/38' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/28' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/24' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/29' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/32' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/22' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/25' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/30' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/33' 2023-11-28T21:39:18.603 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/23' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/35' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/26' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './230/37' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './230' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/21' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/27' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/31' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/36' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/20' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/39' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/34' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/38' 2023-11-28T21:39:18.604 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/28' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/24' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/29' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/32' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/22' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/25' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/30' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/33' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/23' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/35' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/26' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './231/37' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './231' 2023-11-28T21:39:18.605 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/21' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/27' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/31' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/36' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/20' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/39' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/34' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/38' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/28' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/24' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/29' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/32' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/22' 2023-11-28T21:39:18.606 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/25' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/30' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/33' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/23' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/35' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/26' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './232/37' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './232' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/21' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/27' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/31' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/36' 2023-11-28T21:39:18.607 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/20' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/39' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/34' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/38' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/28' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/24' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/29' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/32' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/22' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/25' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/30' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/33' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/23' 2023-11-28T21:39:18.608 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/35' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/26' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './233/37' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './233' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/21' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/27' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/31' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/36' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/20' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/39' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/34' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/38' 2023-11-28T21:39:18.609 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/28' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/24' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/29' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/32' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/22' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/25' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/30' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/33' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/23' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/35' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/26' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed './234/37' 2023-11-28T21:39:18.610 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './234' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/21' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/27' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/31' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/36' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/20' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/39' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/34' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/38' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/28' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/24' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/29' 2023-11-28T21:39:18.611 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/32' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/22' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/25' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/30' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/33' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/23' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/35' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/26' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './235/37' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './235' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/21' 2023-11-28T21:39:18.612 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/27' 2023-11-28T21:39:18.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/31' 2023-11-28T21:39:18.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/36' 2023-11-28T21:39:18.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/20' 2023-11-28T21:39:18.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/39' 2023-11-28T21:39:18.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/34' 2023-11-28T21:39:18.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/38' 2023-11-28T21:39:18.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/28' 2023-11-28T21:39:18.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/24' 2023-11-28T21:39:18.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/29' 2023-11-28T21:39:18.613 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/32' 2023-11-28T21:39:21.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/22' 2023-11-28T21:39:21.234 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/25' 2023-11-28T21:39:21.234 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/30' 2023-11-28T21:39:21.234 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/33' 2023-11-28T21:39:21.234 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/23' 2023-11-28T21:39:21.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/35' 2023-11-28T21:39:21.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/26' 2023-11-28T21:39:21.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed './236/37' 2023-11-28T21:39:21.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './236' 2023-11-28T21:39:21.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/21' 2023-11-28T21:39:21.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/27' 2023-11-28T21:39:21.236 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/31' 2023-11-28T21:39:21.236 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/36' 2023-11-28T21:39:21.236 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/20' 2023-11-28T21:39:21.236 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/39' 2023-11-28T21:39:21.236 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/34' 2023-11-28T21:39:21.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/38' 2023-11-28T21:39:21.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/28' 2023-11-28T21:39:21.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/24' 2023-11-28T21:39:21.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/29' 2023-11-28T21:39:21.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/32' 2023-11-28T21:39:21.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/22' 2023-11-28T21:39:21.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/25' 2023-11-28T21:39:21.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/30' 2023-11-28T21:39:21.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/33' 2023-11-28T21:39:21.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/23' 2023-11-28T21:39:21.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/35' 2023-11-28T21:39:21.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/26' 2023-11-28T21:39:21.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './237/37' 2023-11-28T21:39:21.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './237' 2023-11-28T21:39:21.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/21' 2023-11-28T21:39:21.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/27' 2023-11-28T21:39:21.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/31' 2023-11-28T21:39:21.240 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/36' 2023-11-28T21:39:21.240 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/20' 2023-11-28T21:39:21.240 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/39' 2023-11-28T21:39:21.240 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/34' 2023-11-28T21:39:21.240 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/38' 2023-11-28T21:39:21.241 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/28' 2023-11-28T21:39:21.241 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/24' 2023-11-28T21:39:21.241 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/29' 2023-11-28T21:39:21.241 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/32' 2023-11-28T21:39:21.241 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/22' 2023-11-28T21:39:21.242 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/25' 2023-11-28T21:39:21.242 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/30' 2023-11-28T21:39:21.242 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/33' 2023-11-28T21:39:21.242 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/23' 2023-11-28T21:39:21.242 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/35' 2023-11-28T21:39:21.243 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/26' 2023-11-28T21:39:21.243 INFO:tasks.workunit.client.0.smithi012.stdout:removed './238/37' 2023-11-28T21:39:21.243 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './238' 2023-11-28T21:39:21.243 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/21' 2023-11-28T21:39:21.243 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/27' 2023-11-28T21:39:21.243 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/31' 2023-11-28T21:39:21.244 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/36' 2023-11-28T21:39:21.244 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/20' 2023-11-28T21:39:21.244 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/39' 2023-11-28T21:39:21.244 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/34' 2023-11-28T21:39:21.244 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/38' 2023-11-28T21:39:21.244 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/28' 2023-11-28T21:39:21.245 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/24' 2023-11-28T21:39:21.245 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/29' 2023-11-28T21:39:21.245 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/32' 2023-11-28T21:39:21.245 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/22' 2023-11-28T21:39:21.245 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/25' 2023-11-28T21:39:21.246 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/30' 2023-11-28T21:39:21.246 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/33' 2023-11-28T21:39:21.246 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/23' 2023-11-28T21:39:21.246 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/35' 2023-11-28T21:39:21.246 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/26' 2023-11-28T21:39:21.246 INFO:tasks.workunit.client.0.smithi012.stdout:removed './239/37' 2023-11-28T21:39:21.247 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './239' 2023-11-28T21:39:21.247 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/21' 2023-11-28T21:39:21.247 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/27' 2023-11-28T21:39:21.247 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/31' 2023-11-28T21:39:21.247 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/36' 2023-11-28T21:39:21.247 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/20' 2023-11-28T21:39:21.248 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/39' 2023-11-28T21:39:21.248 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/34' 2023-11-28T21:39:21.248 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/38' 2023-11-28T21:39:21.248 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/28' 2023-11-28T21:39:21.248 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/24' 2023-11-28T21:39:21.248 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/29' 2023-11-28T21:39:21.248 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/32' 2023-11-28T21:39:21.248 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/22' 2023-11-28T21:39:21.248 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/25' 2023-11-28T21:39:21.248 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/30' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/33' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/23' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/35' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/26' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './240/37' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './240' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/21' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/27' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/31' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/36' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/20' 2023-11-28T21:39:21.249 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/39' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/34' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/38' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/28' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/24' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/29' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/32' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/22' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/25' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/30' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/33' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/23' 2023-11-28T21:39:21.250 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/35' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/26' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './241/37' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './241' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/21' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/27' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/31' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/36' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/20' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/39' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/34' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/38' 2023-11-28T21:39:21.251 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/28' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/24' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/29' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/32' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/22' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/25' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/30' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/33' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/23' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/35' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/26' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './242/37' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './242' 2023-11-28T21:39:21.252 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/21' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/27' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/31' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/36' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/20' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/39' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/34' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/38' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/28' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/24' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/29' 2023-11-28T21:39:21.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/32' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/22' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/25' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/30' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/33' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/23' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/35' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/26' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './243/37' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './243' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/21' 2023-11-28T21:39:21.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/27' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/31' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/36' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/20' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/39' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/34' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/38' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/28' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/24' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/29' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/32' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/22' 2023-11-28T21:39:21.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/25' 2023-11-28T21:39:21.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/30' 2023-11-28T21:39:21.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/33' 2023-11-28T21:39:21.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/23' 2023-11-28T21:39:21.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/35' 2023-11-28T21:39:21.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/26' 2023-11-28T21:39:21.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './244/37' 2023-11-28T21:39:21.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './244' 2023-11-28T21:39:21.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/21' 2023-11-28T21:39:21.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/27' 2023-11-28T21:39:21.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/31' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/36' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/20' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/39' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/34' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/38' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/28' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/24' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/29' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/32' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/22' 2023-11-28T21:39:21.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/25' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/30' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/33' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/23' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/35' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/26' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './245/37' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './245' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/21' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/27' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/31' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/36' 2023-11-28T21:39:21.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/20' 2023-11-28T21:39:21.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/39' 2023-11-28T21:39:21.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/34' 2023-11-28T21:39:21.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/38' 2023-11-28T21:39:21.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/28' 2023-11-28T21:39:21.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/24' 2023-11-28T21:39:21.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/29' 2023-11-28T21:39:21.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/32' 2023-11-28T21:39:21.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/22' 2023-11-28T21:39:23.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/25' 2023-11-28T21:39:23.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/30' 2023-11-28T21:39:23.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/33' 2023-11-28T21:39:23.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/23' 2023-11-28T21:39:23.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/35' 2023-11-28T21:39:23.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/26' 2023-11-28T21:39:23.550 INFO:tasks.workunit.client.0.smithi012.stdout:removed './246/37' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './246' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/21' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/27' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/31' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/36' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/20' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/39' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/34' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/38' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/28' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/24' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/29' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/32' 2023-11-28T21:39:23.551 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/22' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/25' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/30' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/33' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/23' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/35' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/26' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './247/37' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './247' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/21' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/27' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/31' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/36' 2023-11-28T21:39:23.552 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/20' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/39' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/34' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/38' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/28' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/24' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/29' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/32' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/22' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/25' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/30' 2023-11-28T21:39:23.553 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/33' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/23' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/35' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/26' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './248/37' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './248' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/21' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/27' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/31' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/36' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/20' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/39' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/34' 2023-11-28T21:39:23.554 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/38' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/28' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/24' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/29' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/32' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/22' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/25' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/30' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/33' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/23' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/35' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/26' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './249/37' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './249' 2023-11-28T21:39:23.555 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/21' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/27' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/31' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/36' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/20' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/39' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/34' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/38' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/28' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/24' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/29' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/32' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/22' 2023-11-28T21:39:23.556 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/25' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/30' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/33' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/23' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/35' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/26' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './250/37' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './250' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/21' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/27' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/31' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/36' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/20' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/39' 2023-11-28T21:39:23.557 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/34' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/38' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/28' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/24' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/29' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/32' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/22' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/25' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/30' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/33' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/23' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/35' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/26' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed './251/37' 2023-11-28T21:39:23.558 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './251' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/21' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/27' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/31' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/36' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/20' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/39' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/34' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/38' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/28' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/24' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/29' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/32' 2023-11-28T21:39:23.559 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/22' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/25' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/30' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/33' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/23' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/35' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/26' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './252/37' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './252' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/21' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/27' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/31' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/36' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/20' 2023-11-28T21:39:23.560 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/39' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/34' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/38' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/28' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/24' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/29' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/32' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/22' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/25' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/30' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/33' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/23' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/35' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/26' 2023-11-28T21:39:23.561 INFO:tasks.workunit.client.0.smithi012.stdout:removed './253/37' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './253' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/21' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/27' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/31' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/36' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/20' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/39' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/34' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/38' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/28' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/24' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/29' 2023-11-28T21:39:23.562 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/32' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/22' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/25' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/30' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/33' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/23' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/35' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/26' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './254/37' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './254' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/21' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/27' 2023-11-28T21:39:23.563 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/31' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/36' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/20' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/39' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/34' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/38' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/28' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/24' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/29' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/32' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/22' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/25' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/30' 2023-11-28T21:39:23.564 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/33' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/23' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/35' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/26' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './255/37' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './255' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/21' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/27' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/31' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/36' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/20' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/39' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/34' 2023-11-28T21:39:23.565 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/38' 2023-11-28T21:39:23.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/28' 2023-11-28T21:39:23.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/24' 2023-11-28T21:39:23.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/29' 2023-11-28T21:39:23.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/32' 2023-11-28T21:39:23.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/22' 2023-11-28T21:39:23.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/25' 2023-11-28T21:39:23.566 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/30' 2023-11-28T21:39:25.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/33' 2023-11-28T21:39:25.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/23' 2023-11-28T21:39:25.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/35' 2023-11-28T21:39:25.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/26' 2023-11-28T21:39:25.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed './256/37' 2023-11-28T21:39:25.803 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './256' 2023-11-28T21:39:25.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/21' 2023-11-28T21:39:25.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/27' 2023-11-28T21:39:25.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/31' 2023-11-28T21:39:25.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/36' 2023-11-28T21:39:25.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/20' 2023-11-28T21:39:25.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/39' 2023-11-28T21:39:25.804 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/34' 2023-11-28T21:39:25.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/38' 2023-11-28T21:39:25.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/28' 2023-11-28T21:39:25.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/24' 2023-11-28T21:39:25.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/29' 2023-11-28T21:39:25.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/32' 2023-11-28T21:39:25.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/22' 2023-11-28T21:39:25.805 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/25' 2023-11-28T21:39:25.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/30' 2023-11-28T21:39:25.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/33' 2023-11-28T21:39:25.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/23' 2023-11-28T21:39:25.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/35' 2023-11-28T21:39:25.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/26' 2023-11-28T21:39:25.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './257/37' 2023-11-28T21:39:25.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './257' 2023-11-28T21:39:25.806 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/21' 2023-11-28T21:39:25.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/27' 2023-11-28T21:39:25.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/31' 2023-11-28T21:39:25.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/36' 2023-11-28T21:39:25.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/20' 2023-11-28T21:39:25.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/39' 2023-11-28T21:39:25.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/34' 2023-11-28T21:39:25.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/38' 2023-11-28T21:39:25.807 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/28' 2023-11-28T21:39:25.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/24' 2023-11-28T21:39:25.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/29' 2023-11-28T21:39:25.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/32' 2023-11-28T21:39:25.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/22' 2023-11-28T21:39:25.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/25' 2023-11-28T21:39:25.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/30' 2023-11-28T21:39:25.808 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/33' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/23' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/35' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/26' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './258/37' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './258' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/21' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/27' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/31' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/36' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/20' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/39' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/34' 2023-11-28T21:39:25.809 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/38' 2023-11-28T21:39:25.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/28' 2023-11-28T21:39:25.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/24' 2023-11-28T21:39:25.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/29' 2023-11-28T21:39:25.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/32' 2023-11-28T21:39:25.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/22' 2023-11-28T21:39:25.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/25' 2023-11-28T21:39:25.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/30' 2023-11-28T21:39:25.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/33' 2023-11-28T21:39:25.810 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/23' 2023-11-28T21:39:25.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/35' 2023-11-28T21:39:25.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/26' 2023-11-28T21:39:25.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './259/37' 2023-11-28T21:39:25.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './259' 2023-11-28T21:39:25.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/21' 2023-11-28T21:39:25.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/27' 2023-11-28T21:39:25.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/31' 2023-11-28T21:39:25.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/36' 2023-11-28T21:39:25.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/20' 2023-11-28T21:39:25.811 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/39' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/34' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/38' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/28' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/24' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/29' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/32' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/22' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/25' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/30' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/33' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/23' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/35' 2023-11-28T21:39:25.812 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/26' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './260/37' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './260' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/21' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/27' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/31' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/36' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/20' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/39' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/34' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/38' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/28' 2023-11-28T21:39:25.813 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/24' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/29' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/32' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/22' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/25' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/30' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/33' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/23' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/35' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/26' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './261/37' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './261' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/21' 2023-11-28T21:39:25.814 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/27' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/31' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/36' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/20' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/39' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/34' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/38' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/28' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/24' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/29' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/32' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/22' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/25' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/30' 2023-11-28T21:39:25.815 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/33' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/23' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/35' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/26' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './262/37' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './262' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/21' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/27' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/31' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/36' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/20' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/39' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/34' 2023-11-28T21:39:25.816 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/38' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/28' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/24' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/29' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/32' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/22' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/25' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/30' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/33' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/23' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/35' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/26' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed './263/37' 2023-11-28T21:39:25.817 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './263' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/21' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/27' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/31' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/36' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/20' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/39' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/34' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/38' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/28' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/24' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/29' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/32' 2023-11-28T21:39:25.818 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/22' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/25' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/30' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/33' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/23' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/35' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/26' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './264/37' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './264' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/21' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/27' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/31' 2023-11-28T21:39:25.819 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/36' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/20' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/39' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/34' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/38' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/28' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/24' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/29' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/32' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/22' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/25' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/30' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/33' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/23' 2023-11-28T21:39:25.820 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/35' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/26' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './265/37' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './265' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/21' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/27' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/31' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/36' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/20' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/39' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/34' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/38' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/28' 2023-11-28T21:39:25.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/24' 2023-11-28T21:39:25.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/29' 2023-11-28T21:39:25.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/32' 2023-11-28T21:39:25.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/22' 2023-11-28T21:39:25.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/25' 2023-11-28T21:39:25.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/30' 2023-11-28T21:39:25.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/33' 2023-11-28T21:39:25.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/23' 2023-11-28T21:39:27.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/35' 2023-11-28T21:39:27.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/26' 2023-11-28T21:39:27.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './266/37' 2023-11-28T21:39:27.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './266' 2023-11-28T21:39:27.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/21' 2023-11-28T21:39:27.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/27' 2023-11-28T21:39:27.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/31' 2023-11-28T21:39:27.285 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/36' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/20' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/39' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/34' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/38' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/28' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/24' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/29' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/32' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/22' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/25' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/30' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/33' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/23' 2023-11-28T21:39:27.286 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/35' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/26' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './267/37' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './267' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/21' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/27' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/31' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/36' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/20' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/39' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/34' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/38' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/28' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/24' 2023-11-28T21:39:27.287 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/29' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/32' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/22' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/25' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/30' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/33' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/23' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/35' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/26' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './268/37' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './268' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/21' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/27' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/31' 2023-11-28T21:39:27.288 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/36' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/20' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/39' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/34' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/38' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/28' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/24' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/29' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/32' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/22' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/25' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/30' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/33' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/23' 2023-11-28T21:39:27.289 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/35' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/26' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './269/37' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './269' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/21' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/27' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/31' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/36' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/20' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/39' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/34' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/38' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/28' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/24' 2023-11-28T21:39:27.290 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/29' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/32' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/22' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/25' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/30' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/33' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/23' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/35' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/26' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './270/37' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './270' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/21' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/27' 2023-11-28T21:39:27.291 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/31' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/36' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/20' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/39' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/34' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/38' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/28' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/24' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/29' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/32' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/22' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/25' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/30' 2023-11-28T21:39:27.292 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/33' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/23' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/35' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/26' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './271/37' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './271' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/21' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/27' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/31' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/36' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/20' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/39' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/34' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/38' 2023-11-28T21:39:27.293 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/28' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/24' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/29' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/32' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/22' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/25' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/30' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/33' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/23' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/35' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/26' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './272/37' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './272' 2023-11-28T21:39:27.294 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/21' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/27' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/31' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/36' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/20' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/39' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/34' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/38' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/28' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/24' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/29' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/32' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/22' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/25' 2023-11-28T21:39:27.295 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/30' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/33' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/23' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/35' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/26' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './273/37' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './273' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/21' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/27' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/31' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/36' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/20' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/39' 2023-11-28T21:39:27.296 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/34' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/38' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/28' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/24' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/29' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/32' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/22' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/25' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/30' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/33' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/23' 2023-11-28T21:39:27.297 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/35' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/26' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './274/37' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './274' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/21' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/27' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/31' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/36' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/20' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/39' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/34' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/38' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/28' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/24' 2023-11-28T21:39:27.298 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/29' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/32' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/22' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/25' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/30' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/33' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/23' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/35' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/26' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './275/37' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './275' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/21' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/27' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/31' 2023-11-28T21:39:27.299 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/36' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/20' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/39' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/34' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/38' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/28' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/24' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/29' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/32' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/22' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/25' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/30' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/33' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/23' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/35' 2023-11-28T21:39:27.300 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/26' 2023-11-28T21:39:28.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './276/37' 2023-11-28T21:39:28.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './276' 2023-11-28T21:39:28.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/21' 2023-11-28T21:39:28.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/27' 2023-11-28T21:39:28.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/31' 2023-11-28T21:39:28.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/36' 2023-11-28T21:39:28.303 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/20' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/39' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/34' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/38' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/28' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/24' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/29' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/32' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/22' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/25' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/30' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/33' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/23' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/35' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/26' 2023-11-28T21:39:28.304 INFO:tasks.workunit.client.0.smithi012.stdout:removed './277/37' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './277' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/21' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/27' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/31' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/36' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/20' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/39' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/34' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/38' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/28' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/24' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/29' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/32' 2023-11-28T21:39:28.305 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/22' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/25' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/30' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/33' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/23' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/35' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/26' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './278/37' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './278' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/21' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/27' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/31' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/36' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/20' 2023-11-28T21:39:28.306 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/39' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/34' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/38' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/28' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/24' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/29' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/32' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/22' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/25' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/30' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/33' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/23' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/35' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/26' 2023-11-28T21:39:28.307 INFO:tasks.workunit.client.0.smithi012.stdout:removed './279/37' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './279' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/21' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/27' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/31' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/36' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/20' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/39' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/34' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/38' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/28' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/24' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/29' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/32' 2023-11-28T21:39:28.308 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/22' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/25' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/30' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/33' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/23' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/35' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/26' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './280/37' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './280' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/21' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/27' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/31' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/36' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/20' 2023-11-28T21:39:28.309 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/39' 2023-11-28T21:39:28.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/34' 2023-11-28T21:39:28.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/38' 2023-11-28T21:39:28.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/28' 2023-11-28T21:39:28.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/24' 2023-11-28T21:39:28.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/29' 2023-11-28T21:39:28.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/32' 2023-11-28T21:39:28.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/22' 2023-11-28T21:39:28.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/25' 2023-11-28T21:39:28.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/30' 2023-11-28T21:39:28.310 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/33' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/23' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/35' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/26' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './281/37' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './281' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/21' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/27' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/31' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/36' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/20' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/39' 2023-11-28T21:39:28.311 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/34' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/38' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/28' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/24' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/29' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/32' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/22' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/25' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/30' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/33' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/23' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/35' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/26' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed './282/37' 2023-11-28T21:39:28.312 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './282' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/21' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/27' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/31' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/36' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/20' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/39' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/34' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/38' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/28' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/24' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/29' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/32' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/22' 2023-11-28T21:39:28.313 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/25' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/30' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/33' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/23' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/35' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/26' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './283/37' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './283' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/21' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/27' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/31' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/36' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/20' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/39' 2023-11-28T21:39:28.314 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/34' 2023-11-28T21:39:28.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/38' 2023-11-28T21:39:28.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/28' 2023-11-28T21:39:28.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/24' 2023-11-28T21:39:28.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/29' 2023-11-28T21:39:28.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/32' 2023-11-28T21:39:28.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/22' 2023-11-28T21:39:28.315 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/25' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/30' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/33' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/23' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/35' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/26' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './284/37' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './284' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/21' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/27' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/31' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/36' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/20' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/39' 2023-11-28T21:39:28.316 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/34' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/38' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/28' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/24' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/29' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/32' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/22' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/25' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/30' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/33' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/23' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/35' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/26' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './285/37' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './285' 2023-11-28T21:39:28.317 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/21' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/27' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/31' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/36' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/20' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/39' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/34' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/38' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/28' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/24' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/29' 2023-11-28T21:39:28.318 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/32' 2023-11-28T21:39:28.323 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/22' 2023-11-28T21:39:28.323 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/25' 2023-11-28T21:39:28.323 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/30' 2023-11-28T21:39:28.323 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/33' 2023-11-28T21:39:28.323 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/23' 2023-11-28T21:39:28.324 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/35' 2023-11-28T21:39:28.324 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/26' 2023-11-28T21:39:28.324 INFO:tasks.workunit.client.0.smithi012.stdout:removed './286/37' 2023-11-28T21:39:28.324 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './286' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/21' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/27' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/31' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/36' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/20' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/39' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/34' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/38' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/28' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/24' 2023-11-28T21:39:34.532 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/29' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/32' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/22' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/25' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/30' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/33' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/23' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/35' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/26' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './287/37' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './287' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/21' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/27' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/31' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/36' 2023-11-28T21:39:34.533 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/20' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/39' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/34' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/38' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/28' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/24' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/29' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/32' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/22' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/25' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/30' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/33' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/23' 2023-11-28T21:39:34.534 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/35' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/26' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './288/37' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './288' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/21' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/27' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/31' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/36' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/20' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/39' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/34' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/38' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/28' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/24' 2023-11-28T21:39:34.535 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/29' 2023-11-28T21:39:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/32' 2023-11-28T21:39:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/22' 2023-11-28T21:39:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/25' 2023-11-28T21:39:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/30' 2023-11-28T21:39:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/33' 2023-11-28T21:39:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/23' 2023-11-28T21:39:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/35' 2023-11-28T21:39:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/26' 2023-11-28T21:39:34.536 INFO:tasks.workunit.client.0.smithi012.stdout:removed './289/37' 2023-11-28T21:39:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './289' 2023-11-28T21:39:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/21' 2023-11-28T21:39:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/27' 2023-11-28T21:39:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/31' 2023-11-28T21:39:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/36' 2023-11-28T21:39:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/20' 2023-11-28T21:39:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/39' 2023-11-28T21:39:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/34' 2023-11-28T21:39:34.537 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/38' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/28' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/24' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/29' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/32' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/22' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/25' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/30' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/33' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/23' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/35' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/26' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed './290/37' 2023-11-28T21:39:34.538 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './290' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/21' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/27' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/31' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/36' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/20' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/39' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/34' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/38' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/28' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/24' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/29' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/32' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/22' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/25' 2023-11-28T21:39:34.539 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/30' 2023-11-28T21:39:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/33' 2023-11-28T21:39:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/23' 2023-11-28T21:39:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/35' 2023-11-28T21:39:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/26' 2023-11-28T21:39:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './291/37' 2023-11-28T21:39:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './291' 2023-11-28T21:39:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/21' 2023-11-28T21:39:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/27' 2023-11-28T21:39:34.541 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/31' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/36' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/20' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/39' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/34' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/38' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/28' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/24' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/29' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/32' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/22' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/25' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/30' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/33' 2023-11-28T21:39:34.542 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/23' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/35' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/26' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './292/37' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './292' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/21' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/27' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/31' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/36' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/20' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/39' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/34' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/38' 2023-11-28T21:39:34.543 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/28' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/24' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/29' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/32' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/22' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/25' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/30' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/33' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/23' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/35' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/26' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './293/37' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './293' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/21' 2023-11-28T21:39:34.544 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/27' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/31' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/36' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/20' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/39' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/34' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/38' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/28' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/24' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/29' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/32' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/22' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/25' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/30' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/33' 2023-11-28T21:39:34.545 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/23' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/35' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/26' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './294/37' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './294' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/21' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/27' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/31' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/36' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/20' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/39' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/34' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/38' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/28' 2023-11-28T21:39:34.546 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/24' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/29' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/32' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/22' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/25' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/30' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/33' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/23' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/35' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/26' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './295/37' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './295' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/21' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/27' 2023-11-28T21:39:34.547 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/31' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/36' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/20' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/39' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/34' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/38' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/28' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/24' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/29' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/32' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/22' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/25' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/30' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/33' 2023-11-28T21:39:34.548 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/23' 2023-11-28T21:39:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/35' 2023-11-28T21:39:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/26' 2023-11-28T21:39:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './296/37' 2023-11-28T21:39:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './296' 2023-11-28T21:39:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/21' 2023-11-28T21:39:34.549 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/27' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/31' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/36' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/20' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/39' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/34' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/38' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/28' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/24' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/29' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/32' 2023-11-28T21:39:35.157 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/22' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/25' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/30' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/33' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/23' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/35' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/26' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './297/37' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './297' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/21' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/27' 2023-11-28T21:39:35.158 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/31' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/36' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/20' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/39' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/34' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/38' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/28' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/24' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/29' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/32' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/22' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/25' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/30' 2023-11-28T21:39:35.159 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/33' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/23' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/35' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/26' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './298/37' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './298' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/21' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/27' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/31' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/36' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/20' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/39' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/34' 2023-11-28T21:39:35.160 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/38' 2023-11-28T21:39:35.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/28' 2023-11-28T21:39:35.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/24' 2023-11-28T21:39:35.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/29' 2023-11-28T21:39:35.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/32' 2023-11-28T21:39:35.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/22' 2023-11-28T21:39:35.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/25' 2023-11-28T21:39:35.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/30' 2023-11-28T21:39:35.161 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/33' 2023-11-28T21:39:35.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/23' 2023-11-28T21:39:35.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/35' 2023-11-28T21:39:35.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/26' 2023-11-28T21:39:35.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './299/37' 2023-11-28T21:39:35.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './299' 2023-11-28T21:39:35.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/19' 2023-11-28T21:39:35.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/15' 2023-11-28T21:39:35.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/20' 2023-11-28T21:39:35.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/12' 2023-11-28T21:39:35.162 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/10' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/11' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/16' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/14' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/18' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/13' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/166/17' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/166' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/19' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/15' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/20' 2023-11-28T21:39:35.163 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/12' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/10' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/11' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/16' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/14' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/18' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/13' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/147/17' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/147' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/19' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/15' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/20' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/12' 2023-11-28T21:39:35.164 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/10' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/11' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/16' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/14' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/18' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/13' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/151/17' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/151' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/19' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/15' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/20' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/12' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/10' 2023-11-28T21:39:35.165 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/11' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/16' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/14' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/18' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/13' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/170/17' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/170' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/19' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/15' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/20' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/12' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/10' 2023-11-28T21:39:35.166 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/11' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/16' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/14' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/18' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/13' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/167/17' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/167' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/19' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/15' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/20' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/12' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/10' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/11' 2023-11-28T21:39:35.167 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/16' 2023-11-28T21:39:35.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/14' 2023-11-28T21:39:35.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/18' 2023-11-28T21:39:35.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/13' 2023-11-28T21:39:35.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/140/17' 2023-11-28T21:39:35.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/140' 2023-11-28T21:39:35.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/19' 2023-11-28T21:39:35.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/15' 2023-11-28T21:39:35.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/20' 2023-11-28T21:39:35.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/12' 2023-11-28T21:39:35.168 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/10' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/11' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/16' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/14' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/18' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/13' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/142/17' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/142' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/19' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/15' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/20' 2023-11-28T21:39:36.023 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/12' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/10' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/11' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/16' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/14' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/18' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/13' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/150/17' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/150' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/19' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/15' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/20' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/12' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/10' 2023-11-28T21:39:36.024 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/11' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/16' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/14' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/18' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/13' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/105/17' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/105' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/19' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/15' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/20' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/12' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/10' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/11' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/16' 2023-11-28T21:39:36.025 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/14' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/18' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/13' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/149/17' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/149' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/19' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/15' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/20' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/12' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/10' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/11' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/16' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/14' 2023-11-28T21:39:36.026 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/18' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/13' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/109/17' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/109' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/19' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/15' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/20' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/12' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/10' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/11' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/16' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/14' 2023-11-28T21:39:36.027 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/18' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/13' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/158/17' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/158' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/19' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/15' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/20' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/12' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/10' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/11' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/16' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/14' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/18' 2023-11-28T21:39:36.028 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/13' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/106/17' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/106' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/19' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/15' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/20' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/12' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/10' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/11' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/16' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/14' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/18' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/13' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/189/17' 2023-11-28T21:39:36.029 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/189' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/19' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/15' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/20' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/12' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/10' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/11' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/16' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/14' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/18' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/13' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/133/17' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/133' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/19' 2023-11-28T21:39:36.030 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/15' 2023-11-28T21:39:36.031 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/20' 2023-11-28T21:39:36.031 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/12' 2023-11-28T21:39:36.031 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/10' 2023-11-28T21:39:36.031 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/11' 2023-11-28T21:39:37.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:39:37.165 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:39:37.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/16' 2023-11-28T21:39:37.635 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/14' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/18' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/13' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/173/17' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/173' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/19' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/15' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/20' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/12' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/10' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/11' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/16' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/14' 2023-11-28T21:39:37.636 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/18' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/13' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/174/17' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/174' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/19' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/15' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/20' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/12' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/10' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/11' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/16' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/14' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/18' 2023-11-28T21:39:37.637 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/13' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/155/17' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/155' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/19' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/15' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/20' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/12' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/10' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/11' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/16' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/14' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/18' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/13' 2023-11-28T21:39:37.638 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/114/17' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/114' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/19' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/15' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/20' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/12' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/10' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/11' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/16' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/14' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/18' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/13' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/159/17' 2023-11-28T21:39:37.639 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/159' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/19' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/15' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/20' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/12' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/10' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/11' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/16' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/14' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/18' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/13' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/187/17' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/187' 2023-11-28T21:39:37.640 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/19' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/15' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/20' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/12' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/10' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/11' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/16' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/14' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/18' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/13' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/185/17' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/185' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/19' 2023-11-28T21:39:37.641 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/15' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/20' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/12' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/10' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/11' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/16' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/14' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/18' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/13' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/120/17' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/120' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/19' 2023-11-28T21:39:37.642 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/15' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/20' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/12' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/10' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/11' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/16' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/14' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/18' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/13' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/110/17' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/110' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/19' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/15' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/20' 2023-11-28T21:39:37.643 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/12' 2023-11-28T21:39:37.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/10' 2023-11-28T21:39:37.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/11' 2023-11-28T21:39:37.644 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/16' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/14' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/18' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/13' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/137/17' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/137' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/19' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/15' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/20' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/12' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/10' 2023-11-28T21:39:39.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/11' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/16' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/14' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/18' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/13' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/198/17' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/198' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/19' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/15' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/20' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/12' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/10' 2023-11-28T21:39:39.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/11' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/16' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/14' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/18' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/13' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/136/17' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/136' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/19' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/15' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/20' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/12' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/10' 2023-11-28T21:39:39.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/11' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/16' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/14' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/18' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/13' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/125/17' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/125' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/19' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/15' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/20' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/12' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/10' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/11' 2023-11-28T21:39:39.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/16' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/14' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/18' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/13' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/194/17' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/194' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/19' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/15' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/20' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/12' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/10' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/11' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/16' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/14' 2023-11-28T21:39:39.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/18' 2023-11-28T21:39:39.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/13' 2023-11-28T21:39:39.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/119/17' 2023-11-28T21:39:39.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/119' 2023-11-28T21:39:39.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/19' 2023-11-28T21:39:39.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/15' 2023-11-28T21:39:39.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/20' 2023-11-28T21:39:39.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/12' 2023-11-28T21:39:39.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/10' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/11' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/16' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/14' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/18' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/13' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/157/17' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/157' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/19' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/15' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/20' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/12' 2023-11-28T21:39:39.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/10' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/11' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/16' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/14' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/18' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/13' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/172/17' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/172' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/19' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/15' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/20' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/12' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/10' 2023-11-28T21:39:39.263 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/11' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/16' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/14' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/18' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/13' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/186/17' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/186' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/19' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/15' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/20' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/12' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/10' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/11' 2023-11-28T21:39:39.264 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/16' 2023-11-28T21:39:39.265 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/14' 2023-11-28T21:39:40.137 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/18' 2023-11-28T21:39:40.138 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/13' 2023-11-28T21:39:40.138 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/190/17' 2023-11-28T21:39:40.138 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/190' 2023-11-28T21:39:40.138 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/19' 2023-11-28T21:39:40.138 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/15' 2023-11-28T21:39:40.138 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/20' 2023-11-28T21:39:40.138 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/12' 2023-11-28T21:39:40.138 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/10' 2023-11-28T21:39:40.138 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/11' 2023-11-28T21:39:40.138 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/16' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/14' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/18' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/13' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/104/17' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/104' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/19' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/15' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/20' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/12' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/10' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/11' 2023-11-28T21:39:40.139 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/16' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/14' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/18' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/13' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/164/17' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/164' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/19' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/15' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/20' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/12' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/10' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/11' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/16' 2023-11-28T21:39:40.140 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/14' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/18' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/13' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/112/17' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/112' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/19' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/15' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/20' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/12' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/10' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/11' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/16' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/14' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/18' 2023-11-28T21:39:40.141 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/13' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/100/17' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/100' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/19' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/15' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/20' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/12' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/10' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/11' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/16' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/14' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/18' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/13' 2023-11-28T21:39:40.142 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/134/17' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/134' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/19' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/15' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/20' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/12' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/10' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/11' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/16' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/14' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/18' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/13' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/115/17' 2023-11-28T21:39:40.143 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/115' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/19' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/15' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/20' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/12' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/10' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/11' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/16' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/14' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/18' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/13' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/116/17' 2023-11-28T21:39:40.144 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/116' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/19' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/15' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/20' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/12' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/10' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/11' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/16' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/14' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/18' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/13' 2023-11-28T21:39:40.145 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/195/17' 2023-11-28T21:39:40.146 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/195' 2023-11-28T21:39:40.146 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/19' 2023-11-28T21:39:40.146 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/15' 2023-11-28T21:39:40.146 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/20' 2023-11-28T21:39:40.146 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/12' 2023-11-28T21:39:40.146 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/10' 2023-11-28T21:39:40.146 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/11' 2023-11-28T21:39:40.146 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/16' 2023-11-28T21:39:40.146 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/14' 2023-11-28T21:39:40.146 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/18' 2023-11-28T21:39:41.253 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/13' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/138/17' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/138' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/19' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/15' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/20' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/12' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/10' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/11' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/16' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/14' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/18' 2023-11-28T21:39:41.254 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/13' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/135/17' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/135' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/19' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/15' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/20' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/12' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/10' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/11' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/16' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/14' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/18' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/13' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/117/17' 2023-11-28T21:39:41.255 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/117' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/19' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/15' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/20' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/12' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/10' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/11' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/16' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/14' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/18' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/13' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/143/17' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/143' 2023-11-28T21:39:41.256 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/19' 2023-11-28T21:39:41.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/15' 2023-11-28T21:39:41.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/20' 2023-11-28T21:39:41.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/12' 2023-11-28T21:39:41.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/10' 2023-11-28T21:39:41.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/11' 2023-11-28T21:39:41.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/16' 2023-11-28T21:39:41.257 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/14' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/18' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/13' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/180/17' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/180' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/19' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/15' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/20' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/12' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/10' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/11' 2023-11-28T21:39:41.258 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/16' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/14' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/18' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/13' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/126/17' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/126' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/19' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/15' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/20' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/12' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/10' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/11' 2023-11-28T21:39:41.259 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/16' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/14' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/18' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/13' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/161/17' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/161' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/19' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/15' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/20' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/12' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/10' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/11' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/16' 2023-11-28T21:39:41.260 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/14' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/18' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/13' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/176/17' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/176' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/19' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/15' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/20' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/12' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/10' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/11' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/16' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/14' 2023-11-28T21:39:41.261 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/18' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/13' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/111/17' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/111' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/19' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/15' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/20' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/12' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/10' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/11' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/16' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/14' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/18' 2023-11-28T21:39:41.262 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/13' 2023-11-28T21:39:42.126 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:39:42.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f982e7aa640 1 Processor -- start 2023-11-28T21:39:42.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f982e7aa640 1 -- start start 2023-11-28T21:39:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f982e7aa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9828153e00 0x7f982806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f982e7aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f982806bc00 0x7f98280cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f982e7aa640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98280c7710 con 0x7f98280cd950 2023-11-28T21:39:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f982e7aa640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98281541e0 con 0x7f9828153e00 2023-11-28T21:39:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f982e7aa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982815a010 con 0x7f982806bc00 2023-11-28T21:39:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f9827fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9828153e00 0x7f982806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f9827fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9828153e00 0x7f982806b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42892/0 (socket says 172.21.15.12:42892) 2023-11-28T21:39:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f9827fff640 1 -- 172.21.15.12:0/3745667878 learned_addr learned my addr 172.21.15.12:0/3745667878 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:39:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f981ffff640 1 --2- 172.21.15.12:0/3745667878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f982806bc00 0x7f98280cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f9827fff640 1 -- 172.21.15.12:0/3745667878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f982806bc00 msgr2=0x7f98280cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:39:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f9827fff640 1 --2- 172.21.15.12:0/3745667878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f982806bc00 0x7f98280cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f9827fff640 1 -- 172.21.15.12:0/3745667878 >> v1:172.21.15.144:6789/0 conn(0x7f98280cd950 legacy=0x7f982815d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:39:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.269+0000 7f9827fff640 1 -- 172.21.15.12:0/3745667878 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982815a2f0 con 0x7f9828153e00 2023-11-28T21:39:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f9827fff640 1 --2- 172.21.15.12:0/3745667878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9828153e00 0x7f982806b6c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9818009870 tx=0x7f981801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb23e05814423c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f98277fe640 1 -- 172.21.15.12:0/3745667878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98180a89b0 con 0x7f9828153e00 2023-11-28T21:39:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f98277fe640 1 -- 172.21.15.12:0/3745667878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98180a8b10 con 0x7f9828153e00 2023-11-28T21:39:42.280 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f98277fe640 1 -- 172.21.15.12:0/3745667878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98180a8ea0 con 0x7f9828153e00 2023-11-28T21:39:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f982e7aa640 1 -- 172.21.15.12:0/3745667878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9828153e00 msgr2=0x7f982806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f982e7aa640 1 --2- 172.21.15.12:0/3745667878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9828153e00 0x7f982806b6c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9818009870 tx=0x7f981801ecf0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f982e7aa640 1 -- 172.21.15.12:0/3745667878 shutdown_connections 2023-11-28T21:39:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f982e7aa640 1 --2- 172.21.15.12:0/3745667878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f982806bc00 0x7f98280cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f982e7aa640 1 --2- 172.21.15.12:0/3745667878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9828153e00 0x7f982806b6c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f982e7aa640 1 -- 172.21.15.12:0/3745667878 >> 172.21.15.12:0/3745667878 conn(0x7f9828066360 msgr2=0x7f9828066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f982e7aa640 1 -- 172.21.15.12:0/3745667878 shutdown_connections 2023-11-28T21:39:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.273+0000 7f982e7aa640 1 -- 172.21.15.12:0/3745667878 wait complete. 2023-11-28T21:39:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982e7aa640 1 Processor -- start 2023-11-28T21:39:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982e7aa640 1 -- start start 2023-11-28T21:39:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982e7aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f982806bc00 0x7f9828157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982e7aa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98280cd950 0x7f98281f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982e7aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9828153e00 0x7f9828155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982e7aa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9828157050 con 0x7f982806bc00 2023-11-28T21:39:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982e7aa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9828066bd0 con 0x7f98280cd950 2023-11-28T21:39:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982e7aa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98281562a0 con 0x7f9828153e00 2023-11-28T21:39:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f9827fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f982806bc00 0x7f9828157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f9827fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f982806bc00 0x7f9828157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42900/0 (socket says 172.21.15.12:42900) 2023-11-28T21:39:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f9827fff640 1 -- 172.21.15.12:0/1924556452 learned_addr learned my addr 172.21.15.12:0/1924556452 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:39:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f981ffff640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98280cd950 0x7f98281f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f9827fff640 1 -- 172.21.15.12:0/1924556452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9828153e00 msgr2=0x7f9828155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982cd47640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9828153e00 0x7f9828155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f9827fff640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9828153e00 0x7f9828155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f9827fff640 1 -- 172.21.15.12:0/1924556452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98280cd950 msgr2=0x7f98281f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f9827fff640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98280cd950 0x7f98281f2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f9827fff640 1 -- 172.21.15.12:0/1924556452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180094d0 con 0x7f982806bc00 2023-11-28T21:39:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f981ffff640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98280cd950 0x7f98281f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:39:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f9827fff640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f982806bc00 0x7f9828157470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f98180b3040 tx=0x7f981801ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f98257fa640 1 -- 172.21.15.12:0/1924556452 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98180a89b0 con 0x7f982806bc00 2023-11-28T21:39:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f98257fa640 1 -- 172.21.15.12:0/1924556452 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98180bb480 con 0x7f982806bc00 2023-11-28T21:39:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982e7aa640 1 -- 172.21.15.12:0/1924556452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98281569b0 con 0x7f982806bc00 2023-11-28T21:39:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f98257fa640 1 -- 172.21.15.12:0/1924556452 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98180bb850 con 0x7f982806bc00 2023-11-28T21:39:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.277+0000 7f982e7aa640 1 -- 172.21.15.12:0/1924556452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98280c90b0 con 0x7f982806bc00 2023-11-28T21:39:42.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.281+0000 7f982e7aa640 1 -- 172.21.15.12:0/1924556452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f982806c620 con 0x7f982806bc00 2023-11-28T21:39:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.281+0000 7f98257fa640 1 -- 172.21.15.12:0/1924556452 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f98180bb9b0 con 0x7f982806bc00 2023-11-28T21:39:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.281+0000 7f98257fa640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9808066470 0x7f9808068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.281+0000 7f98257fa640 1 -- 172.21.15.12:0/1924556452 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6883+0+0 (secure 0 0 0) 0x7f98181229d0 con 0x7f982806bc00 2023-11-28T21:39:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.281+0000 7f981ffff640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9808066470 0x7f9808068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.281+0000 7f981ffff640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9808066470 0x7f9808068930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9810006fd0 tx=0x7f9810008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:42.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.285+0000 7f98257fa640 1 -- 172.21.15.12:0/1924556452 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98180ef9a0 con 0x7f982806bc00 2023-11-28T21:39:42.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.473+0000 7f982e7aa640 1 -- 172.21.15.12:0/1924556452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f982806c620 con 0x7f982806bc00 2023-11-28T21:39:42.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f98257fa640 1 -- 172.21.15.12:0/1924556452 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f98180b5090 con 0x7f982806bc00 2023-11-28T21:39:42.483 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:39:42.483 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5307,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:39:42.483 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2023-11-28T21:39:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 -- 172.21.15.12:0/1924556452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9808066470 msgr2=0x7f9808068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9808066470 0x7f9808068930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9810006fd0 tx=0x7f9810008040 comp rx=0 tx=0).stop 2023-11-28T21:39:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 -- 172.21.15.12:0/1924556452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f982806bc00 msgr2=0x7f9828157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f982806bc00 0x7f9828157470 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f98180b3040 tx=0x7f981801ba90 comp rx=0 tx=0).stop 2023-11-28T21:39:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 -- 172.21.15.12:0/1924556452 shutdown_connections 2023-11-28T21:39:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9808066470 0x7f9808068930 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9828153e00 0x7f9828155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98280cd950 0x7f98281f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 --2- 172.21.15.12:0/1924556452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f982806bc00 0x7f9828157470 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 -- 172.21.15.12:0/1924556452 >> 172.21.15.12:0/1924556452 conn(0x7f9828066360 msgr2=0x7f982815c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 -- 172.21.15.12:0/1924556452 shutdown_connections 2023-11-28T21:39:42.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.477+0000 7f982e7aa640 1 -- 172.21.15.12:0/1924556452 wait complete. 2023-11-28T21:39:42.504 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2023-11-28T21:39:42.505 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-11-28T21:39:42.505 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:39:42.530 INFO:tasks.ceph.mds.f:Stopped 2023-11-28T21:39:42.530 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:39:42.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfc032a640 1 Processor -- start 2023-11-28T21:39:42.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfc032a640 1 -- start start 2023-11-28T21:39:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfc032a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdfb8154100 0x7fdfb815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfc032a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb815e270 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfc032a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb80c4e60 con 0x7fdfb80c9fd0 2023-11-28T21:39:42.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfc032a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb81544e0 con 0x7fdfb8154100 2023-11-28T21:39:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfbe0c6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdfb8154100 0x7fdfb815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfbe8c7640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdfb80ca3f0 0x7fdfb80c52f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53438/0 (socket says 172.21.15.12:53438) 2023-11-28T21:39:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfbe8c7640 1 -- 172.21.15.12:0/3911202940 learned_addr learned my addr 172.21.15.12:0/3911202940 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:39:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfbe0c6640 1 -- 172.21.15.12:0/3911202940 >> v1:172.21.15.144:6789/0 conn(0x7fdfb80c9fd0 legacy=0x7fdfb806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfbe0c6640 1 -- 172.21.15.12:0/3911202940 >> v1:172.21.15.12:6789/0 conn(0x7fdfb80ca3f0 legacy=0x7fdfb80c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.669+0000 7fdfbe0c6640 1 -- 172.21.15.12:0/3911202940 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb815d9d0 con 0x7fdfb8154100 2023-11-28T21:39:42.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfbe0c6640 1 --2- 172.21.15.12:0/3911202940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb8154100 0x7fdfb815d320 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40097c0 tx=0x7fdfb401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdee4b4057669228 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:42.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfbd0c4640 1 -- 172.21.15.12:0/3911202940 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40ab650 con 0x7fdfb8154100 2023-11-28T21:39:42.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfbd0c4640 1 -- 172.21.15.12:0/3911202940 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb40ab7b0 con 0x7fdfb8154100 2023-11-28T21:39:42.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfbd0c4640 1 -- 172.21.15.12:0/3911202940 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40abb40 con 0x7fdfb8154100 2023-11-28T21:39:42.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfc032a640 1 -- 172.21.15.12:0/3911202940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb8154100 msgr2=0x7fdfb815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfc032a640 1 --2- 172.21.15.12:0/3911202940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb8154100 0x7fdfb815d320 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40097c0 tx=0x7fdfb401c450 comp rx=0 tx=0).stop 2023-11-28T21:39:42.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfc032a640 1 -- 172.21.15.12:0/3911202940 shutdown_connections 2023-11-28T21:39:42.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfc032a640 1 --2- 172.21.15.12:0/3911202940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb8154100 0x7fdfb815d320 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfc032a640 1 -- 172.21.15.12:0/3911202940 >> 172.21.15.12:0/3911202940 conn(0x7fdfb806c690 msgr2=0x7fdfb8066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:42.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfc032a640 1 -- 172.21.15.12:0/3911202940 shutdown_connections 2023-11-28T21:39:42.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.673+0000 7fdfc032a640 1 -- 172.21.15.12:0/3911202940 wait complete. 2023-11-28T21:39:42.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 Processor -- start 2023-11-28T21:39:42.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 -- start start 2023-11-28T21:39:42.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb80c9fd0 0x7fdfb80b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb80ca3f0 0x7fdfb80b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb8154100 0x7fdfb8139620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb813a770 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb815d070 con 0x7fdfb8154100 2023-11-28T21:39:42.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb815aea0 con 0x7fdfb80c9fd0 2023-11-28T21:39:42.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbd8c5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb80ca3f0 0x7fdfb80b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbd8c5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb80ca3f0 0x7fdfb80b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42924/0 (socket says 172.21.15.12:42924) 2023-11-28T21:39:42.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbd8c5640 1 -- 172.21.15.12:0/247452996 learned_addr learned my addr 172.21.15.12:0/247452996 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:39:42.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbe0c6640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb80c9fd0 0x7fdfb80b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbe8c7640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb8154100 0x7fdfb8139620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbd8c5640 1 -- 172.21.15.12:0/247452996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb80c9fd0 msgr2=0x7fdfb80b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbd8c5640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb80c9fd0 0x7fdfb80b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbd8c5640 1 -- 172.21.15.12:0/247452996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb8154100 msgr2=0x7fdfb8139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbd8c5640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb8154100 0x7fdfb8139620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbd8c5640 1 -- 172.21.15.12:0/247452996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb4009420 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfbd8c5640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb80ca3f0 0x7fdfb80b4ac0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800c3c0 tx=0x7fdfa800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:42.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfaeffd640 1 -- 172.21.15.12:0/247452996 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa80052d0 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfaeffd640 1 -- 172.21.15.12:0/247452996 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa8005430 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfaeffd640 1 -- 172.21.15.12:0/247452996 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8003840 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 -- 172.21.15.12:0/247452996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb8151710 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 -- 172.21.15.12:0/247452996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb815b0f0 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.677+0000 7fdfc032a640 1 -- 172.21.15.12:0/247452996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb8067c40 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.681+0000 7fdfaeffd640 1 -- 172.21.15.12:0/247452996 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdfa80039a0 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.681+0000 7fdfaeffd640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdf9c066470 0x7fdf9c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:42.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.681+0000 7fdfaeffd640 1 -- 172.21.15.12:0/247452996 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6883+0+0 (secure 0 0 0) 0x7fdfa80838b0 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.681+0000 7fdfbe0c6640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdf9c066470 0x7fdf9c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:42.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.681+0000 7fdfbe0c6640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdf9c066470 0x7fdf9c068930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40af000 tx=0x7fdfb40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:42.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.685+0000 7fdfaeffd640 1 -- 172.21.15.12:0/247452996 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdfa80507f0 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.873+0000 7fdfc032a640 1 -- 172.21.15.12:0/247452996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfb8074f60 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.873+0000 7fdfaeffd640 1 -- 172.21.15.12:0/247452996 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fdfa8053e10 con 0x7fdfb80ca3f0 2023-11-28T21:39:42.881 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:39:42.881 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5307,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:39:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2023-11-28T21:39:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.873+0000 7fdfc032a640 1 -- 172.21.15.12:0/247452996 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdf9c066470 msgr2=0x7fdf9c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdf9c066470 0x7fdf9c068930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40af000 tx=0x7fdfb40a7000 comp rx=0 tx=0).stop 2023-11-28T21:39:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 -- 172.21.15.12:0/247452996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb80ca3f0 msgr2=0x7fdfb80b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb80ca3f0 0x7fdfb80b4ac0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800c3c0 tx=0x7fdfa800c880 comp rx=0 tx=0).stop 2023-11-28T21:39:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 -- 172.21.15.12:0/247452996 shutdown_connections 2023-11-28T21:39:42.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdf9c066470 0x7fdf9c068930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb8154100 0x7fdfb8139620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdfb80ca3f0 0x7fdfb80b4ac0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 --2- 172.21.15.12:0/247452996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb80c9fd0 0x7fdfb80b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 -- 172.21.15.12:0/247452996 >> 172.21.15.12:0/247452996 conn(0x7fdfb806c690 msgr2=0x7fdfb8146ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 -- 172.21.15.12:0/247452996 shutdown_connections 2023-11-28T21:39:42.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:42.877+0000 7fdfc032a640 1 -- 172.21.15.12:0/247452996 wait complete. 2023-11-28T21:39:42.901 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-11-28T21:39:44.902 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:39:45.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb6950f640 1 Processor -- start 2023-11-28T21:39:45.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb6950f640 1 -- start start 2023-11-28T21:39:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb6950f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb64153f30 0x7feb640ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb6950f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb6406bdd0 0x7feb6415d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb6950f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6415d4a0 con 0x7feb640ca6b0 2023-11-28T21:39:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb6950f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6415a130 con 0x7feb6406bdd0 2023-11-28T21:39:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb6950f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6415a3f0 con 0x7feb64153f30 2023-11-28T21:39:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb637fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb6406bdd0 0x7feb6415d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:45.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb637fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb6406bdd0 0x7feb6415d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40390/0 (socket says 172.21.15.12:40390) 2023-11-28T21:39:45.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb637fe640 1 -- 172.21.15.12:0/1753420076 learned_addr learned my addr 172.21.15.12:0/1753420076 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:39:45.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb62ffd640 1 --2- 172.21.15.12:0/1753420076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb64153f30 0x7feb640ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:45.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb637fe640 1 -- 172.21.15.12:0/1753420076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb64153f30 msgr2=0x7feb640ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:39:45.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb637fe640 1 --2- 172.21.15.12:0/1753420076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb64153f30 0x7feb640ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:45.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb637fe640 1 -- 172.21.15.12:0/1753420076 >> v1:172.21.15.144:6789/0 conn(0x7feb640ca6b0 legacy=0x7feb6406b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:45.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.049+0000 7feb637fe640 1 -- 172.21.15.12:0/1753420076 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6415a6d0 con 0x7feb6406bdd0 2023-11-28T21:39:45.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb637fe640 1 --2- 172.21.15.12:0/1753420076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb6406bdd0 0x7feb6415d930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009e50 tx=0x7feb5c01f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be079c9b5359a790 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:45.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb61ffb640 1 -- 172.21.15.12:0/1753420076 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0af400 con 0x7feb6406bdd0 2023-11-28T21:39:45.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb61ffb640 1 -- 172.21.15.12:0/1753420076 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c0ab050 con 0x7feb6406bdd0 2023-11-28T21:39:45.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb61ffb640 1 -- 172.21.15.12:0/1753420076 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0af780 con 0x7feb6406bdd0 2023-11-28T21:39:45.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb6950f640 1 -- 172.21.15.12:0/1753420076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb6406bdd0 msgr2=0x7feb6415d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:45.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb6950f640 1 --2- 172.21.15.12:0/1753420076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb6406bdd0 0x7feb6415d930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009e50 tx=0x7feb5c01f1f0 comp rx=0 tx=0).stop 2023-11-28T21:39:45.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb6950f640 1 -- 172.21.15.12:0/1753420076 shutdown_connections 2023-11-28T21:39:45.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb6950f640 1 --2- 172.21.15.12:0/1753420076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb6406bdd0 0x7feb6415d930 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:45.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb6950f640 1 --2- 172.21.15.12:0/1753420076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb64153f30 0x7feb640ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:45.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb6950f640 1 -- 172.21.15.12:0/1753420076 >> 172.21.15.12:0/1753420076 conn(0x7feb64066530 msgr2=0x7feb64066940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:45.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb6950f640 1 -- 172.21.15.12:0/1753420076 shutdown_connections 2023-11-28T21:39:45.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.053+0000 7feb6950f640 1 -- 172.21.15.12:0/1753420076 wait complete. 2023-11-28T21:39:45.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 Processor -- start 2023-11-28T21:39:45.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 -- start start 2023-11-28T21:39:45.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6406bdd0 0x7feb641557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:45.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb640ca6b0 0x7feb640c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:45.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb64153f30 0x7feb640cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:45.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb640c58e0 con 0x7feb64153f30 2023-11-28T21:39:45.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb64157590 con 0x7feb6406bdd0 2023-11-28T21:39:45.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb640c5600 con 0x7feb640ca6b0 2023-11-28T21:39:45.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb64153f30 0x7feb640cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:45.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb64153f30 0x7feb640cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40394/0 (socket says 172.21.15.12:40394) 2023-11-28T21:39:45.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb637fe640 1 -- 172.21.15.12:0/2057308684 learned_addr learned my addr 172.21.15.12:0/2057308684 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb62ffd640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6406bdd0 0x7feb641557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb627fc640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb640ca6b0 0x7feb640c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb637fe640 1 -- 172.21.15.12:0/2057308684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb640ca6b0 msgr2=0x7feb640c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb637fe640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb640ca6b0 0x7feb640c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb637fe640 1 -- 172.21.15.12:0/2057308684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6406bdd0 msgr2=0x7feb641557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb637fe640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6406bdd0 0x7feb641557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb637fe640 1 -- 172.21.15.12:0/2057308684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c009b30 con 0x7feb64153f30 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb62ffd640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6406bdd0 0x7feb641557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb637fe640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb64153f30 0x7feb640cc3d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7feb5c01ce20 tx=0x7feb5c01bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb4b7fe640 1 -- 172.21.15.12:0/2057308684 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0afde0 con 0x7feb64153f30 2023-11-28T21:39:45.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb4b7fe640 1 -- 172.21.15.12:0/2057308684 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c009510 con 0x7feb64153f30 2023-11-28T21:39:45.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 -- 172.21.15.12:0/2057308684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb64154710 con 0x7feb64153f30 2023-11-28T21:39:45.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb4b7fe640 1 -- 172.21.15.12:0/2057308684 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0ba660 con 0x7feb64153f30 2023-11-28T21:39:45.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 -- 172.21.15.12:0/2057308684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb640c6160 con 0x7feb64153f30 2023-11-28T21:39:45.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.057+0000 7feb6950f640 1 -- 172.21.15.12:0/2057308684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb64074f60 con 0x7feb64153f30 2023-11-28T21:39:45.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.061+0000 7feb4b7fe640 1 -- 172.21.15.12:0/2057308684 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7feb5c0b6040 con 0x7feb64153f30 2023-11-28T21:39:45.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.061+0000 7feb4b7fe640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7feb500664a0 0x7feb50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:45.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.061+0000 7feb4b7fe640 1 -- 172.21.15.12:0/2057308684 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6883+0+0 (secure 0 0 0) 0x7feb5c1225b0 con 0x7feb64153f30 2023-11-28T21:39:45.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.061+0000 7feb62ffd640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7feb500664a0 0x7feb50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:45.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.061+0000 7feb62ffd640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7feb500664a0 0x7feb50068960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb5400b9f0 tx=0x7feb54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:45.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.065+0000 7feb4b7fe640 1 -- 172.21.15.12:0/2057308684 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feb5c0ef5b0 con 0x7feb64153f30 2023-11-28T21:39:45.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.253+0000 7feb6950f640 1 -- 172.21.15.12:0/2057308684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb640705e0 con 0x7feb64153f30 2023-11-28T21:39:45.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.253+0000 7feb4b7fe640 1 -- 172.21.15.12:0/2057308684 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6506 (secure 0 0 0) 0x7feb5c0f2bd0 con 0x7feb64153f30 2023-11-28T21:39:45.261 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:39:45.262 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5307,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:39:45.262 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2023-11-28T21:39:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 -- 172.21.15.12:0/2057308684 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7feb500664a0 msgr2=0x7feb50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7feb500664a0 0x7feb50068960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb5400b9f0 tx=0x7feb54008040 comp rx=0 tx=0).stop 2023-11-28T21:39:45.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 -- 172.21.15.12:0/2057308684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb64153f30 msgr2=0x7feb640cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb64153f30 0x7feb640cc3d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7feb5c01ce20 tx=0x7feb5c01bf90 comp rx=0 tx=0).stop 2023-11-28T21:39:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 -- 172.21.15.12:0/2057308684 shutdown_connections 2023-11-28T21:39:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7feb500664a0 0x7feb50068960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb64153f30 0x7feb640cc3d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb640ca6b0 0x7feb640c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 --2- 172.21.15.12:0/2057308684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6406bdd0 0x7feb641557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 -- 172.21.15.12:0/2057308684 >> 172.21.15.12:0/2057308684 conn(0x7feb64066530 msgr2=0x7feb64159280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 -- 172.21.15.12:0/2057308684 shutdown_connections 2023-11-28T21:39:45.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:45.257+0000 7feb6950f640 1 -- 172.21.15.12:0/2057308684 wait complete. 2023-11-28T21:39:45.283 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-11-28T21:39:47.283 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:39:53.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c289e6640 1 Processor -- start 2023-11-28T21:39:53.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c289e6640 1 -- start start 2023-11-28T21:39:53.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c289e6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4c240c52f0 0x7f4c240c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:53.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c289e6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2415e270 con 0x7f4c24154100 2023-11-28T21:39:53.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c289e6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c240c4e60 con 0x7f4c2415d320 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c289e6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2415d740 con 0x7f4c240c52f0 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c2259c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4c24154100 0x7f4c240c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41700/0 (socket says 172.21.15.12:41700) 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c2259c640 1 -- 172.21.15.12:0/585682031 learned_addr learned my addr 172.21.15.12:0/585682031 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c22d9d640 1 --2- 172.21.15.12:0/585682031 >> v2:172.21.15.144:3300/0 conn(0x7f4c240c52f0 0x7f4c240c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286915726 0 0) 0x7f4c2415e270 con 0x7f4c24154100 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c10003530 con 0x7f4c24154100 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1212867872 0 0) 0x7f4c10003530 con 0x7f4c24154100 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 >> v2:172.21.15.144:3300/0 conn(0x7f4c240c52f0 msgr2=0x7f4c240c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 --2- 172.21.15.12:0/585682031 >> v2:172.21.15.144:3300/0 conn(0x7f4c240c52f0 0x7f4c240c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 >> v1:172.21.15.144:6789/0 conn(0x7f4c2415d320 legacy=0x7f4c2406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:53.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c241544e0 con 0x7f4c24154100 2023-11-28T21:39:53.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f4c14001680 con 0x7f4c24154100 2023-11-28T21:39:53.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 >> v1:172.21.15.12:6789/0 conn(0x7f4c24154100 legacy=0x7f4c240c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:53.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10003a40 0x7f4c10003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:53.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c100047d0 0x7f4c10004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:53.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c100051e0 0x7f4c1000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:53.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2415d740 con 0x7f4c10003a40 2023-11-28T21:39:53.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1000c110 con 0x7f4c100047d0 2023-11-28T21:39:53.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.973+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1000c2d0 con 0x7f4c100051e0 2023-11-28T21:39:53.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c2259c640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10003a40 0x7f4c10003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:53.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c1bfff640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c100047d0 0x7f4c10004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:53.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c22d9d640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c100051e0 0x7f4c1000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:53.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c2259c640 1 -- 172.21.15.12:0/585682031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c100051e0 msgr2=0x7f4c1000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:39:53.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c2259c640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c100051e0 0x7f4c1000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:53.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c2259c640 1 -- 172.21.15.12:0/585682031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c100047d0 msgr2=0x7f4c10004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:39:53.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c2259c640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c100047d0 0x7f4c10004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:53.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c2259c640 1 -- 172.21.15.12:0/585682031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c10003530 con 0x7f4c10003a40 2023-11-28T21:39:53.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c22d9d640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c100051e0 0x7f4c1000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:39:53.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c22d9d640 1 -- 172.21.15.12:0/585682031 reap_dead start 2023-11-28T21:39:53.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c2259c640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10003a40 0x7f4c10003e60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400c0e0 tx=0x7f4c1401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f52aca85c42912d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:54.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14005ae0 con 0x7f4c10003a40 2023-11-28T21:39:54.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c14005c40 con 0x7f4c10003a40 2023-11-28T21:39:54.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.977+0000 7f4c21d9b640 1 -- 172.21.15.12:0/585682031 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c140a59c0 con 0x7f4c10003a40 2023-11-28T21:39:54.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 -- 172.21.15.12:0/585682031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10003a40 msgr2=0x7f4c10003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:54.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10003a40 0x7f4c10003e60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400c0e0 tx=0x7f4c1401cab0 comp rx=0 tx=0).stop 2023-11-28T21:39:54.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 -- 172.21.15.12:0/585682031 shutdown_connections 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 --2- 172.21.15.12:0/585682031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c10003a40 0x7f4c10003e60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 -- 172.21.15.12:0/585682031 >> 172.21.15.12:0/585682031 conn(0x7f4c2406c690 msgr2=0x7f4c24066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 -- 172.21.15.12:0/585682031 shutdown_connections 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 -- 172.21.15.12:0/585682031 wait complete. 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 Processor -- start 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 -- start start 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c10003a40 0x7f4c2415cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c24146730 0x7f4c24148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c241456f0 0x7f4c241f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c241473b0 con 0x7f4c24146730 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24147ef0 con 0x7f4c10003a40 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c289e6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24145b80 con 0x7f4c241456f0 2023-11-28T21:39:54.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c1bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c24146730 0x7f4c24148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c1bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c24146730 0x7f4c24148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40400/0 (socket says 172.21.15.12:40400) 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c1bfff640 1 -- 172.21.15.12:0/438010451 learned_addr learned my addr 172.21.15.12:0/438010451 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.981+0000 7f4c2259c640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c10003a40 0x7f4c2415cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c22d9d640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c241456f0 0x7f4c241f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c1bfff640 1 -- 172.21.15.12:0/438010451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c241456f0 msgr2=0x7f4c241f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c1bfff640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c241456f0 0x7f4c241f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c1bfff640 1 -- 172.21.15.12:0/438010451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c10003a40 msgr2=0x7f4c2415cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c1bfff640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c10003a40 0x7f4c2415cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c1bfff640 1 -- 172.21.15.12:0/438010451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c241f3fc0 con 0x7f4c24146730 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c1bfff640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c24146730 0x7f4c24148380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00f620 tx=0x7f4c0c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c1affd640 1 -- 172.21.15.12:0/438010451 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0052e0 con 0x7f4c24146730 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c1affd640 1 -- 172.21.15.12:0/438010451 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c005440 con 0x7f4c24146730 2023-11-28T21:39:54.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c289e6640 1 -- 172.21.15.12:0/438010451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c24146270 con 0x7f4c24146730 2023-11-28T21:39:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c1affd640 1 -- 172.21.15.12:0/438010451 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c003820 con 0x7f4c24146730 2023-11-28T21:39:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c289e6640 1 -- 172.21.15.12:0/438010451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c241478a0 con 0x7f4c24146730 2023-11-28T21:39:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c289e6640 1 -- 172.21.15.12:0/438010451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c241480a0 con 0x7f4c24146730 2023-11-28T21:39:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.985+0000 7f4c1affd640 1 -- 172.21.15.12:0/438010451 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4c0c003980 con 0x7f4c24146730 2023-11-28T21:39:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.989+0000 7f4c1affd640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c040664a0 0x7f4c04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.989+0000 7f4c1affd640 1 -- 172.21.15.12:0/438010451 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6883+0+0 (secure 0 0 0) 0x7f4c0c0840f0 con 0x7f4c24146730 2023-11-28T21:39:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.989+0000 7f4c2259c640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c040664a0 0x7f4c04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.989+0000 7f4c2259c640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c040664a0 0x7f4c04068960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4c140ab000 tx=0x7f4c140b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:54.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:53.993+0000 7f4c1affd640 1 -- 172.21.15.12:0/438010451 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4c241480a0 con 0x7f4c24146730 2023-11-28T21:39:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 -- 172.21.15.12:0/438010451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c241480a0 con 0x7f4c24146730 2023-11-28T21:39:54.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c1affd640 1 -- 172.21.15.12:0/438010451 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f4c241480a0 con 0x7f4c24146730 2023-11-28T21:39:54.188 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:39:54.188 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5307,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:39:54.188 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2023-11-28T21:39:54.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 -- 172.21.15.12:0/438010451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c040664a0 msgr2=0x7f4c04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:54.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c040664a0 0x7f4c04068960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4c140ab000 tx=0x7f4c140b1040 comp rx=0 tx=0).stop 2023-11-28T21:39:54.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 -- 172.21.15.12:0/438010451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c24146730 msgr2=0x7f4c24148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:54.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c24146730 0x7f4c24148380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00f620 tx=0x7f4c0c00fae0 comp rx=0 tx=0).stop 2023-11-28T21:39:54.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 -- 172.21.15.12:0/438010451 shutdown_connections 2023-11-28T21:39:54.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4c040664a0 0x7f4c04068960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:54.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c241456f0 0x7f4c241f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:54.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c24146730 0x7f4c24148380 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:54.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 --2- 172.21.15.12:0/438010451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c10003a40 0x7f4c2415cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:54.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 -- 172.21.15.12:0/438010451 >> 172.21.15.12:0/438010451 conn(0x7f4c2406c690 msgr2=0x7f4c24152fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:54.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 -- 172.21.15.12:0/438010451 shutdown_connections 2023-11-28T21:39:54.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:54.181+0000 7f4c289e6640 1 -- 172.21.15.12:0/438010451 wait complete. 2023-11-28T21:39:54.209 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-11-28T21:39:56.210 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:39:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.357+0000 7fb6ab4ea640 1 Processor -- start 2023-11-28T21:39:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.357+0000 7fb6ab4ea640 1 -- start start 2023-11-28T21:39:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.357+0000 7fb6ab4ea640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6a41500d0 0x7fb6a40c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.357+0000 7fb6ab4ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a4156af0 0x7fb6a40c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.357+0000 7fb6ab4ea640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb6a40c3500 0x7fb6a406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.357+0000 7fb6ab4ea640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a40c7170 con 0x7fb6a41500d0 2023-11-28T21:39:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.357+0000 7fb6ab4ea640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a40c6ce0 con 0x7fb6a40c3500 2023-11-28T21:39:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.357+0000 7fb6ab4ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a40c5df0 con 0x7fb6a4156af0 2023-11-28T21:39:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a9286640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6a41500d0 0x7fb6a40c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a9286640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb6a41500d0 0x7fb6a40c27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38284/0 (socket says 172.21.15.12:38284) 2023-11-28T21:39:56.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a9286640 1 -- 172.21.15.12:0/3061666198 learned_addr learned my addr 172.21.15.12:0/3061666198 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:39:56.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a8a85640 1 --2- 172.21.15.12:0/3061666198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a4156af0 0x7fb6a40c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:56.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a9a87640 1 --2- 172.21.15.12:0/3061666198 >> v2:172.21.15.144:3300/0 conn(0x7fb6a40c3500 0x7fb6a406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:56.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a9286640 1 -- 172.21.15.12:0/3061666198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a4156af0 msgr2=0x7fb6a40c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:39:56.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a9286640 1 --2- 172.21.15.12:0/3061666198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a4156af0 0x7fb6a40c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a9286640 1 -- 172.21.15.12:0/3061666198 >> v2:172.21.15.144:3300/0 conn(0x7fb6a40c3500 msgr2=0x7fb6a406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:39:56.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a9286640 1 --2- 172.21.15.12:0/3061666198 >> v2:172.21.15.144:3300/0 conn(0x7fb6a40c3500 0x7fb6a406d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a9286640 1 -- 172.21.15.12:0/3061666198 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a40c60d0 con 0x7fb6a41500d0 2023-11-28T21:39:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb6a9286640 1 --2- 172.21.15.12:0/3061666198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 0x7fb6a40c27c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4074360 tx=0x7fb6a001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d21c6d5f483bddd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb69bfff640 1 -- 172.21.15.12:0/3061666198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00ab600 con 0x7fb6a41500d0 2023-11-28T21:39:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb69bfff640 1 -- 172.21.15.12:0/3061666198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a00a7050 con 0x7fb6a41500d0 2023-11-28T21:39:56.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.361+0000 7fb69bfff640 1 -- 172.21.15.12:0/3061666198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00ab980 con 0x7fb6a41500d0 2023-11-28T21:39:56.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/3061666198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 msgr2=0x7fb6a40c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:56.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 --2- 172.21.15.12:0/3061666198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 0x7fb6a40c27c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4074360 tx=0x7fb6a001c440 comp rx=0 tx=0).stop 2023-11-28T21:39:56.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/3061666198 shutdown_connections 2023-11-28T21:39:56.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 --2- 172.21.15.12:0/3061666198 >> v2:172.21.15.144:3300/0 conn(0x7fb6a40c3500 0x7fb6a406d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 --2- 172.21.15.12:0/3061666198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a4156af0 0x7fb6a40c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 --2- 172.21.15.12:0/3061666198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 0x7fb6a40c27c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/3061666198 >> 172.21.15.12:0/3061666198 conn(0x7fb6a4066360 msgr2=0x7fb6a4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:56.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/3061666198 shutdown_connections 2023-11-28T21:39:56.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/3061666198 wait complete. 2023-11-28T21:39:56.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 Processor -- start 2023-11-28T21:39:56.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 -- start start 2023-11-28T21:39:56.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a40c3500 0x7fb6a41f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:56.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 0x7fb6a414aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:56.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6a4156af0 0x7fb6a414b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a4070bb0 con 0x7fb6a41500d0 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a41f65f0 con 0x7fb6a4156af0 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6ab4ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a41f6310 con 0x7fb6a40c3500 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6a8a85640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 0x7fb6a414aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6a8a85640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 0x7fb6a414aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38290/0 (socket says 172.21.15.12:38290) 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6a8a85640 1 -- 172.21.15.12:0/2037081132 learned_addr learned my addr 172.21.15.12:0/2037081132 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6a9286640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a40c3500 0x7fb6a41f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6a9a87640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6a4156af0 0x7fb6a414b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6a8a85640 1 -- 172.21.15.12:0/2037081132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a40c3500 msgr2=0x7fb6a41f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6a8a85640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a40c3500 0x7fb6a41f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6a8a85640 1 -- 172.21.15.12:0/2037081132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6a4156af0 msgr2=0x7fb6a414b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:56.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6a8a85640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6a4156af0 0x7fb6a414b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.365+0000 7fb6a8a85640 1 -- 172.21.15.12:0/2037081132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a00094d0 con 0x7fb6a41500d0 2023-11-28T21:39:56.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.369+0000 7fb6a8a85640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 0x7fb6a414aae0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb69400e930 tx=0x7fb69400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:56.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.369+0000 7fb699ffb640 1 -- 172.21.15.12:0/2037081132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694009690 con 0x7fb6a41500d0 2023-11-28T21:39:56.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.369+0000 7fb699ffb640 1 -- 172.21.15.12:0/2037081132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69400ca00 con 0x7fb6a41500d0 2023-11-28T21:39:56.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.369+0000 7fb699ffb640 1 -- 172.21.15.12:0/2037081132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69400cd70 con 0x7fb6a41500d0 2023-11-28T21:39:56.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.369+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/2037081132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a4158420 con 0x7fb6a41500d0 2023-11-28T21:39:56.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.369+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/2037081132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a4155a00 con 0x7fb6a41500d0 2023-11-28T21:39:56.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.369+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/2037081132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a4074f60 con 0x7fb6a41500d0 2023-11-28T21:39:56.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.369+0000 7fb699ffb640 1 -- 172.21.15.12:0/2037081132 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb694013470 con 0x7fb6a41500d0 2023-11-28T21:39:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.373+0000 7fb699ffb640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb688066470 0x7fb688068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.373+0000 7fb699ffb640 1 -- 172.21.15.12:0/2037081132 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6883+0+0 (secure 0 0 0) 0x7fb6940844e0 con 0x7fb6a41500d0 2023-11-28T21:39:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.373+0000 7fb6a9286640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb688066470 0x7fb688068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.373+0000 7fb6a9286640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb688066470 0x7fb688068930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00ad000 tx=0x7fb6a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:56.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.377+0000 7fb699ffb640 1 -- 172.21.15.12:0/2037081132 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb694051590 con 0x7fb6a41500d0 2023-11-28T21:39:56.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/2037081132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6a40c7320 con 0x7fb6a41500d0 2023-11-28T21:39:56.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb699ffb640 1 -- 172.21.15.12:0/2037081132 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fb694054bb0 con 0x7fb6a41500d0 2023-11-28T21:39:56.572 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:39:56.572 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5307,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:38:03.475148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_4261":{"gid":4261,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.144:6833/3053317078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3053317078},{"type":"v1","addr":"172.21.15.144:6833","nonce":3053317078}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:39:56.572 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 21 2023-11-28T21:39:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/2037081132 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb688066470 msgr2=0x7fb688068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb688066470 0x7fb688068930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00ad000 tx=0x7fb6a0008040 comp rx=0 tx=0).stop 2023-11-28T21:39:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/2037081132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 msgr2=0x7fb6a414aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 0x7fb6a414aae0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb69400e930 tx=0x7fb69400edf0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/2037081132 shutdown_connections 2023-11-28T21:39:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb688066470 0x7fb688068930 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6a4156af0 0x7fb6a414b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a41500d0 0x7fb6a414aae0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 --2- 172.21.15.12:0/2037081132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a40c3500 0x7fb6a41f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/2037081132 >> 172.21.15.12:0/2037081132 conn(0x7fb6a4066360 msgr2=0x7fb6a414f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:56.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.565+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/2037081132 shutdown_connections 2023-11-28T21:39:56.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:56.569+0000 7fb6ab4ea640 1 -- 172.21.15.12:0/2037081132 wait complete. 2023-11-28T21:39:56.593 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-11-28T21:39:58.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.741+0000 7f3b35316640 1 Processor -- start 2023-11-28T21:39:58.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.741+0000 7f3b35316640 1 -- start start 2023-11-28T21:39:58.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b35316640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3b30150c20 0x7f3b30156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b35316640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 0x7f3b30158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b35316640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b30067410 con 0x7f3b3006e300 2023-11-28T21:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b35316640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3006e090 con 0x7f3b30150c20 2023-11-28T21:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b35316640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30157f30 con 0x7f3b3006cf40 2023-11-28T21:39:58.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b2f7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 0x7f3b30158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b2effd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3b30150c20 0x7f3b30156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b27fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f3b3006e300 0x7f3b300c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:56548/0 (socket says 172.21.15.12:56548) 2023-11-28T21:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b2f7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 0x7f3b30158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:50822/0 (socket says 172.21.15.12:50822) 2023-11-28T21:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b27fff640 1 -- 172.21.15.12:0/1333241791 learned_addr learned my addr 172.21.15.12:0/1333241791 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b2effd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3b30150c20 0x7f3b30156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:57372/0 (socket says 172.21.15.12:57372) 2023-11-28T21:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b2effd640 1 -- 172.21.15.12:0/1333241791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 msgr2=0x7f3b30158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b2effd640 1 --2- 172.21.15.12:0/1333241791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 0x7f3b30158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b2effd640 1 -- 172.21.15.12:0/1333241791 >> v1:172.21.15.144:6789/0 conn(0x7f3b3006e300 legacy=0x7f3b300c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:58.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.745+0000 7f3b2effd640 1 -- 172.21.15.12:0/1333241791 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b30151000 con 0x7f3b30150c20 2023-11-28T21:39:58.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b2effd640 1 --2- 172.21.15.12:0/1333241791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b30150c20 0x7f3b30156910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3b200097c0 tx=0x7f3b2001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=882f01d0eb86d0a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:58.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b2e7fc640 1 -- 172.21.15.12:0/1333241791 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200ad640 con 0x7f3b30150c20 2023-11-28T21:39:58.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b2e7fc640 1 -- 172.21.15.12:0/1333241791 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b200ad7a0 con 0x7f3b30150c20 2023-11-28T21:39:58.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b2e7fc640 1 -- 172.21.15.12:0/1333241791 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200adb30 con 0x7f3b30150c20 2023-11-28T21:39:58.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b35316640 1 -- 172.21.15.12:0/1333241791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b30150c20 msgr2=0x7f3b30156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:58.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b35316640 1 --2- 172.21.15.12:0/1333241791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b30150c20 0x7f3b30156910 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3b200097c0 tx=0x7f3b2001c480 comp rx=0 tx=0).stop 2023-11-28T21:39:58.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b35316640 1 -- 172.21.15.12:0/1333241791 shutdown_connections 2023-11-28T21:39:58.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b35316640 1 --2- 172.21.15.12:0/1333241791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 0x7f3b30158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:58.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b35316640 1 --2- 172.21.15.12:0/1333241791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b30150c20 0x7f3b30156910 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:58.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b35316640 1 -- 172.21.15.12:0/1333241791 >> 172.21.15.12:0/1333241791 conn(0x7f3b30066360 msgr2=0x7f3b30066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:58.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b35316640 1 -- 172.21.15.12:0/1333241791 shutdown_connections 2023-11-28T21:39:58.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.749+0000 7f3b35316640 1 -- 172.21.15.12:0/1333241791 wait complete. 2023-11-28T21:39:58.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 Processor -- start 2023-11-28T21:39:58.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 -- start start 2023-11-28T21:39:58.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 0x7f3b3014b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b3006e300 0x7f3b301f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b30150c20 0x7f3b3014d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3014d8d0 con 0x7f3b3006e300 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3014ac80 con 0x7f3b30150c20 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3014af60 con 0x7f3b3006cf40 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b27fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b3006e300 0x7f3b301f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b27fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b3006e300 0x7f3b301f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38296/0 (socket says 172.21.15.12:38296) 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b27fff640 1 -- 172.21.15.12:0/492995598 learned_addr learned my addr 172.21.15.12:0/492995598 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b2effd640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 0x7f3b3014b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b2f7fe640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b30150c20 0x7f3b3014d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b27fff640 1 -- 172.21.15.12:0/492995598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 msgr2=0x7f3b3014b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b27fff640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 0x7f3b3014b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:58.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b27fff640 1 -- 172.21.15.12:0/492995598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b30150c20 msgr2=0x7f3b3014d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:58.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b27fff640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b30150c20 0x7f3b3014d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:58.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b27fff640 1 -- 172.21.15.12:0/492995598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b20009420 con 0x7f3b3006e300 2023-11-28T21:39:58.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b27fff640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b3006e300 0x7f3b301f3840 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800c430 tx=0x7f3b1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:58.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b277fe640 1 -- 172.21.15.12:0/492995598 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18005cf0 con 0x7f3b3006e300 2023-11-28T21:39:58.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b277fe640 1 -- 172.21.15.12:0/492995598 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b18005e50 con 0x7f3b3006e300 2023-11-28T21:39:58.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b277fe640 1 -- 172.21.15.12:0/492995598 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b180072f0 con 0x7f3b3006e300 2023-11-28T21:39:58.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 -- 172.21.15.12:0/492995598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3014b240 con 0x7f3b3006e300 2023-11-28T21:39:58.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 -- 172.21.15.12:0/492995598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b301f3320 con 0x7f3b3006e300 2023-11-28T21:39:58.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.753+0000 7f3b35316640 1 -- 172.21.15.12:0/492995598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b301f3540 con 0x7f3b3006e300 2023-11-28T21:39:58.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.757+0000 7f3b277fe640 1 -- 172.21.15.12:0/492995598 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3b18007450 con 0x7f3b3006e300 2023-11-28T21:39:58.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.757+0000 7f3b277fe640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3b10066470 0x7f3b10068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:58.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.757+0000 7f3b277fe640 1 -- 172.21.15.12:0/492995598 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f3b18083890 con 0x7f3b3006e300 2023-11-28T21:39:58.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.757+0000 7f3b2effd640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3b10066470 0x7f3b10068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:58.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.757+0000 7f3b2effd640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3b10066470 0x7f3b10068930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3b200af000 tx=0x7f3b200a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:58.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.761+0000 7f3b277fe640 1 -- 172.21.15.12:0/492995598 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3b18050910 con 0x7f3b3006e300 2023-11-28T21:39:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.949+0000 7f3b35316640 1 -- 172.21.15.12:0/492995598 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b30074f60 con 0x7f3b3006e300 2023-11-28T21:39:58.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.949+0000 7f3b277fe640 1 -- 172.21.15.12:0/492995598 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f3b18017090 con 0x7f3b3006e300 2023-11-28T21:39:58.957 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:39:58.957 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:39:58.381244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:39:58.957 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 22 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 -- 172.21.15.12:0/492995598 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3b10066470 msgr2=0x7f3b10068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3b10066470 0x7f3b10068930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3b200af000 tx=0x7f3b200a7070 comp rx=0 tx=0).stop 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 -- 172.21.15.12:0/492995598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b3006e300 msgr2=0x7f3b301f3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b3006e300 0x7f3b301f3840 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800c430 tx=0x7f3b1800c8f0 comp rx=0 tx=0).stop 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 -- 172.21.15.12:0/492995598 shutdown_connections 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3b10066470 0x7f3b10068930 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b30150c20 0x7f3b3014d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b3006e300 0x7f3b301f3840 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 --2- 172.21.15.12:0/492995598 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b3006cf40 0x7f3b3014b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 -- 172.21.15.12:0/492995598 >> 172.21.15.12:0/492995598 conn(0x7f3b30066360 msgr2=0x7f3b300c6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 -- 172.21.15.12:0/492995598 shutdown_connections 2023-11-28T21:39:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:58.953+0000 7f3b35316640 1 -- 172.21.15.12:0/492995598 wait complete. 2023-11-28T21:39:58.978 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2023-11-28T21:39:58.978 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:39:58.978 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:39:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.113+0000 7f1c2a4fe640 1 Processor -- start 2023-11-28T21:39:59.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.113+0000 7f1c2a4fe640 1 -- start start 2023-11-28T21:39:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c2a4fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1c24158b20 0x7f1c24156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c2a4fe640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c24067410 con 0x7f1c24150c20 2023-11-28T21:39:59.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c2a4fe640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2406e090 con 0x7f1c2406e300 2023-11-28T21:39:59.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c2a4fe640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c24157f30 con 0x7f1c24158b20 2023-11-28T21:39:59.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c28a9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1c24158b20 0x7f1c24156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:59.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c23fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1c24150c20 0x7f1c240c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40830/0 (socket says 172.21.15.12:40830) 2023-11-28T21:39:59.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c23fff640 1 -- 172.21.15.12:0/312662856 learned_addr learned my addr 172.21.15.12:0/312662856 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:39:59.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c28a9b640 1 -- 172.21.15.12:0/312662856 >> v1:172.21.15.144:6789/0 conn(0x7f1c2406e300 legacy=0x7f1c24158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:59.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c28a9b640 1 -- 172.21.15.12:0/312662856 >> v1:172.21.15.12:6789/0 conn(0x7f1c24150c20 legacy=0x7f1c240c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:59.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c28a9b640 1 -- 172.21.15.12:0/312662856 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c24151000 con 0x7f1c24158b20 2023-11-28T21:39:59.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.117+0000 7f1c22ffd640 1 -- 172.21.15.12:0/312662856 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259301453 0 0) 0x7f1c24067410 con 0x7f1c24150c20 2023-11-28T21:39:59.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c28a9b640 1 --2- 172.21.15.12:0/312662856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 0x7f1c24156910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009d90 tx=0x7f1c1c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4728889a4f9f9a1a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:59.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c22ffd640 1 -- 172.21.15.12:0/312662856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0a5920 con 0x7f1c24158b20 2023-11-28T21:39:59.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c22ffd640 1 -- 172.21.15.12:0/312662856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0a5a80 con 0x7f1c24158b20 2023-11-28T21:39:59.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c22ffd640 1 -- 172.21.15.12:0/312662856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0a5e10 con 0x7f1c24158b20 2023-11-28T21:39:59.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/312662856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 msgr2=0x7f1c24156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:59.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c2a4fe640 1 --2- 172.21.15.12:0/312662856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 0x7f1c24156910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009d90 tx=0x7f1c1c01c950 comp rx=0 tx=0).stop 2023-11-28T21:39:59.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/312662856 shutdown_connections 2023-11-28T21:39:59.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c2a4fe640 1 --2- 172.21.15.12:0/312662856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 0x7f1c24156910 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:59.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/312662856 >> 172.21.15.12:0/312662856 conn(0x7f1c24066360 msgr2=0x7f1c24066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:59.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/312662856 shutdown_connections 2023-11-28T21:39:59.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/312662856 wait complete. 2023-11-28T21:39:59.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c2a4fe640 1 Processor -- start 2023-11-28T21:39:59.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.121+0000 7f1c2a4fe640 1 -- start start 2023-11-28T21:39:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c2a4fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c2406e300 0x7f1c2414a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c2a4fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c24150c20 0x7f1c241488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c2a4fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 0x7f1c241f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c2a4fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c240ca520 con 0x7f1c24158b20 2023-11-28T21:39:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c2a4fe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24153b70 con 0x7f1c2406e300 2023-11-28T21:39:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c2a4fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2406ad70 con 0x7f1c24150c20 2023-11-28T21:39:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c28a9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 0x7f1c241f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c28a9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 0x7f1c241f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38322/0 (socket says 172.21.15.12:38322) 2023-11-28T21:39:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c28a9b640 1 -- 172.21.15.12:0/3522117629 learned_addr learned my addr 172.21.15.12:0/3522117629 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:39:59.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c23fff640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c2406e300 0x7f1c2414a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c237fe640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c24150c20 0x7f1c241488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c28a9b640 1 -- 172.21.15.12:0/3522117629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c24150c20 msgr2=0x7f1c241488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c28a9b640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c24150c20 0x7f1c241488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c28a9b640 1 -- 172.21.15.12:0/3522117629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c2406e300 msgr2=0x7f1c2414a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c28a9b640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c2406e300 0x7f1c2414a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c28a9b640 1 -- 172.21.15.12:0/3522117629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c009a70 con 0x7f1c24158b20 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c237fe640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c24150c20 0x7f1c241488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c28a9b640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 0x7f1c241f3020 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009d90 tx=0x7f1c1c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c20ff9640 1 -- 172.21.15.12:0/3522117629 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0ade40 con 0x7f1c24158b20 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c20ff9640 1 -- 172.21.15.12:0/3522117629 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0b7410 con 0x7f1c24158b20 2023-11-28T21:39:59.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/3522117629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c241f3560 con 0x7f1c24158b20 2023-11-28T21:39:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c20ff9640 1 -- 172.21.15.12:0/3522117629 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0b77c0 con 0x7f1c24158b20 2023-11-28T21:39:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/3522117629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2406b280 con 0x7f1c24158b20 2023-11-28T21:39:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.125+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/3522117629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c24074f60 con 0x7f1c24158b20 2023-11-28T21:39:59.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.129+0000 7f1c20ff9640 1 -- 172.21.15.12:0/3522117629 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1c1c0b4050 con 0x7f1c24158b20 2023-11-28T21:39:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.129+0000 7f1c20ff9640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c10066470 0x7f1c10068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:39:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.129+0000 7f1c20ff9640 1 -- 172.21.15.12:0/3522117629 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f1c1c11f210 con 0x7f1c24158b20 2023-11-28T21:39:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.129+0000 7f1c23fff640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c10066470 0x7f1c10068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:39:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.129+0000 7f1c23fff640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c10066470 0x7f1c10068930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c14000f70 tx=0x7f1c14007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:39:59.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.133+0000 7f1c20ff9640 1 -- 172.21.15.12:0/3522117629 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1c1c0ec2a0 con 0x7f1c24158b20 2023-11-28T21:39:59.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.321+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/3522117629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c241f3560 con 0x7f1c24158b20 2023-11-28T21:39:59.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.321+0000 7f1c20ff9640 1 -- 172.21.15.12:0/3522117629 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f1c241f3560 con 0x7f1c24158b20 2023-11-28T21:39:59.328 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:39:59.328 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:39:58.381244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:39:59.328 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 22 2023-11-28T21:39:59.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/3522117629 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c10066470 msgr2=0x7f1c10068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:59.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c10066470 0x7f1c10068930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1c14000f70 tx=0x7f1c14007240 comp rx=0 tx=0).stop 2023-11-28T21:39:59.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/3522117629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 msgr2=0x7f1c241f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:39:59.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 0x7f1c241f3020 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009d90 tx=0x7f1c1c0196f0 comp rx=0 tx=0).stop 2023-11-28T21:39:59.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/3522117629 shutdown_connections 2023-11-28T21:39:59.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1c10066470 0x7f1c10068930 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:59.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1c24158b20 0x7f1c241f3020 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:59.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c24150c20 0x7f1c241488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:59.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 --2- 172.21.15.12:0/3522117629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c2406e300 0x7f1c2414a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:39:59.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/3522117629 >> 172.21.15.12:0/3522117629 conn(0x7f1c24066360 msgr2=0x7f1c2414b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:39:59.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/3522117629 shutdown_connections 2023-11-28T21:39:59.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:39:59.325+0000 7f1c2a4fe640 1 -- 172.21.15.12:0/3522117629 wait complete. 2023-11-28T21:39:59.350 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 22, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 22, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5307": { "addr": "172.21.15.12:6838/3741155090", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3741155090, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 3741155090, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5307, "incarnation": 22, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 39, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:39:58.381244+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5307 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6839/3009112102", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3009112102, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3009112102, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5232, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:40:01.351 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:01.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.501+0000 7f462a674640 1 Processor -- start 2023-11-28T21:40:01.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f462a674640 1 -- start start 2023-11-28T21:40:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f462a674640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f462415a980 0x7f46240b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f462a674640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f46240b4010 0x7f46240cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f462a674640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462415d390 con 0x7f4624153fb0 2023-11-28T21:40:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f462a674640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46240b2b60 con 0x7f46240b4010 2023-11-28T21:40:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f462a674640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462415e1b0 con 0x7f462415a980 2023-11-28T21:40:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4629672640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4624153fb0 0x7f46240ca4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40846/0 (socket says 172.21.15.12:40846) 2023-11-28T21:40:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4629672640 1 -- 172.21.15.12:0/2149453612 learned_addr learned my addr 172.21.15.12:0/2149453612 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4628e71640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f462415a980 0x7f46240b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4629e73640 1 --2- 172.21.15.12:0/2149453612 >> v2:172.21.15.144:3300/0 conn(0x7f46240b4010 0x7f46240cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015894050 0 0) 0x7f462415d390 con 0x7f4624153fb0 2023-11-28T21:40:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4610003530 con 0x7f4624153fb0 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 703116084 0 0) 0x7f4610003530 con 0x7f4624153fb0 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f462415a980 msgr2=0x7f46240b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f462415a980 0x7f46240b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 >> v2:172.21.15.144:3300/0 conn(0x7f46240b4010 msgr2=0x7f46240cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 --2- 172.21.15.12:0/2149453612 >> v2:172.21.15.144:3300/0 conn(0x7f46240b4010 0x7f46240cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462415d630 con 0x7f4624153fb0 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f4620001680 con 0x7f4624153fb0 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 >> v1:172.21.15.12:6789/0 conn(0x7f4624153fb0 legacy=0x7f46240ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f462415a980 0x7f4610003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4610003f00 0x7f46100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4610004d10 0x7f4610005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240b2b60 con 0x7f4610004d10 2023-11-28T21:40:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462415e1b0 con 0x7f4610003f00 2023-11-28T21:40:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461000b990 con 0x7f462415a980 2023-11-28T21:40:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4629672640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f462415a980 0x7f4610003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4629e73640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4610004d10 0x7f4610005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4628e71640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4610003f00 0x7f46100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4629e73640 1 -- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f462415a980 msgr2=0x7f4610003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4629e73640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f462415a980 0x7f4610003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4629e73640 1 -- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4610003f00 msgr2=0x7f46100047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4629e73640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4610003f00 0x7f46100047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.505+0000 7f4629e73640 1 -- 172.21.15.12:0/2149453612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610003530 con 0x7f4610004d10 2023-11-28T21:40:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f4629e73640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4610004d10 0x7f4610005190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f461c009fc0 tx=0x7f461c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6b972c0578052e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0a5cb0 con 0x7f4610004d10 2023-11-28T21:40:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c00f040 con 0x7f4610004d10 2023-11-28T21:40:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f461bfff640 1 -- 172.21.15.12:0/2149453612 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0ad680 con 0x7f4610004d10 2023-11-28T21:40:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f462a674640 1 -- 172.21.15.12:0/2149453612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4610004d10 msgr2=0x7f4610005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f462a674640 1 --2- 172.21.15.12:0/2149453612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4610004d10 0x7f4610005190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f461c009fc0 tx=0x7f461c01cc40 comp rx=0 tx=0).stop 2023-11-28T21:40:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f4629e73640 1 -- 172.21.15.12:0/2149453612 reap_dead start 2023-11-28T21:40:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f462a674640 1 -- 172.21.15.12:0/2149453612 shutdown_connections 2023-11-28T21:40:01.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f462a674640 1 -- 172.21.15.12:0/2149453612 >> 172.21.15.12:0/2149453612 conn(0x7f4624071ce0 msgr2=0x7f46240720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:01.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f462a674640 1 -- 172.21.15.12:0/2149453612 shutdown_connections 2023-11-28T21:40:01.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.509+0000 7f462a674640 1 -- 172.21.15.12:0/2149453612 wait complete. 2023-11-28T21:40:01.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f462a674640 1 Processor -- start 2023-11-28T21:40:01.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f462a674640 1 -- start start 2023-11-28T21:40:01.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f462a674640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4610004d10 0x7f4624156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:01.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f462a674640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46240cf280 0x7f46241547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:01.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f462a674640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46240cf6c0 0x7f4624154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f462a674640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46241f02d0 con 0x7f46240cf6c0 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f462a674640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46241ef3d0 con 0x7f46240cf280 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f462a674640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46241ef650 con 0x7f4610004d10 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4629e73640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46240cf6c0 0x7f4624154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4629e73640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46240cf6c0 0x7f4624154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38338/0 (socket says 172.21.15.12:38338) 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4629e73640 1 -- 172.21.15.12:0/1084277404 learned_addr learned my addr 172.21.15.12:0/1084277404 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4629672640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4610004d10 0x7f4624156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4628e71640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46240cf280 0x7f46241547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4629e73640 1 -- 172.21.15.12:0/1084277404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4610004d10 msgr2=0x7f4624156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4629e73640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4610004d10 0x7f4624156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4629e73640 1 -- 172.21.15.12:0/1084277404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46240cf280 msgr2=0x7f46241547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4629e73640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46240cf280 0x7f46241547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:01.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4629e73640 1 -- 172.21.15.12:0/1084277404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46241ef930 con 0x7f46240cf6c0 2023-11-28T21:40:01.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4628e71640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46240cf280 0x7f46241547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:01.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4629e73640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46240cf6c0 0x7f4624154d10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f461c0b0040 tx=0x7f461c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4619ffb640 1 -- 172.21.15.12:0/1084277404 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0039a0 con 0x7f46240cf6c0 2023-11-28T21:40:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4619ffb640 1 -- 172.21.15.12:0/1084277404 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c003b00 con 0x7f46240cf6c0 2023-11-28T21:40:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f462a674640 1 -- 172.21.15.12:0/1084277404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46241efc10 con 0x7f46240cf6c0 2023-11-28T21:40:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.513+0000 7f4619ffb640 1 -- 172.21.15.12:0/1084277404 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c003e70 con 0x7f46240cf6c0 2023-11-28T21:40:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.517+0000 7f462a674640 1 -- 172.21.15.12:0/1084277404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462415ceb0 con 0x7f46240cf6c0 2023-11-28T21:40:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.517+0000 7f462a674640 1 -- 172.21.15.12:0/1084277404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46240c6600 con 0x7f46240cf6c0 2023-11-28T21:40:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.517+0000 7f4619ffb640 1 -- 172.21.15.12:0/1084277404 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f461c0b4040 con 0x7f46240cf6c0 2023-11-28T21:40:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.517+0000 7f4619ffb640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46080664a0 0x7f4608068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.517+0000 7f4619ffb640 1 -- 172.21.15.12:0/1084277404 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f461c11e390 con 0x7f46240cf6c0 2023-11-28T21:40:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.517+0000 7f4629672640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46080664a0 0x7f4608068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:01.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.517+0000 7f4629672640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46080664a0 0x7f4608068960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4620000c10 tx=0x7f4620007900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:01.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.521+0000 7f4619ffb640 1 -- 172.21.15.12:0/1084277404 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f461c0eb2f0 con 0x7f46240cf6c0 2023-11-28T21:40:01.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.713+0000 7f462a674640 1 -- 172.21.15.12:0/1084277404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4624074f60 con 0x7f46240cf6c0 2023-11-28T21:40:01.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.713+0000 7f4619ffb640 1 -- 172.21.15.12:0/1084277404 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f461c0ed220 con 0x7f46240cf6c0 2023-11-28T21:40:01.721 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:01.721 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:39:58.381244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:01.721 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 22 2023-11-28T21:40:01.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 -- 172.21.15.12:0/1084277404 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46080664a0 msgr2=0x7f4608068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:01.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46080664a0 0x7f4608068960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4620000c10 tx=0x7f4620007900 comp rx=0 tx=0).stop 2023-11-28T21:40:01.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 -- 172.21.15.12:0/1084277404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46240cf6c0 msgr2=0x7f4624154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:01.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46240cf6c0 0x7f4624154d10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f461c0b0040 tx=0x7f461c009750 comp rx=0 tx=0).stop 2023-11-28T21:40:01.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 -- 172.21.15.12:0/1084277404 shutdown_connections 2023-11-28T21:40:01.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46080664a0 0x7f4608068960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:01.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46240cf6c0 0x7f4624154d10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:01.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46240cf280 0x7f46241547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:01.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 --2- 172.21.15.12:0/1084277404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4610004d10 0x7f4624156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:01.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 -- 172.21.15.12:0/1084277404 >> 172.21.15.12:0/1084277404 conn(0x7f4624071ce0 msgr2=0x7f4624152790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:01.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 -- 172.21.15.12:0/1084277404 shutdown_connections 2023-11-28T21:40:01.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:01.717+0000 7f462a674640 1 -- 172.21.15.12:0/1084277404 wait complete. 2023-11-28T21:40:01.742 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:40:03.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:03.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.885+0000 7f4f2e0dc640 1 Processor -- start 2023-11-28T21:40:03.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.885+0000 7f4f2e0dc640 1 -- start start 2023-11-28T21:40:03.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2e0dc640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4f2815a980 0x7f4f280b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:03.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2e0dc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f280b4010 0x7f4f280cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:03.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2e0dc640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f280b44a0 con 0x7f4f28153fb0 2023-11-28T21:40:03.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2e0dc640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f280b2b60 con 0x7f4f2815a980 2023-11-28T21:40:03.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2e0dc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2815e1b0 con 0x7f4f280b4010 2023-11-28T21:40:03.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2d0da640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4f28153fb0 0x7f4f280ca4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40854/0 (socket says 172.21.15.12:40854) 2023-11-28T21:40:03.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2d0da640 1 -- 172.21.15.12:0/4009884885 learned_addr learned my addr 172.21.15.12:0/4009884885 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:03.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2d8db640 1 --2- 172.21.15.12:0/4009884885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f280b4010 0x7f4f280cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:03.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2c8d9640 1 --2- 172.21.15.12:0/4009884885 >> v2:172.21.15.144:3300/0 conn(0x7f4f2815a980 0x7f4f280b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:03.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f1ffff640 1 -- 172.21.15.12:0/4009884885 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826694868 0 0) 0x7f4f280b44a0 con 0x7f4f28153fb0 2023-11-28T21:40:03.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f1ffff640 1 -- 172.21.15.12:0/4009884885 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f14003530 con 0x7f4f28153fb0 2023-11-28T21:40:03.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2c8d9640 1 -- 172.21.15.12:0/4009884885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f280b4010 msgr2=0x7f4f280cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:03.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2c8d9640 1 --2- 172.21.15.12:0/4009884885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f280b4010 0x7f4f280cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:03.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2c8d9640 1 -- 172.21.15.12:0/4009884885 >> v1:172.21.15.12:6789/0 conn(0x7f4f28153fb0 legacy=0x7f4f280ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:03.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.889+0000 7f4f2c8d9640 1 -- 172.21.15.12:0/4009884885 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2815d390 con 0x7f4f2815a980 2023-11-28T21:40:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f2c8d9640 1 --2- 172.21.15.12:0/4009884885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f2815a980 0x7f4f280b3ad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f180097f0 tx=0x7f4f1801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fec8b80c5c3b8b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f1ffff640 1 -- 172.21.15.12:0/4009884885 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f180ab650 con 0x7f4f2815a980 2023-11-28T21:40:03.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f1ffff640 1 -- 172.21.15.12:0/4009884885 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f180ab7b0 con 0x7f4f2815a980 2023-11-28T21:40:03.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f1ffff640 1 -- 172.21.15.12:0/4009884885 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f180abb40 con 0x7f4f2815a980 2023-11-28T21:40:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4009884885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f2815a980 msgr2=0x7f4f280b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f2e0dc640 1 --2- 172.21.15.12:0/4009884885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f2815a980 0x7f4f280b3ad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f180097f0 tx=0x7f4f1801c450 comp rx=0 tx=0).stop 2023-11-28T21:40:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4009884885 shutdown_connections 2023-11-28T21:40:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f2e0dc640 1 --2- 172.21.15.12:0/4009884885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f280b4010 0x7f4f280cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f2e0dc640 1 --2- 172.21.15.12:0/4009884885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f2815a980 0x7f4f280b3ad0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4009884885 >> 172.21.15.12:0/4009884885 conn(0x7f4f28071ce0 msgr2=0x7f4f280720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4009884885 shutdown_connections 2023-11-28T21:40:03.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.893+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4009884885 wait complete. 2023-11-28T21:40:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.897+0000 7f4f2e0dc640 1 Processor -- start 2023-11-28T21:40:03.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.897+0000 7f4f2e0dc640 1 -- start start 2023-11-28T21:40:03.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.897+0000 7f4f2e0dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f280b4010 0x7f4f2815c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:03.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.897+0000 7f4f2e0dc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f28153fb0 0x7f4f280ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:03.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.897+0000 7f4f2e0dc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2815a980 0x7f4f281450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:03.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.897+0000 7f4f2e0dc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f281f6a70 con 0x7f4f2815a980 2023-11-28T21:40:03.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.897+0000 7f4f2e0dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2815b900 con 0x7f4f280b4010 2023-11-28T21:40:03.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.897+0000 7f4f2e0dc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28159a10 con 0x7f4f28153fb0 2023-11-28T21:40:03.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2d8db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2815a980 0x7f4f281450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:03.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2c8d9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f28153fb0 0x7f4f280ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:03.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2d8db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2815a980 0x7f4f281450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38354/0 (socket says 172.21.15.12:38354) 2023-11-28T21:40:03.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2d8db640 1 -- 172.21.15.12:0/4116606439 learned_addr learned my addr 172.21.15.12:0/4116606439 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:03.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2d0da640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f280b4010 0x7f4f2815c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:03.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2d8db640 1 -- 172.21.15.12:0/4116606439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f28153fb0 msgr2=0x7f4f280ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:03.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2d8db640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f28153fb0 0x7f4f280ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:03.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2d8db640 1 -- 172.21.15.12:0/4116606439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f280b4010 msgr2=0x7f4f2815c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:03.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2d8db640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f280b4010 0x7f4f2815c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:03.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2d8db640 1 -- 172.21.15.12:0/4116606439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18009420 con 0x7f4f2815a980 2023-11-28T21:40:03.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2d8db640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2815a980 0x7f4f281450a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000efc0 tx=0x7f4f2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:03.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f1dffb640 1 -- 172.21.15.12:0/4116606439 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f20014480 con 0x7f4f2815a980 2023-11-28T21:40:03.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f1dffb640 1 -- 172.21.15.12:0/4116606439 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2000f040 con 0x7f4f2815a980 2023-11-28T21:40:03.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4116606439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f280cf070 con 0x7f4f2815a980 2023-11-28T21:40:03.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f1dffb640 1 -- 172.21.15.12:0/4116606439 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f20014820 con 0x7f4f2815a980 2023-11-28T21:40:03.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4116606439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f281f6140 con 0x7f4f2815a980 2023-11-28T21:40:03.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.901+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4116606439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f281f6c20 con 0x7f4f2815a980 2023-11-28T21:40:03.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.905+0000 7f4f1dffb640 1 -- 172.21.15.12:0/4116606439 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4f20014980 con 0x7f4f2815a980 2023-11-28T21:40:03.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.905+0000 7f4f1dffb640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4f0c0664a0 0x7f4f0c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:03.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.905+0000 7f4f1dffb640 1 -- 172.21.15.12:0/4116606439 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f4f20084da0 con 0x7f4f2815a980 2023-11-28T21:40:03.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.905+0000 7f4f2d0da640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4f0c0664a0 0x7f4f0c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:03.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.905+0000 7f4f2d0da640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4f0c0664a0 0x7f4f0c068960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f240011f0 tx=0x7f4f24007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:03.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:03.909+0000 7f4f1dffb640 1 -- 172.21.15.12:0/4116606439 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4f281f6c20 con 0x7f4f2815a980 2023-11-28T21:40:04.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.105+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4116606439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f281f6c20 con 0x7f4f2815a980 2023-11-28T21:40:04.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.105+0000 7f4f1dffb640 1 -- 172.21.15.12:0/4116606439 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f4f281f6c20 con 0x7f4f2815a980 2023-11-28T21:40:04.113 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:04.114 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:39:58.381244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:04.114 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 22 2023-11-28T21:40:04.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4116606439 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4f0c0664a0 msgr2=0x7f4f0c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:04.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4f0c0664a0 0x7f4f0c068960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f240011f0 tx=0x7f4f24007d40 comp rx=0 tx=0).stop 2023-11-28T21:40:04.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4116606439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2815a980 msgr2=0x7f4f281450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:04.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2815a980 0x7f4f281450a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000efc0 tx=0x7f4f2000c490 comp rx=0 tx=0).stop 2023-11-28T21:40:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4116606439 shutdown_connections 2023-11-28T21:40:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4f0c0664a0 0x7f4f0c068960 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2815a980 0x7f4f281450a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f28153fb0 0x7f4f280ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 --2- 172.21.15.12:0/4116606439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f280b4010 0x7f4f2815c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4116606439 >> 172.21.15.12:0/4116606439 conn(0x7f4f28071ce0 msgr2=0x7f4f281532d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4116606439 shutdown_connections 2023-11-28T21:40:04.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:04.109+0000 7f4f2e0dc640 1 -- 172.21.15.12:0/4116606439 wait complete. 2023-11-28T21:40:04.136 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:40:06.137 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:06.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.281+0000 7f63c7972640 1 Processor -- start 2023-11-28T21:40:06.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c7972640 1 -- start start 2023-11-28T21:40:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c7972640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f63c0151ed0 0x7f63c00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c7972640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 0x7f63c0072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c7972640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c0157b70 con 0x7f63c0151af0 2023-11-28T21:40:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c7972640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c00707f0 con 0x7f63c0151ed0 2023-11-28T21:40:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c7972640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00c7b90 con 0x7f63c006e300 2023-11-28T21:40:06.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c4f0d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f63c0151ed0 0x7f63c00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:06.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c570e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f63c0151af0 0x7f63c00c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48692/0 (socket says 172.21.15.12:48692) 2023-11-28T21:40:06.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c570e640 1 -- 172.21.15.12:0/2090107044 learned_addr learned my addr 172.21.15.12:0/2090107044 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:06.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c5f0f640 1 --2- 172.21.15.12:0/2090107044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 0x7f63c0072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:06.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c4f0d640 1 -- 172.21.15.12:0/2090107044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 msgr2=0x7f63c0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:06.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c4f0d640 1 --2- 172.21.15.12:0/2090107044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 0x7f63c0072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:06.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c4f0d640 1 -- 172.21.15.12:0/2090107044 >> v1:172.21.15.12:6789/0 conn(0x7f63c0151af0 legacy=0x7f63c00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:06.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63b7fff640 1 -- 172.21.15.12:0/2090107044 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804992575 0 0) 0x7f63c0157b70 con 0x7f63c0151af0 2023-11-28T21:40:06.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c4f0d640 1 -- 172.21.15.12:0/2090107044 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c0157e50 con 0x7f63c0151ed0 2023-11-28T21:40:06.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.285+0000 7f63c5f0f640 1 --2- 172.21.15.12:0/2090107044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 0x7f63c0072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:40:06.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c4f0d640 1 --2- 172.21.15.12:0/2090107044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c0151ed0 0x7f63c00c7650 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f63b00097c0 tx=0x7f63b001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2832240046eb1487 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:06.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63b7fff640 1 -- 172.21.15.12:0/2090107044 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00ab600 con 0x7f63c0151ed0 2023-11-28T21:40:06.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63b7fff640 1 -- 172.21.15.12:0/2090107044 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b00a7050 con 0x7f63c0151ed0 2023-11-28T21:40:06.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63b7fff640 1 -- 172.21.15.12:0/2090107044 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00ab980 con 0x7f63c0151ed0 2023-11-28T21:40:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c7972640 1 -- 172.21.15.12:0/2090107044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c0151ed0 msgr2=0x7f63c00c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c7972640 1 --2- 172.21.15.12:0/2090107044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c0151ed0 0x7f63c00c7650 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f63b00097c0 tx=0x7f63b001c480 comp rx=0 tx=0).stop 2023-11-28T21:40:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c7972640 1 -- 172.21.15.12:0/2090107044 shutdown_connections 2023-11-28T21:40:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c7972640 1 --2- 172.21.15.12:0/2090107044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 0x7f63c0072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c7972640 1 --2- 172.21.15.12:0/2090107044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c0151ed0 0x7f63c00c7650 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c7972640 1 -- 172.21.15.12:0/2090107044 >> 172.21.15.12:0/2090107044 conn(0x7f63c0066360 msgr2=0x7f63c0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c7972640 1 -- 172.21.15.12:0/2090107044 shutdown_connections 2023-11-28T21:40:06.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c7972640 1 -- 172.21.15.12:0/2090107044 wait complete. 2023-11-28T21:40:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c7972640 1 Processor -- start 2023-11-28T21:40:06.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.289+0000 7f63c7972640 1 -- start start 2023-11-28T21:40:06.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c7972640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 0x7f63c013a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:06.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c7972640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63c0151af0 0x7f63c014c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:06.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c7972640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c014bda0 0x7f63c014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:06.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c7972640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c01f3490 con 0x7f63c014bda0 2023-11-28T21:40:06.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c7972640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c01f3960 con 0x7f63c0151af0 2023-11-28T21:40:06.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c7972640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c01f36c0 con 0x7f63c006e300 2023-11-28T21:40:06.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c5f0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c014bda0 0x7f63c014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:06.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c5f0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c014bda0 0x7f63c014cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47054/0 (socket says 172.21.15.12:47054) 2023-11-28T21:40:06.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c5f0f640 1 -- 172.21.15.12:0/294017414 learned_addr learned my addr 172.21.15.12:0/294017414 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:06.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c570e640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 0x7f63c013a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:06.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c4f0d640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63c0151af0 0x7f63c014c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:06.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c5f0f640 1 -- 172.21.15.12:0/294017414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 msgr2=0x7f63c013a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:06.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c5f0f640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 0x7f63c013a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:06.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c5f0f640 1 -- 172.21.15.12:0/294017414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63c0151af0 msgr2=0x7f63c014c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:06.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c5f0f640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63c0151af0 0x7f63c014c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:06.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c5f0f640 1 -- 172.21.15.12:0/294017414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b0009420 con 0x7f63c014bda0 2023-11-28T21:40:06.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c4f0d640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63c0151af0 0x7f63c014c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:06.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c5f0f640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c014bda0 0x7f63c014cec0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f63b800ce80 tx=0x7f63b800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:06.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63b5ffb640 1 -- 172.21.15.12:0/294017414 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b800d840 con 0x7f63c014bda0 2023-11-28T21:40:06.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63b5ffb640 1 -- 172.21.15.12:0/294017414 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b800d9e0 con 0x7f63c014bda0 2023-11-28T21:40:06.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63b5ffb640 1 -- 172.21.15.12:0/294017414 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b800dd90 con 0x7f63c014bda0 2023-11-28T21:40:06.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c7972640 1 -- 172.21.15.12:0/294017414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c014d770 con 0x7f63c014bda0 2023-11-28T21:40:06.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c7972640 1 -- 172.21.15.12:0/294017414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c014dd30 con 0x7f63c014bda0 2023-11-28T21:40:06.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.293+0000 7f63c7972640 1 -- 172.21.15.12:0/294017414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c0074f60 con 0x7f63c014bda0 2023-11-28T21:40:06.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.297+0000 7f63b5ffb640 1 -- 172.21.15.12:0/294017414 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f63b8015440 con 0x7f63c014bda0 2023-11-28T21:40:06.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.297+0000 7f63b5ffb640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63a4066470 0x7f63a4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:06.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.297+0000 7f63b5ffb640 1 -- 172.21.15.12:0/294017414 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f63b8084bf0 con 0x7f63c014bda0 2023-11-28T21:40:06.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.297+0000 7f63c570e640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63a4066470 0x7f63a4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:06.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.297+0000 7f63c570e640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63a4066470 0x7f63a4068930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f63bc000f70 tx=0x7f63bc007b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:06.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.301+0000 7f63b5ffb640 1 -- 172.21.15.12:0/294017414 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f63b8051d00 con 0x7f63c014bda0 2023-11-28T21:40:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.489+0000 7f63c7972640 1 -- 172.21.15.12:0/294017414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63c0146650 con 0x7f63c014bda0 2023-11-28T21:40:06.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.489+0000 7f63b5ffb640 1 -- 172.21.15.12:0/294017414 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5718 (secure 0 0 0) 0x7f63b801a020 con 0x7f63c014bda0 2023-11-28T21:40:06.498 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:06.498 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:05.415073+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:06.498 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 23 2023-11-28T21:40:06.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 -- 172.21.15.12:0/294017414 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63a4066470 msgr2=0x7f63a4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63a4066470 0x7f63a4068930 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f63bc000f70 tx=0x7f63bc007b60 comp rx=0 tx=0).stop 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 -- 172.21.15.12:0/294017414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c014bda0 msgr2=0x7f63c014cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c014bda0 0x7f63c014cec0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f63b800ce80 tx=0x7f63b800a710 comp rx=0 tx=0).stop 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 -- 172.21.15.12:0/294017414 shutdown_connections 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63a4066470 0x7f63a4068930 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63c014bda0 0x7f63c014cec0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63c0151af0 0x7f63c014c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 --2- 172.21.15.12:0/294017414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63c006e300 0x7f63c013a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 -- 172.21.15.12:0/294017414 >> 172.21.15.12:0/294017414 conn(0x7f63c0066360 msgr2=0x7f63c0146e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 -- 172.21.15.12:0/294017414 shutdown_connections 2023-11-28T21:40:06.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:06.493+0000 7f63c7972640 1 -- 172.21.15.12:0/294017414 wait complete. 2023-11-28T21:40:06.519 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:40:07.214 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:40:07.218 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:40:08.521 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:08.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.669+0000 7fbc223c4640 1 Processor -- start 2023-11-28T21:40:08.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.669+0000 7fbc223c4640 1 -- start start 2023-11-28T21:40:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.669+0000 7fbc223c4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc1c151ed0 0x7fbc1c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.669+0000 7fbc223c4640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1c157b70 con 0x7fbc1c06e300 2023-11-28T21:40:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.669+0000 7fbc223c4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1c157e50 con 0x7fbc1c151af0 2023-11-28T21:40:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.669+0000 7fbc223c4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c0707f0 con 0x7fbc1c151ed0 2023-11-28T21:40:08.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.669+0000 7fbc20961640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbc1c06e300 0x7fbc1c072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48694/0 (socket says 172.21.15.12:48694) 2023-11-28T21:40:08.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.669+0000 7fbc20961640 1 -- 172.21.15.12:0/1049793479 learned_addr learned my addr 172.21.15.12:0/1049793479 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:08.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1b7fe640 1 --2- 172.21.15.12:0/1049793479 >> v2:172.21.15.144:3300/0 conn(0x7fbc1c151ed0 0x7fbc1c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:08.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203448913 0 0) 0x7fbc1c157b70 con 0x7fbc1c06e300 2023-11-28T21:40:08.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc04003530 con 0x7fbc1c06e300 2023-11-28T21:40:08.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2600829580 0 0) 0x7fbc04003530 con 0x7fbc1c06e300 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 >> v2:172.21.15.144:3300/0 conn(0x7fbc1c151ed0 msgr2=0x7fbc1c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 --2- 172.21.15.12:0/1049793479 >> v2:172.21.15.144:3300/0 conn(0x7fbc1c151ed0 0x7fbc1c0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 >> v1:172.21.15.144:6789/0 conn(0x7fbc1c151af0 legacy=0x7fbc1c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c158130 con 0x7fbc1c06e300 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fbc14001a00 con 0x7fbc1c06e300 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 >> v1:172.21.15.12:6789/0 conn(0x7fbc1c06e300 legacy=0x7fbc1c072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc04003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc040047d0 0x7fbc04004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc040051e0 0x7fbc0400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0707f0 con 0x7fbc04003a40 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0400c110 con 0x7fbc040051e0 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0400c2d0 con 0x7fbc040047d0 2023-11-28T21:40:08.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1bfff640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc04003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:08.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc20961640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc040051e0 0x7fbc0400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:08.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1b7fe640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc040047d0 0x7fbc04004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:08.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1bfff640 1 -- 172.21.15.12:0/1049793479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc040047d0 msgr2=0x7fbc04004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:08.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1bfff640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc040047d0 0x7fbc04004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1bfff640 1 -- 172.21.15.12:0/1049793479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc040051e0 msgr2=0x7fbc0400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:08.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1bfff640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc040051e0 0x7fbc0400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc1bfff640 1 -- 172.21.15.12:0/1049793479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc04003530 con 0x7fbc04003a40 2023-11-28T21:40:08.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc20961640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc040051e0 0x7fbc0400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:08.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.673+0000 7fbc20961640 1 -- 172.21.15.12:0/1049793479 reap_dead start 2023-11-28T21:40:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc1bfff640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc04003e60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c001780 tx=0x7fbc0c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222cd7db536c1ca8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0052d0 con 0x7fbc04003a40 2023-11-28T21:40:08.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c0a7050 con 0x7fbc04003a40 2023-11-28T21:40:08.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc1affd640 1 -- 172.21.15.12:0/1049793479 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c003600 con 0x7fbc04003a40 2023-11-28T21:40:08.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc223c4640 1 -- 172.21.15.12:0/1049793479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 msgr2=0x7fbc04003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:08.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc223c4640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc04003e60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c001780 tx=0x7fbc0c01c3e0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc223c4640 1 -- 172.21.15.12:0/1049793479 shutdown_connections 2023-11-28T21:40:08.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc223c4640 1 --2- 172.21.15.12:0/1049793479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc04003e60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc223c4640 1 -- 172.21.15.12:0/1049793479 >> 172.21.15.12:0/1049793479 conn(0x7fbc1c066360 msgr2=0x7fbc1c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:08.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc223c4640 1 -- 172.21.15.12:0/1049793479 shutdown_connections 2023-11-28T21:40:08.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc223c4640 1 -- 172.21.15.12:0/1049793479 wait complete. 2023-11-28T21:40:08.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc223c4640 1 Processor -- start 2023-11-28T21:40:08.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.677+0000 7fbc223c4640 1 -- start start 2023-11-28T21:40:08.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc223c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc1c145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:08.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc223c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1c14d800 0x7fbc1c14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:08.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc223c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1c14c6c0 0x7fbc1c1f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:08.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc223c4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c152de0 con 0x7fbc04003a40 2023-11-28T21:40:08.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc223c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0cb0c0 con 0x7fbc1c14c6c0 2023-11-28T21:40:08.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc223c4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c154210 con 0x7fbc1c14d800 2023-11-28T21:40:08.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc1bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc1c145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:08.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc1bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc1c145990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47060/0 (socket says 172.21.15.12:47060) 2023-11-28T21:40:08.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc1bfff640 1 -- 172.21.15.12:0/343625820 learned_addr learned my addr 172.21.15.12:0/343625820 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:08.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc1b7fe640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1c14d800 0x7fbc1c14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:08.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc20961640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1c14c6c0 0x7fbc1c1f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:08.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc1bfff640 1 -- 172.21.15.12:0/343625820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1c14d800 msgr2=0x7fbc1c14c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:08.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc1bfff640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1c14d800 0x7fbc1c14c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc1bfff640 1 -- 172.21.15.12:0/343625820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1c14c6c0 msgr2=0x7fbc1c1f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:08.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc1bfff640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1c14c6c0 0x7fbc1c1f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc1bfff640 1 -- 172.21.15.12:0/343625820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c14dc90 con 0x7fbc04003a40 2023-11-28T21:40:08.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc1bfff640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc1c145990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005040 tx=0x7fbc04005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:08.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc18ff9640 1 -- 172.21.15.12:0/343625820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0072f0 con 0x7fbc04003a40 2023-11-28T21:40:08.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc18ff9640 1 -- 172.21.15.12:0/343625820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c007450 con 0x7fbc04003a40 2023-11-28T21:40:08.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc18ff9640 1 -- 172.21.15.12:0/343625820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0067e0 con 0x7fbc04003a40 2023-11-28T21:40:08.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc223c4640 1 -- 172.21.15.12:0/343625820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c0ca540 con 0x7fbc04003a40 2023-11-28T21:40:08.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc223c4640 1 -- 172.21.15.12:0/343625820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c0cab00 con 0x7fbc04003a40 2023-11-28T21:40:08.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc223c4640 1 -- 172.21.15.12:0/343625820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c074f60 con 0x7fbc04003a40 2023-11-28T21:40:08.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.681+0000 7fbc18ff9640 1 -- 172.21.15.12:0/343625820 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fbc0c006940 con 0x7fbc04003a40 2023-11-28T21:40:08.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.685+0000 7fbc18ff9640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc08066470 0x7fbc08068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:08.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.685+0000 7fbc18ff9640 1 -- 172.21.15.12:0/343625820 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fbc0c11e290 con 0x7fbc04003a40 2023-11-28T21:40:08.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.685+0000 7fbc1b7fe640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc08066470 0x7fbc08068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:08.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.685+0000 7fbc1b7fe640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc08066470 0x7fbc08068930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c146bd0 tx=0x7fbc10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:08.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.689+0000 7fbc18ff9640 1 -- 172.21.15.12:0/343625820 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc0c0eb300 con 0x7fbc04003a40 2023-11-28T21:40:08.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.877+0000 7fbc223c4640 1 -- 172.21.15.12:0/343625820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc1c14bda0 con 0x7fbc04003a40 2023-11-28T21:40:08.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.877+0000 7fbc18ff9640 1 -- 172.21.15.12:0/343625820 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5718 (secure 0 0 0) 0x7fbc0c0aae80 con 0x7fbc04003a40 2023-11-28T21:40:08.884 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:08.884 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:05.415073+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:08.885 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 23 2023-11-28T21:40:08.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 -- 172.21.15.12:0/343625820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc08066470 msgr2=0x7fbc08068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:08.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc08066470 0x7fbc08068930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c146bd0 tx=0x7fbc10008040 comp rx=0 tx=0).stop 2023-11-28T21:40:08.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 -- 172.21.15.12:0/343625820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 msgr2=0x7fbc1c145990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:08.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc1c145990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005040 tx=0x7fbc04005800 comp rx=0 tx=0).stop 2023-11-28T21:40:08.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 -- 172.21.15.12:0/343625820 shutdown_connections 2023-11-28T21:40:08.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc08066470 0x7fbc08068930 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1c14c6c0 0x7fbc1c1f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1c14d800 0x7fbc1c14c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 --2- 172.21.15.12:0/343625820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc04003a40 0x7fbc1c145990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:08.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 -- 172.21.15.12:0/343625820 >> 172.21.15.12:0/343625820 conn(0x7fbc1c066360 msgr2=0x7fbc1c14ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:08.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 -- 172.21.15.12:0/343625820 shutdown_connections 2023-11-28T21:40:08.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:08.881+0000 7fbc223c4640 1 -- 172.21.15.12:0/343625820 wait complete. 2023-11-28T21:40:08.905 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:40:10.907 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:11.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.053+0000 7f5696032640 1 Processor -- start 2023-11-28T21:40:11.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.053+0000 7f5696032640 1 -- start start 2023-11-28T21:40:11.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f5696032640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5690151af0 0x7f56900c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:11.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f5696032640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5690151ed0 0x7f56900c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:11.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f5696032640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5690157b70 con 0x7f569006e300 2023-11-28T21:40:11.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f5696032640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56900707f0 con 0x7f5690151af0 2023-11-28T21:40:11.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f5696032640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56900c7b90 con 0x7f5690151ed0 2023-11-28T21:40:11.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f568f7fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5690151af0 0x7f56900c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:11.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f568effd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5690151ed0 0x7f56900c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:11.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f568f7fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5690151af0 0x7f56900c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:59308/0 (socket says 172.21.15.12:59308) 2023-11-28T21:40:11.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f568effd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5690151ed0 0x7f56900c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:43022/0 (socket says 172.21.15.12:43022) 2023-11-28T21:40:11.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f568f7fe640 1 -- 172.21.15.12:0/2661202058 learned_addr learned my addr 172.21.15.12:0/2661202058 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:11.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f568effd640 1 -- 172.21.15.12:0/2661202058 >> v2:172.21.15.144:3300/0 conn(0x7f5690151af0 msgr2=0x7f56900c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:11.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f568effd640 1 --2- 172.21.15.12:0/2661202058 >> v2:172.21.15.144:3300/0 conn(0x7f5690151af0 0x7f56900c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:11.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f568effd640 1 -- 172.21.15.12:0/2661202058 >> v1:172.21.15.144:6789/0 conn(0x7f569006e300 legacy=0x7f5690072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:11.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f568effd640 1 -- 172.21.15.12:0/2661202058 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5690157e50 con 0x7f5690151ed0 2023-11-28T21:40:11.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.057+0000 7f568f7fe640 1 --2- 172.21.15.12:0/2661202058 >> v2:172.21.15.144:3300/0 conn(0x7f5690151af0 0x7f56900c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:40:11.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f568effd640 1 --2- 172.21.15.12:0/2661202058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5690151ed0 0x7f56900c49d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f56840097c0 tx=0x7f568401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf0d51e5aaf0550 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:11.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f568e7fc640 1 -- 172.21.15.12:0/2661202058 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56840ab600 con 0x7f5690151ed0 2023-11-28T21:40:11.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f568e7fc640 1 -- 172.21.15.12:0/2661202058 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56840a7050 con 0x7f5690151ed0 2023-11-28T21:40:11.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f568e7fc640 1 -- 172.21.15.12:0/2661202058 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56840ab980 con 0x7f5690151ed0 2023-11-28T21:40:11.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f5696032640 1 -- 172.21.15.12:0/2661202058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5690151ed0 msgr2=0x7f56900c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:11.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f5696032640 1 --2- 172.21.15.12:0/2661202058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5690151ed0 0x7f56900c49d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f56840097c0 tx=0x7f568401c480 comp rx=0 tx=0).stop 2023-11-28T21:40:11.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f5696032640 1 -- 172.21.15.12:0/2661202058 shutdown_connections 2023-11-28T21:40:11.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f5696032640 1 --2- 172.21.15.12:0/2661202058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5690151ed0 0x7f56900c49d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:11.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f5696032640 1 --2- 172.21.15.12:0/2661202058 >> v2:172.21.15.144:3300/0 conn(0x7f5690151af0 0x7f56900c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:11.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f5696032640 1 -- 172.21.15.12:0/2661202058 >> 172.21.15.12:0/2661202058 conn(0x7f5690066360 msgr2=0x7f5690066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:11.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f5696032640 1 -- 172.21.15.12:0/2661202058 shutdown_connections 2023-11-28T21:40:11.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.061+0000 7f5696032640 1 -- 172.21.15.12:0/2661202058 wait complete. 2023-11-28T21:40:11.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 Processor -- start 2023-11-28T21:40:11.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 -- start start 2023-11-28T21:40:11.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f569006e300 0x7f569014ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690151af0 0x7f56901433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56901e9e10 0x7f5690143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5690144930 con 0x7f56901e9e10 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5690145120 con 0x7f5690151af0 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5690150d20 con 0x7f569006e300 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56901e9e10 0x7f5690143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690151af0 0x7f56901433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568f7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f569006e300 0x7f569014ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56901e9e10 0x7f5690143930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47068/0 (socket says 172.21.15.12:47068) 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568ffff640 1 -- 172.21.15.12:0/4043069594 learned_addr learned my addr 172.21.15.12:0/4043069594 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568ffff640 1 -- 172.21.15.12:0/4043069594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f569006e300 msgr2=0x7f569014ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:11.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568ffff640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f569006e300 0x7f569014ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568ffff640 1 -- 172.21.15.12:0/4043069594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690151af0 msgr2=0x7f56901433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568ffff640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690151af0 0x7f56901433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568ffff640 1 -- 172.21.15.12:0/4043069594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56800094d0 con 0x7f56901e9e10 2023-11-28T21:40:11.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f568ffff640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56901e9e10 0x7f5690143930 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f568800c730 tx=0x7f568800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5677fff640 1 -- 172.21.15.12:0/4043069594 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5688003b80 con 0x7f56901e9e10 2023-11-28T21:40:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5677fff640 1 -- 172.21.15.12:0/4043069594 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5688003ce0 con 0x7f56901e9e10 2023-11-28T21:40:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5677fff640 1 -- 172.21.15.12:0/4043069594 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5688005310 con 0x7f56901e9e10 2023-11-28T21:40:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 -- 172.21.15.12:0/4043069594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5684009420 con 0x7f56901e9e10 2023-11-28T21:40:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 -- 172.21.15.12:0/4043069594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5690145be0 con 0x7f56901e9e10 2023-11-28T21:40:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.065+0000 7f5696032640 1 -- 172.21.15.12:0/4043069594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5690145d70 con 0x7f56901e9e10 2023-11-28T21:40:11.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.069+0000 7f5677fff640 1 -- 172.21.15.12:0/4043069594 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5688005680 con 0x7f56901e9e10 2023-11-28T21:40:11.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.069+0000 7f5677fff640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f567c066470 0x7f567c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.069+0000 7f5677fff640 1 -- 172.21.15.12:0/4043069594 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f5688084100 con 0x7f56901e9e10 2023-11-28T21:40:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.069+0000 7f568f7fe640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f567c066470 0x7f567c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:11.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.069+0000 7f568f7fe640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f567c066470 0x7f567c068930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5680006cb0 tx=0x7f5680009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:11.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.073+0000 7f5677fff640 1 -- 172.21.15.12:0/4043069594 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5688051210 con 0x7f56901e9e10 2023-11-28T21:40:11.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 -- 172.21.15.12:0/4043069594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56900c79f0 con 0x7f56901e9e10 2023-11-28T21:40:11.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5677fff640 1 -- 172.21.15.12:0/4043069594 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f56900c79f0 con 0x7f56901e9e10 2023-11-28T21:40:11.271 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:11.271 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:10.567892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:rejoin","state_seq":27,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:11.271 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 24 2023-11-28T21:40:11.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 -- 172.21.15.12:0/4043069594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f567c066470 msgr2=0x7f567c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:11.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f567c066470 0x7f567c068930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5680006cb0 tx=0x7f5680009240 comp rx=0 tx=0).stop 2023-11-28T21:40:11.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 -- 172.21.15.12:0/4043069594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56901e9e10 msgr2=0x7f5690143930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:11.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56901e9e10 0x7f5690143930 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f568800c730 tx=0x7f568800cbf0 comp rx=0 tx=0).stop 2023-11-28T21:40:11.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 -- 172.21.15.12:0/4043069594 shutdown_connections 2023-11-28T21:40:11.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f567c066470 0x7f567c068930 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:11.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56901e9e10 0x7f5690143930 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:11.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5690151af0 0x7f56901433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:11.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 --2- 172.21.15.12:0/4043069594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f569006e300 0x7f569014ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:11.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 -- 172.21.15.12:0/4043069594 >> 172.21.15.12:0/4043069594 conn(0x7f5690066360 msgr2=0x7f569006ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:11.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 -- 172.21.15.12:0/4043069594 shutdown_connections 2023-11-28T21:40:11.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:11.265+0000 7f5696032640 1 -- 172.21.15.12:0/4043069594 wait complete. 2023-11-28T21:40:11.293 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 24, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 24, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5307": { "addr": "172.21.15.12:6838/3741155090", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3741155090, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 3741155090, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5307, "incarnation": 22, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 27 } }, "last_failure": 0, "last_failure_osd_epoch": 39, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:40:10.567892+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5307 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6839/3009112102", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3009112102, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3009112102, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5232, "incarnation": 0, "join_fscid": -1, "name": "e", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:40:13.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:13.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.445+0000 7f86f4f1d640 1 Processor -- start 2023-11-28T21:40:13.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.445+0000 7f86f4f1d640 1 -- start start 2023-11-28T21:40:13.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86f4f1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f86f0158b20 0x7f86f0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:13.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86f4f1d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f0067410 con 0x7f86f0150c20 2023-11-28T21:40:13.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86f4f1d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f006e090 con 0x7f86f006e300 2023-11-28T21:40:13.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86f4f1d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f0157f30 con 0x7f86f0158b20 2023-11-28T21:40:13.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86ee59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f86f0150c20 0x7f86f00c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48696/0 (socket says 172.21.15.12:48696) 2023-11-28T21:40:13.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86ee59c640 1 -- 172.21.15.12:0/2215572133 learned_addr learned my addr 172.21.15.12:0/2215572133 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:13.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86eed9d640 1 --2- 172.21.15.12:0/2215572133 >> v2:172.21.15.12:3300/0 conn(0x7f86f0158b20 0x7f86f0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:13.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86edd9b640 1 -- 172.21.15.12:0/2215572133 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076913353 0 0) 0x7f86f0067410 con 0x7f86f0150c20 2023-11-28T21:40:13.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86edd9b640 1 -- 172.21.15.12:0/2215572133 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86dc005740 con 0x7f86f0150c20 2023-11-28T21:40:13.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86eed9d640 1 -- 172.21.15.12:0/2215572133 >> v1:172.21.15.144:6789/0 conn(0x7f86f006e300 legacy=0x7f86f0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:13.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86eed9d640 1 -- 172.21.15.12:0/2215572133 >> v1:172.21.15.12:6789/0 conn(0x7f86f0150c20 legacy=0x7f86f00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:13.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.449+0000 7f86eed9d640 1 -- 172.21.15.12:0/2215572133 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f0151000 con 0x7f86f0158b20 2023-11-28T21:40:13.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86eed9d640 1 --2- 172.21.15.12:0/2215572133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 0x7f86f0156910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009d90 tx=0x7f86e801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e0d712e30dee0b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:13.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86edd9b640 1 -- 172.21.15.12:0/2215572133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e80a5820 con 0x7f86f0158b20 2023-11-28T21:40:13.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86edd9b640 1 -- 172.21.15.12:0/2215572133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e80a5980 con 0x7f86f0158b20 2023-11-28T21:40:13.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86edd9b640 1 -- 172.21.15.12:0/2215572133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e80a5d10 con 0x7f86f0158b20 2023-11-28T21:40:13.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86f4f1d640 1 -- 172.21.15.12:0/2215572133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 msgr2=0x7f86f0156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:13.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86f4f1d640 1 --2- 172.21.15.12:0/2215572133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 0x7f86f0156910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009d90 tx=0x7f86e801c950 comp rx=0 tx=0).stop 2023-11-28T21:40:13.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86f4f1d640 1 -- 172.21.15.12:0/2215572133 shutdown_connections 2023-11-28T21:40:13.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86f4f1d640 1 --2- 172.21.15.12:0/2215572133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 0x7f86f0156910 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:13.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86f4f1d640 1 -- 172.21.15.12:0/2215572133 >> 172.21.15.12:0/2215572133 conn(0x7f86f0066360 msgr2=0x7f86f0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:13.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86f4f1d640 1 -- 172.21.15.12:0/2215572133 shutdown_connections 2023-11-28T21:40:13.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86f4f1d640 1 -- 172.21.15.12:0/2215572133 wait complete. 2023-11-28T21:40:13.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86f4f1d640 1 Processor -- start 2023-11-28T21:40:13.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.453+0000 7f86f4f1d640 1 -- start start 2023-11-28T21:40:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86f4f1d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86f006e300 0x7f86f01f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86f4f1d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f0150c20 0x7f86f0071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86f4f1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 0x7f86f014d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86f4f1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01f6d60 con 0x7f86f0158b20 2023-11-28T21:40:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86f4f1d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f014cef0 con 0x7f86f0150c20 2023-11-28T21:40:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86f4f1d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01609d0 con 0x7f86f006e300 2023-11-28T21:40:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86eed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 0x7f86f014d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86eed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 0x7f86f014d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47086/0 (socket says 172.21.15.12:47086) 2023-11-28T21:40:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86eed9d640 1 -- 172.21.15.12:0/4011864690 learned_addr learned my addr 172.21.15.12:0/4011864690 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86ee59c640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86f006e300 0x7f86f01f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86e7fff640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f0150c20 0x7f86f0071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86eed9d640 1 -- 172.21.15.12:0/4011864690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86f006e300 msgr2=0x7f86f01f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86eed9d640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86f006e300 0x7f86f01f6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86eed9d640 1 -- 172.21.15.12:0/4011864690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f0150c20 msgr2=0x7f86f0071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86eed9d640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f0150c20 0x7f86f0071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86eed9d640 1 -- 172.21.15.12:0/4011864690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e8009a70 con 0x7f86f0158b20 2023-11-28T21:40:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86eed9d640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 0x7f86f014d380 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f86e801a580 tx=0x7f86e80196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86e6ffd640 1 -- 172.21.15.12:0/4011864690 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e80b9650 con 0x7f86f0158b20 2023-11-28T21:40:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86e6ffd640 1 -- 172.21.15.12:0/4011864690 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e80b97b0 con 0x7f86f0158b20 2023-11-28T21:40:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86e6ffd640 1 -- 172.21.15.12:0/4011864690 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e80b9b20 con 0x7f86f0158b20 2023-11-28T21:40:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86f4f1d640 1 -- 172.21.15.12:0/4011864690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f0160630 con 0x7f86f0158b20 2023-11-28T21:40:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86f4f1d640 1 -- 172.21.15.12:0/4011864690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f0160f70 con 0x7f86f0158b20 2023-11-28T21:40:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86f4f1d640 1 -- 172.21.15.12:0/4011864690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f0074f60 con 0x7f86f0158b20 2023-11-28T21:40:13.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.457+0000 7f86e6ffd640 1 -- 172.21.15.12:0/4011864690 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f86e80b9c80 con 0x7f86f0158b20 2023-11-28T21:40:13.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.461+0000 7f86e6ffd640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86d0066470 0x7f86d0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:13.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.461+0000 7f86e6ffd640 1 -- 172.21.15.12:0/4011864690 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f86e811f410 con 0x7f86f0158b20 2023-11-28T21:40:13.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.461+0000 7f86ee59c640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86d0066470 0x7f86d0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:13.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.461+0000 7f86ee59c640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86d0066470 0x7f86d0068930 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86e00011f0 tx=0x7f86e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.465+0000 7f86e6ffd640 1 -- 172.21.15.12:0/4011864690 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86e80ec3e0 con 0x7f86f0158b20 2023-11-28T21:40:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 -- 172.21.15.12:0/4011864690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86f014d030 con 0x7f86f0158b20 2023-11-28T21:40:13.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86e6ffd640 1 -- 172.21.15.12:0/4011864690 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f86e80efa00 con 0x7f86f0158b20 2023-11-28T21:40:13.664 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:13.664 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:10.567892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:rejoin","state_seq":27,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:13.664 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 24 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 -- 172.21.15.12:0/4011864690 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86d0066470 msgr2=0x7f86d0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86d0066470 0x7f86d0068930 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86e00011f0 tx=0x7f86e0009040 comp rx=0 tx=0).stop 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 -- 172.21.15.12:0/4011864690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 msgr2=0x7f86f014d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 0x7f86f014d380 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f86e801a580 tx=0x7f86e80196f0 comp rx=0 tx=0).stop 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 -- 172.21.15.12:0/4011864690 shutdown_connections 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86d0066470 0x7f86d0068930 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86f0158b20 0x7f86f014d380 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86f0150c20 0x7f86f0071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 --2- 172.21.15.12:0/4011864690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86f006e300 0x7f86f01f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.657+0000 7f86f4f1d640 1 -- 172.21.15.12:0/4011864690 >> 172.21.15.12:0/4011864690 conn(0x7f86f0066360 msgr2=0x7f86f006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.661+0000 7f86f4f1d640 1 -- 172.21.15.12:0/4011864690 shutdown_connections 2023-11-28T21:40:13.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:13.661+0000 7f86f4f1d640 1 -- 172.21.15.12:0/4011864690 wait complete. 2023-11-28T21:40:13.685 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:40:15.686 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.833+0000 7fc2addd8640 1 Processor -- start 2023-11-28T21:40:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.833+0000 7fc2addd8640 1 -- start start 2023-11-28T21:40:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2addd8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2a806e300 0x7fc2a8156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2addd8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a8067410 con 0x7fc2a8150c20 2023-11-28T21:40:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2addd8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a806e090 con 0x7fc2a806cf40 2023-11-28T21:40:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2addd8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a8157f30 con 0x7fc2a806e300 2023-11-28T21:40:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a77fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc2a8150c20 0x7fc2a80c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38600/0 (socket says 172.21.15.12:38600) 2023-11-28T21:40:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a77fe640 1 -- 172.21.15.12:0/2693368114 learned_addr learned my addr 172.21.15.12:0/2693368114 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a6ffd640 1 --2- 172.21.15.12:0/2693368114 >> v2:172.21.15.144:3300/0 conn(0x7fc2a806e300 0x7fc2a8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687574020 0 0) 0x7fc2a8067410 con 0x7fc2a8150c20 2023-11-28T21:40:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc290003530 con 0x7fc2a8150c20 2023-11-28T21:40:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434056739 0 0) 0x7fc290003530 con 0x7fc2a8150c20 2023-11-28T21:40:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 >> v2:172.21.15.144:3300/0 conn(0x7fc2a806e300 msgr2=0x7fc2a8156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 --2- 172.21.15.12:0/2693368114 >> v2:172.21.15.144:3300/0 conn(0x7fc2a806e300 0x7fc2a8156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 >> v1:172.21.15.144:6789/0 conn(0x7fc2a806cf40 legacy=0x7fc2a8158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8151000 con 0x7fc2a8150c20 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fc298001680 con 0x7fc2a8150c20 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 >> v1:172.21.15.12:6789/0 conn(0x7fc2a8150c20 legacy=0x7fc2a80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc290003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2900047d0 0x7fc290004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2900051e0 0x7fc29000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8157f30 con 0x7fc290003a40 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29000c110 con 0x7fc2900047d0 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29000c2d0 con 0x7fc2900051e0 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a77fe640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc290003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a6ffd640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2900047d0 0x7fc290004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a7fff640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2900051e0 0x7fc29000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a77fe640 1 -- 172.21.15.12:0/2693368114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2900051e0 msgr2=0x7fc29000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a77fe640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2900051e0 0x7fc29000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a77fe640 1 -- 172.21.15.12:0/2693368114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2900047d0 msgr2=0x7fc290004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a77fe640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2900047d0 0x7fc290004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a77fe640 1 -- 172.21.15.12:0/2693368114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290003530 con 0x7fc290003a40 2023-11-28T21:40:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a7fff640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2900051e0 0x7fc29000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:40:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.837+0000 7fc2a7fff640 1 -- 172.21.15.12:0/2693368114 reap_dead start 2023-11-28T21:40:15.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2a77fe640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc290003e60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c0e0 tx=0x7fc29801cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39708bdd321b84bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:15.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298005ae0 con 0x7fc290003a40 2023-11-28T21:40:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc298005c40 con 0x7fc290003a40 2023-11-28T21:40:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2a67fc640 1 -- 172.21.15.12:0/2693368114 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2980072b0 con 0x7fc290003a40 2023-11-28T21:40:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2addd8640 1 -- 172.21.15.12:0/2693368114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 msgr2=0x7fc290003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2addd8640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc290003e60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c0e0 tx=0x7fc29801cb60 comp rx=0 tx=0).stop 2023-11-28T21:40:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2addd8640 1 -- 172.21.15.12:0/2693368114 shutdown_connections 2023-11-28T21:40:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2addd8640 1 --2- 172.21.15.12:0/2693368114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc290003e60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2addd8640 1 -- 172.21.15.12:0/2693368114 >> 172.21.15.12:0/2693368114 conn(0x7fc2a8066360 msgr2=0x7fc2a8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2addd8640 1 -- 172.21.15.12:0/2693368114 shutdown_connections 2023-11-28T21:40:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2addd8640 1 -- 172.21.15.12:0/2693368114 wait complete. 2023-11-28T21:40:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.841+0000 7fc2addd8640 1 Processor -- start 2023-11-28T21:40:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2addd8640 1 -- start start 2023-11-28T21:40:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2addd8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc2a80c17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2addd8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a81e65f0 0x7fc2a81f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2addd8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a81e6a30 0x7fc2a81f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2addd8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80c1f20 con 0x7fc290003a40 2023-11-28T21:40:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2addd8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80c1cf0 con 0x7fc2a81e65f0 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2addd8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8154130 con 0x7fc2a81e6a30 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc2a80c17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc2a80c17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48524/0 (socket says 172.21.15.12:48524) 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a77fe640 1 -- 172.21.15.12:0/600354241 learned_addr learned my addr 172.21.15.12:0/600354241 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a7fff640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a81e6a30 0x7fc2a81f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a6ffd640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a81e65f0 0x7fc2a81f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a77fe640 1 -- 172.21.15.12:0/600354241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a81e6a30 msgr2=0x7fc2a81f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a77fe640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a81e6a30 0x7fc2a81f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a77fe640 1 -- 172.21.15.12:0/600354241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a81e65f0 msgr2=0x7fc2a81f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a77fe640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a81e65f0 0x7fc2a81f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a77fe640 1 -- 172.21.15.12:0/600354241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80b9ea0 con 0x7fc290003a40 2023-11-28T21:40:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2a77fe640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc2a80c17b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc29801c990 tx=0x7fc290005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc28ffff640 1 -- 172.21.15.12:0/600354241 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2980b4070 con 0x7fc290003a40 2023-11-28T21:40:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc28ffff640 1 -- 172.21.15.12:0/600354241 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc298005980 con 0x7fc290003a40 2023-11-28T21:40:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2addd8640 1 -- 172.21.15.12:0/600354241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a81535b0 con 0x7fc290003a40 2023-11-28T21:40:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc28ffff640 1 -- 172.21.15.12:0/600354241 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2980072d0 con 0x7fc290003a40 2023-11-28T21:40:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2addd8640 1 -- 172.21.15.12:0/600354241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a8153b70 con 0x7fc290003a40 2023-11-28T21:40:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.845+0000 7fc2addd8640 1 -- 172.21.15.12:0/600354241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a8074f60 con 0x7fc290003a40 2023-11-28T21:40:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.849+0000 7fc28ffff640 1 -- 172.21.15.12:0/600354241 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc298007430 con 0x7fc290003a40 2023-11-28T21:40:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.849+0000 7fc28ffff640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2940664a0 0x7fc294068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.849+0000 7fc28ffff640 1 -- 172.21.15.12:0/600354241 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fc29811dd70 con 0x7fc290003a40 2023-11-28T21:40:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.849+0000 7fc2a6ffd640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2940664a0 0x7fc294068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.849+0000 7fc2a6ffd640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2940664a0 0x7fc294068960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2a81f0be0 tx=0x7fc29c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:15.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:15.853+0000 7fc28ffff640 1 -- 172.21.15.12:0/600354241 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc2980eacb0 con 0x7fc290003a40 2023-11-28T21:40:16.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.041+0000 7fc2addd8640 1 -- 172.21.15.12:0/600354241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2a81f2fb0 con 0x7fc290003a40 2023-11-28T21:40:16.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.041+0000 7fc28ffff640 1 -- 172.21.15.12:0/600354241 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fc2980ee2d0 con 0x7fc290003a40 2023-11-28T21:40:16.048 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:16.048 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:10.567892+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:rejoin","state_seq":27,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:16.049 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 24 2023-11-28T21:40:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 -- 172.21.15.12:0/600354241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2940664a0 msgr2=0x7fc294068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2940664a0 0x7fc294068960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2a81f0be0 tx=0x7fc29c008040 comp rx=0 tx=0).stop 2023-11-28T21:40:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 -- 172.21.15.12:0/600354241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 msgr2=0x7fc2a80c17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc2a80c17b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc29801c990 tx=0x7fc290005800 comp rx=0 tx=0).stop 2023-11-28T21:40:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 -- 172.21.15.12:0/600354241 shutdown_connections 2023-11-28T21:40:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2940664a0 0x7fc294068960 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:16.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2a81e6a30 0x7fc2a81f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2a81e65f0 0x7fc2a81f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 --2- 172.21.15.12:0/600354241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc290003a40 0x7fc2a80c17b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 -- 172.21.15.12:0/600354241 >> 172.21.15.12:0/600354241 conn(0x7fc2a8066360 msgr2=0x7fc2a80c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 -- 172.21.15.12:0/600354241 shutdown_connections 2023-11-28T21:40:16.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:16.045+0000 7fc2addd8640 1 -- 172.21.15.12:0/600354241 wait complete. 2023-11-28T21:40:16.069 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:40:18.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:18.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20eb980640 1 Processor -- start 2023-11-28T21:40:18.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20eb980640 1 -- start start 2023-11-28T21:40:18.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20eb980640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f20e406e300 0x7f20e4156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:18.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20eb980640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e4067410 con 0x7f20e406cf40 2023-11-28T21:40:18.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20eb980640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e406e090 con 0x7f20e4150c20 2023-11-28T21:40:18.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20eb980640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e4157f30 con 0x7f20e406e300 2023-11-28T21:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e9f1d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f20e406cf40 0x7f20e4158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38604/0 (socket says 172.21.15.12:38604) 2023-11-28T21:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e9f1d640 1 -- 172.21.15.12:0/1281302974 learned_addr learned my addr 172.21.15.12:0/1281302974 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e8f1b640 1 --2- 172.21.15.12:0/1281302974 >> v2:172.21.15.144:3300/0 conn(0x7f20e406e300 0x7f20e4156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533778780 0 0) 0x7f20e4067410 con 0x7f20e406cf40 2023-11-28T21:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20d0003530 con 0x7f20e406cf40 2023-11-28T21:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3007906088 0 0) 0x7f20d0003530 con 0x7f20e406cf40 2023-11-28T21:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 >> v2:172.21.15.144:3300/0 conn(0x7f20e406e300 msgr2=0x7f20e4156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:18.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 --2- 172.21.15.12:0/1281302974 >> v2:172.21.15.144:3300/0 conn(0x7f20e406e300 0x7f20e4156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 >> v1:172.21.15.144:6789/0 conn(0x7f20e4150c20 legacy=0x7f20e40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4151000 con 0x7f20e406cf40 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f20dc001a00 con 0x7f20e406cf40 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 >> v1:172.21.15.12:6789/0 conn(0x7f20e406cf40 legacy=0x7f20e4158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d00039f0 0x7f20d0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20d00047d0 0x7f20d0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20d000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4157f30 con 0x7f20d0005190 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d000c0c0 con 0x7f20d00039f0 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d000c280 con 0x7f20d00047d0 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e9f1d640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20d000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:18.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e8f1b640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20d00047d0 0x7f20d0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e971c640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d00039f0 0x7f20d0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e9f1d640 1 -- 172.21.15.12:0/1281302974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20d00047d0 msgr2=0x7f20d0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e9f1d640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20d00047d0 0x7f20d0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e9f1d640 1 -- 172.21.15.12:0/1281302974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d00039f0 msgr2=0x7f20d0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e9f1d640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20d00039f0 0x7f20d0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:18.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.221+0000 7f20e9f1d640 1 -- 172.21.15.12:0/1281302974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d0003530 con 0x7f20d0005190 2023-11-28T21:40:18.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20e9f1d640 1 -- 172.21.15.12:0/1281302974 reap_dead start 2023-11-28T21:40:18.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20e9f1d640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20d000b990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00a830 tx=0x7f20dc01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed3b2013d4aa765 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0072d0 con 0x7f20d0005190 2023-11-28T21:40:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc0a8070 con 0x7f20d0005190 2023-11-28T21:40:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20dbfff640 1 -- 172.21.15.12:0/1281302974 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0076b0 con 0x7f20d0005190 2023-11-28T21:40:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20eb980640 1 -- 172.21.15.12:0/1281302974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 msgr2=0x7f20d000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:18.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20eb980640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20d000b990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00a830 tx=0x7f20dc01d340 comp rx=0 tx=0).stop 2023-11-28T21:40:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20eb980640 1 -- 172.21.15.12:0/1281302974 shutdown_connections 2023-11-28T21:40:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20eb980640 1 --2- 172.21.15.12:0/1281302974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20d000b990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20eb980640 1 -- 172.21.15.12:0/1281302974 >> 172.21.15.12:0/1281302974 conn(0x7f20e4066360 msgr2=0x7f20e4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20eb980640 1 -- 172.21.15.12:0/1281302974 shutdown_connections 2023-11-28T21:40:18.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.225+0000 7f20eb980640 1 -- 172.21.15.12:0/1281302974 wait complete. 2023-11-28T21:40:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20eb980640 1 Processor -- start 2023-11-28T21:40:18.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20eb980640 1 -- start start 2023-11-28T21:40:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20eb980640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20e41edf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20eb980640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20e4154240 0x7f20e414dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20eb980640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4152f60 0x7f20e414a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:18.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20eb980640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4153410 con 0x7f20d0005190 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20eb980640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e414d170 con 0x7f20e4152f60 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20eb980640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e414cbb0 con 0x7f20e4154240 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e971c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20e41edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e971c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20e41edf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48554/0 (socket says 172.21.15.12:48554) 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e971c640 1 -- 172.21.15.12:0/2549204267 learned_addr learned my addr 172.21.15.12:0/2549204267 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e8f1b640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20e4154240 0x7f20e414dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e9f1d640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4152f60 0x7f20e414a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e971c640 1 -- 172.21.15.12:0/2549204267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20e4154240 msgr2=0x7f20e414dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e971c640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20e4154240 0x7f20e414dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e971c640 1 -- 172.21.15.12:0/2549204267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4152f60 msgr2=0x7f20e414a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e971c640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4152f60 0x7f20e414a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:18.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e971c640 1 -- 172.21.15.12:0/2549204267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e414ce90 con 0x7f20d0005190 2023-11-28T21:40:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e8f1b640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20e4154240 0x7f20e414dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20e971c640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20e41edf70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f20e000f540 tx=0x7f20e000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20d9ffb640 1 -- 172.21.15.12:0/2549204267 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0005320 con 0x7f20d0005190 2023-11-28T21:40:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20d9ffb640 1 -- 172.21.15.12:0/2549204267 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e0005480 con 0x7f20d0005190 2023-11-28T21:40:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20eb980640 1 -- 172.21.15.12:0/2549204267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e41ed490 con 0x7f20d0005190 2023-11-28T21:40:18.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.229+0000 7f20d9ffb640 1 -- 172.21.15.12:0/2549204267 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0007250 con 0x7f20d0005190 2023-11-28T21:40:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.233+0000 7f20eb980640 1 -- 172.21.15.12:0/2549204267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e41eda50 con 0x7f20d0005190 2023-11-28T21:40:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.233+0000 7f20eb980640 1 -- 172.21.15.12:0/2549204267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e41edc10 con 0x7f20d0005190 2023-11-28T21:40:18.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.233+0000 7f20d9ffb640 1 -- 172.21.15.12:0/2549204267 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f20e00073b0 con 0x7f20d0005190 2023-11-28T21:40:18.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.233+0000 7f20d9ffb640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c80664a0 0x7f20c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:18.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.233+0000 7f20d9ffb640 1 -- 172.21.15.12:0/2549204267 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f20e00837d0 con 0x7f20d0005190 2023-11-28T21:40:18.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.233+0000 7f20e8f1b640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c80664a0 0x7f20c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:18.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.237+0000 7f20e8f1b640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c80664a0 0x7f20c8068960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f20e414b620 tx=0x7f20d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:18.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.237+0000 7f20d9ffb640 1 -- 172.21.15.12:0/2549204267 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f20e0050850 con 0x7f20d0005190 2023-11-28T21:40:18.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.429+0000 7f20eb980640 1 -- 172.21.15.12:0/2549204267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20e4074f60 con 0x7f20d0005190 2023-11-28T21:40:18.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.429+0000 7f20d9ffb640 1 -- 172.21.15.12:0/2549204267 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5721 (secure 0 0 0) 0x7f20e0053e70 con 0x7f20d0005190 2023-11-28T21:40:18.437 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:18.437 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:15.632311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:clientreplay","state_seq":29,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:18.437 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-11-28T21:40:18.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 -- 172.21.15.12:0/2549204267 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c80664a0 msgr2=0x7f20c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:18.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c80664a0 0x7f20c8068960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f20e414b620 tx=0x7f20d4008040 comp rx=0 tx=0).stop 2023-11-28T21:40:18.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 -- 172.21.15.12:0/2549204267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 msgr2=0x7f20e41edf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:18.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20e41edf70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f20e000f540 tx=0x7f20e000fa00 comp rx=0 tx=0).stop 2023-11-28T21:40:18.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 -- 172.21.15.12:0/2549204267 shutdown_connections 2023-11-28T21:40:18.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c80664a0 0x7f20c8068960 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:18.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20e4152f60 0x7f20e414a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:18.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20e4154240 0x7f20e414dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:18.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 --2- 172.21.15.12:0/2549204267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20d0005190 0x7f20e41edf70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:18.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 -- 172.21.15.12:0/2549204267 >> 172.21.15.12:0/2549204267 conn(0x7f20e4066360 msgr2=0x7f20e4072670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:18.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 -- 172.21.15.12:0/2549204267 shutdown_connections 2023-11-28T21:40:18.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:18.433+0000 7f20eb980640 1 -- 172.21.15.12:0/2549204267 wait complete. 2023-11-28T21:40:18.459 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:40:20.460 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:20.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e396bb640 1 Processor -- start 2023-11-28T21:40:20.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e396bb640 1 -- start start 2023-11-28T21:40:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e396bb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5e34153e00 0x7f5e340ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e396bb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5e340caec0 0x7f5e340cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e396bb640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3415ab40 con 0x7f5e340cba30 2023-11-28T21:40:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e396bb640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3406a850 con 0x7f5e34153e00 2023-11-28T21:40:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e396bb640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e340b2670 con 0x7f5e340caec0 2023-11-28T21:40:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e32ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5e34153e00 0x7f5e340ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e2bfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5e340cba30 0x7f5e3406ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38616/0 (socket says 172.21.15.12:38616) 2023-11-28T21:40:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e2bfff640 1 -- 172.21.15.12:0/2916073319 learned_addr learned my addr 172.21.15.12:0/2916073319 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:20.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e337fe640 1 --2- 172.21.15.12:0/2916073319 >> v2:172.21.15.144:3300/0 conn(0x7f5e340caec0 0x7f5e340cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e32ffd640 1 -- 172.21.15.12:0/2916073319 >> v2:172.21.15.144:3300/0 conn(0x7f5e340caec0 msgr2=0x7f5e340cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e32ffd640 1 --2- 172.21.15.12:0/2916073319 >> v2:172.21.15.144:3300/0 conn(0x7f5e340caec0 0x7f5e340cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e32ffd640 1 -- 172.21.15.12:0/2916073319 >> v1:172.21.15.12:6789/0 conn(0x7f5e340cba30 legacy=0x7f5e3406ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:20.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.609+0000 7f5e32ffd640 1 -- 172.21.15.12:0/2916073319 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34159fe0 con 0x7f5e34153e00 2023-11-28T21:40:20.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e32ffd640 1 --2- 172.21.15.12:0/2916073319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e34153e00 0x7f5e340ca3d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5e240097f0 tx=0x7f5e2401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a634507cd0516f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:20.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e327fc640 1 -- 172.21.15.12:0/2916073319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e240a8930 con 0x7f5e34153e00 2023-11-28T21:40:20.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e327fc640 1 -- 172.21.15.12:0/2916073319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e240a8a90 con 0x7f5e34153e00 2023-11-28T21:40:20.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e327fc640 1 -- 172.21.15.12:0/2916073319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e240a8e20 con 0x7f5e34153e00 2023-11-28T21:40:20.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e396bb640 1 -- 172.21.15.12:0/2916073319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e34153e00 msgr2=0x7f5e340ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:20.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e396bb640 1 --2- 172.21.15.12:0/2916073319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e34153e00 0x7f5e340ca3d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5e240097f0 tx=0x7f5e2401ec30 comp rx=0 tx=0).stop 2023-11-28T21:40:20.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e396bb640 1 -- 172.21.15.12:0/2916073319 shutdown_connections 2023-11-28T21:40:20.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e396bb640 1 --2- 172.21.15.12:0/2916073319 >> v2:172.21.15.144:3300/0 conn(0x7f5e340caec0 0x7f5e340cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:20.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e396bb640 1 --2- 172.21.15.12:0/2916073319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e34153e00 0x7f5e340ca3d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:20.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e396bb640 1 -- 172.21.15.12:0/2916073319 >> 172.21.15.12:0/2916073319 conn(0x7f5e340cf4e0 msgr2=0x7f5e340cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:20.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e396bb640 1 -- 172.21.15.12:0/2916073319 shutdown_connections 2023-11-28T21:40:20.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.613+0000 7f5e396bb640 1 -- 172.21.15.12:0/2916073319 wait complete. 2023-11-28T21:40:20.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e396bb640 1 Processor -- start 2023-11-28T21:40:20.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e396bb640 1 -- start start 2023-11-28T21:40:20.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e396bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e340caec0 0x7f5e340c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:20.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e396bb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e340cba30 0x7f5e340b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:20.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e396bb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34153e00 0x7f5e3414b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:20.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e396bb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3414abf0 con 0x7f5e340cba30 2023-11-28T21:40:20.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e396bb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340c84c0 con 0x7f5e34153e00 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e396bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340c8730 con 0x7f5e340caec0 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e340cba30 0x7f5e340b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e340cba30 0x7f5e340b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48580/0 (socket says 172.21.15.12:48580) 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2bfff640 1 -- 172.21.15.12:0/1412292100 learned_addr learned my addr 172.21.15.12:0/1412292100 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e32ffd640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e340caec0 0x7f5e340c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e337fe640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34153e00 0x7f5e3414b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2bfff640 1 -- 172.21.15.12:0/1412292100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e340caec0 msgr2=0x7f5e340c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2bfff640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e340caec0 0x7f5e340c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2bfff640 1 -- 172.21.15.12:0/1412292100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34153e00 msgr2=0x7f5e3414b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2bfff640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34153e00 0x7f5e3414b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2bfff640 1 -- 172.21.15.12:0/1412292100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e24009420 con 0x7f5e340cba30 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2bfff640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e340cba30 0x7f5e340b4e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00c530 tx=0x7f5e1c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:20.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2b7fe640 1 -- 172.21.15.12:0/1412292100 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c005cf0 con 0x7f5e340cba30 2023-11-28T21:40:20.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2b7fe640 1 -- 172.21.15.12:0/1412292100 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c005e50 con 0x7f5e340cba30 2023-11-28T21:40:20.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e2b7fe640 1 -- 172.21.15.12:0/1412292100 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c007270 con 0x7f5e340cba30 2023-11-28T21:40:20.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e396bb640 1 -- 172.21.15.12:0/1412292100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e340c8a10 con 0x7f5e340cba30 2023-11-28T21:40:20.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.617+0000 7f5e396bb640 1 -- 172.21.15.12:0/1412292100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3414ae00 con 0x7f5e340cba30 2023-11-28T21:40:20.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.621+0000 7f5e396bb640 1 -- 172.21.15.12:0/1412292100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e340b2d70 con 0x7f5e340cba30 2023-11-28T21:40:20.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.621+0000 7f5e2b7fe640 1 -- 172.21.15.12:0/1412292100 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5e1c0073d0 con 0x7f5e340cba30 2023-11-28T21:40:20.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.621+0000 7f5e2b7fe640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e14066470 0x7f5e14068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:20.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.621+0000 7f5e2b7fe640 1 -- 172.21.15.12:0/1412292100 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f5e1c083770 con 0x7f5e340cba30 2023-11-28T21:40:20.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.621+0000 7f5e32ffd640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e14066470 0x7f5e14068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:20.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.621+0000 7f5e32ffd640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e14066470 0x7f5e14068930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e240b2000 tx=0x7f5e24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:20.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.625+0000 7f5e2b7fe640 1 -- 172.21.15.12:0/1412292100 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5e1c0507f0 con 0x7f5e340cba30 2023-11-28T21:40:20.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 -- 172.21.15.12:0/1412292100 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e34074f60 con 0x7f5e340cba30 2023-11-28T21:40:20.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e2b7fe640 1 -- 172.21.15.12:0/1412292100 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5721 (secure 0 0 0) 0x7f5e1c053e10 con 0x7f5e340cba30 2023-11-28T21:40:20.823 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:20.823 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:15.632311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:clientreplay","state_seq":29,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:20.824 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 25 2023-11-28T21:40:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 -- 172.21.15.12:0/1412292100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e14066470 msgr2=0x7f5e14068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e14066470 0x7f5e14068930 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e240b2000 tx=0x7f5e24008040 comp rx=0 tx=0).stop 2023-11-28T21:40:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 -- 172.21.15.12:0/1412292100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e340cba30 msgr2=0x7f5e340b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e340cba30 0x7f5e340b4e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00c530 tx=0x7f5e1c00c9f0 comp rx=0 tx=0).stop 2023-11-28T21:40:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 -- 172.21.15.12:0/1412292100 shutdown_connections 2023-11-28T21:40:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5e14066470 0x7f5e14068930 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e34153e00 0x7f5e3414b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e340cba30 0x7f5e340b4e60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:20.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 --2- 172.21.15.12:0/1412292100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e340caec0 0x7f5e340c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:20.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 -- 172.21.15.12:0/1412292100 >> 172.21.15.12:0/1412292100 conn(0x7f5e340cf4e0 msgr2=0x7f5e34159630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:20.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 -- 172.21.15.12:0/1412292100 shutdown_connections 2023-11-28T21:40:20.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:20.817+0000 7f5e396bb640 1 -- 172.21.15.12:0/1412292100 wait complete. 2023-11-28T21:40:20.845 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:40:22.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:23.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.993+0000 7f971dd72640 1 Processor -- start 2023-11-28T21:40:23.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.993+0000 7f971dd72640 1 -- start start 2023-11-28T21:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f971dd72640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9718150c20 0x7f9718156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f971dd72640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f971806cf40 0x7f9718158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f971dd72640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9718067410 con 0x7f971806e300 2023-11-28T21:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f971dd72640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971806e090 con 0x7f971806cf40 2023-11-28T21:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f971dd72640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9718157f30 con 0x7f9718150c20 2023-11-28T21:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f9716ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f971806e300 0x7f97180c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38618/0 (socket says 172.21.15.12:38618) 2023-11-28T21:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f9716ffd640 1 -- 172.21.15.12:0/3344793194 learned_addr learned my addr 172.21.15.12:0/3344793194 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f97177fe640 1 --2- 172.21.15.12:0/3344793194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9718150c20 0x7f9718156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:23.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f9717fff640 1 --2- 172.21.15.12:0/3344793194 >> v2:172.21.15.144:3300/0 conn(0x7f971806cf40 0x7f9718158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:23.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f97167fc640 1 -- 172.21.15.12:0/3344793194 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138392745 0 0) 0x7f9718067410 con 0x7f971806e300 2023-11-28T21:40:23.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f97167fc640 1 -- 172.21.15.12:0/3344793194 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9700005740 con 0x7f971806e300 2023-11-28T21:40:23.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f97177fe640 1 -- 172.21.15.12:0/3344793194 >> v2:172.21.15.144:3300/0 conn(0x7f971806cf40 msgr2=0x7f9718158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:23.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f97177fe640 1 --2- 172.21.15.12:0/3344793194 >> v2:172.21.15.144:3300/0 conn(0x7f971806cf40 0x7f9718158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:23.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f97177fe640 1 -- 172.21.15.12:0/3344793194 >> v1:172.21.15.12:6789/0 conn(0x7f971806e300 legacy=0x7f97180c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:23.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f97167fc640 1 -- 172.21.15.12:0/3344793194 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3204368030 0 0) 0x7f9700005740 con 0x7f971806e300 2023-11-28T21:40:23.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:22.997+0000 7f97177fe640 1 -- 172.21.15.12:0/3344793194 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9718151000 con 0x7f9718150c20 2023-11-28T21:40:23.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f97177fe640 1 --2- 172.21.15.12:0/3344793194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9718150c20 0x7f9718156910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f97080097c0 tx=0x7f970801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c2d3ecacc36663c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:23.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f97167fc640 1 -- 172.21.15.12:0/3344793194 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c001680 con 0x7f9718150c20 2023-11-28T21:40:23.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f97167fc640 1 -- 172.21.15.12:0/3344793194 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97080a7050 con 0x7f9718150c20 2023-11-28T21:40:23.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f97167fc640 1 -- 172.21.15.12:0/3344793194 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97080ab600 con 0x7f9718150c20 2023-11-28T21:40:23.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f971dd72640 1 -- 172.21.15.12:0/3344793194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9718150c20 msgr2=0x7f9718156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:23.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f971dd72640 1 --2- 172.21.15.12:0/3344793194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9718150c20 0x7f9718156910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f97080097c0 tx=0x7f970801c3a0 comp rx=0 tx=0).stop 2023-11-28T21:40:23.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f971dd72640 1 -- 172.21.15.12:0/3344793194 shutdown_connections 2023-11-28T21:40:23.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f971dd72640 1 --2- 172.21.15.12:0/3344793194 >> v2:172.21.15.144:3300/0 conn(0x7f971806cf40 0x7f9718158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:23.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f971dd72640 1 --2- 172.21.15.12:0/3344793194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9718150c20 0x7f9718156910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f971dd72640 1 -- 172.21.15.12:0/3344793194 >> 172.21.15.12:0/3344793194 conn(0x7f9718066360 msgr2=0x7f9718066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f971dd72640 1 -- 172.21.15.12:0/3344793194 shutdown_connections 2023-11-28T21:40:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f971dd72640 1 -- 172.21.15.12:0/3344793194 wait complete. 2023-11-28T21:40:23.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.001+0000 7f971dd72640 1 Processor -- start 2023-11-28T21:40:23.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f971dd72640 1 -- start start 2023-11-28T21:40:23.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f971dd72640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f971806cf40 0x7f97181faac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f971dd72640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f971806e300 0x7f97180c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f971dd72640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9718150c20 0x7f97180c1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f971dd72640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971814ab50 con 0x7f9718150c20 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f971dd72640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97181fb000 con 0x7f971806cf40 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f971dd72640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97180c4950 con 0x7f971806e300 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f9717fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9718150c20 0x7f97180c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f9717fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9718150c20 0x7f97180c1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48584/0 (socket says 172.21.15.12:48584) 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f9717fff640 1 -- 172.21.15.12:0/3734231451 learned_addr learned my addr 172.21.15.12:0/3734231451 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f9716ffd640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f971806e300 0x7f97180c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f97177fe640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f971806cf40 0x7f97181faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f9717fff640 1 -- 172.21.15.12:0/3734231451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f971806e300 msgr2=0x7f97180c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:23.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f9717fff640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f971806e300 0x7f97180c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f9717fff640 1 -- 172.21.15.12:0/3734231451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f971806cf40 msgr2=0x7f97181faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f9717fff640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f971806cf40 0x7f97181faac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f9717fff640 1 -- 172.21.15.12:0/3734231451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9708009420 con 0x7f9718150c20 2023-11-28T21:40:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f97177fe640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f971806cf40 0x7f97181faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:23.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f9717fff640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9718150c20 0x7f97180c1950 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f971000efc0 tx=0x7f971000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f96fffff640 1 -- 172.21.15.12:0/3734231451 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710014480 con 0x7f9718150c20 2023-11-28T21:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f96fffff640 1 -- 172.21.15.12:0/3734231451 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971000f040 con 0x7f9718150c20 2023-11-28T21:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f96fffff640 1 -- 172.21.15.12:0/3734231451 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9710014820 con 0x7f9718150c20 2023-11-28T21:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f971dd72640 1 -- 172.21.15.12:0/3734231451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9718149f40 con 0x7f9718150c20 2023-11-28T21:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f971dd72640 1 -- 172.21.15.12:0/3734231451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971814a500 con 0x7f9718150c20 2023-11-28T21:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.005+0000 7f971dd72640 1 -- 172.21.15.12:0/3734231451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971814acc0 con 0x7f9718150c20 2023-11-28T21:40:23.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.009+0000 7f96fffff640 1 -- 172.21.15.12:0/3734231451 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9710014980 con 0x7f9718150c20 2023-11-28T21:40:23.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.009+0000 7f96fffff640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9704066470 0x7f9704068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.009+0000 7f96fffff640 1 -- 172.21.15.12:0/3734231451 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f97100851f0 con 0x7f9718150c20 2023-11-28T21:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.009+0000 7f97177fe640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9704066470 0x7f9704068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:23.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.009+0000 7f97177fe640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9704066470 0x7f9704068930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f97080ad000 tx=0x7f97080a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:23.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.013+0000 7f96fffff640 1 -- 172.21.15.12:0/3734231451 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9710052130 con 0x7f9718150c20 2023-11-28T21:40:23.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.201+0000 7f971dd72640 1 -- 172.21.15.12:0/3734231451 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9718074f60 con 0x7f9718150c20 2023-11-28T21:40:23.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.201+0000 7f96fffff640 1 -- 172.21.15.12:0/3734231451 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f9710018090 con 0x7f9718150c20 2023-11-28T21:40:23.209 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:23.209 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:21.761090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:23.209 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 26 2023-11-28T21:40:23.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 -- 172.21.15.12:0/3734231451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9704066470 msgr2=0x7f9704068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:23.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9704066470 0x7f9704068930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f97080ad000 tx=0x7f97080a6000 comp rx=0 tx=0).stop 2023-11-28T21:40:23.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 -- 172.21.15.12:0/3734231451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9718150c20 msgr2=0x7f97180c1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:23.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9718150c20 0x7f97180c1950 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f971000efc0 tx=0x7f971000c490 comp rx=0 tx=0).stop 2023-11-28T21:40:23.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 -- 172.21.15.12:0/3734231451 shutdown_connections 2023-11-28T21:40:23.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9704066470 0x7f9704068930 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:23.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9718150c20 0x7f97180c1950 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:23.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f971806e300 0x7f97180c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:23.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 --2- 172.21.15.12:0/3734231451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f971806cf40 0x7f97181faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:23.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 -- 172.21.15.12:0/3734231451 >> 172.21.15.12:0/3734231451 conn(0x7f9718066360 msgr2=0x7f9718070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:23.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 -- 172.21.15.12:0/3734231451 shutdown_connections 2023-11-28T21:40:23.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:23.205+0000 7f971dd72640 1 -- 172.21.15.12:0/3734231451 wait complete. 2023-11-28T21:40:23.230 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=4261 2023-11-28T21:40:23.230 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.f 2023-11-28T21:40:27.450 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/191/17' 2023-11-28T21:40:27.450 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/191' 2023-11-28T21:40:27.450 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/19' 2023-11-28T21:40:27.450 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/15' 2023-11-28T21:40:27.450 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/20' 2023-11-28T21:40:27.451 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/12' 2023-11-28T21:40:27.451 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/10' 2023-11-28T21:40:27.451 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/11' 2023-11-28T21:40:27.451 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/16' 2023-11-28T21:40:27.451 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/14' 2023-11-28T21:40:27.452 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/18' 2023-11-28T21:40:27.452 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/13' 2023-11-28T21:40:27.452 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/184/17' 2023-11-28T21:40:27.452 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/184' 2023-11-28T21:40:27.452 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/19' 2023-11-28T21:40:27.453 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/15' 2023-11-28T21:40:27.453 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/20' 2023-11-28T21:40:27.453 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/12' 2023-11-28T21:40:27.453 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/10' 2023-11-28T21:40:27.454 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/11' 2023-11-28T21:40:27.454 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/16' 2023-11-28T21:40:27.454 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/14' 2023-11-28T21:40:27.454 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/18' 2023-11-28T21:40:27.454 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/13' 2023-11-28T21:40:27.454 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/192/17' 2023-11-28T21:40:27.455 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/192' 2023-11-28T21:40:27.455 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/19' 2023-11-28T21:40:27.455 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/15' 2023-11-28T21:40:27.455 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/20' 2023-11-28T21:40:27.455 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/12' 2023-11-28T21:40:27.455 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/10' 2023-11-28T21:40:27.455 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/11' 2023-11-28T21:40:27.455 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/16' 2023-11-28T21:40:27.456 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/14' 2023-11-28T21:40:27.456 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/18' 2023-11-28T21:40:27.456 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/13' 2023-11-28T21:40:27.456 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/103/17' 2023-11-28T21:40:27.457 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/103' 2023-11-28T21:40:27.457 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/19' 2023-11-28T21:40:27.457 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/15' 2023-11-28T21:40:27.457 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/20' 2023-11-28T21:40:27.457 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/12' 2023-11-28T21:40:27.457 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/10' 2023-11-28T21:40:27.457 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/11' 2023-11-28T21:40:27.458 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/16' 2023-11-28T21:40:27.458 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/14' 2023-11-28T21:40:27.458 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/18' 2023-11-28T21:40:27.458 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/13' 2023-11-28T21:40:27.458 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/128/17' 2023-11-28T21:40:27.458 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/128' 2023-11-28T21:40:27.461 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/19' 2023-11-28T21:40:27.461 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/15' 2023-11-28T21:40:27.461 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/20' 2023-11-28T21:40:27.461 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/12' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/10' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/11' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/16' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/14' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/18' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/13' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/144/17' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/144' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/19' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/15' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/20' 2023-11-28T21:40:27.462 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/12' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/10' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/11' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/16' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/14' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/18' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/13' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/146/17' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/146' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/19' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/15' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/20' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/12' 2023-11-28T21:40:27.463 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/10' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/11' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/16' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/14' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/18' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/13' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/141/17' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/141' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/19' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/15' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/20' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/12' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/10' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/11' 2023-11-28T21:40:27.464 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/16' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/14' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/18' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/13' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/182/17' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/182' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/19' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/15' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/20' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/12' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/10' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/11' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/16' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/14' 2023-11-28T21:40:27.465 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/18' 2023-11-28T21:40:27.466 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/13' 2023-11-28T21:40:27.466 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/107/17' 2023-11-28T21:40:28.221 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/107' 2023-11-28T21:40:28.221 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/19' 2023-11-28T21:40:28.222 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/15' 2023-11-28T21:40:28.222 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/20' 2023-11-28T21:40:28.222 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/12' 2023-11-28T21:40:28.222 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/10' 2023-11-28T21:40:28.222 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/11' 2023-11-28T21:40:28.222 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/16' 2023-11-28T21:40:28.223 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/14' 2023-11-28T21:40:28.223 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/18' 2023-11-28T21:40:28.223 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/13' 2023-11-28T21:40:28.224 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/178/17' 2023-11-28T21:40:28.224 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/178' 2023-11-28T21:40:28.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/19' 2023-11-28T21:40:28.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/15' 2023-11-28T21:40:28.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/20' 2023-11-28T21:40:28.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/12' 2023-11-28T21:40:28.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/10' 2023-11-28T21:40:28.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/11' 2023-11-28T21:40:28.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/16' 2023-11-28T21:40:28.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/14' 2023-11-28T21:40:28.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/18' 2023-11-28T21:40:28.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/13' 2023-11-28T21:40:28.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/156/17' 2023-11-28T21:40:28.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/156' 2023-11-28T21:40:28.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/19' 2023-11-28T21:40:28.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/15' 2023-11-28T21:40:28.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/20' 2023-11-28T21:40:28.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/12' 2023-11-28T21:40:28.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/10' 2023-11-28T21:40:28.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/11' 2023-11-28T21:40:28.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/16' 2023-11-28T21:40:28.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/14' 2023-11-28T21:40:28.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/18' 2023-11-28T21:40:28.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/13' 2023-11-28T21:40:28.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/131/17' 2023-11-28T21:40:28.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/131' 2023-11-28T21:40:28.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/19' 2023-11-28T21:40:28.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/15' 2023-11-28T21:40:28.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/20' 2023-11-28T21:40:28.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/12' 2023-11-28T21:40:28.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/10' 2023-11-28T21:40:28.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/11' 2023-11-28T21:40:28.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/16' 2023-11-28T21:40:28.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/14' 2023-11-28T21:40:28.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/18' 2023-11-28T21:40:28.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/13' 2023-11-28T21:40:28.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/123/17' 2023-11-28T21:40:28.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/123' 2023-11-28T21:40:28.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/19' 2023-11-28T21:40:28.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/15' 2023-11-28T21:40:28.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/20' 2023-11-28T21:40:28.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/12' 2023-11-28T21:40:28.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/10' 2023-11-28T21:40:28.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/11' 2023-11-28T21:40:28.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/16' 2023-11-28T21:40:28.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/14' 2023-11-28T21:40:28.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/18' 2023-11-28T21:40:28.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/13' 2023-11-28T21:40:28.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/160/17' 2023-11-28T21:40:28.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/160' 2023-11-28T21:40:28.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/19' 2023-11-28T21:40:28.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/15' 2023-11-28T21:40:28.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/20' 2023-11-28T21:40:28.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/12' 2023-11-28T21:40:28.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/10' 2023-11-28T21:40:28.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/11' 2023-11-28T21:40:28.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/16' 2023-11-28T21:40:28.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/14' 2023-11-28T21:40:28.234 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/18' 2023-11-28T21:40:28.234 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/13' 2023-11-28T21:40:28.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/165/17' 2023-11-28T21:40:28.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/165' 2023-11-28T21:40:28.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/19' 2023-11-28T21:40:28.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/15' 2023-11-28T21:40:28.235 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/20' 2023-11-28T21:40:28.236 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/12' 2023-11-28T21:40:28.236 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/10' 2023-11-28T21:40:28.236 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/11' 2023-11-28T21:40:28.236 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/16' 2023-11-28T21:40:28.236 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/14' 2023-11-28T21:40:28.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/18' 2023-11-28T21:40:28.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/13' 2023-11-28T21:40:28.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/101/17' 2023-11-28T21:40:28.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/101' 2023-11-28T21:40:28.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/19' 2023-11-28T21:40:28.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/15' 2023-11-28T21:40:28.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/20' 2023-11-28T21:40:28.237 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/12' 2023-11-28T21:40:28.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/10' 2023-11-28T21:40:28.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/11' 2023-11-28T21:40:28.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/16' 2023-11-28T21:40:28.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/14' 2023-11-28T21:40:28.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/18' 2023-11-28T21:40:28.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/13' 2023-11-28T21:40:28.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/148/17' 2023-11-28T21:40:28.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/148' 2023-11-28T21:40:28.238 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/19' 2023-11-28T21:40:28.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/15' 2023-11-28T21:40:28.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/20' 2023-11-28T21:40:28.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/12' 2023-11-28T21:40:28.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/10' 2023-11-28T21:40:28.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/11' 2023-11-28T21:40:28.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/16' 2023-11-28T21:40:28.239 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/14' 2023-11-28T21:40:28.240 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/18' 2023-11-28T21:40:28.240 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/13' 2023-11-28T21:40:28.241 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/168/17' 2023-11-28T21:40:28.241 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/168' 2023-11-28T21:40:29.231 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2023-11-28T21:40:29.231 INFO:tasks.ceph.mds.f:Restarting daemon 2023-11-28T21:40:29.232 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-11-28T21:40:29.237 INFO:tasks.ceph.mds.f:Started 2023-11-28T21:40:29.238 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:29.311 INFO:tasks.ceph.mds.f.smithi144.stdout:starting mds.f at 2023-11-28T21:40:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f653b59d640 1 Processor -- start 2023-11-28T21:40:29.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f653b59d640 1 -- start start 2023-11-28T21:40:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f653b59d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6534150c20 0x7f6534156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f653b59d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f653406e300 0x7f65340c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f653b59d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6534067410 con 0x7f653406cf40 2023-11-28T21:40:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f653b59d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653406e090 con 0x7f6534150c20 2023-11-28T21:40:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f653b59d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6534157f30 con 0x7f653406e300 2023-11-28T21:40:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f6539339640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6534150c20 0x7f6534156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f6538b38640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f653406e300 0x7f65340c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f6539339640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6534150c20 0x7f6534156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34908/0 (socket says 172.21.15.12:34908) 2023-11-28T21:40:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.389+0000 7f6539339640 1 -- 172.21.15.12:0/2364406908 learned_addr learned my addr 172.21.15.12:0/2364406908 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f6539339640 1 -- 172.21.15.12:0/2364406908 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f653406e300 msgr2=0x7f65340c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f6539339640 1 --2- 172.21.15.12:0/2364406908 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f653406e300 0x7f65340c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f6539339640 1 -- 172.21.15.12:0/2364406908 >> v1:172.21.15.144:6789/0 conn(0x7f653406cf40 legacy=0x7f6534158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f6539339640 1 -- 172.21.15.12:0/2364406908 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6534151000 con 0x7f6534150c20 2023-11-28T21:40:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f6539339640 1 --2- 172.21.15.12:0/2364406908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534150c20 0x7f6534156910 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6530009870 tx=0x7f653001e750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183a5374d9915953 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f652bfff640 1 -- 172.21.15.12:0/2364406908 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65300ab630 con 0x7f6534150c20 2023-11-28T21:40:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f652bfff640 1 -- 172.21.15.12:0/2364406908 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65300ab790 con 0x7f6534150c20 2023-11-28T21:40:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f652bfff640 1 -- 172.21.15.12:0/2364406908 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65300abb20 con 0x7f6534150c20 2023-11-28T21:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f653b59d640 1 -- 172.21.15.12:0/2364406908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534150c20 msgr2=0x7f6534156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f653b59d640 1 --2- 172.21.15.12:0/2364406908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534150c20 0x7f6534156910 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6530009870 tx=0x7f653001e750 comp rx=0 tx=0).stop 2023-11-28T21:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f653b59d640 1 -- 172.21.15.12:0/2364406908 shutdown_connections 2023-11-28T21:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f653b59d640 1 --2- 172.21.15.12:0/2364406908 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f653406e300 0x7f65340c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f653b59d640 1 --2- 172.21.15.12:0/2364406908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6534150c20 0x7f6534156910 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.393+0000 7f653b59d640 1 -- 172.21.15.12:0/2364406908 >> 172.21.15.12:0/2364406908 conn(0x7f6534066360 msgr2=0x7f6534066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f653b59d640 1 -- 172.21.15.12:0/2364406908 shutdown_connections 2023-11-28T21:40:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f653b59d640 1 -- 172.21.15.12:0/2364406908 wait complete. 2023-11-28T21:40:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f653b59d640 1 Processor -- start 2023-11-28T21:40:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f653b59d640 1 -- start start 2023-11-28T21:40:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f653b59d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f653406cf40 0x7f65340c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f653b59d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653406e300 0x7f653413ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f653b59d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6534150c20 0x7f6534160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f653b59d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653413e4a0 con 0x7f653406cf40 2023-11-28T21:40:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f653b59d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65340c9880 con 0x7f653406e300 2023-11-28T21:40:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f653b59d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65340c90c0 con 0x7f6534150c20 2023-11-28T21:40:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6539339640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f653406cf40 0x7f65340c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6539339640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f653406cf40 0x7f65340c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34918/0 (socket says 172.21.15.12:34918) 2023-11-28T21:40:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6539339640 1 -- 172.21.15.12:0/923025273 learned_addr learned my addr 172.21.15.12:0/923025273 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6538b38640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653406e300 0x7f653413ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6539b3a640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6534150c20 0x7f6534160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6539339640 1 -- 172.21.15.12:0/923025273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6534150c20 msgr2=0x7f6534160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6539339640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6534150c20 0x7f6534160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6539339640 1 -- 172.21.15.12:0/923025273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653406e300 msgr2=0x7f653413ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6539339640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653406e300 0x7f653413ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6539339640 1 -- 172.21.15.12:0/923025273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65300094d0 con 0x7f653406cf40 2023-11-28T21:40:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6538b38640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653406e300 0x7f653413ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6539339640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f653406cf40 0x7f65340c9340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f653001c380 tx=0x7f653001b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:29.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.397+0000 7f6529ffb640 1 -- 172.21.15.12:0/923025273 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65300b3800 con 0x7f653406cf40 2023-11-28T21:40:29.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.401+0000 7f6529ffb640 1 -- 172.21.15.12:0/923025273 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65300b3960 con 0x7f653406cf40 2023-11-28T21:40:29.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.401+0000 7f6529ffb640 1 -- 172.21.15.12:0/923025273 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65300b3cf0 con 0x7f653406cf40 2023-11-28T21:40:29.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.401+0000 7f653b59d640 1 -- 172.21.15.12:0/923025273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653415f400 con 0x7f653406cf40 2023-11-28T21:40:29.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.401+0000 7f653b59d640 1 -- 172.21.15.12:0/923025273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653415fca0 con 0x7f653406cf40 2023-11-28T21:40:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.401+0000 7f653b59d640 1 -- 172.21.15.12:0/923025273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6534074f60 con 0x7f653406cf40 2023-11-28T21:40:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.401+0000 7f6529ffb640 1 -- 172.21.15.12:0/923025273 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f65300aa390 con 0x7f653406cf40 2023-11-28T21:40:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.401+0000 7f6529ffb640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 0x7f6518068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.401+0000 7f6529ffb640 1 -- 172.21.15.12:0/923025273 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f653011e870 con 0x7f653406cf40 2023-11-28T21:40:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.401+0000 7f6538b38640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 0x7f6518068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.405+0000 7f6538b38640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 0x7f6518068930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65341f6860 tx=0x7f6524008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.405+0000 7f6529ffb640 1 -- 172.21.15.12:0/923025273 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f65300eb860 con 0x7f653406cf40 2023-11-28T21:40:29.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.697+0000 7f653b59d640 1 -- 172.21.15.12:0/923025273 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65340c2c30 con 0x7f653406cf40 2023-11-28T21:40:29.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.697+0000 7f6529ffb640 1 -- 172.21.15.12:0/923025273 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f65300eee80 con 0x7f653406cf40 2023-11-28T21:40:29.704 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:29.704 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:21.761090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:29.704 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-11-28T21:40:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.697+0000 7f653b59d640 1 -- 172.21.15.12:0/923025273 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 msgr2=0x7f6518068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.697+0000 7f653b59d640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 0x7f6518068930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65341f6860 tx=0x7f6524008040 comp rx=0 tx=0).stop 2023-11-28T21:40:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.697+0000 7f653b59d640 1 -- 172.21.15.12:0/923025273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f653406cf40 msgr2=0x7f65340c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.697+0000 7f653b59d640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f653406cf40 0x7f65340c9340 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f653001c380 tx=0x7f653001b4f0 comp rx=0 tx=0).stop 2023-11-28T21:40:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.701+0000 7f653b59d640 1 -- 172.21.15.12:0/923025273 shutdown_connections 2023-11-28T21:40:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.701+0000 7f653b59d640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6518066470 0x7f6518068930 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.701+0000 7f653b59d640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6534150c20 0x7f6534160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.701+0000 7f653b59d640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f653406e300 0x7f653413ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.701+0000 7f653b59d640 1 --2- 172.21.15.12:0/923025273 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f653406cf40 0x7f65340c9340 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.701+0000 7f653b59d640 1 -- 172.21.15.12:0/923025273 >> 172.21.15.12:0/923025273 conn(0x7f6534066360 msgr2=0x7f653406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.701+0000 7f653b59d640 1 -- 172.21.15.12:0/923025273 shutdown_connections 2023-11-28T21:40:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:29.701+0000 7f653b59d640 1 -- 172.21.15.12:0/923025273 wait complete. 2023-11-28T21:40:29.726 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2023-11-28T21:40:29.726 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 18 secs before thrashing 2023-11-28T21:40:29.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/19' 2023-11-28T21:40:29.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/15' 2023-11-28T21:40:29.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/20' 2023-11-28T21:40:29.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/12' 2023-11-28T21:40:29.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/10' 2023-11-28T21:40:29.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/11' 2023-11-28T21:40:29.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/16' 2023-11-28T21:40:29.991 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/14' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/18' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/13' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/179/17' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/179' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/19' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/15' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/20' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/12' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/10' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/11' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/16' 2023-11-28T21:40:29.992 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/14' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/18' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/13' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/132/17' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/132' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/19' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/15' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/20' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/12' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/10' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/11' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/16' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/14' 2023-11-28T21:40:29.993 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/18' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/13' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/199/17' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/199' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/19' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/15' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/20' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/12' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/10' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/11' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/16' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/14' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/18' 2023-11-28T21:40:29.994 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/13' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/163/17' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/163' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/19' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/15' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/20' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/12' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/10' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/11' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/16' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/14' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/18' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/13' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/153/17' 2023-11-28T21:40:29.995 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/153' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/19' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/15' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/20' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/12' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/10' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/11' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/16' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/14' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/18' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/13' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/122/17' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/122' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/19' 2023-11-28T21:40:29.996 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/15' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/20' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/12' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/10' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/11' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/16' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/14' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/18' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/13' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/162/17' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/162' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/19' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/15' 2023-11-28T21:40:29.997 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/20' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/12' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/10' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/11' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/16' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/14' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/18' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/13' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/197/17' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/197' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/19' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/15' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/20' 2023-11-28T21:40:29.998 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/12' 2023-11-28T21:40:29.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/10' 2023-11-28T21:40:29.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/11' 2023-11-28T21:40:29.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/16' 2023-11-28T21:40:29.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/14' 2023-11-28T21:40:29.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/18' 2023-11-28T21:40:29.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/13' 2023-11-28T21:40:29.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/113/17' 2023-11-28T21:40:29.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/113' 2023-11-28T21:40:29.999 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/19' 2023-11-28T21:40:31.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/15' 2023-11-28T21:40:31.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/20' 2023-11-28T21:40:31.372 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/12' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/10' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/11' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/16' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/14' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/18' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/13' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/152/17' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/152' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/19' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/15' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/20' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/12' 2023-11-28T21:40:31.373 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/10' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/11' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/16' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/14' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/18' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/13' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/121/17' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/121' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/19' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/15' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/20' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/12' 2023-11-28T21:40:31.374 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/10' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/11' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/16' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/14' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/18' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/13' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/118/17' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/118' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/19' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/15' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/20' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/12' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/10' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/11' 2023-11-28T21:40:31.375 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/16' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/14' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/18' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/13' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/196/17' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/196' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/19' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/15' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/20' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/12' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/10' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/11' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/16' 2023-11-28T21:40:31.376 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/14' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/18' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/13' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/124/17' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/124' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/19' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/15' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/20' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/12' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/10' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/11' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/16' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/14' 2023-11-28T21:40:31.377 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/18' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/13' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/188/17' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/188' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/19' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/15' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/20' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/12' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/10' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/11' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/16' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/14' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/18' 2023-11-28T21:40:31.378 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/13' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/145/17' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/145' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/19' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/15' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/20' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/12' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/10' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/11' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/16' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/14' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/18' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/13' 2023-11-28T21:40:31.379 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/130/17' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/130' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/19' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/15' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/20' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/12' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/10' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/11' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/16' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/14' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/18' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/13' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/193/17' 2023-11-28T21:40:31.380 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/193' 2023-11-28T21:40:31.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/19' 2023-11-28T21:40:31.381 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/15' 2023-11-28T21:40:31.381 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-11-28T21:40:31.373+0000 7f9859df7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/20' 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/12' 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/10' 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/11' 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/16' 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/14' 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/18' 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/13' 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/175/17' 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/175' 2023-11-28T21:40:32.225 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/19' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/15' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/20' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/12' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/10' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/11' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/16' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/14' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/18' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/13' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/181/17' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/181' 2023-11-28T21:40:32.226 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/19' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/15' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/20' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/12' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/10' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/11' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/16' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/14' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/18' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/13' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/127/17' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/127' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/19' 2023-11-28T21:40:32.227 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/15' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/20' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/12' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/10' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/11' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/16' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/14' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/18' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/13' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/108/17' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/108' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/19' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/15' 2023-11-28T21:40:32.228 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/20' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/12' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/10' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/11' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/16' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/14' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/18' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/13' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/183/17' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/183' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/19' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/15' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/20' 2023-11-28T21:40:32.229 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/12' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/10' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/11' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/16' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/14' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/18' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/13' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/177/17' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/177' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/19' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/15' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/20' 2023-11-28T21:40:32.230 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/12' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/10' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/11' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/16' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/14' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/18' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/13' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/169/17' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/169' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/19' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/15' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/20' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/12' 2023-11-28T21:40:32.231 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/10' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/11' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/16' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/14' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/18' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/13' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/154/17' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/154' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/19' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/15' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/20' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/12' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/10' 2023-11-28T21:40:32.232 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/11' 2023-11-28T21:40:32.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/16' 2023-11-28T21:40:32.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/14' 2023-11-28T21:40:32.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/18' 2023-11-28T21:40:32.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/13' 2023-11-28T21:40:32.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/171/17' 2023-11-28T21:40:32.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/171' 2023-11-28T21:40:32.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/19' 2023-11-28T21:40:32.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/15' 2023-11-28T21:40:32.233 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/20' 2023-11-28T21:40:32.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/12' 2023-11-28T21:40:32.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/10' 2023-11-28T21:40:32.821 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/11' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/16' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/14' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/18' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/13' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/139/17' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/139' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/19' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/15' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/20' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/12' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/10' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/11' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/16' 2023-11-28T21:40:32.822 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/14' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/18' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/13' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/102/17' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/102' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/19' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/15' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/20' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/12' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/10' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/11' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/16' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/14' 2023-11-28T21:40:32.823 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/18' 2023-11-28T21:40:32.824 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/13' 2023-11-28T21:40:32.824 INFO:tasks.workunit.client.0.smithi012.stdout:removed './copyofsnap1/test1/129/17' 2023-11-28T21:40:32.824 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1/129' 2023-11-28T21:40:32.824 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1/test1' 2023-11-28T21:40:32.824 INFO:tasks.workunit.client.0.smithi012.stdout:removed directory './copyofsnap1' 2023-11-28T21:40:32.824 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-11-28T21:40:32.825 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-28T21:40:32.825 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-28T21:40:32.855 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2023-11-28T21:40:32.855 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2023-11-28T21:40:32.914 INFO:tasks.workunit.client.0.smithi012.stderr:+ set -e 2023-11-28T21:40:32.915 INFO:tasks.workunit.client.0.smithi012.stderr:+ touch foo 2023-11-28T21:40:32.916 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod +x foo 2023-11-28T21:40:32.917 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir .snap/s 2023-11-28T21:40:32.933 INFO:tasks.workunit.client.0.smithi012.stderr:+ find .snap/s/foo -executable 2023-11-28T21:40:32.933 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep foo 2023-11-28T21:40:32.935 INFO:tasks.workunit.client.0.smithi012.stdout:.snap/s/foo 2023-11-28T21:40:32.935 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir .snap/s 2023-11-28T21:40:32.943 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm foo 2023-11-28T21:40:32.946 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-11-28T21:40:32.947 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-11-28T21:40:32.947 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-28T21:40:32.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-28T21:40:33.013 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2023-11-28T21:40:33.014 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2023-11-28T21:40:33.077 INFO:tasks.workunit.client.0.smithi012.stderr:+ set -e 2023-11-28T21:40:33.078 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir foo 2023-11-28T21:40:33.080 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo x 2023-11-28T21:40:33.080 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:33.999 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir foo/.snap/ss 2023-11-28T21:40:34.020 INFO:tasks.workunit.client.0.smithi012.stderr:+ ln foo/x foo/xx 2023-11-28T21:40:34.023 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat foo/.snap/ss/x 2023-11-28T21:40:34.030 INFO:tasks.workunit.client.0.smithi012.stdout:x 2023-11-28T21:40:34.030 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir foo/.snap/ss 2023-11-28T21:40:34.039 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo a 2023-11-28T21:40:34.039 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo b 2023-11-28T21:40:34.039 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir foo/.snap/s 2023-11-28T21:40:34.052 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat foo/.snap/s/a 2023-11-28T21:40:34.063 INFO:tasks.workunit.client.0.smithi012.stderr:+ r=a 2023-11-28T21:40:34.064 INFO:tasks.workunit.client.0.smithi012.stderr:+ [ -z a ] 2023-11-28T21:40:34.064 INFO:tasks.workunit.client.0.smithi012.stderr:+ ln foo/b foo/b2 2023-11-28T21:40:34.065 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat foo/.snap/s/b 2023-11-28T21:40:34.084 INFO:tasks.workunit.client.0.smithi012.stdout:b 2023-11-28T21:40:34.085 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo this used to hang: 2023-11-28T21:40:34.085 INFO:tasks.workunit.client.0.smithi012.stdout:this used to hang: 2023-11-28T21:40:34.085 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo more 2023-11-28T21:40:34.086 INFO:tasks.workunit.client.0.smithi012.stdout:oh, it didn't hang! good job. 2023-11-28T21:40:34.086 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo oh, it didn't hang! good job. 2023-11-28T21:40:34.086 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat foo/b 2023-11-28T21:40:34.086 INFO:tasks.workunit.client.0.smithi012.stdout:b 2023-11-28T21:40:34.086 INFO:tasks.workunit.client.0.smithi012.stdout:more 2023-11-28T21:40:34.087 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir foo/.snap/s 2023-11-28T21:40:34.094 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm -r foo 2023-11-28T21:40:34.109 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-11-28T21:40:34.109 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-11-28T21:40:34.110 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-28T21:40:34.110 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-28T21:40:34.638 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2023-11-28T21:40:34.638 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2023-11-28T21:40:35.061 INFO:tasks.workunit.client.0.smithi012.stderr:+ set -e 2023-11-28T21:40:35.061 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:35.063 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir .snap/bar 2023-11-28T21:40:35.079 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/foo 2023-11-28T21:40:35.091 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/foo 2023-11-28T21:40:35.100 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:35.102 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/bar/a 2023-11-28T21:40:35.106 INFO:tasks.workunit.client.0.smithi012.stdout: File: .snap/bar/a 2023-11-28T21:40:35.106 INFO:tasks.workunit.client.0.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-11-28T21:40:35.106 INFO:tasks.workunit.client.0.smithi012.stdout:Device: fdh/253d Inode: 1099511633796 Links: 2 2023-11-28T21:40:35.106 INFO:tasks.workunit.client.0.smithi012.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1271/ ubuntu) 2023-11-28T21:40:35.106 INFO:tasks.workunit.client.0.smithi012.stdout:Access: 2023-11-28 21:40:35.061090448 +0000 2023-11-28T21:40:35.106 INFO:tasks.workunit.client.0.smithi012.stdout:Modify: 2023-11-28 21:40:35.061090448 +0000 2023-11-28T21:40:35.106 INFO:tasks.workunit.client.0.smithi012.stdout:Change: 2023-11-28 21:40:35.061090448 +0000 2023-11-28T21:40:35.107 INFO:tasks.workunit.client.0.smithi012.stdout: Birth: 2023-11-28 21:40:35.061090448 +0000 2023-11-28T21:40:35.107 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir .snap/bar 2023-11-28T21:40:35.119 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-11-28T21:40:35.119 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-11-28T21:40:35.120 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-28T21:40:35.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-28T21:40:35.145 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2023-11-28T21:40:35.145 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2023-11-28T21:40:35.206 INFO:tasks.workunit.client.0.smithi012.stderr:+ set -e 2023-11-28T21:40:35.207 INFO:tasks.workunit.client.0.smithi012.stderr:+ seq 1 20 2023-11-28T21:40:35.208 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:35.210 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:35.211 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:35.224 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:35.232 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:35.233 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:35.541 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:35.558 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:35.558 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:35.558 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:35.559 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:35.559 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:35.559 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:35.559 INFO:tasks.workunit.client.0.smithi012.stdout: 48773 ? 00:00:00 ps 2023-11-28T21:40:35.559 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:35.576 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:35.578 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:35.586 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:35.588 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:35.589 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:35.604 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:35.612 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:35.614 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:35.702 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:35.719 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:35.719 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:35.719 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:35.719 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:35.720 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:35.720 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:35.720 INFO:tasks.workunit.client.0.smithi012.stdout: 48783 ? 00:00:00 ps 2023-11-28T21:40:35.720 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:35.735 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:35.737 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:35.744 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:35.747 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:35.748 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:35.760 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:35.770 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:35.771 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:35.958 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:35.974 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:35.974 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:35.974 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:35.974 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:35.974 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:35.974 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:35.975 INFO:tasks.workunit.client.0.smithi012.stdout: 48793 ? 00:00:00 ps 2023-11-28T21:40:35.975 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:35.994 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:35.996 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:36.002 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:36.005 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:36.006 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:36.018 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:36.024 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:36.025 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:36.099 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:36.114 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:36.115 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:36.115 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:36.115 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:36.115 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:36.115 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:36.115 INFO:tasks.workunit.client.0.smithi012.stdout: 48803 ? 00:00:00 ps 2023-11-28T21:40:36.116 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:36.125 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:36.127 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:36.133 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:36.136 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:36.137 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:36.145 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:36.151 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:36.152 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:36.224 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:36.239 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:36.240 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:36.240 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:36.240 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:36.240 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:36.240 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:36.240 INFO:tasks.workunit.client.0.smithi012.stdout: 48813 ? 00:00:00 ps 2023-11-28T21:40:36.240 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:36.251 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:36.253 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:36.259 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:36.261 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:36.263 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:36.272 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:36.278 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:36.280 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:36.577 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:36.593 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:36.594 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:36.594 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:36.594 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:36.594 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:36.594 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:36.594 INFO:tasks.workunit.client.0.smithi012.stdout: 48823 ? 00:00:00 ps 2023-11-28T21:40:36.594 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:36.606 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:36.608 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:36.613 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:36.616 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:36.617 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:36.626 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:36.632 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:36.633 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:36.750 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:36.766 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:36.766 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:36.766 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:36.766 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:36.766 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:36.766 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:36.766 INFO:tasks.workunit.client.0.smithi012.stdout: 48834 ? 00:00:00 ps 2023-11-28T21:40:36.767 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:36.778 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:36.780 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:36.785 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:36.788 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:36.789 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:36.797 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:36.803 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:36.804 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:37.269 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:40:37.274 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:40:37.704 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:37.726 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:37.726 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:37.726 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:37.726 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:37.727 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:37.727 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:37.727 INFO:tasks.workunit.client.0.smithi012.stdout: 48847 ? 00:00:00 ps 2023-11-28T21:40:37.727 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:37.842 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:37.846 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:37.871 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:37.875 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:37.877 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:38.141 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:38.151 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:38.154 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:38.779 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:38.796 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:38.797 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:38.797 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:38.797 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:38.797 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:38.797 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:38.797 INFO:tasks.workunit.client.0.smithi012.stdout: 48857 ? 00:00:00 ps 2023-11-28T21:40:38.797 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:38.820 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:38.823 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:38.831 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:38.834 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:38.835 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:38.845 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:38.852 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:38.854 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:38.918 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:39.009 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:39.009 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:39.009 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:39.009 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:39.009 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:39.010 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:39.010 INFO:tasks.workunit.client.0.smithi012.stdout: 48867 ? 00:00:00 ps 2023-11-28T21:40:39.011 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:39.011 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:39.011 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:39.011 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:39.011 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:39.011 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:39.011 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:39.011 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:39.011 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:39.034 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:39.050 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:39.051 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:39.051 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:39.051 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:39.051 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:39.051 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:39.051 INFO:tasks.workunit.client.0.smithi012.stdout: 48877 ? 00:00:00 ps 2023-11-28T21:40:39.051 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:39.063 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:39.066 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:39.073 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:39.076 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:39.077 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:39.087 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:39.093 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:39.094 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:39.143 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:39.158 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:39.158 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:39.158 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:39.159 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:39.159 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:39.159 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:39.159 INFO:tasks.workunit.client.0.smithi012.stdout: 48887 ? 00:00:00 ps 2023-11-28T21:40:39.159 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:39.172 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:39.175 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:39.181 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:39.184 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:39.185 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:39.194 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:39.201 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:39.202 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:39.301 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:39.317 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:39.317 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:39.317 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:39.317 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:39.318 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:39.318 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:39.318 INFO:tasks.workunit.client.0.smithi012.stdout: 48897 ? 00:00:00 ps 2023-11-28T21:40:39.318 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:39.331 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:39.334 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:39.340 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:39.343 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:39.344 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:39.354 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:39.360 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:39.361 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:39.427 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:39.442 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:39.442 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:39.442 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:39.442 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:39.442 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:39.443 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:39.443 INFO:tasks.workunit.client.0.smithi012.stdout: 48907 ? 00:00:00 ps 2023-11-28T21:40:39.443 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:39.455 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:39.458 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:39.464 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:39.466 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:39.468 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:39.482 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:39.494 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:39.495 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:39.552 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:39.568 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:39.568 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:39.568 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:39.568 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:39.568 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:39.569 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:39.569 INFO:tasks.workunit.client.0.smithi012.stdout: 48917 ? 00:00:00 ps 2023-11-28T21:40:39.569 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:39.581 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:39.583 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:39.589 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:39.591 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:39.592 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:39.607 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:39.614 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:39.615 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:39.694 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:39.710 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:39.710 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:39.710 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:39.710 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:39.710 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:39.710 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:39.710 INFO:tasks.workunit.client.0.smithi012.stdout: 48927 ? 00:00:00 ps 2023-11-28T21:40:39.711 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:39.723 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:39.726 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:39.732 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:39.734 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:39.735 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:39.745 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:39.752 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:39.753 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:39.819 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:39.835 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:39.835 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:39.835 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:39.835 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:39.836 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:39.836 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:39.836 INFO:tasks.workunit.client.0.smithi012.stdout: 48937 ? 00:00:00 ps 2023-11-28T21:40:39.836 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:39.850 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:39.853 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:39.860 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:39.862 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:39.863 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:39.872 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:39.879 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:39.880 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:39.962 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:39.977 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:39.978 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:39.978 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:39.978 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:39.978 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:39.978 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:39.978 INFO:tasks.workunit.client.0.smithi012.stdout: 48947 ? 00:00:00 ps 2023-11-28T21:40:39.978 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:39.990 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:39.993 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:40.000 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:40.001 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:40.002 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:40.012 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:40.019 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:40.020 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:40.097 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:40.113 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:40.113 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:40.113 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:40.113 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:40.114 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:40.114 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:40.114 INFO:tasks.workunit.client.0.smithi012.stdout: 48957 ? 00:00:00 ps 2023-11-28T21:40:40.115 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:40.124 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:40.127 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:40.132 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a 2023-11-28T21:40:40.135 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/one 2023-11-28T21:40:40.136 INFO:tasks.workunit.client.0.smithi012.stderr:+ cat 2023-11-28T21:40:40.146 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir a/.snap/two 2023-11-28T21:40:40.153 INFO:tasks.workunit.client.0.smithi012.stderr:+ chmod 777 a/foo 2023-11-28T21:40:40.154 INFO:tasks.workunit.client.0.smithi012.stderr:+ sync 2023-11-28T21:40:40.230 INFO:tasks.workunit.client.0.smithi012.stderr:+ ps 2023-11-28T21:40:40.245 INFO:tasks.workunit.client.0.smithi012.stdout: PID TTY TIME CMD 2023-11-28T21:40:40.246 INFO:tasks.workunit.client.0.smithi012.stdout: 1195 ? 00:00:00 systemd 2023-11-28T21:40:40.246 INFO:tasks.workunit.client.0.smithi012.stdout: 1196 ? 00:00:00 (sd-pam) 2023-11-28T21:40:40.247 INFO:tasks.workunit.client.0.smithi012.stdout: 27854 ? 00:00:01 sshd 2023-11-28T21:40:40.247 INFO:tasks.workunit.client.0.smithi012.stdout: 48761 ? 00:00:00 timeout 2023-11-28T21:40:40.247 INFO:tasks.workunit.client.0.smithi012.stdout: 48764 ? 00:00:00 snaptest-double 2023-11-28T21:40:40.247 INFO:tasks.workunit.client.0.smithi012.stdout: 48967 ? 00:00:00 ps 2023-11-28T21:40:40.247 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a/.snap/one a/.snap/two 2023-11-28T21:40:40.262 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm a/foo 2023-11-28T21:40:40.265 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir a 2023-11-28T21:40:40.275 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-11-28T21:40:40.276 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-11-28T21:40:40.277 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-28T21:40:40.277 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-28T21:40:40.304 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2023-11-28T21:40:40.304 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2023-11-28T21:40:40.366 INFO:tasks.workunit.client.0.smithi012.stderr:+ mkdir .snap/foo 2023-11-28T21:40:40.374 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo We want ENOENT, not ESTALE, here. 2023-11-28T21:40:40.374 INFO:tasks.workunit.client.0.smithi012.stdout:We want ENOENT, not ESTALE, here. 2023-11-28T21:40:40.375 INFO:tasks.workunit.client.0.smithi012.stderr:+ seq 1 100 2023-11-28T21:40:40.376 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/1 2023-11-28T21:40:40.376 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.378 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/1': No such file or directory 2023-11-28T21:40:40.379 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/2 2023-11-28T21:40:40.379 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.381 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/2': No such file or directory 2023-11-28T21:40:40.381 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/3 2023-11-28T21:40:40.382 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.383 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/3': No such file or directory 2023-11-28T21:40:40.384 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/4 2023-11-28T21:40:40.384 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.385 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/4': No such file or directory 2023-11-28T21:40:40.386 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/5 2023-11-28T21:40:40.386 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.388 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/5': No such file or directory 2023-11-28T21:40:40.389 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/6 2023-11-28T21:40:40.389 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.390 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/6': No such file or directory 2023-11-28T21:40:40.391 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/7 2023-11-28T21:40:40.391 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.392 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/7': No such file or directory 2023-11-28T21:40:40.393 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/8 2023-11-28T21:40:40.393 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.395 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/8': No such file or directory 2023-11-28T21:40:40.395 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/9 2023-11-28T21:40:40.396 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.397 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/9': No such file or directory 2023-11-28T21:40:40.398 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/10 2023-11-28T21:40:40.398 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.399 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/10': No such file or directory 2023-11-28T21:40:40.400 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/11 2023-11-28T21:40:40.400 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.402 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/11': No such file or directory 2023-11-28T21:40:40.402 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/12 2023-11-28T21:40:40.403 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.404 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/12': No such file or directory 2023-11-28T21:40:40.405 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/13 2023-11-28T21:40:40.405 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.406 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/13': No such file or directory 2023-11-28T21:40:40.407 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/14 2023-11-28T21:40:40.407 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.409 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/14': No such file or directory 2023-11-28T21:40:40.409 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/15 2023-11-28T21:40:40.410 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.411 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/15': No such file or directory 2023-11-28T21:40:40.412 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/16 2023-11-28T21:40:40.412 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.413 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/16': No such file or directory 2023-11-28T21:40:40.414 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/17 2023-11-28T21:40:40.414 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.415 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/17': No such file or directory 2023-11-28T21:40:40.418 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/18 2023-11-28T21:40:40.418 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.419 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/18': No such file or directory 2023-11-28T21:40:40.419 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/19 2023-11-28T21:40:40.419 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.420 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/19': No such file or directory 2023-11-28T21:40:40.421 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/20 2023-11-28T21:40:40.421 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.423 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/20': No such file or directory 2023-11-28T21:40:40.423 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/21 2023-11-28T21:40:40.423 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.425 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/21': No such file or directory 2023-11-28T21:40:40.426 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/22 2023-11-28T21:40:40.426 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.427 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/22': No such file or directory 2023-11-28T21:40:40.428 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/23 2023-11-28T21:40:40.428 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.430 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/23': No such file or directory 2023-11-28T21:40:40.431 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/24 2023-11-28T21:40:40.431 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.432 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/24': No such file or directory 2023-11-28T21:40:40.433 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/25 2023-11-28T21:40:40.433 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.434 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/25': No such file or directory 2023-11-28T21:40:40.436 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/26 2023-11-28T21:40:40.436 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.437 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/26': No such file or directory 2023-11-28T21:40:40.438 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/27 2023-11-28T21:40:40.439 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.440 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/27': No such file or directory 2023-11-28T21:40:40.441 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/28 2023-11-28T21:40:40.442 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.442 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/28': No such file or directory 2023-11-28T21:40:40.443 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/29 2023-11-28T21:40:40.443 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.444 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/29': No such file or directory 2023-11-28T21:40:40.445 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/30 2023-11-28T21:40:40.445 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.447 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/30': No such file or directory 2023-11-28T21:40:40.447 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/31 2023-11-28T21:40:40.448 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.449 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/31': No such file or directory 2023-11-28T21:40:40.450 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/32 2023-11-28T21:40:40.450 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.451 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/32': No such file or directory 2023-11-28T21:40:40.452 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/33 2023-11-28T21:40:40.452 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.454 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/33': No such file or directory 2023-11-28T21:40:40.455 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/34 2023-11-28T21:40:40.455 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.458 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/34': No such file or directory 2023-11-28T21:40:40.459 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/35 2023-11-28T21:40:40.459 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.459 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/35': No such file or directory 2023-11-28T21:40:40.460 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/36 2023-11-28T21:40:40.460 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.461 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/36': No such file or directory 2023-11-28T21:40:40.461 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/37 2023-11-28T21:40:40.462 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.463 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/37': No such file or directory 2023-11-28T21:40:40.464 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/38 2023-11-28T21:40:40.464 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.465 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/38': No such file or directory 2023-11-28T21:40:40.466 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/39 2023-11-28T21:40:40.466 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.467 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/39': No such file or directory 2023-11-28T21:40:40.469 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/40 2023-11-28T21:40:40.469 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.470 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/40': No such file or directory 2023-11-28T21:40:40.471 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/41 2023-11-28T21:40:40.471 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.472 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/41': No such file or directory 2023-11-28T21:40:40.473 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/42 2023-11-28T21:40:40.474 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.475 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/42': No such file or directory 2023-11-28T21:40:40.478 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/43 2023-11-28T21:40:40.478 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.478 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/43': No such file or directory 2023-11-28T21:40:40.479 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/44 2023-11-28T21:40:40.479 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.479 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/44': No such file or directory 2023-11-28T21:40:40.480 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/45 2023-11-28T21:40:40.480 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.482 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/45': No such file or directory 2023-11-28T21:40:40.482 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/46 2023-11-28T21:40:40.483 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.484 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/46': No such file or directory 2023-11-28T21:40:40.485 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/47 2023-11-28T21:40:40.485 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.486 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/47': No such file or directory 2023-11-28T21:40:40.487 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/48 2023-11-28T21:40:40.487 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.489 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/48': No such file or directory 2023-11-28T21:40:40.490 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/49 2023-11-28T21:40:40.490 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.491 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/49': No such file or directory 2023-11-28T21:40:40.492 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/50 2023-11-28T21:40:40.492 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.494 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/50': No such file or directory 2023-11-28T21:40:40.495 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/51 2023-11-28T21:40:40.495 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.496 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/51': No such file or directory 2023-11-28T21:40:40.497 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/52 2023-11-28T21:40:40.498 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.499 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/52': No such file or directory 2023-11-28T21:40:40.500 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/53 2023-11-28T21:40:40.500 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.501 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/53': No such file or directory 2023-11-28T21:40:40.502 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/54 2023-11-28T21:40:40.502 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.503 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/54': No such file or directory 2023-11-28T21:40:40.504 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/55 2023-11-28T21:40:40.505 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.505 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/55': No such file or directory 2023-11-28T21:40:40.506 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/56 2023-11-28T21:40:40.506 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.508 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/56': No such file or directory 2023-11-28T21:40:40.509 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/57 2023-11-28T21:40:40.509 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.510 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/57': No such file or directory 2023-11-28T21:40:40.511 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/58 2023-11-28T21:40:40.511 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.512 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/58': No such file or directory 2023-11-28T21:40:40.513 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/59 2023-11-28T21:40:40.513 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.515 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/59': No such file or directory 2023-11-28T21:40:40.516 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/60 2023-11-28T21:40:40.516 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.517 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/60': No such file or directory 2023-11-28T21:40:40.518 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/61 2023-11-28T21:40:40.518 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.520 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/61': No such file or directory 2023-11-28T21:40:40.520 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/62 2023-11-28T21:40:40.520 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.522 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/62': No such file or directory 2023-11-28T21:40:40.523 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/63 2023-11-28T21:40:40.523 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.524 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/63': No such file or directory 2023-11-28T21:40:40.525 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/64 2023-11-28T21:40:40.525 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.526 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/64': No such file or directory 2023-11-28T21:40:40.527 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/65 2023-11-28T21:40:40.527 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.529 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/65': No such file or directory 2023-11-28T21:40:40.530 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/66 2023-11-28T21:40:40.530 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.531 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/66': No such file or directory 2023-11-28T21:40:40.532 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/67 2023-11-28T21:40:40.532 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.533 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/67': No such file or directory 2023-11-28T21:40:40.534 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/68 2023-11-28T21:40:40.534 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.536 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/68': No such file or directory 2023-11-28T21:40:40.536 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/69 2023-11-28T21:40:40.537 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.538 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/69': No such file or directory 2023-11-28T21:40:40.539 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/70 2023-11-28T21:40:40.539 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.541 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/70': No such file or directory 2023-11-28T21:40:40.541 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/71 2023-11-28T21:40:40.542 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.543 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/71': No such file or directory 2023-11-28T21:40:40.544 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/72 2023-11-28T21:40:40.544 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.545 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/72': No such file or directory 2023-11-28T21:40:40.546 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/73 2023-11-28T21:40:40.546 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.548 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/73': No such file or directory 2023-11-28T21:40:40.549 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/74 2023-11-28T21:40:40.549 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.550 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/74': No such file or directory 2023-11-28T21:40:40.551 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/75 2023-11-28T21:40:40.551 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.552 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/75': No such file or directory 2023-11-28T21:40:40.553 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/76 2023-11-28T21:40:40.553 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.555 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/76': No such file or directory 2023-11-28T21:40:40.556 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/77 2023-11-28T21:40:40.556 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.557 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/77': No such file or directory 2023-11-28T21:40:40.558 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/78 2023-11-28T21:40:40.559 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.559 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/78': No such file or directory 2023-11-28T21:40:40.561 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/79 2023-11-28T21:40:40.561 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.562 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/79': No such file or directory 2023-11-28T21:40:40.563 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/80 2023-11-28T21:40:40.563 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.564 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/80': No such file or directory 2023-11-28T21:40:40.565 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/81 2023-11-28T21:40:40.565 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.566 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/81': No such file or directory 2023-11-28T21:40:40.567 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/82 2023-11-28T21:40:40.567 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.569 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/82': No such file or directory 2023-11-28T21:40:40.570 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/83 2023-11-28T21:40:40.570 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.571 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/83': No such file or directory 2023-11-28T21:40:40.572 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/84 2023-11-28T21:40:40.572 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.573 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/84': No such file or directory 2023-11-28T21:40:40.575 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/85 2023-11-28T21:40:40.575 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.576 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/85': No such file or directory 2023-11-28T21:40:40.577 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/86 2023-11-28T21:40:40.577 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.578 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/86': No such file or directory 2023-11-28T21:40:40.579 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/87 2023-11-28T21:40:40.579 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.581 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/87': No such file or directory 2023-11-28T21:40:40.582 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/88 2023-11-28T21:40:40.582 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.583 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/88': No such file or directory 2023-11-28T21:40:40.584 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/89 2023-11-28T21:40:40.584 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.585 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/89': No such file or directory 2023-11-28T21:40:40.586 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/90 2023-11-28T21:40:40.587 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.587 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/90': No such file or directory 2023-11-28T21:40:40.589 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/91 2023-11-28T21:40:40.589 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.590 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/91': No such file or directory 2023-11-28T21:40:40.591 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/92 2023-11-28T21:40:40.591 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.594 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/92': No such file or directory 2023-11-28T21:40:40.595 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/93 2023-11-28T21:40:40.595 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.595 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/93': No such file or directory 2023-11-28T21:40:40.596 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/94 2023-11-28T21:40:40.596 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.597 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/94': No such file or directory 2023-11-28T21:40:40.598 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/95 2023-11-28T21:40:40.598 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.599 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/95': No such file or directory 2023-11-28T21:40:40.600 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/96 2023-11-28T21:40:40.601 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.602 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/96': No such file or directory 2023-11-28T21:40:40.602 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/97 2023-11-28T21:40:40.603 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.604 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/97': No such file or directory 2023-11-28T21:40:40.605 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/98 2023-11-28T21:40:40.605 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.606 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/98': No such file or directory 2023-11-28T21:40:40.607 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/99 2023-11-28T21:40:40.608 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.609 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/99': No such file or directory 2023-11-28T21:40:40.610 INFO:tasks.workunit.client.0.smithi012.stderr:+ stat .snap/foo/100 2023-11-28T21:40:40.610 INFO:tasks.workunit.client.0.smithi012.stderr:+ grep No such file 2023-11-28T21:40:40.611 INFO:tasks.workunit.client.0.smithi012.stdout:stat: cannot statx '.snap/foo/100': No such file or directory 2023-11-28T21:40:40.612 INFO:tasks.workunit.client.0.smithi012.stderr:+ rmdir .snap/foo 2023-11-28T21:40:40.618 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo OK 2023-11-28T21:40:40.619 INFO:tasks.workunit.client.0.smithi012.stdout:OK 2023-11-28T21:40:40.620 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-28T21:40:40.620 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-28T21:40:40.688 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2023-11-28T21:40:40.688 DEBUG:teuthology.orchestra.run.smithi012:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2023-11-28T21:40:40.750 INFO:tasks.workunit.client.0.smithi012.stderr:+ set -e 2023-11-28T21:40:40.750 INFO:tasks.workunit.client.0.smithi012.stderr:+ sudo git config --global http.sslVerify false 2023-11-28T21:40:40.765 INFO:tasks.workunit.client.0.smithi012.stderr:+ sudo git config --global http.postBuffer 1048576000 2023-11-28T21:40:40.779 INFO:tasks.workunit.client.0.smithi012.stderr:+ retried=false 2023-11-28T21:40:40.780 INFO:tasks.workunit.client.0.smithi012.stderr:+ trap -- retry EXIT 2023-11-28T21:40:40.780 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm -rf ceph 2023-11-28T21:40:40.780 INFO:tasks.workunit.client.0.smithi012.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2023-11-28T21:40:40.785 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into 'ceph'... 2023-11-28T21:40:47.727 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14e69c640 1 Processor -- start 2023-11-28T21:40:47.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14e69c640 1 -- start start 2023-11-28T21:40:47.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14e69c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe148151ed0 0x7fe1480c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:47.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14e69c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe14806e300 0x7fe148072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:47.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14e69c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe148157b70 con 0x7fe148151af0 2023-11-28T21:40:47.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14e69c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1480707f0 con 0x7fe14806e300 2023-11-28T21:40:47.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14e69c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1480c7b90 con 0x7fe148151ed0 2023-11-28T21:40:47.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe1477fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe148151ed0 0x7fe1480c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:47.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14cc39640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe14806e300 0x7fe148072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:47.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe1477fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe148151ed0 0x7fe1480c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:54774/0 (socket says 172.21.15.12:54774) 2023-11-28T21:40:47.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe1477fe640 1 -- 172.21.15.12:0/2148008861 learned_addr learned my addr 172.21.15.12:0/2148008861 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:47.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe147fff640 1 --1- 172.21.15.12:0/2148008861 >> v1:172.21.15.144:6789/0 conn(0x7fe148151af0 0x7fe1480c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:44048/0 (socket says 172.21.15.12:44048) 2023-11-28T21:40:47.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14cc39640 1 -- 172.21.15.12:0/2148008861 >> v2:172.21.15.144:3300/0 conn(0x7fe148151ed0 msgr2=0x7fe1480c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:47.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14cc39640 1 --2- 172.21.15.12:0/2148008861 >> v2:172.21.15.144:3300/0 conn(0x7fe148151ed0 0x7fe1480c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:47.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14cc39640 1 -- 172.21.15.12:0/2148008861 >> v1:172.21.15.144:6789/0 conn(0x7fe148151af0 legacy=0x7fe1480c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:47.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.880+0000 7fe14cc39640 1 -- 172.21.15.12:0/2148008861 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe148157e50 con 0x7fe14806e300 2023-11-28T21:40:47.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe14cc39640 1 --2- 172.21.15.12:0/2148008861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 0x7fe148072070 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe1480c8e60 tx=0x7fe14001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc561f5a82f434a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:47.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe146ffd640 1 -- 172.21.15.12:0/2148008861 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1400a57d0 con 0x7fe14806e300 2023-11-28T21:40:47.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe146ffd640 1 -- 172.21.15.12:0/2148008861 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1400a5930 con 0x7fe14806e300 2023-11-28T21:40:47.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe146ffd640 1 -- 172.21.15.12:0/2148008861 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1400a5cc0 con 0x7fe14806e300 2023-11-28T21:40:47.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe14e69c640 1 -- 172.21.15.12:0/2148008861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 msgr2=0x7fe148072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe14e69c640 1 --2- 172.21.15.12:0/2148008861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 0x7fe148072070 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe1480c8e60 tx=0x7fe14001c910 comp rx=0 tx=0).stop 2023-11-28T21:40:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe14e69c640 1 -- 172.21.15.12:0/2148008861 shutdown_connections 2023-11-28T21:40:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe14e69c640 1 --2- 172.21.15.12:0/2148008861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 0x7fe148072070 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe14e69c640 1 --2- 172.21.15.12:0/2148008861 >> v2:172.21.15.144:3300/0 conn(0x7fe148151ed0 0x7fe1480c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe14e69c640 1 -- 172.21.15.12:0/2148008861 >> 172.21.15.12:0/2148008861 conn(0x7fe148066360 msgr2=0x7fe148066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe14e69c640 1 -- 172.21.15.12:0/2148008861 shutdown_connections 2023-11-28T21:40:47.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.884+0000 7fe14e69c640 1 -- 172.21.15.12:0/2148008861 wait complete. 2023-11-28T21:40:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14e69c640 1 Processor -- start 2023-11-28T21:40:47.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14e69c640 1 -- start start 2023-11-28T21:40:47.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14e69c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 0x7fe1481414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:47.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14e69c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe148151af0 0x7fe1480c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14e69c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe148142980 0x7fe148161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14e69c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1480c29c0 con 0x7fe14806e300 2023-11-28T21:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14e69c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14814f7d0 con 0x7fe148142980 2023-11-28T21:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14e69c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe148142e30 con 0x7fe148151af0 2023-11-28T21:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe147fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 0x7fe1481414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe147fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 0x7fe1481414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39128/0 (socket says 172.21.15.12:39128) 2023-11-28T21:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe147fff640 1 -- 172.21.15.12:0/342763108 learned_addr learned my addr 172.21.15.12:0/342763108 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe1477fe640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe148151af0 0x7fe1480c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14cc39640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe148142980 0x7fe148161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:47.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe147fff640 1 -- 172.21.15.12:0/342763108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe148151af0 msgr2=0x7fe1480c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe147fff640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe148151af0 0x7fe1480c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe147fff640 1 -- 172.21.15.12:0/342763108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe148142980 msgr2=0x7fe148161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe147fff640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe148142980 0x7fe148161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe147fff640 1 -- 172.21.15.12:0/342763108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe140009a70 con 0x7fe14806e300 2023-11-28T21:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe1477fe640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe148151af0 0x7fe1480c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe147fff640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 0x7fe1481414a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe13800c490 tx=0x7fe13800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe144ff9640 1 -- 172.21.15.12:0/342763108 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138007480 con 0x7fe14806e300 2023-11-28T21:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe144ff9640 1 -- 172.21.15.12:0/342763108 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1380075e0 con 0x7fe14806e300 2023-11-28T21:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe144ff9640 1 -- 172.21.15.12:0/342763108 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138004a10 con 0x7fe14806e300 2023-11-28T21:40:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14e69c640 1 -- 172.21.15.12:0/342763108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1480c3580 con 0x7fe14806e300 2023-11-28T21:40:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.888+0000 7fe14e69c640 1 -- 172.21.15.12:0/342763108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe148141c50 con 0x7fe14806e300 2023-11-28T21:40:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.892+0000 7fe14e69c640 1 -- 172.21.15.12:0/342763108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe148074f60 con 0x7fe14806e300 2023-11-28T21:40:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.892+0000 7fe144ff9640 1 -- 172.21.15.12:0/342763108 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe138004b70 con 0x7fe14806e300 2023-11-28T21:40:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.892+0000 7fe144ff9640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1340664a0 0x7fe134068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.892+0000 7fe144ff9640 1 -- 172.21.15.12:0/342763108 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6861+0+0 (secure 0 0 0) 0x7fe138083250 con 0x7fe14806e300 2023-11-28T21:40:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.892+0000 7fe1477fe640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1340664a0 0x7fe134068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.892+0000 7fe1477fe640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1340664a0 0x7fe134068960 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe1481f1c30 tx=0x7fe13c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:47.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:47.896+0000 7fe144ff9640 1 -- 172.21.15.12:0/342763108 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe1380502f0 con 0x7fe14806e300 2023-11-28T21:40:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.084+0000 7fe14e69c640 1 -- 172.21.15.12:0/342763108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1480c3580 con 0x7fe14806e300 2023-11-28T21:40:48.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.084+0000 7fe144ff9640 1 -- 172.21.15.12:0/342763108 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fe1480c3580 con 0x7fe14806e300 2023-11-28T21:40:48.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:48.090 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:21.761090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:48.090 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-11-28T21:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 -- 172.21.15.12:0/342763108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1340664a0 msgr2=0x7fe134068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1340664a0 0x7fe134068960 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe1481f1c30 tx=0x7fe13c008040 comp rx=0 tx=0).stop 2023-11-28T21:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 -- 172.21.15.12:0/342763108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 msgr2=0x7fe1481414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 0x7fe1481414a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe13800c490 tx=0x7fe13800c950 comp rx=0 tx=0).stop 2023-11-28T21:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 -- 172.21.15.12:0/342763108 shutdown_connections 2023-11-28T21:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1340664a0 0x7fe134068960 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe148142980 0x7fe148161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe148151af0 0x7fe1480c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 --2- 172.21.15.12:0/342763108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe14806e300 0x7fe1481414a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 -- 172.21.15.12:0/342763108 >> 172.21.15.12:0/342763108 conn(0x7fe148066360 msgr2=0x7fe14806af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 -- 172.21.15.12:0/342763108 shutdown_connections 2023-11-28T21:40:48.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.088+0000 7fe14e69c640 1 -- 172.21.15.12:0/342763108 wait complete. 2023-11-28T21:40:48.111 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2023-11-28T21:40:48.111 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T21:40:48.111 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:40:48.129 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T21:40:48.129 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.308+0000 7fae8350f640 1 Processor -- start 2023-11-28T21:40:48.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.308+0000 7fae8350f640 1 -- start start 2023-11-28T21:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.308+0000 7fae8350f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae7c158420 0x7fae7c06d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.308+0000 7fae8350f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 0x7fae7c0c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.308+0000 7fae8350f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c0c3460 con 0x7fae7c15e010 2023-11-28T21:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.308+0000 7fae8350f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7c158800 con 0x7fae7c158420 2023-11-28T21:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.308+0000 7fae8350f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c162610 con 0x7fae7c0c38f0 2023-11-28T21:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae80aaa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 0x7fae7c0c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae812ab640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae7c158420 0x7fae7c06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae812ab640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae7c158420 0x7fae7c06d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:54798/0 (socket says 172.21.15.12:54798) 2023-11-28T21:40:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae80aaa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 0x7fae7c0c3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:38568/0 (socket says 172.21.15.12:38568) 2023-11-28T21:40:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae812ab640 1 -- 172.21.15.12:0/184604864 learned_addr learned my addr 172.21.15.12:0/184604864 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae81aac640 1 --1- 172.21.15.12:0/184604864 >> v1:172.21.15.144:6789/0 conn(0x7fae7c15e010 0x7fae7c15e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:44064/0 (socket says 172.21.15.12:44064) 2023-11-28T21:40:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae812ab640 1 -- 172.21.15.12:0/184604864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 msgr2=0x7fae7c0c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae812ab640 1 --2- 172.21.15.12:0/184604864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 0x7fae7c0c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae812ab640 1 -- 172.21.15.12:0/184604864 >> v1:172.21.15.144:6789/0 conn(0x7fae7c15e010 legacy=0x7fae7c15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae812ab640 1 -- 172.21.15.12:0/184604864 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0c2b70 con 0x7fae7c158420 2023-11-28T21:40:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.312+0000 7fae812ab640 1 --2- 172.21.15.12:0/184604864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c158420 0x7fae7c06d950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fae78009870 tx=0x7fae7801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc2ac8bdc91b15b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae73fff640 1 -- 172.21.15.12:0/184604864 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae780ab600 con 0x7fae7c158420 2023-11-28T21:40:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae73fff640 1 -- 172.21.15.12:0/184604864 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae780a7050 con 0x7fae7c158420 2023-11-28T21:40:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae73fff640 1 -- 172.21.15.12:0/184604864 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae780ab980 con 0x7fae7c158420 2023-11-28T21:40:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 -- 172.21.15.12:0/184604864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c158420 msgr2=0x7fae7c06d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 --2- 172.21.15.12:0/184604864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c158420 0x7fae7c06d950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fae78009870 tx=0x7fae7801c540 comp rx=0 tx=0).stop 2023-11-28T21:40:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 -- 172.21.15.12:0/184604864 shutdown_connections 2023-11-28T21:40:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 --2- 172.21.15.12:0/184604864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 0x7fae7c0c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 --2- 172.21.15.12:0/184604864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c158420 0x7fae7c06d950 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 -- 172.21.15.12:0/184604864 >> 172.21.15.12:0/184604864 conn(0x7fae7c066360 msgr2=0x7fae7c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 -- 172.21.15.12:0/184604864 shutdown_connections 2023-11-28T21:40:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 -- 172.21.15.12:0/184604864 wait complete. 2023-11-28T21:40:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 Processor -- start 2023-11-28T21:40:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 -- start start 2023-11-28T21:40:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 0x7fae7c156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c158420 0x7fae7c0c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae7c15e010 0x7fae7c14db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c14d690 con 0x7fae7c15e010 2023-11-28T21:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c14ea00 con 0x7fae7c158420 2023-11-28T21:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae8350f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c14f560 con 0x7fae7c0c38f0 2023-11-28T21:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.316+0000 7fae81aac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae7c15e010 0x7fae7c14db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae81aac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae7c15e010 0x7fae7c14db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39142/0 (socket says 172.21.15.12:39142) 2023-11-28T21:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae81aac640 1 -- 172.21.15.12:0/4120207455 learned_addr learned my addr 172.21.15.12:0/4120207455 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae812ab640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 0x7fae7c156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae80aaa640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c158420 0x7fae7c0c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:48.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae81aac640 1 -- 172.21.15.12:0/4120207455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 msgr2=0x7fae7c156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:48.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae81aac640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 0x7fae7c156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae81aac640 1 -- 172.21.15.12:0/4120207455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c158420 msgr2=0x7fae7c0c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae81aac640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c158420 0x7fae7c0c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae81aac640 1 -- 172.21.15.12:0/4120207455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae780094d0 con 0x7fae7c15e010 2023-11-28T21:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae81aac640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae7c15e010 0x7fae7c14db20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c700 tx=0x7fae7400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae71ffb640 1 -- 172.21.15.12:0/4120207455 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7400ce10 con 0x7fae7c15e010 2023-11-28T21:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae71ffb640 1 -- 172.21.15.12:0/4120207455 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7400a640 con 0x7fae7c15e010 2023-11-28T21:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae71ffb640 1 -- 172.21.15.12:0/4120207455 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7400a9b0 con 0x7fae7c15e010 2023-11-28T21:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae8350f640 1 -- 172.21.15.12:0/4120207455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7c14ecc0 con 0x7fae7c15e010 2023-11-28T21:40:48.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae8350f640 1 -- 172.21.15.12:0/4120207455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c14f280 con 0x7fae7c15e010 2023-11-28T21:40:48.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae8350f640 1 -- 172.21.15.12:0/4120207455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c074f60 con 0x7fae7c15e010 2023-11-28T21:40:48.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.320+0000 7fae71ffb640 1 -- 172.21.15.12:0/4120207455 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fae7400ab10 con 0x7fae7c15e010 2023-11-28T21:40:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.324+0000 7fae71ffb640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae600664f0 0x7fae600689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.324+0000 7fae71ffb640 1 -- 172.21.15.12:0/4120207455 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6861+0+0 (secure 0 0 0) 0x7fae74083960 con 0x7fae7c15e010 2023-11-28T21:40:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.324+0000 7fae812ab640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae600664f0 0x7fae600689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:48.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.324+0000 7fae812ab640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae600664f0 0x7fae600689b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae7c06fb30 tx=0x7fae78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:48.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.328+0000 7fae71ffb640 1 -- 172.21.15.12:0/4120207455 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae74050a80 con 0x7fae7c15e010 2023-11-28T21:40:48.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.520+0000 7fae8350f640 1 -- 172.21.15.12:0/4120207455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae7c0c7a10 con 0x7fae7c15e010 2023-11-28T21:40:48.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.520+0000 7fae71ffb640 1 -- 172.21.15.12:0/4120207455 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fae74017090 con 0x7fae7c15e010 2023-11-28T21:40:48.525 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:48.525 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:21.761090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:48.525 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-11-28T21:40:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 -- 172.21.15.12:0/4120207455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae600664f0 msgr2=0x7fae600689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae600664f0 0x7fae600689b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fae7c06fb30 tx=0x7fae78008040 comp rx=0 tx=0).stop 2023-11-28T21:40:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 -- 172.21.15.12:0/4120207455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae7c15e010 msgr2=0x7fae7c14db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae7c15e010 0x7fae7c14db20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fae7400c700 tx=0x7fae7400cbc0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 -- 172.21.15.12:0/4120207455 shutdown_connections 2023-11-28T21:40:48.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae600664f0 0x7fae600689b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae7c15e010 0x7fae7c14db20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c158420 0x7fae7c0c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 --2- 172.21.15.12:0/4120207455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae7c0c38f0 0x7fae7c156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 -- 172.21.15.12:0/4120207455 >> 172.21.15.12:0/4120207455 conn(0x7fae7c066360 msgr2=0x7fae7c070620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 -- 172.21.15.12:0/4120207455 shutdown_connections 2023-11-28T21:40:48.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:48.524+0000 7fae8350f640 1 -- 172.21.15.12:0/4120207455 wait complete. 2023-11-28T21:40:48.547 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:40:50.548 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:50.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc73af2a640 1 Processor -- start 2023-11-28T21:40:50.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc73af2a640 1 -- start start 2023-11-28T21:40:50.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc73af2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73406e300 0x7fc734156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:50.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc73af2a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc73406cf40 0x7fc734158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:50.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc73af2a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc734067410 con 0x7fc734150c20 2023-11-28T21:40:50.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc73af2a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73406e090 con 0x7fc73406cf40 2023-11-28T21:40:50.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc73af2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc734157f30 con 0x7fc73406e300 2023-11-28T21:40:50.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc738cc6640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc734150c20 0x7fc7340c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:46808/0 (socket says 172.21.15.12:46808) 2023-11-28T21:40:50.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc738cc6640 1 -- 172.21.15.12:0/607379619 learned_addr learned my addr 172.21.15.12:0/607379619 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:50.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc733fff640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73406e300 0x7fc734156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:50.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7394c7640 1 --2- 172.21.15.12:0/607379619 >> v2:172.21.15.144:3300/0 conn(0x7fc73406cf40 0x7fc734158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:50.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998408530 0 0) 0x7fc734067410 con 0x7fc734150c20 2023-11-28T21:40:50.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc724005740 con 0x7fc734150c20 2023-11-28T21:40:50.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2746247140 0 0) 0x7fc724005740 con 0x7fc734150c20 2023-11-28T21:40:50.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73406e300 msgr2=0x7fc734156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73406e300 0x7fc734156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 >> v2:172.21.15.144:3300/0 conn(0x7fc73406cf40 msgr2=0x7fc734158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 --2- 172.21.15.12:0/607379619 >> v2:172.21.15.144:3300/0 conn(0x7fc73406cf40 0x7fc734158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc734151000 con 0x7fc734150c20 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fc728001680 con 0x7fc734150c20 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 >> v1:172.21.15.12:6789/0 conn(0x7fc734150c20 legacy=0x7fc7340c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc724005c00 0x7fc724006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 0x7fc724006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73406e300 0x7fc7240073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73406e090 con 0x7fc7240069e0 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734157f30 con 0x7fc724005c00 2023-11-28T21:40:50.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72400dbd0 con 0x7fc73406e300 2023-11-28T21:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc733fff640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 0x7fc724006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc738cc6640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc724005c00 0x7fc724006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7394c7640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73406e300 0x7fc7240073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc733fff640 1 -- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73406e300 msgr2=0x7fc7240073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc733fff640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73406e300 0x7fc7240073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc733fff640 1 -- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc724005c00 msgr2=0x7fc724006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc733fff640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc724005c00 0x7fc724006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc733fff640 1 -- 172.21.15.12:0/607379619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc724005740 con 0x7fc7240069e0 2023-11-28T21:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc7394c7640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73406e300 0x7fc7240073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:40:50.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.700+0000 7fc738cc6640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc724005c00 0x7fc724006020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:50.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc733fff640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 0x7fc724006e60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc720009ae0 tx=0x7fc72001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4480bfe9af6d42dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:50.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200a6460 con 0x7fc7240069e0 2023-11-28T21:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7200a65c0 con 0x7fc7240069e0 2023-11-28T21:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc7337fe640 1 -- 172.21.15.12:0/607379619 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200a6970 con 0x7fc7240069e0 2023-11-28T21:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc73af2a640 1 -- 172.21.15.12:0/607379619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 msgr2=0x7fc724006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc73af2a640 1 --2- 172.21.15.12:0/607379619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 0x7fc724006e60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc720009ae0 tx=0x7fc72001ca50 comp rx=0 tx=0).stop 2023-11-28T21:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc7394c7640 1 -- 172.21.15.12:0/607379619 reap_dead start 2023-11-28T21:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc73af2a640 1 -- 172.21.15.12:0/607379619 shutdown_connections 2023-11-28T21:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc73af2a640 1 -- 172.21.15.12:0/607379619 >> 172.21.15.12:0/607379619 conn(0x7fc734066360 msgr2=0x7fc734066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc73af2a640 1 -- 172.21.15.12:0/607379619 shutdown_connections 2023-11-28T21:40:50.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.704+0000 7fc73af2a640 1 -- 172.21.15.12:0/607379619 wait complete. 2023-11-28T21:40:50.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc73af2a640 1 Processor -- start 2023-11-28T21:40:50.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc73af2a640 1 -- start start 2023-11-28T21:40:50.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc73af2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 0x7fc7341ef820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:50.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc73af2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7340c5120 0x7fc7340c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc73af2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7340ca010 0x7fc7340c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc73af2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340cb140 con 0x7fc7240069e0 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc73af2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340c37d0 con 0x7fc7340c5120 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc73af2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340ca5c0 con 0x7fc7340ca010 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc738cc6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 0x7fc7341ef820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc738cc6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 0x7fc7341ef820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39150/0 (socket says 172.21.15.12:39150) 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc738cc6640 1 -- 172.21.15.12:0/425160706 learned_addr learned my addr 172.21.15.12:0/425160706 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc733fff640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7340c5120 0x7fc7340c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc7394c7640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7340ca010 0x7fc7340c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc738cc6640 1 -- 172.21.15.12:0/425160706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7340ca010 msgr2=0x7fc7340c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:50.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc738cc6640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7340ca010 0x7fc7340c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc738cc6640 1 -- 172.21.15.12:0/425160706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7340c5120 msgr2=0x7fc7340c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc738cc6640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7340c5120 0x7fc7340c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc738cc6640 1 -- 172.21.15.12:0/425160706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7340ca8a0 con 0x7fc7240069e0 2023-11-28T21:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc738cc6640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 0x7fc7341ef820 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc7280058b0 tx=0x7fc728011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc7317fa640 1 -- 172.21.15.12:0/425160706 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72800fae0 con 0x7fc7240069e0 2023-11-28T21:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc7317fa640 1 -- 172.21.15.12:0/425160706 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72800fc40 con 0x7fc7240069e0 2023-11-28T21:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc7317fa640 1 -- 172.21.15.12:0/425160706 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7280016e0 con 0x7fc7240069e0 2023-11-28T21:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.708+0000 7fc73af2a640 1 -- 172.21.15.12:0/425160706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7340cab80 con 0x7fc7240069e0 2023-11-28T21:40:50.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.712+0000 7fc73af2a640 1 -- 172.21.15.12:0/425160706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7340c3a20 con 0x7fc7240069e0 2023-11-28T21:40:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.712+0000 7fc73af2a640 1 -- 172.21.15.12:0/425160706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc734074f60 con 0x7fc7240069e0 2023-11-28T21:40:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.712+0000 7fc7317fa640 1 -- 172.21.15.12:0/425160706 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc72801f460 con 0x7fc7240069e0 2023-11-28T21:40:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.712+0000 7fc7317fa640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc718066470 0x7fc718068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.712+0000 7fc7317fa640 1 -- 172.21.15.12:0/425160706 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6805+0+0 (secure 0 0 0) 0x7fc728087610 con 0x7fc7240069e0 2023-11-28T21:40:50.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.712+0000 7fc733fff640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc718066470 0x7fc718068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:50.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.712+0000 7fc733fff640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc718066470 0x7fc718068930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc7200ad000 tx=0x7fc7200b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:50.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.716+0000 7fc7317fa640 1 -- 172.21.15.12:0/425160706 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc7280545a0 con 0x7fc7240069e0 2023-11-28T21:40:50.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.904+0000 7fc73af2a640 1 -- 172.21.15.12:0/425160706 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7341582d0 con 0x7fc7240069e0 2023-11-28T21:40:50.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc7317fa640 1 -- 172.21.15.12:0/425160706 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fc728057bc0 con 0x7fc7240069e0 2023-11-28T21:40:50.910 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:50.910 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:21.761090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:50.910 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-11-28T21:40:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 -- 172.21.15.12:0/425160706 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc718066470 msgr2=0x7fc718068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc718066470 0x7fc718068930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc7200ad000 tx=0x7fc7200b2040 comp rx=0 tx=0).stop 2023-11-28T21:40:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 -- 172.21.15.12:0/425160706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 msgr2=0x7fc7341ef820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 0x7fc7341ef820 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc7280058b0 tx=0x7fc728011d20 comp rx=0 tx=0).stop 2023-11-28T21:40:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 -- 172.21.15.12:0/425160706 shutdown_connections 2023-11-28T21:40:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc718066470 0x7fc718068930 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:50.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7340ca010 0x7fc7340c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7340c5120 0x7fc7340c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 --2- 172.21.15.12:0/425160706 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7240069e0 0x7fc7341ef820 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 -- 172.21.15.12:0/425160706 >> 172.21.15.12:0/425160706 conn(0x7fc734066360 msgr2=0x7fc734070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 -- 172.21.15.12:0/425160706 shutdown_connections 2023-11-28T21:40:50.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:50.908+0000 7fc73af2a640 1 -- 172.21.15.12:0/425160706 wait complete. 2023-11-28T21:40:50.931 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:40:52.932 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:53.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd04432640 1 Processor -- start 2023-11-28T21:40:53.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd04432640 1 -- start start 2023-11-28T21:40:53.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd04432640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 0x7efcfc0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:53.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd04432640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efcfc15dc80 0x7efcfc0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:53.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd04432640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc15acc0 con 0x7efcfc0cfba0 2023-11-28T21:40:53.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd04432640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc159cb0 con 0x7efcfc15dc80 2023-11-28T21:40:53.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd04432640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc159ea0 con 0x7efcfc153e40 2023-11-28T21:40:53.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd029cf640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efcfc0cfba0 0x7efcfc0cb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:46816/0 (socket says 172.21.15.12:46816) 2023-11-28T21:40:53.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd029cf640 1 -- 172.21.15.12:0/3460794278 learned_addr learned my addr 172.21.15.12:0/3460794278 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:53.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd021ce640 1 --2- 172.21.15.12:0/3460794278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 0x7efcfc0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:53.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd019cd640 1 --2- 172.21.15.12:0/3460794278 >> v2:172.21.15.144:3300/0 conn(0x7efcfc15dc80 0x7efcfc0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:53.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd011cc640 1 -- 172.21.15.12:0/3460794278 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815112176 0 0) 0x7efcfc15acc0 con 0x7efcfc0cfba0 2023-11-28T21:40:53.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd011cc640 1 -- 172.21.15.12:0/3460794278 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce8003530 con 0x7efcfc0cfba0 2023-11-28T21:40:53.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd019cd640 1 -- 172.21.15.12:0/3460794278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 msgr2=0x7efcfc0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:53.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd019cd640 1 --2- 172.21.15.12:0/3460794278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 0x7efcfc0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:53.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.084+0000 7efd019cd640 1 -- 172.21.15.12:0/3460794278 >> v1:172.21.15.12:6789/0 conn(0x7efcfc0cfba0 legacy=0x7efcfc0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:53.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.088+0000 7efd011cc640 1 -- 172.21.15.12:0/3460794278 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2242400913 0 0) 0x7efce8003530 con 0x7efcfc0cfba0 2023-11-28T21:40:53.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.088+0000 7efd019cd640 1 -- 172.21.15.12:0/3460794278 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc15a140 con 0x7efcfc15dc80 2023-11-28T21:40:53.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.088+0000 7efd021ce640 1 --2- 172.21.15.12:0/3460794278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 0x7efcfc0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:40:53.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.088+0000 7efd019cd640 1 --2- 172.21.15.12:0/3460794278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcfc15dc80 0x7efcfc0cf660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efcec0097c0 tx=0x7efcec01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eae2dbe1fc1bf866 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:53.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.088+0000 7efd011cc640 1 -- 172.21.15.12:0/3460794278 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4001a00 con 0x7efcfc15dc80 2023-11-28T21:40:53.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.088+0000 7efd011cc640 1 -- 172.21.15.12:0/3460794278 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec0a7050 con 0x7efcfc15dc80 2023-11-28T21:40:53.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.088+0000 7efd011cc640 1 -- 172.21.15.12:0/3460794278 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0ab600 con 0x7efcfc15dc80 2023-11-28T21:40:53.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 -- 172.21.15.12:0/3460794278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcfc15dc80 msgr2=0x7efcfc0cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:53.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 --2- 172.21.15.12:0/3460794278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcfc15dc80 0x7efcfc0cf660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efcec0097c0 tx=0x7efcec01c3a0 comp rx=0 tx=0).stop 2023-11-28T21:40:53.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 -- 172.21.15.12:0/3460794278 shutdown_connections 2023-11-28T21:40:53.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 --2- 172.21.15.12:0/3460794278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcfc15dc80 0x7efcfc0cf660 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:53.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 --2- 172.21.15.12:0/3460794278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 0x7efcfc0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:53.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 -- 172.21.15.12:0/3460794278 >> 172.21.15.12:0/3460794278 conn(0x7efcfc06b7b0 msgr2=0x7efcfc066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:53.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 -- 172.21.15.12:0/3460794278 shutdown_connections 2023-11-28T21:40:53.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 -- 172.21.15.12:0/3460794278 wait complete. 2023-11-28T21:40:53.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 Processor -- start 2023-11-28T21:40:53.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 -- start start 2023-11-28T21:40:53.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcfc0cfba0 0x7efcfc1f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:53.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 0x7efcfc146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:53.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc15dc80 0x7efcfc148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:53.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0b2f90 con 0x7efcfc15dc80 2023-11-28T21:40:53.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1465f0 con 0x7efcfc0cfba0 2023-11-28T21:40:53.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd04432640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc146030 con 0x7efcfc153e40 2023-11-28T21:40:53.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd029cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc15dc80 0x7efcfc148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:53.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd029cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc15dc80 0x7efcfc148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39156/0 (socket says 172.21.15.12:39156) 2023-11-28T21:40:53.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd029cf640 1 -- 172.21.15.12:0/26912494 learned_addr learned my addr 172.21.15.12:0/26912494 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:53.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd021ce640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcfc0cfba0 0x7efcfc1f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:53.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd019cd640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 0x7efcfc146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:53.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd029cf640 1 -- 172.21.15.12:0/26912494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 msgr2=0x7efcfc146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:53.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd029cf640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 0x7efcfc146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:53.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd029cf640 1 -- 172.21.15.12:0/26912494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcfc0cfba0 msgr2=0x7efcfc1f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:53.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd029cf640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcfc0cfba0 0x7efcfc1f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:53.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd029cf640 1 -- 172.21.15.12:0/26912494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf8009420 con 0x7efcfc15dc80 2023-11-28T21:40:53.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.092+0000 7efd019cd640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 0x7efcfc146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.096+0000 7efd029cf640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc15dc80 0x7efcfc148480 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efcf400cc00 tx=0x7efcf400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.096+0000 7efcf2ffd640 1 -- 172.21.15.12:0/26912494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf400ce10 con 0x7efcfc15dc80 2023-11-28T21:40:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.096+0000 7efcf2ffd640 1 -- 172.21.15.12:0/26912494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf4012400 con 0x7efcfc15dc80 2023-11-28T21:40:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.096+0000 7efcf2ffd640 1 -- 172.21.15.12:0/26912494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf40127b0 con 0x7efcfc15dc80 2023-11-28T21:40:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.096+0000 7efd04432640 1 -- 172.21.15.12:0/26912494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcec009420 con 0x7efcfc15dc80 2023-11-28T21:40:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.096+0000 7efd04432640 1 -- 172.21.15.12:0/26912494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc1f20c0 con 0x7efcfc15dc80 2023-11-28T21:40:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.096+0000 7efd04432640 1 -- 172.21.15.12:0/26912494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc074f60 con 0x7efcfc15dc80 2023-11-28T21:40:53.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.096+0000 7efcf2ffd640 1 -- 172.21.15.12:0/26912494 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7efcf4012910 con 0x7efcfc15dc80 2023-11-28T21:40:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.096+0000 7efcf2ffd640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efce00664c0 0x7efce0068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:53.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.100+0000 7efcf2ffd640 1 -- 172.21.15.12:0/26912494 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6805+0+0 (secure 0 0 0) 0x7efcf4084910 con 0x7efcfc15dc80 2023-11-28T21:40:53.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.100+0000 7efd021ce640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efce00664c0 0x7efce0068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:53.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.100+0000 7efd021ce640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efce00664c0 0x7efce0068980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009cf0 tx=0x7efcf8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:53.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.104+0000 7efcf2ffd640 1 -- 172.21.15.12:0/26912494 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efcf4051920 con 0x7efcfc15dc80 2023-11-28T21:40:53.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 -- 172.21.15.12:0/26912494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcfc1589a0 con 0x7efcfc15dc80 2023-11-28T21:40:53.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efcf2ffd640 1 -- 172.21.15.12:0/26912494 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6506 (secure 0 0 0) 0x7efcf4054f40 con 0x7efcfc15dc80 2023-11-28T21:40:53.295 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:53.295 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:21.761090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:53.295 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-11-28T21:40:53.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 -- 172.21.15.12:0/26912494 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efce00664c0 msgr2=0x7efce0068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:53.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efce00664c0 0x7efce0068980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009cf0 tx=0x7efcf8009190 comp rx=0 tx=0).stop 2023-11-28T21:40:53.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 -- 172.21.15.12:0/26912494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc15dc80 msgr2=0x7efcfc148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:53.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc15dc80 0x7efcfc148480 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efcf400cc00 tx=0x7efcf400a430 comp rx=0 tx=0).stop 2023-11-28T21:40:53.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 -- 172.21.15.12:0/26912494 shutdown_connections 2023-11-28T21:40:53.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efce00664c0 0x7efce0068980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:53.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcfc15dc80 0x7efcfc148480 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:53.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcfc153e40 0x7efcfc146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:53.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 --2- 172.21.15.12:0/26912494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcfc0cfba0 0x7efcfc1f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:53.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 -- 172.21.15.12:0/26912494 >> 172.21.15.12:0/26912494 conn(0x7efcfc06b7b0 msgr2=0x7efcfc071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:53.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 -- 172.21.15.12:0/26912494 shutdown_connections 2023-11-28T21:40:53.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:53.292+0000 7efd04432640 1 -- 172.21.15.12:0/26912494 wait complete. 2023-11-28T21:40:53.317 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:40:55.318 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:55.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.468+0000 7fd2b98d6640 1 Processor -- start 2023-11-28T21:40:55.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.468+0000 7fd2b98d6640 1 -- start start 2023-11-28T21:40:55.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b98d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2b4150c20 0x7fd2b4156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:55.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b98d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b4067410 con 0x7fd2b406cf40 2023-11-28T21:40:55.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b98d6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b406e090 con 0x7fd2b406e300 2023-11-28T21:40:55.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b98d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4157f30 con 0x7fd2b4150c20 2023-11-28T21:40:55.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b37fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd2b406cf40 0x7fd2b4158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36476/0 (socket says 172.21.15.12:36476) 2023-11-28T21:40:55.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b37fe640 1 -- 172.21.15.12:0/3650201238 learned_addr learned my addr 172.21.15.12:0/3650201238 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:55.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b2ffd640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2b4150c20 0x7fd2b4156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:55.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412006391 0 0) 0x7fd2b4067410 con 0x7fd2b406cf40 2023-11-28T21:40:55.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2a0003530 con 0x7fd2b406cf40 2023-11-28T21:40:55.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487418601 0 0) 0x7fd2b406e090 con 0x7fd2b406e300 2023-11-28T21:40:55.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2b4067410 con 0x7fd2b406e300 2023-11-28T21:40:55.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3511117772 0 0) 0x7fd2a0003530 con 0x7fd2b406cf40 2023-11-28T21:40:55.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2b4150c20 msgr2=0x7fd2b4156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:55.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2b4150c20 0x7fd2b4156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 >> v1:172.21.15.144:6789/0 conn(0x7fd2b406e300 legacy=0x7fd2b40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b4151000 con 0x7fd2b406cf40 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fd2ac001a00 con 0x7fd2b406cf40 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b2ffd640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2b4150c20 0x7fd2b4156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 >> v1:172.21.15.12:6789/0 conn(0x7fd2b406cf40 legacy=0x7fd2b4158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2b4150c20 0x7fd2a0003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a00040f0 0x7fd2a00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a0004d10 0x7fd2a0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4157f30 con 0x7fd2a0004d10 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a000bad0 con 0x7fd2a00040f0 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a000bc90 con 0x7fd2b4150c20 2023-11-28T21:40:55.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b37fe640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a0004d10 0x7fd2a0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:55.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b2ffd640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2b4150c20 0x7fd2a0003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:55.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2abfff640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a00040f0 0x7fd2a00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:55.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b37fe640 1 -- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2b4150c20 msgr2=0x7fd2a0003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:55.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b37fe640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2b4150c20 0x7fd2a0003bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:55.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b37fe640 1 -- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a00040f0 msgr2=0x7fd2a00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:55.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b37fe640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a00040f0 0x7fd2a00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:55.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2b37fe640 1 -- 172.21.15.12:0/3650201238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4009420 con 0x7fd2a0004d10 2023-11-28T21:40:55.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.472+0000 7fd2abfff640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a00040f0 0x7fd2a00047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:55.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b37fe640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a0004d10 0x7fd2a0005190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00a7c0 tx=0x7fd2ac01fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34de4dfd0a2cc25 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:55.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0aa630 con 0x7fd2a0004d10 2023-11-28T21:40:55.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac0aa7b0 con 0x7fd2a0004d10 2023-11-28T21:40:55.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b27fc640 1 -- 172.21.15.12:0/3650201238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0aaba0 con 0x7fd2a0004d10 2023-11-28T21:40:55.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b98d6640 1 -- 172.21.15.12:0/3650201238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a0004d10 msgr2=0x7fd2a0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:55.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b98d6640 1 --2- 172.21.15.12:0/3650201238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2a0004d10 0x7fd2a0005190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00a7c0 tx=0x7fd2ac01fa00 comp rx=0 tx=0).stop 2023-11-28T21:40:55.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b37fe640 1 -- 172.21.15.12:0/3650201238 reap_dead start 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b98d6640 1 -- 172.21.15.12:0/3650201238 shutdown_connections 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b98d6640 1 -- 172.21.15.12:0/3650201238 >> 172.21.15.12:0/3650201238 conn(0x7fd2b4066360 msgr2=0x7fd2b4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b98d6640 1 -- 172.21.15.12:0/3650201238 shutdown_connections 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b98d6640 1 -- 172.21.15.12:0/3650201238 wait complete. 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.476+0000 7fd2b98d6640 1 Processor -- start 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b98d6640 1 -- start start 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b98d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2a0004d10 0x7fd2b414f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b98d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2b4150090 0x7fd2b41e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b98d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b41e6a10 0x7fd2b41f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b98d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4153530 con 0x7fd2b4150090 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b98d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b414fe90 con 0x7fd2b41e6a10 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b98d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b40c4980 con 0x7fd2a0004d10 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2abfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2b4150090 0x7fd2b41e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:55.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2abfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2b4150090 0x7fd2b41e64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56302/0 (socket says 172.21.15.12:56302) 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2abfff640 1 -- 172.21.15.12:0/937743961 learned_addr learned my addr 172.21.15.12:0/937743961 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b2ffd640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2a0004d10 0x7fd2b414f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b37fe640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b41e6a10 0x7fd2b41f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2abfff640 1 -- 172.21.15.12:0/937743961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2a0004d10 msgr2=0x7fd2b414f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2abfff640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2a0004d10 0x7fd2b414f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2abfff640 1 -- 172.21.15.12:0/937743961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b41e6a10 msgr2=0x7fd2b41f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2abfff640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b41e6a10 0x7fd2b41f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2abfff640 1 -- 172.21.15.12:0/937743961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b41efe40 con 0x7fd2b4150090 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2abfff640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2b4150090 0x7fd2b41e64d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00e7b0 tx=0x7fd29c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b0ff9640 1 -- 172.21.15.12:0/937743961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c017070 con 0x7fd2b4150090 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b0ff9640 1 -- 172.21.15.12:0/937743961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c00ee60 con 0x7fd2b4150090 2023-11-28T21:40:55.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b98d6640 1 -- 172.21.15.12:0/937743961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b41f0120 con 0x7fd2b4150090 2023-11-28T21:40:55.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b0ff9640 1 -- 172.21.15.12:0/937743961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0052e0 con 0x7fd2b4150090 2023-11-28T21:40:55.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b98d6640 1 -- 172.21.15.12:0/937743961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b41f06e0 con 0x7fd2b4150090 2023-11-28T21:40:55.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.480+0000 7fd2b98d6640 1 -- 172.21.15.12:0/937743961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b41f30c0 con 0x7fd2b4150090 2023-11-28T21:40:55.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.484+0000 7fd2b0ff9640 1 -- 172.21.15.12:0/937743961 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd29c007020 con 0x7fd2b4150090 2023-11-28T21:40:55.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.484+0000 7fd2b0ff9640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd294066470 0x7fd294068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:55.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.484+0000 7fd2b0ff9640 1 -- 172.21.15.12:0/937743961 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6805+0+0 (secure 0 0 0) 0x7fd29c083cf0 con 0x7fd2b4150090 2023-11-28T21:40:55.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.484+0000 7fd2b2ffd640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd294066470 0x7fd294068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:55.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.484+0000 7fd2b2ffd640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd294066470 0x7fd294068930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009cb0 tx=0x7fd2a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:55.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.488+0000 7fd2b0ff9640 1 -- 172.21.15.12:0/937743961 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2b41f30c0 con 0x7fd2b4150090 2023-11-28T21:40:55.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 -- 172.21.15.12:0/937743961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2b4074f60 con 0x7fd2b4150090 2023-11-28T21:40:55.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b0ff9640 1 -- 172.21.15.12:0/937743961 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fd29c054710 con 0x7fd2b4150090 2023-11-28T21:40:55.683 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:55.683 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:21.761090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:55.684 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-11-28T21:40:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 -- 172.21.15.12:0/937743961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd294066470 msgr2=0x7fd294068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd294066470 0x7fd294068930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009cb0 tx=0x7fd2a4008040 comp rx=0 tx=0).stop 2023-11-28T21:40:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 -- 172.21.15.12:0/937743961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2b4150090 msgr2=0x7fd2b41e64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2b4150090 0x7fd2b41e64d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00e7b0 tx=0x7fd29c00ec70 comp rx=0 tx=0).stop 2023-11-28T21:40:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 -- 172.21.15.12:0/937743961 shutdown_connections 2023-11-28T21:40:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd294066470 0x7fd294068930 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b41e6a10 0x7fd2b41f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2b4150090 0x7fd2b41e64d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 --2- 172.21.15.12:0/937743961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2a0004d10 0x7fd2b414f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 -- 172.21.15.12:0/937743961 >> 172.21.15.12:0/937743961 conn(0x7fd2b4066360 msgr2=0x7fd2b4156400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 -- 172.21.15.12:0/937743961 shutdown_connections 2023-11-28T21:40:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:55.680+0000 7fd2b98d6640 1 -- 172.21.15.12:0/937743961 wait complete. 2023-11-28T21:40:55.705 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:40:57.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:40:57.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.856+0000 7f94b8d9d640 1 Processor -- start 2023-11-28T21:40:57.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.856+0000 7f94b8d9d640 1 -- start start 2023-11-28T21:40:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b8d9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f94b4151af0 0x7f94b40c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b8d9d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b4157b70 con 0x7f94b406e300 2023-11-28T21:40:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b8d9d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b4157e50 con 0x7f94b4151ed0 2023-11-28T21:40:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b8d9d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b40707f0 con 0x7f94b4151af0 2023-11-28T21:40:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b2d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f94b406e300 0x7f94b4072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36480/0 (socket says 172.21.15.12:36480) 2023-11-28T21:40:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b2d9d640 1 -- 172.21.15.12:0/3316765260 learned_addr learned my addr 172.21.15.12:0/3316765260 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:40:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b259c640 1 --2- 172.21.15.12:0/3316765260 >> v2:172.21.15.144:3300/0 conn(0x7f94b4151af0 0x7f94b40c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535950365 0 0) 0x7f94b4157b70 con 0x7f94b406e300 2023-11-28T21:40:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a0003530 con 0x7f94b406e300 2023-11-28T21:40:57.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2138105922 0 0) 0x7f94a0003530 con 0x7f94b406e300 2023-11-28T21:40:57.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 >> v2:172.21.15.144:3300/0 conn(0x7f94b4151af0 msgr2=0x7f94b40c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:57.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 --2- 172.21.15.12:0/3316765260 >> v2:172.21.15.144:3300/0 conn(0x7f94b4151af0 0x7f94b40c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:57.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 >> v1:172.21.15.144:6789/0 conn(0x7f94b4151ed0 legacy=0x7f94b40c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:57.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4158130 con 0x7f94b406e300 2023-11-28T21:40:57.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f94ac001a00 con 0x7f94b406e300 2023-11-28T21:40:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 >> v1:172.21.15.12:6789/0 conn(0x7f94b406e300 legacy=0x7f94b4072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a0003a40 0x7f94a0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94a00047d0 0x7f94a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a00051e0 0x7f94a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b40707f0 con 0x7f94a0003a40 2023-11-28T21:40:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a000c110 con 0x7f94a00051e0 2023-11-28T21:40:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a000c2d0 con 0x7f94a00047d0 2023-11-28T21:40:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b259c640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a0003a40 0x7f94a0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b2d9d640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a00051e0 0x7f94a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:57.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94abfff640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94a00047d0 0x7f94a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b259c640 1 -- 172.21.15.12:0/3316765260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94a00047d0 msgr2=0x7f94a0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b259c640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94a00047d0 0x7f94a0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b259c640 1 -- 172.21.15.12:0/3316765260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a00051e0 msgr2=0x7f94a000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:40:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b259c640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a00051e0 0x7f94a000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b259c640 1 -- 172.21.15.12:0/3316765260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0003530 con 0x7f94a0003a40 2023-11-28T21:40:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b2d9d640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a00051e0 0x7f94a000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:40:57.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.860+0000 7f94b2d9d640 1 -- 172.21.15.12:0/3316765260 reap_dead start 2023-11-28T21:40:57.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b259c640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a0003a40 0x7f94a0003e60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009ae0 tx=0x7f94a401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edec9b6059eb1195 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:57.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a40a6430 con 0x7f94a0003a40 2023-11-28T21:40:57.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a40a6590 con 0x7f94a0003a40 2023-11-28T21:40:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b1d9b640 1 -- 172.21.15.12:0/3316765260 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a40a6940 con 0x7f94a0003a40 2023-11-28T21:40:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3316765260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a0003a40 msgr2=0x7f94a0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b8d9d640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a0003a40 0x7f94a0003e60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009ae0 tx=0x7f94a401ca50 comp rx=0 tx=0).stop 2023-11-28T21:40:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3316765260 shutdown_connections 2023-11-28T21:40:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b8d9d640 1 --2- 172.21.15.12:0/3316765260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94a0003a40 0x7f94a0003e60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3316765260 >> 172.21.15.12:0/3316765260 conn(0x7f94b4066360 msgr2=0x7f94b4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3316765260 shutdown_connections 2023-11-28T21:40:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3316765260 wait complete. 2023-11-28T21:40:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.864+0000 7f94b8d9d640 1 Processor -- start 2023-11-28T21:40:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b8d9d640 1 -- start start 2023-11-28T21:40:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b8d9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94a0003a40 0x7f94b415b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b8d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94b4149d10 0x7f94b414a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b8d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b413f990 0x7f94b413fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b8d9d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b415a5a0 con 0x7f94b4149d10 2023-11-28T21:40:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b8d9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b414a690 con 0x7f94b413f990 2023-11-28T21:40:57.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b8d9d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b4159640 con 0x7f94a0003a40 2023-11-28T21:40:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94abfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94b4149d10 0x7f94b414a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94abfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94b4149d10 0x7f94b414a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56322/0 (socket says 172.21.15.12:56322) 2023-11-28T21:40:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94abfff640 1 -- 172.21.15.12:0/3119686595 learned_addr learned my addr 172.21.15.12:0/3119686595 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:40:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b2d9d640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b413f990 0x7f94b413fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b259c640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94a0003a40 0x7f94b415b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94abfff640 1 -- 172.21.15.12:0/3119686595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94a0003a40 msgr2=0x7f94b415b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94abfff640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94a0003a40 0x7f94b415b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94abfff640 1 -- 172.21.15.12:0/3119686595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b413f990 msgr2=0x7f94b413fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94abfff640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b413f990 0x7f94b413fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:57.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94abfff640 1 -- 172.21.15.12:0/3119686595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4159920 con 0x7f94b4149d10 2023-11-28T21:40:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b2d9d640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b413f990 0x7f94b413fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:40:57.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94abfff640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94b4149d10 0x7f94b414a150 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e550 tx=0x7f949c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94aaffd640 1 -- 172.21.15.12:0/3119686595 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c0052e0 con 0x7f94b4149d10 2023-11-28T21:40:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94aaffd640 1 -- 172.21.15.12:0/3119686595 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c005440 con 0x7f94b4149d10 2023-11-28T21:40:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3119686595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b4159c00 con 0x7f94b4149d10 2023-11-28T21:40:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94aaffd640 1 -- 172.21.15.12:0/3119686595 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c003820 con 0x7f94b4149d10 2023-11-28T21:40:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3119686595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b415a1c0 con 0x7f94b4149d10 2023-11-28T21:40:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.868+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3119686595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b4074f60 con 0x7f94b4149d10 2023-11-28T21:40:57.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.872+0000 7f94aaffd640 1 -- 172.21.15.12:0/3119686595 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f949c003980 con 0x7f94b4149d10 2023-11-28T21:40:57.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.872+0000 7f94aaffd640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94940664a0 0x7f9494068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:40:57.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.872+0000 7f94aaffd640 1 -- 172.21.15.12:0/3119686595 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6805+0+0 (secure 0 0 0) 0x7f949c083c10 con 0x7f94b4149d10 2023-11-28T21:40:57.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.872+0000 7f94b259c640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94940664a0 0x7f9494068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:40:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.872+0000 7f94b259c640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94940664a0 0x7f9494068960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f94a40ad000 tx=0x7f94a40046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:40:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:57.876+0000 7f94aaffd640 1 -- 172.21.15.12:0/3119686595 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f949c050c70 con 0x7f94b4149d10 2023-11-28T21:40:58.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.068+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3119686595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94b4072c10 con 0x7f94b4149d10 2023-11-28T21:40:58.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.068+0000 7f94aaffd640 1 -- 172.21.15.12:0/3119686595 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f949c054290 con 0x7f94b4149d10 2023-11-28T21:40:58.073 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:40:58.073 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:21.761090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:40:58.074 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-11-28T21:40:58.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3119686595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94940664a0 msgr2=0x7f9494068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:58.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94940664a0 0x7f9494068960 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f94a40ad000 tx=0x7f94a40046d0 comp rx=0 tx=0).stop 2023-11-28T21:40:58.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3119686595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94b4149d10 msgr2=0x7f94b414a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:40:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94b4149d10 0x7f94b414a150 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f949c00e550 tx=0x7f949c00ea10 comp rx=0 tx=0).stop 2023-11-28T21:40:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3119686595 shutdown_connections 2023-11-28T21:40:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f94940664a0 0x7f9494068960 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94b413f990 0x7f94b413fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f94b4149d10 0x7f94b414a150 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 --2- 172.21.15.12:0/3119686595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94a0003a40 0x7f94b415b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:40:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3119686595 >> 172.21.15.12:0/3119686595 conn(0x7f94b4066360 msgr2=0x7f94b40c6260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:40:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3119686595 shutdown_connections 2023-11-28T21:40:58.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:40:58.072+0000 7f94b8d9d640 1 -- 172.21.15.12:0/3119686595 wait complete. 2023-11-28T21:40:58.095 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:41:00.096 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:00.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafad7e6640 1 Processor -- start 2023-11-28T21:41:00.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafad7e6640 1 -- start start 2023-11-28T21:41:00.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafad7e6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fafa8153e00 0x7fafa80ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:00.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafad7e6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa80cba30 0x7fafa806ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafad7e6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa815ab40 con 0x7fafa806b1b0 2023-11-28T21:41:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafad7e6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa806a850 con 0x7fafa8153e00 2023-11-28T21:41:00.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafad7e6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80b2670 con 0x7fafa80cba30 2023-11-28T21:41:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafa77fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fafa806b1b0 0x7fafa80caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36482/0 (socket says 172.21.15.12:36482) 2023-11-28T21:41:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafa77fe640 1 -- 172.21.15.12:0/2525734480 learned_addr learned my addr 172.21.15.12:0/2525734480 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:41:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafa6ffd640 1 --2- 172.21.15.12:0/2525734480 >> v2:172.21.15.144:3300/0 conn(0x7fafa8153e00 0x7fafa80ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7faf9ffff640 1 --2- 172.21.15.12:0/2525734480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa80cba30 0x7fafa806ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafa67fc640 1 -- 172.21.15.12:0/2525734480 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952392281 0 0) 0x7fafa815ab40 con 0x7fafa806b1b0 2023-11-28T21:41:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafa67fc640 1 -- 172.21.15.12:0/2525734480 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf94003530 con 0x7fafa806b1b0 2023-11-28T21:41:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafa6ffd640 1 -- 172.21.15.12:0/2525734480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa80cba30 msgr2=0x7fafa806ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafa6ffd640 1 --2- 172.21.15.12:0/2525734480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa80cba30 0x7fafa806ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafa6ffd640 1 -- 172.21.15.12:0/2525734480 >> v1:172.21.15.12:6789/0 conn(0x7fafa806b1b0 legacy=0x7fafa80caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:00.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.248+0000 7fafa6ffd640 1 -- 172.21.15.12:0/2525734480 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa8159fe0 con 0x7fafa8153e00 2023-11-28T21:41:00.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafa6ffd640 1 --2- 172.21.15.12:0/2525734480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa8153e00 0x7fafa80ca3d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf980097c0 tx=0x7faf9801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0fba567ce6478bc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:00.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafa67fc640 1 -- 172.21.15.12:0/2525734480 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf980a8970 con 0x7fafa8153e00 2023-11-28T21:41:00.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafa67fc640 1 -- 172.21.15.12:0/2525734480 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf980a8ad0 con 0x7fafa8153e00 2023-11-28T21:41:00.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafa67fc640 1 -- 172.21.15.12:0/2525734480 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf980a8e60 con 0x7fafa8153e00 2023-11-28T21:41:00.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafad7e6640 1 -- 172.21.15.12:0/2525734480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa8153e00 msgr2=0x7fafa80ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:00.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafad7e6640 1 --2- 172.21.15.12:0/2525734480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa8153e00 0x7fafa80ca3d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faf980097c0 tx=0x7faf9801ec70 comp rx=0 tx=0).stop 2023-11-28T21:41:00.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafad7e6640 1 -- 172.21.15.12:0/2525734480 shutdown_connections 2023-11-28T21:41:00.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafad7e6640 1 --2- 172.21.15.12:0/2525734480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa80cba30 0x7fafa806ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:00.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafad7e6640 1 --2- 172.21.15.12:0/2525734480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa8153e00 0x7fafa80ca3d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:00.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafad7e6640 1 -- 172.21.15.12:0/2525734480 >> 172.21.15.12:0/2525734480 conn(0x7fafa80cf4e0 msgr2=0x7fafa80cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:00.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafad7e6640 1 -- 172.21.15.12:0/2525734480 shutdown_connections 2023-11-28T21:41:00.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.252+0000 7fafad7e6640 1 -- 172.21.15.12:0/2525734480 wait complete. 2023-11-28T21:41:00.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafad7e6640 1 Processor -- start 2023-11-28T21:41:00.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafad7e6640 1 -- start start 2023-11-28T21:41:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafad7e6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafa806b1b0 0x7fafa81f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafad7e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa80cba30 0x7fafa814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafad7e6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa8153e00 0x7fafa80c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafad7e6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa815e030 con 0x7fafa806b1b0 2023-11-28T21:41:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafad7e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80c80e0 con 0x7fafa80cba30 2023-11-28T21:41:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafad7e6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80c86a0 con 0x7fafa8153e00 2023-11-28T21:41:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafa806b1b0 0x7fafa81f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafa806b1b0 0x7fafa81f27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56330/0 (socket says 172.21.15.12:56330) 2023-11-28T21:41:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa6ffd640 1 -- 172.21.15.12:0/441146563 learned_addr learned my addr 172.21.15.12:0/441146563 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:00.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7faf9ffff640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa80cba30 0x7fafa814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:00.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa77fe640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa8153e00 0x7fafa80c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:00.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa6ffd640 1 -- 172.21.15.12:0/441146563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa8153e00 msgr2=0x7fafa80c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:00.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa6ffd640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa8153e00 0x7fafa80c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:00.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa6ffd640 1 -- 172.21.15.12:0/441146563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa80cba30 msgr2=0x7fafa814d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:00.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa6ffd640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa80cba30 0x7fafa814d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:00.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa6ffd640 1 -- 172.21.15.12:0/441146563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf98009420 con 0x7fafa806b1b0 2023-11-28T21:41:00.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa77fe640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa8153e00 0x7fafa80c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:41:00.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafa6ffd640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafa806b1b0 0x7fafa81f27a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faf980b3040 tx=0x7faf9801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7faf9f7fe640 1 -- 172.21.15.12:0/441146563 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf980b0d60 con 0x7fafa806b1b0 2023-11-28T21:41:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7faf9f7fe640 1 -- 172.21.15.12:0/441146563 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf980b9430 con 0x7fafa806b1b0 2023-11-28T21:41:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7faf9f7fe640 1 -- 172.21.15.12:0/441146563 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf980b97e0 con 0x7fafa806b1b0 2023-11-28T21:41:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafad7e6640 1 -- 172.21.15.12:0/441146563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa81f2ce0 con 0x7fafa806b1b0 2023-11-28T21:41:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.256+0000 7fafad7e6640 1 -- 172.21.15.12:0/441146563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa81f3250 con 0x7fafa806b1b0 2023-11-28T21:41:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.260+0000 7fafad7e6640 1 -- 172.21.15.12:0/441146563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa8074f60 con 0x7fafa806b1b0 2023-11-28T21:41:00.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.260+0000 7faf9f7fe640 1 -- 172.21.15.12:0/441146563 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faf980b7040 con 0x7fafa806b1b0 2023-11-28T21:41:00.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.260+0000 7faf9f7fe640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf880664a0 0x7faf88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:00.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.260+0000 7faf9f7fe640 1 -- 172.21.15.12:0/441146563 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6805+0+0 (secure 0 0 0) 0x7faf98121c50 con 0x7fafa806b1b0 2023-11-28T21:41:00.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.260+0000 7faf9ffff640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf880664a0 0x7faf88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:00.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.260+0000 7faf9ffff640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf880664a0 0x7faf88068960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fafa8158520 tx=0x7faf90009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:00.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.264+0000 7faf9f7fe640 1 -- 172.21.15.12:0/441146563 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faf980eec60 con 0x7fafa806b1b0 2023-11-28T21:41:00.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.452+0000 7fafad7e6640 1 -- 172.21.15.12:0/441146563 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafa80caca0 con 0x7fafa806b1b0 2023-11-28T21:41:00.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.452+0000 7faf9f7fe640 1 -- 172.21.15.12:0/441146563 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6506 (secure 0 0 0) 0x7faf980b5020 con 0x7fafa806b1b0 2023-11-28T21:41:00.457 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:00.457 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:21.761090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:00.457 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 -- 172.21.15.12:0/441146563 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf880664a0 msgr2=0x7faf88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf880664a0 0x7faf88068960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fafa8158520 tx=0x7faf90009190 comp rx=0 tx=0).stop 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 -- 172.21.15.12:0/441146563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafa806b1b0 msgr2=0x7fafa81f27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafa806b1b0 0x7fafa81f27a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7faf980b3040 tx=0x7faf9801ba10 comp rx=0 tx=0).stop 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 -- 172.21.15.12:0/441146563 shutdown_connections 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf880664a0 0x7faf88068960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa8153e00 0x7fafa80c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafa80cba30 0x7fafa814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 --2- 172.21.15.12:0/441146563 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fafa806b1b0 0x7fafa81f27a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 -- 172.21.15.12:0/441146563 >> 172.21.15.12:0/441146563 conn(0x7fafa80cf4e0 msgr2=0x7fafa8159490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:00.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 -- 172.21.15.12:0/441146563 shutdown_connections 2023-11-28T21:41:00.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:00.456+0000 7fafad7e6640 1 -- 172.21.15.12:0/441146563 wait complete. 2023-11-28T21:41:00.478 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:41:02.480 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:02.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fdb06b640 1 Processor -- start 2023-11-28T21:41:02.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fdb06b640 1 -- start start 2023-11-28T21:41:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fdb06b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3fd4150c20 0x7f3fd4156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fdb06b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406e300 0x7f3fd40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:02.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fdb06b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd4067410 con 0x7f3fd406cf40 2023-11-28T21:41:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fdb06b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd406e090 con 0x7f3fd4150c20 2023-11-28T21:41:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fdb06b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd4157f30 con 0x7f3fd406e300 2023-11-28T21:41:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd9608640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3fd406cf40 0x7f3fd4158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36490/0 (socket says 172.21.15.12:36490) 2023-11-28T21:41:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd9608640 1 -- 172.21.15.12:0/3925408475 learned_addr learned my addr 172.21.15.12:0/3925408475 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:41:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd3fff640 1 --2- 172.21.15.12:0/3925408475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406e300 0x7f3fd40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:02.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd8e07640 1 --2- 172.21.15.12:0/3925408475 >> v2:172.21.15.144:3300/0 conn(0x7f3fd4150c20 0x7f3fd4156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:02.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd37fe640 1 -- 172.21.15.12:0/3925408475 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357390147 0 0) 0x7f3fd4067410 con 0x7f3fd406cf40 2023-11-28T21:41:02.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd37fe640 1 -- 172.21.15.12:0/3925408475 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc4003530 con 0x7f3fd406cf40 2023-11-28T21:41:02.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd3fff640 1 -- 172.21.15.12:0/3925408475 >> v2:172.21.15.144:3300/0 conn(0x7f3fd4150c20 msgr2=0x7f3fd4156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:02.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd3fff640 1 --2- 172.21.15.12:0/3925408475 >> v2:172.21.15.144:3300/0 conn(0x7f3fd4150c20 0x7f3fd4156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:02.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd3fff640 1 -- 172.21.15.12:0/3925408475 >> v1:172.21.15.12:6789/0 conn(0x7f3fd406cf40 legacy=0x7f3fd4158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:02.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd37fe640 1 -- 172.21.15.12:0/3925408475 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4259594702 0 0) 0x7f3fc4003530 con 0x7f3fd406cf40 2023-11-28T21:41:02.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.632+0000 7f3fd3fff640 1 -- 172.21.15.12:0/3925408475 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4151000 con 0x7f3fd406e300 2023-11-28T21:41:02.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fd3fff640 1 --2- 172.21.15.12:0/3925408475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406e300 0x7f3fd40c6dd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00097c0 tx=0x7f3fc001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ecf5a0ea3072d99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:02.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fd37fe640 1 -- 172.21.15.12:0/3925408475 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc001a00 con 0x7f3fd406e300 2023-11-28T21:41:02.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fd37fe640 1 -- 172.21.15.12:0/3925408475 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc00a7050 con 0x7f3fd406e300 2023-11-28T21:41:02.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fd37fe640 1 -- 172.21.15.12:0/3925408475 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00ab600 con 0x7f3fd406e300 2023-11-28T21:41:02.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fdb06b640 1 -- 172.21.15.12:0/3925408475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406e300 msgr2=0x7f3fd40c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:02.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fdb06b640 1 --2- 172.21.15.12:0/3925408475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406e300 0x7f3fd40c6dd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00097c0 tx=0x7f3fc001c3a0 comp rx=0 tx=0).stop 2023-11-28T21:41:02.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fdb06b640 1 -- 172.21.15.12:0/3925408475 shutdown_connections 2023-11-28T21:41:02.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fdb06b640 1 --2- 172.21.15.12:0/3925408475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406e300 0x7f3fd40c6dd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:02.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fdb06b640 1 --2- 172.21.15.12:0/3925408475 >> v2:172.21.15.144:3300/0 conn(0x7f3fd4150c20 0x7f3fd4156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:02.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fdb06b640 1 -- 172.21.15.12:0/3925408475 >> 172.21.15.12:0/3925408475 conn(0x7f3fd4066360 msgr2=0x7f3fd4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:02.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fdb06b640 1 -- 172.21.15.12:0/3925408475 shutdown_connections 2023-11-28T21:41:02.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.636+0000 7f3fdb06b640 1 -- 172.21.15.12:0/3925408475 wait complete. 2023-11-28T21:41:02.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fdb06b640 1 Processor -- start 2023-11-28T21:41:02.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fdb06b640 1 -- start start 2023-11-28T21:41:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fdb06b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406cf40 0x7f3fd41538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fdb06b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fd406e300 0x7f3fd40c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fdb06b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4150c20 0x7f3fd4151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fdb06b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd4153df0 con 0x7f3fd4150c20 2023-11-28T21:41:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fdb06b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41f2de0 con 0x7f3fd406e300 2023-11-28T21:41:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fdb06b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd41f30a0 con 0x7f3fd406cf40 2023-11-28T21:41:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd9608640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4150c20 0x7f3fd4151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd9608640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4150c20 0x7f3fd4151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56342/0 (socket says 172.21.15.12:56342) 2023-11-28T21:41:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd9608640 1 -- 172.21.15.12:0/976345843 learned_addr learned my addr 172.21.15.12:0/976345843 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd3fff640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fd406e300 0x7f3fd40c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd8e07640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406cf40 0x7f3fd41538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd9608640 1 -- 172.21.15.12:0/976345843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406cf40 msgr2=0x7f3fd41538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd9608640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406cf40 0x7f3fd41538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd9608640 1 -- 172.21.15.12:0/976345843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fd406e300 msgr2=0x7f3fd40c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd9608640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fd406e300 0x7f3fd40c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd9608640 1 -- 172.21.15.12:0/976345843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0009420 con 0x7f3fd4150c20 2023-11-28T21:41:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd9608640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4150c20 0x7f3fd4151e30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc001840 tx=0x7f3fcc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.640+0000 7f3fd17fa640 1 -- 172.21.15.12:0/976345843 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc012400 con 0x7f3fd4150c20 2023-11-28T21:41:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.644+0000 7f3fd17fa640 1 -- 172.21.15.12:0/976345843 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fcc012560 con 0x7f3fd4150c20 2023-11-28T21:41:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.644+0000 7f3fd17fa640 1 -- 172.21.15.12:0/976345843 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc012910 con 0x7f3fd4150c20 2023-11-28T21:41:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.644+0000 7f3fdb06b640 1 -- 172.21.15.12:0/976345843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd41f3380 con 0x7f3fd4150c20 2023-11-28T21:41:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.644+0000 7f3fdb06b640 1 -- 172.21.15.12:0/976345843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd41f3940 con 0x7f3fd4150c20 2023-11-28T21:41:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.644+0000 7f3fdb06b640 1 -- 172.21.15.12:0/976345843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd4066ef0 con 0x7f3fd4150c20 2023-11-28T21:41:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.644+0000 7f3fd17fa640 1 -- 172.21.15.12:0/976345843 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3fcc012a70 con 0x7f3fd4150c20 2023-11-28T21:41:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.644+0000 7f3fd17fa640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3fb8066470 0x7f3fb8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.644+0000 7f3fd17fa640 1 -- 172.21.15.12:0/976345843 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6805+0+0 (secure 0 0 0) 0x7f3fcc0849f0 con 0x7f3fd4150c20 2023-11-28T21:41:02.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.644+0000 7f3fd8e07640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3fb8066470 0x7f3fb8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:02.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.644+0000 7f3fd8e07640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3fb8066470 0x7f3fb8068930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009ac0 tx=0x7f3fc8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.648+0000 7f3fd17fa640 1 -- 172.21.15.12:0/976345843 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3fcc051a30 con 0x7f3fd4150c20 2023-11-28T21:41:02.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.836+0000 7f3fdb06b640 1 -- 172.21.15.12:0/976345843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fd4074f60 con 0x7f3fd4150c20 2023-11-28T21:41:02.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.836+0000 7f3fd17fa640 1 -- 172.21.15.12:0/976345843 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f3fcc055050 con 0x7f3fd4150c20 2023-11-28T21:41:02.841 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:02.841 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5232,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:40:21.761090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5307},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5307":{"gid":5307,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":31,"addr":"172.21.15.12:6838/3741155090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3741155090},{"type":"v1","addr":"172.21.15.12:6838","nonce":3741155090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:02.841 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2023-11-28T21:41:02.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 -- 172.21.15.12:0/976345843 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3fb8066470 msgr2=0x7f3fb8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:02.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3fb8066470 0x7f3fb8068930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009ac0 tx=0x7f3fc8009190 comp rx=0 tx=0).stop 2023-11-28T21:41:02.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 -- 172.21.15.12:0/976345843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4150c20 msgr2=0x7f3fd4151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:02.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4150c20 0x7f3fd4151e30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc001840 tx=0x7f3fcc00a430 comp rx=0 tx=0).stop 2023-11-28T21:41:02.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 -- 172.21.15.12:0/976345843 shutdown_connections 2023-11-28T21:41:02.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3fb8066470 0x7f3fb8068930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:02.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4150c20 0x7f3fd4151e30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:02.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fd406e300 0x7f3fd40c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:02.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 --2- 172.21.15.12:0/976345843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd406cf40 0x7f3fd41538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:02.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 -- 172.21.15.12:0/976345843 >> 172.21.15.12:0/976345843 conn(0x7f3fd4066360 msgr2=0x7f3fd40ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:02.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 -- 172.21.15.12:0/976345843 shutdown_connections 2023-11-28T21:41:02.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:02.840+0000 7f3fdb06b640 1 -- 172.21.15.12:0/976345843 wait complete. 2023-11-28T21:41:02.862 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:41:04.863 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:05.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.016+0000 7f30878fd640 1 Processor -- start 2023-11-28T21:41:05.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.016+0000 7f30878fd640 1 -- start start 2023-11-28T21:41:05.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.016+0000 7f30878fd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3080158420 0x7f308006d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.016+0000 7f30878fd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30800c3460 con 0x7f30800c38f0 2023-11-28T21:41:05.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.016+0000 7f30878fd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30800c2b70 con 0x7f3080161600 2023-11-28T21:41:05.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.016+0000 7f30878fd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3080158800 con 0x7f3080158420 2023-11-28T21:41:05.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.016+0000 7f3084e98640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f30800c38f0 0x7f30800c3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40908/0 (socket says 172.21.15.12:40908) 2023-11-28T21:41:05.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.016+0000 7f3084e98640 1 -- 172.21.15.12:0/518860542 learned_addr learned my addr 172.21.15.12:0/518860542 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:41:05.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.020+0000 7f3085699640 1 --2- 172.21.15.12:0/518860542 >> v2:172.21.15.144:3300/0 conn(0x7f3080158420 0x7f308006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.020+0000 7f3077fff640 1 -- 172.21.15.12:0/518860542 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980311691 0 0) 0x7f30800c3460 con 0x7f30800c38f0 2023-11-28T21:41:05.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.020+0000 7f3077fff640 1 -- 172.21.15.12:0/518860542 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f306c003530 con 0x7f30800c38f0 2023-11-28T21:41:05.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.020+0000 7f3085699640 1 -- 172.21.15.12:0/518860542 >> v1:172.21.15.144:6789/0 conn(0x7f3080161600 legacy=0x7f3080161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.020+0000 7f3077fff640 1 -- 172.21.15.12:0/518860542 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3897310131 0 0) 0x7f306c003530 con 0x7f30800c38f0 2023-11-28T21:41:05.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.020+0000 7f3085699640 1 -- 172.21.15.12:0/518860542 >> v1:172.21.15.12:6789/0 conn(0x7f30800c38f0 legacy=0x7f30800c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.020+0000 7f3085699640 1 -- 172.21.15.12:0/518860542 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308006e090 con 0x7f3080158420 2023-11-28T21:41:05.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.020+0000 7f3085699640 1 --2- 172.21.15.12:0/518860542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3080158420 0x7f308006d950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f307c0097c0 tx=0x7f307c01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b7b02d644406e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:05.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f3077fff640 1 -- 172.21.15.12:0/518860542 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070001680 con 0x7f3080158420 2023-11-28T21:41:05.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f3077fff640 1 -- 172.21.15.12:0/518860542 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c0a7050 con 0x7f3080158420 2023-11-28T21:41:05.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f3077fff640 1 -- 172.21.15.12:0/518860542 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c0ab600 con 0x7f3080158420 2023-11-28T21:41:05.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 -- 172.21.15.12:0/518860542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3080158420 msgr2=0x7f308006d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 --2- 172.21.15.12:0/518860542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3080158420 0x7f308006d950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f307c0097c0 tx=0x7f307c01c3a0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 -- 172.21.15.12:0/518860542 shutdown_connections 2023-11-28T21:41:05.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 --2- 172.21.15.12:0/518860542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3080158420 0x7f308006d950 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 -- 172.21.15.12:0/518860542 >> 172.21.15.12:0/518860542 conn(0x7f3080066360 msgr2=0x7f3080066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:05.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 -- 172.21.15.12:0/518860542 shutdown_connections 2023-11-28T21:41:05.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 -- 172.21.15.12:0/518860542 wait complete. 2023-11-28T21:41:05.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 Processor -- start 2023-11-28T21:41:05.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 -- start start 2023-11-28T21:41:05.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30800c38f0 0x7f30801497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3080158420 0x7f30800ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3080161600 0x7f30800caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3080151cf0 con 0x7f3080161600 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30801fa270 con 0x7f30800c38f0 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f30878fd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3080154650 con 0x7f3080158420 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f3085e9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3080161600 0x7f30800caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.024+0000 7f3085e9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3080161600 0x7f30800caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50780/0 (socket says 172.21.15.12:50780) 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3085e9a640 1 -- 172.21.15.12:0/3078901134 learned_addr learned my addr 172.21.15.12:0/3078901134 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3085699640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30800c38f0 0x7f30801497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3084e98640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3080158420 0x7f30800ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3085e9a640 1 -- 172.21.15.12:0/3078901134 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3080158420 msgr2=0x7f30800ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3085e9a640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3080158420 0x7f30800ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3085e9a640 1 -- 172.21.15.12:0/3078901134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30800c38f0 msgr2=0x7f30801497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3085e9a640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30800c38f0 0x7f30801497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3085e9a640 1 -- 172.21.15.12:0/3078901134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c009420 con 0x7f3080161600 2023-11-28T21:41:05.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3085e9a640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3080161600 0x7f30800caca0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f307800c730 tx=0x7f307800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:05.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3075ffb640 1 -- 172.21.15.12:0/3078901134 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078003b80 con 0x7f3080161600 2023-11-28T21:41:05.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3075ffb640 1 -- 172.21.15.12:0/3078901134 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3078003ce0 con 0x7f3080161600 2023-11-28T21:41:05.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3075ffb640 1 -- 172.21.15.12:0/3078901134 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307800aa30 con 0x7f3080161600 2023-11-28T21:41:05.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f30878fd640 1 -- 172.21.15.12:0/3078901134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3080154930 con 0x7f3080161600 2023-11-28T21:41:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f30878fd640 1 -- 172.21.15.12:0/3078901134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3080154ef0 con 0x7f3080161600 2023-11-28T21:41:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f30878fd640 1 -- 172.21.15.12:0/3078901134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30801fa420 con 0x7f3080161600 2023-11-28T21:41:05.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.028+0000 7f3075ffb640 1 -- 172.21.15.12:0/3078901134 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f307800ab90 con 0x7f3080161600 2023-11-28T21:41:05.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.032+0000 7f3075ffb640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f30640664a0 0x7f3064068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.032+0000 7f3075ffb640 1 -- 172.21.15.12:0/3078901134 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f3078083ff0 con 0x7f3080161600 2023-11-28T21:41:05.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.032+0000 7f3085699640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f30640664a0 0x7f3064068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.032+0000 7f3085699640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f30640664a0 0x7f3064068960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f307c0ad000 tx=0x7f307c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:05.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.036+0000 7f3075ffb640 1 -- 172.21.15.12:0/3078901134 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f30801fa420 con 0x7f3080161600 2023-11-28T21:41:05.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.224+0000 7f30878fd640 1 -- 172.21.15.12:0/3078901134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30801fa3e0 con 0x7f3080161600 2023-11-28T21:41:05.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.224+0000 7f3075ffb640 1 -- 172.21.15.12:0/3078901134 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f3078050fe0 con 0x7f3080161600 2023-11-28T21:41:05.229 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:05.229 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:03.398744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:05.230 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2023-11-28T21:41:05.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 -- 172.21.15.12:0/3078901134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f30640664a0 msgr2=0x7f3064068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f30640664a0 0x7f3064068960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f307c0ad000 tx=0x7f307c0a6000 comp rx=0 tx=0).stop 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 -- 172.21.15.12:0/3078901134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3080161600 msgr2=0x7f30800caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3080161600 0x7f30800caca0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f307800c730 tx=0x7f307800cbf0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 -- 172.21.15.12:0/3078901134 shutdown_connections 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f30640664a0 0x7f3064068960 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3080161600 0x7f30800caca0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3080158420 0x7f30800ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 --2- 172.21.15.12:0/3078901134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30800c38f0 0x7f30801497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 -- 172.21.15.12:0/3078901134 >> 172.21.15.12:0/3078901134 conn(0x7f3080066360 msgr2=0x7f30800c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 -- 172.21.15.12:0/3078901134 shutdown_connections 2023-11-28T21:41:05.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.228+0000 7f30878fd640 1 -- 172.21.15.12:0/3078901134 wait complete. 2023-11-28T21:41:05.251 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2023-11-28T21:41:05.251 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:41:05.251 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:05.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804ed01640 1 Processor -- start 2023-11-28T21:41:05.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804ed01640 1 -- start start 2023-11-28T21:41:05.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804ed01640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806e300 0x7f8048156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804ed01640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f804806cf40 0x7f8048158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804ed01640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8048067410 con 0x7f8048150c20 2023-11-28T21:41:05.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804ed01640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804806e090 con 0x7f804806cf40 2023-11-28T21:41:05.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804ed01640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8048157f30 con 0x7f804806e300 2023-11-28T21:41:05.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804d29e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f804806cf40 0x7f8048158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804ca9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8048150c20 0x7f80480c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40916/0 (socket says 172.21.15.12:40916) 2023-11-28T21:41:05.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804ca9d640 1 -- 172.21.15.12:0/4126204574 learned_addr learned my addr 172.21.15.12:0/4126204574 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:41:05.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f8047fff640 1 --2- 172.21.15.12:0/4126204574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806e300 0x7f8048156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804d29e640 1 -- 172.21.15.12:0/4126204574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806e300 msgr2=0x7f8048156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:05.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804d29e640 1 --2- 172.21.15.12:0/4126204574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806e300 0x7f8048156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804d29e640 1 -- 172.21.15.12:0/4126204574 >> v1:172.21.15.12:6789/0 conn(0x7f8048150c20 legacy=0x7f80480c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.392+0000 7f804d29e640 1 -- 172.21.15.12:0/4126204574 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048151000 con 0x7f804806cf40 2023-11-28T21:41:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f804d29e640 1 --2- 172.21.15.12:0/4126204574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804806cf40 0x7f8048158410 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8040009d90 tx=0x7f804001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abfeaecf0b399c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:05.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f80477fe640 1 -- 172.21.15.12:0/4126204574 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400a7a20 con 0x7f804806cf40 2023-11-28T21:41:05.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f80477fe640 1 -- 172.21.15.12:0/4126204574 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80400a7b80 con 0x7f804806cf40 2023-11-28T21:41:05.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f80477fe640 1 -- 172.21.15.12:0/4126204574 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400af440 con 0x7f804806cf40 2023-11-28T21:41:05.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f804ed01640 1 -- 172.21.15.12:0/4126204574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804806cf40 msgr2=0x7f8048158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f804ed01640 1 --2- 172.21.15.12:0/4126204574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804806cf40 0x7f8048158410 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8040009d90 tx=0x7f804001eb60 comp rx=0 tx=0).stop 2023-11-28T21:41:05.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f804ed01640 1 -- 172.21.15.12:0/4126204574 shutdown_connections 2023-11-28T21:41:05.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f804ed01640 1 --2- 172.21.15.12:0/4126204574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f804806cf40 0x7f8048158410 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f804ed01640 1 --2- 172.21.15.12:0/4126204574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806e300 0x7f8048156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f804ed01640 1 -- 172.21.15.12:0/4126204574 >> 172.21.15.12:0/4126204574 conn(0x7f8048066360 msgr2=0x7f8048066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:05.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f804ed01640 1 -- 172.21.15.12:0/4126204574 shutdown_connections 2023-11-28T21:41:05.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.396+0000 7f804ed01640 1 -- 172.21.15.12:0/4126204574 wait complete. 2023-11-28T21:41:05.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804ed01640 1 Processor -- start 2023-11-28T21:41:05.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804ed01640 1 -- start start 2023-11-28T21:41:05.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804ed01640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806cf40 0x7f8048156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804ed01640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f804806e300 0x7f80481532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804ed01640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8048150c20 0x7f80480c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804ed01640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80481563d0 con 0x7f8048150c20 2023-11-28T21:41:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804ed01640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80481f6fe0 con 0x7f804806e300 2023-11-28T21:41:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804ed01640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80481538b0 con 0x7f804806cf40 2023-11-28T21:41:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804d29e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8048150c20 0x7f80480c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804d29e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8048150c20 0x7f80480c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50788/0 (socket says 172.21.15.12:50788) 2023-11-28T21:41:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804d29e640 1 -- 172.21.15.12:0/527872938 learned_addr learned my addr 172.21.15.12:0/527872938 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f8047fff640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f804806e300 0x7f80481532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804ca9d640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806cf40 0x7f8048156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804d29e640 1 -- 172.21.15.12:0/527872938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806cf40 msgr2=0x7f8048156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804d29e640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806cf40 0x7f8048156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804d29e640 1 -- 172.21.15.12:0/527872938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f804806e300 msgr2=0x7f80481532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804d29e640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f804806e300 0x7f80481532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804d29e640 1 -- 172.21.15.12:0/527872938 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040009a70 con 0x7f8048150c20 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f8047fff640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f804806e300 0x7f80481532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804ca9d640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806cf40 0x7f8048156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.400+0000 7f804d29e640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8048150c20 0x7f80480c46a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80400b2040 tx=0x7f804001b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.404+0000 7f80457fa640 1 -- 172.21.15.12:0/527872938 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400a7a20 con 0x7f8048150c20 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.404+0000 7f80457fa640 1 -- 172.21.15.12:0/527872938 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80400afbc0 con 0x7f8048150c20 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.404+0000 7f804ed01640 1 -- 172.21.15.12:0/527872938 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80481f60f0 con 0x7f8048150c20 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.404+0000 7f80457fa640 1 -- 172.21.15.12:0/527872938 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400b8440 con 0x7f8048150c20 2023-11-28T21:41:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.404+0000 7f804ed01640 1 -- 172.21.15.12:0/527872938 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80481f66b0 con 0x7f8048150c20 2023-11-28T21:41:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.404+0000 7f804ed01640 1 -- 172.21.15.12:0/527872938 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8048151240 con 0x7f8048150c20 2023-11-28T21:41:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.404+0000 7f80457fa640 1 -- 172.21.15.12:0/527872938 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f80400b6070 con 0x7f8048150c20 2023-11-28T21:41:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.404+0000 7f80457fa640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f802c066470 0x7f802c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.404+0000 7f80457fa640 1 -- 172.21.15.12:0/527872938 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f804011f410 con 0x7f8048150c20 2023-11-28T21:41:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.404+0000 7f804ca9d640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f802c066470 0x7f802c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.408+0000 7f804ca9d640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f802c066470 0x7f802c068930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f803c005890 tx=0x7f803c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.408+0000 7f80457fa640 1 -- 172.21.15.12:0/527872938 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f80400ec590 con 0x7f8048150c20 2023-11-28T21:41:05.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.600+0000 7f804ed01640 1 -- 172.21.15.12:0/527872938 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8048074f60 con 0x7f8048150c20 2023-11-28T21:41:05.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f80457fa640 1 -- 172.21.15.12:0/527872938 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f80400efbb0 con 0x7f8048150c20 2023-11-28T21:41:05.606 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:05.606 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:03.398744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:05.606 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2023-11-28T21:41:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 -- 172.21.15.12:0/527872938 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f802c066470 msgr2=0x7f802c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f802c066470 0x7f802c068930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f803c005890 tx=0x7f803c009040 comp rx=0 tx=0).stop 2023-11-28T21:41:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 -- 172.21.15.12:0/527872938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8048150c20 msgr2=0x7f80480c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8048150c20 0x7f80480c46a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80400b2040 tx=0x7f804001b900 comp rx=0 tx=0).stop 2023-11-28T21:41:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 -- 172.21.15.12:0/527872938 shutdown_connections 2023-11-28T21:41:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f802c066470 0x7f802c068930 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8048150c20 0x7f80480c46a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f804806e300 0x7f80481532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 --2- 172.21.15.12:0/527872938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f804806cf40 0x7f8048156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:05.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 -- 172.21.15.12:0/527872938 >> 172.21.15.12:0/527872938 conn(0x7f8048066360 msgr2=0x7f80480c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:05.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 -- 172.21.15.12:0/527872938 shutdown_connections 2023-11-28T21:41:05.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:05.604+0000 7f804ed01640 1 -- 172.21.15.12:0/527872938 wait complete. 2023-11-28T21:41:05.628 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 28, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 28, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5232": { "addr": "172.21.15.12:6839/3009112102", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3009112102, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3009112102, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5232, "incarnation": 28, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 44, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:41:03.398744+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5232 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:41:07.351 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:41:07.355 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:41:07.628 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:07.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e7b2a640 1 Processor -- start 2023-11-28T21:41:07.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e7b2a640 1 -- start start 2023-11-28T21:41:07.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e7b2a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f19e0151af0 0x7f19e00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:07.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e7b2a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19e0151ed0 0x7f19e00c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:07.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e7b2a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e0157b70 con 0x7f19e006e300 2023-11-28T21:41:07.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e7b2a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e00707f0 con 0x7f19e0151ed0 2023-11-28T21:41:07.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e7b2a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e00c7b90 con 0x7f19e0151af0 2023-11-28T21:41:07.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e50c5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19e0151ed0 0x7f19e00c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:07.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e50c5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19e0151ed0 0x7f19e00c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50802/0 (socket says 172.21.15.12:50802) 2023-11-28T21:41:07.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e50c5640 1 -- 172.21.15.12:0/1807360297 learned_addr learned my addr 172.21.15.12:0/1807360297 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:07.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e60c7640 1 --1- 172.21.15.12:0/1807360297 >> v1:172.21.15.12:6789/0 conn(0x7f19e006e300 0x7f19e0072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40918/0 (socket says 172.21.15.12:40918) 2023-11-28T21:41:07.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.776+0000 7f19e58c6640 1 --2- 172.21.15.12:0/1807360297 >> v2:172.21.15.144:3300/0 conn(0x7f19e0151af0 0x7f19e00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:07.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e50c5640 1 -- 172.21.15.12:0/1807360297 >> v2:172.21.15.144:3300/0 conn(0x7f19e0151af0 msgr2=0x7f19e00c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:07.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e50c5640 1 --2- 172.21.15.12:0/1807360297 >> v2:172.21.15.144:3300/0 conn(0x7f19e0151af0 0x7f19e00c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:07.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e50c5640 1 -- 172.21.15.12:0/1807360297 >> v1:172.21.15.12:6789/0 conn(0x7f19e006e300 legacy=0x7f19e0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:07.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e50c5640 1 -- 172.21.15.12:0/1807360297 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0157e50 con 0x7f19e0151ed0 2023-11-28T21:41:07.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e48c4640 1 -- 172.21.15.12:0/1807360297 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899017475 0 0) 0x7f19e0157b70 con 0x7f19e006e300 2023-11-28T21:41:07.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e50c5640 1 --2- 172.21.15.12:0/1807360297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0151ed0 0x7f19e00c49d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f19e00c7d20 tx=0x7f19d001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19cd96bda9c84b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:07.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e48c4640 1 -- 172.21.15.12:0/1807360297 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00ab600 con 0x7f19e0151ed0 2023-11-28T21:41:07.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e48c4640 1 -- 172.21.15.12:0/1807360297 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d00a7050 con 0x7f19e0151ed0 2023-11-28T21:41:07.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e48c4640 1 -- 172.21.15.12:0/1807360297 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00ab980 con 0x7f19e0151ed0 2023-11-28T21:41:07.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e7b2a640 1 -- 172.21.15.12:0/1807360297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0151ed0 msgr2=0x7f19e00c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:07.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.780+0000 7f19e7b2a640 1 --2- 172.21.15.12:0/1807360297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0151ed0 0x7f19e00c49d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f19e00c7d20 tx=0x7f19d001c500 comp rx=0 tx=0).stop 2023-11-28T21:41:07.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 -- 172.21.15.12:0/1807360297 shutdown_connections 2023-11-28T21:41:07.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 --2- 172.21.15.12:0/1807360297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0151ed0 0x7f19e00c49d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:07.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 --2- 172.21.15.12:0/1807360297 >> v2:172.21.15.144:3300/0 conn(0x7f19e0151af0 0x7f19e00c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:07.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 -- 172.21.15.12:0/1807360297 >> 172.21.15.12:0/1807360297 conn(0x7f19e0066360 msgr2=0x7f19e0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:07.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 -- 172.21.15.12:0/1807360297 shutdown_connections 2023-11-28T21:41:07.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 -- 172.21.15.12:0/1807360297 wait complete. 2023-11-28T21:41:07.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 Processor -- start 2023-11-28T21:41:07.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 -- start start 2023-11-28T21:41:07.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e006e300 0x7f19e014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:07.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19e0151af0 0x7f19e01f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:07.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e00c9ad0 0x7f19e014f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:07.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e014c200 con 0x7f19e006e300 2023-11-28T21:41:07.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0144e10 con 0x7f19e00c9ad0 2023-11-28T21:41:07.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e7b2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e014d530 con 0x7f19e0151af0 2023-11-28T21:41:07.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e58c6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e006e300 0x7f19e014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e58c6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e006e300 0x7f19e014d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50808/0 (socket says 172.21.15.12:50808) 2023-11-28T21:41:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e58c6640 1 -- 172.21.15.12:0/55803014 learned_addr learned my addr 172.21.15.12:0/55803014 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e60c7640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e00c9ad0 0x7f19e014f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e50c5640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19e0151af0 0x7f19e01f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e58c6640 1 -- 172.21.15.12:0/55803014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19e0151af0 msgr2=0x7f19e01f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e58c6640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19e0151af0 0x7f19e01f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e58c6640 1 -- 172.21.15.12:0/55803014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e00c9ad0 msgr2=0x7f19e014f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e58c6640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e00c9ad0 0x7f19e014f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e58c6640 1 -- 172.21.15.12:0/55803014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d00094d0 con 0x7f19e006e300 2023-11-28T21:41:07.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19e58c6640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e006e300 0x7f19e014d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d830 tx=0x7f19dc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.784+0000 7f19d67fc640 1 -- 172.21.15.12:0/55803014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00b6c0 con 0x7f19e006e300 2023-11-28T21:41:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.788+0000 7f19d67fc640 1 -- 172.21.15.12:0/55803014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc00b820 con 0x7f19e006e300 2023-11-28T21:41:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.788+0000 7f19d67fc640 1 -- 172.21.15.12:0/55803014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00bb90 con 0x7f19e006e300 2023-11-28T21:41:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.788+0000 7f19e7b2a640 1 -- 172.21.15.12:0/55803014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e0150810 con 0x7f19e006e300 2023-11-28T21:41:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.788+0000 7f19e7b2a640 1 -- 172.21.15.12:0/55803014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e0150530 con 0x7f19e006e300 2023-11-28T21:41:07.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.788+0000 7f19e7b2a640 1 -- 172.21.15.12:0/55803014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e0074f60 con 0x7f19e006e300 2023-11-28T21:41:07.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.788+0000 7f19d67fc640 1 -- 172.21.15.12:0/55803014 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f19dc01e460 con 0x7f19e006e300 2023-11-28T21:41:07.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.788+0000 7f19d67fc640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19c40664a0 0x7f19c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:07.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.788+0000 7f19d67fc640 1 -- 172.21.15.12:0/55803014 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f19dc084b20 con 0x7f19e006e300 2023-11-28T21:41:07.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.788+0000 7f19e50c5640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19c40664a0 0x7f19c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:07.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.788+0000 7f19e50c5640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19c40664a0 0x7f19c4068960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f19d00ad000 tx=0x7f19d00a4f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:07.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.792+0000 7f19d67fc640 1 -- 172.21.15.12:0/55803014 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f19dc051a90 con 0x7f19e006e300 2023-11-28T21:41:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 -- 172.21.15.12:0/55803014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19e01552f0 con 0x7f19e006e300 2023-11-28T21:41:07.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19d67fc640 1 -- 172.21.15.12:0/55803014 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f19dc0550b0 con 0x7f19e006e300 2023-11-28T21:41:07.987 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:07.987 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:03.398744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:07.988 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2023-11-28T21:41:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 -- 172.21.15.12:0/55803014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19c40664a0 msgr2=0x7f19c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19c40664a0 0x7f19c4068960 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f19d00ad000 tx=0x7f19d00a4f90 comp rx=0 tx=0).stop 2023-11-28T21:41:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 -- 172.21.15.12:0/55803014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e006e300 msgr2=0x7f19e014d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e006e300 0x7f19e014d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d830 tx=0x7f19dc00dcf0 comp rx=0 tx=0).stop 2023-11-28T21:41:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 -- 172.21.15.12:0/55803014 shutdown_connections 2023-11-28T21:41:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19c40664a0 0x7f19c4068960 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19e00c9ad0 0x7f19e014f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19e0151af0 0x7f19e01f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 --2- 172.21.15.12:0/55803014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e006e300 0x7f19e014d9c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 -- 172.21.15.12:0/55803014 >> 172.21.15.12:0/55803014 conn(0x7f19e0066360 msgr2=0x7f19e0157290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 -- 172.21.15.12:0/55803014 shutdown_connections 2023-11-28T21:41:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:07.984+0000 7f19e7b2a640 1 -- 172.21.15.12:0/55803014 wait complete. 2023-11-28T21:41:08.008 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:41:10.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:10.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedbb5f5640 1 Processor -- start 2023-11-28T21:41:10.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedbb5f5640 1 -- start start 2023-11-28T21:41:10.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedbb5f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb406abf0 0x7fedb4072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:10.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedbb5f5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb406c4b0 con 0x7fedb406a810 2023-11-28T21:41:10.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedbb5f5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb4071300 con 0x7fedb40cce30 2023-11-28T21:41:10.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedbb5f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb4071020 con 0x7fedb406abf0 2023-11-28T21:41:10.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedb9391640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fedb406a810 0x7fedb415ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40926/0 (socket says 172.21.15.12:40926) 2023-11-28T21:41:10.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedb9391640 1 -- 172.21.15.12:0/299147041 learned_addr learned my addr 172.21.15.12:0/299147041 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:41:10.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedb8b90640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb406abf0 0x7fedb4072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:10.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361709152 0 0) 0x7fedb0001120 con 0x7fedb406a810 2023-11-28T21:41:10.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feda0003530 con 0x7fedb406a810 2023-11-28T21:41:10.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893228031 0 0) 0x7fedb4071300 con 0x7fedb40cce30 2023-11-28T21:41:10.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.164+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb0001120 con 0x7fedb40cce30 2023-11-28T21:41:10.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3201087182 0 0) 0x7feda0003530 con 0x7fedb406a810 2023-11-28T21:41:10.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb406abf0 msgr2=0x7fedb4072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:10.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb406abf0 0x7fedb4072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:10.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 >> v1:172.21.15.144:6789/0 conn(0x7fedb40cce30 legacy=0x7fedb4142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:10.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb4071ba0 con 0x7fedb406a810 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fedb0001810 con 0x7fedb406a810 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 >> v1:172.21.15.12:6789/0 conn(0x7fedb406a810 legacy=0x7fedb415ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb406abf0 0x7feda0003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feda0004110 0x7feda00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda0004d10 0x7feda0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb4071020 con 0x7feda0004d10 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda000bad0 con 0x7feda0004110 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda000bc90 con 0x7fedb406abf0 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedb9391640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb406abf0 0x7feda0003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedb9b92640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda0004d10 0x7feda0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedb8b90640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feda0004110 0x7feda00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:10.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedb9b92640 1 -- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb406abf0 msgr2=0x7feda0003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:10.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedb9b92640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb406abf0 0x7feda0003bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:10.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedb9b92640 1 -- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feda0004110 msgr2=0x7feda00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:10.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedb9b92640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feda0004110 0x7feda00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:10.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.168+0000 7fedb9b92640 1 -- 172.21.15.12:0/299147041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda0003530 con 0x7feda0004d10 2023-11-28T21:41:10.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedb9b92640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda0004d10 0x7feda0005190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fedac001fd0 tx=0x7fedac01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f23c3816d87690 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:10.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac003b80 con 0x7feda0004d10 2023-11-28T21:41:10.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac003d00 con 0x7feda0004d10 2023-11-28T21:41:10.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedabfff640 1 -- 172.21.15.12:0/299147041 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac005310 con 0x7feda0004d10 2023-11-28T21:41:10.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedbb5f5640 1 -- 172.21.15.12:0/299147041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda0004d10 msgr2=0x7feda0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:10.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedbb5f5640 1 --2- 172.21.15.12:0/299147041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feda0004d10 0x7feda0005190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fedac001fd0 tx=0x7fedac01c8e0 comp rx=0 tx=0).stop 2023-11-28T21:41:10.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedb9b92640 1 -- 172.21.15.12:0/299147041 reap_dead start 2023-11-28T21:41:10.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedbb5f5640 1 -- 172.21.15.12:0/299147041 shutdown_connections 2023-11-28T21:41:10.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedbb5f5640 1 -- 172.21.15.12:0/299147041 >> 172.21.15.12:0/299147041 conn(0x7fedb40cc310 msgr2=0x7fedb40cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:10.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedbb5f5640 1 -- 172.21.15.12:0/299147041 shutdown_connections 2023-11-28T21:41:10.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.172+0000 7fedbb5f5640 1 -- 172.21.15.12:0/299147041 wait complete. 2023-11-28T21:41:10.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedbb5f5640 1 Processor -- start 2023-11-28T21:41:10.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedbb5f5640 1 -- start start 2023-11-28T21:41:10.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedbb5f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feda0004d10 0x7fedb413b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:10.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedbb5f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb415a280 0x7fedb4158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:10.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedbb5f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb4158980 0x7fedb4139480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:10.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedbb5f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb413c8a0 con 0x7fedb415a280 2023-11-28T21:41:10.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedbb5f5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb41f0450 con 0x7feda0004d10 2023-11-28T21:41:10.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedbb5f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb41ef8d0 con 0x7fedb4158980 2023-11-28T21:41:10.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb8b90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb415a280 0x7fedb4158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:10.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb8b90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb415a280 0x7fedb4158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50836/0 (socket says 172.21.15.12:50836) 2023-11-28T21:41:10.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb8b90640 1 -- 172.21.15.12:0/1021400417 learned_addr learned my addr 172.21.15.12:0/1021400417 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:10.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb9391640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feda0004d10 0x7fedb413b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:10.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb9b92640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb4158980 0x7fedb4139480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:10.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb8b90640 1 -- 172.21.15.12:0/1021400417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb4158980 msgr2=0x7fedb4139480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:10.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb8b90640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb4158980 0x7fedb4139480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:10.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb8b90640 1 -- 172.21.15.12:0/1021400417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feda0004d10 msgr2=0x7fedb413b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:10.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb8b90640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feda0004d10 0x7fedb413b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:10.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb8b90640 1 -- 172.21.15.12:0/1021400417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb41efbb0 con 0x7fedb415a280 2023-11-28T21:41:10.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb9391640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feda0004d10 0x7fedb413b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:41:10.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedb8b90640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb415a280 0x7fedb4158440 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feda40119d0 tx=0x7feda4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:10.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7feda9ffb640 1 -- 172.21.15.12:0/1021400417 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda400fb50 con 0x7fedb415a280 2023-11-28T21:41:10.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7feda9ffb640 1 -- 172.21.15.12:0/1021400417 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda400fcb0 con 0x7fedb415a280 2023-11-28T21:41:10.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7feda9ffb640 1 -- 172.21.15.12:0/1021400417 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4014440 con 0x7fedb415a280 2023-11-28T21:41:10.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedbb5f5640 1 -- 172.21.15.12:0/1021400417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb41efe90 con 0x7fedb415a280 2023-11-28T21:41:10.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.176+0000 7fedbb5f5640 1 -- 172.21.15.12:0/1021400417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb4157f20 con 0x7fedb415a280 2023-11-28T21:41:10.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.180+0000 7fedbb5f5640 1 -- 172.21.15.12:0/1021400417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb41580e0 con 0x7fedb415a280 2023-11-28T21:41:10.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.180+0000 7feda9ffb640 1 -- 172.21.15.12:0/1021400417 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7feda40145a0 con 0x7fedb415a280 2023-11-28T21:41:10.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.180+0000 7feda9ffb640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fed98066470 0x7fed98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:10.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.180+0000 7feda9ffb640 1 -- 172.21.15.12:0/1021400417 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7feda4084c50 con 0x7fedb415a280 2023-11-28T21:41:10.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.180+0000 7fedb9391640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fed98066470 0x7fed98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:10.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.180+0000 7fedb9391640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fed98066470 0x7fed98068930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fedb0001060 tx=0x7fedb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:10.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.184+0000 7feda9ffb640 1 -- 172.21.15.12:0/1021400417 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feda4051c80 con 0x7fedb415a280 2023-11-28T21:41:10.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.452+0000 7fedbb5f5640 1 -- 172.21.15.12:0/1021400417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedb4157d80 con 0x7fedb415a280 2023-11-28T21:41:10.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.452+0000 7feda9ffb640 1 -- 172.21.15.12:0/1021400417 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5714 (secure 0 0 0) 0x7fedb4157d80 con 0x7fedb415a280 2023-11-28T21:41:10.455 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:10.455 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:03.398744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:10.456 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2023-11-28T21:41:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 -- 172.21.15.12:0/1021400417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fed98066470 msgr2=0x7fed98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fed98066470 0x7fed98068930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fedb0001060 tx=0x7fedb0009040 comp rx=0 tx=0).stop 2023-11-28T21:41:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 -- 172.21.15.12:0/1021400417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb415a280 msgr2=0x7fedb4158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:10.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb415a280 0x7fedb4158440 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feda40119d0 tx=0x7feda4011e90 comp rx=0 tx=0).stop 2023-11-28T21:41:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 -- 172.21.15.12:0/1021400417 shutdown_connections 2023-11-28T21:41:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fed98066470 0x7fed98068930 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedb4158980 0x7fedb4139480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fedb415a280 0x7fedb4158440 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 --2- 172.21.15.12:0/1021400417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feda0004d10 0x7fedb413b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 -- 172.21.15.12:0/1021400417 >> 172.21.15.12:0/1021400417 conn(0x7fedb40cc310 msgr2=0x7fedb40cac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 -- 172.21.15.12:0/1021400417 shutdown_connections 2023-11-28T21:41:10.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:10.456+0000 7fedbb5f5640 1 -- 172.21.15.12:0/1021400417 wait complete. 2023-11-28T21:41:10.488 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:41:12.489 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:12.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce933d1640 1 Processor -- start 2023-11-28T21:41:12.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce933d1640 1 -- start start 2023-11-28T21:41:12.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce933d1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c06b130 0x7fce8c0c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:12.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce933d1640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fce8c0cb020 0x7fce8c159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:12.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce933d1640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8c0cb4b0 con 0x7fce8c0cac40 2023-11-28T21:41:12.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce933d1640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c06b550 con 0x7fce8c0cb020 2023-11-28T21:41:12.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce933d1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c15a5e0 con 0x7fce8c06b130 2023-11-28T21:41:12.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce9116d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fce8c0cac40 0x7fce8c0ca030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:40930/0 (socket says 172.21.15.12:40930) 2023-11-28T21:41:12.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce9116d640 1 -- 172.21.15.12:0/3125110583 learned_addr learned my addr 172.21.15.12:0/3125110583 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:41:12.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce9096c640 1 --2- 172.21.15.12:0/3125110583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c06b130 0x7fce8c0c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:12.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce9196e640 1 --2- 172.21.15.12:0/3125110583 >> v2:172.21.15.144:3300/0 conn(0x7fce8c0cb020 0x7fce8c159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:12.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce9096c640 1 -- 172.21.15.12:0/3125110583 >> v2:172.21.15.144:3300/0 conn(0x7fce8c0cb020 msgr2=0x7fce8c159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:12.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce9096c640 1 --2- 172.21.15.12:0/3125110583 >> v2:172.21.15.144:3300/0 conn(0x7fce8c0cb020 0x7fce8c159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:12.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce9096c640 1 -- 172.21.15.12:0/3125110583 >> v1:172.21.15.12:6789/0 conn(0x7fce8c0cac40 legacy=0x7fce8c0ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:12.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.648+0000 7fce9096c640 1 -- 172.21.15.12:0/3125110583 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c15a8c0 con 0x7fce8c06b130 2023-11-28T21:41:12.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.652+0000 7fce9096c640 1 --2- 172.21.15.12:0/3125110583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c06b130 0x7fce8c0c4980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0097c0 tx=0x7fce7c01ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28cc88e4f52d6e58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.652+0000 7fce83fff640 1 -- 172.21.15.12:0/3125110583 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0a8970 con 0x7fce8c06b130 2023-11-28T21:41:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.652+0000 7fce83fff640 1 -- 172.21.15.12:0/3125110583 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c0a8ad0 con 0x7fce8c06b130 2023-11-28T21:41:12.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.652+0000 7fce83fff640 1 -- 172.21.15.12:0/3125110583 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0a8e60 con 0x7fce8c06b130 2023-11-28T21:41:12.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.652+0000 7fce933d1640 1 -- 172.21.15.12:0/3125110583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c06b130 msgr2=0x7fce8c0c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:12.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.652+0000 7fce933d1640 1 --2- 172.21.15.12:0/3125110583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c06b130 0x7fce8c0c4980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0097c0 tx=0x7fce7c01ec70 comp rx=0 tx=0).stop 2023-11-28T21:41:12.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.652+0000 7fce933d1640 1 -- 172.21.15.12:0/3125110583 shutdown_connections 2023-11-28T21:41:12.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.652+0000 7fce933d1640 1 --2- 172.21.15.12:0/3125110583 >> v2:172.21.15.144:3300/0 conn(0x7fce8c0cb020 0x7fce8c159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:12.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.652+0000 7fce933d1640 1 --2- 172.21.15.12:0/3125110583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c06b130 0x7fce8c0c4980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:12.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.652+0000 7fce933d1640 1 -- 172.21.15.12:0/3125110583 >> 172.21.15.12:0/3125110583 conn(0x7fce8c0720e0 msgr2=0x7fce8c0724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:12.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce933d1640 1 -- 172.21.15.12:0/3125110583 shutdown_connections 2023-11-28T21:41:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce933d1640 1 -- 172.21.15.12:0/3125110583 wait complete. 2023-11-28T21:41:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce933d1640 1 Processor -- start 2023-11-28T21:41:12.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce933d1640 1 -- start start 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce933d1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce8c06b130 0x7fce8c14fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce933d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce8c0cac40 0x7fce8c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce933d1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c0cb020 0x7fce8c149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce933d1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c1e5300 con 0x7fce8c06b130 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce933d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c1e49a0 con 0x7fce8c0cac40 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce933d1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c150450 con 0x7fce8c0cb020 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9116d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce8c06b130 0x7fce8c14fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9116d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce8c06b130 0x7fce8c14fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50848/0 (socket says 172.21.15.12:50848) 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9116d640 1 -- 172.21.15.12:0/674614493 learned_addr learned my addr 172.21.15.12:0/674614493 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9096c640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce8c0cac40 0x7fce8c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9196e640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c0cb020 0x7fce8c149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:12.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9116d640 1 -- 172.21.15.12:0/674614493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c0cb020 msgr2=0x7fce8c149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9116d640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c0cb020 0x7fce8c149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9116d640 1 -- 172.21.15.12:0/674614493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce8c0cac40 msgr2=0x7fce8c152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9116d640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce8c0cac40 0x7fce8c152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9116d640 1 -- 172.21.15.12:0/674614493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c009420 con 0x7fce8c06b130 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9196e640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c0cb020 0x7fce8c149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.656+0000 7fce9116d640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce8c06b130 0x7fce8c14fdf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c500 tx=0x7fce8800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce81ffb640 1 -- 172.21.15.12:0/674614493 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88005cf0 con 0x7fce8c06b130 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce81ffb640 1 -- 172.21.15.12:0/674614493 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce88005e50 con 0x7fce8c06b130 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce81ffb640 1 -- 172.21.15.12:0/674614493 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce880072f0 con 0x7fce8c06b130 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce933d1640 1 -- 172.21.15.12:0/674614493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c1e4c40 con 0x7fce8c06b130 2023-11-28T21:41:12.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce933d1640 1 -- 172.21.15.12:0/674614493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c151180 con 0x7fce8c06b130 2023-11-28T21:41:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce933d1640 1 -- 172.21.15.12:0/674614493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c067c70 con 0x7fce8c06b130 2023-11-28T21:41:12.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce81ffb640 1 -- 172.21.15.12:0/674614493 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fce88007450 con 0x7fce8c06b130 2023-11-28T21:41:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce81ffb640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fce70066470 0x7fce70068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce81ffb640 1 -- 172.21.15.12:0/674614493 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fce880837a0 con 0x7fce8c06b130 2023-11-28T21:41:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce9096c640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fce70066470 0x7fce70068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:12.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.660+0000 7fce9096c640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fce70066470 0x7fce70068930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0b2000 tx=0x7fce7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:12.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.664+0000 7fce81ffb640 1 -- 172.21.15.12:0/674614493 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fce88050850 con 0x7fce8c06b130 2023-11-28T21:41:12.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.856+0000 7fce933d1640 1 -- 172.21.15.12:0/674614493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce8c074f60 con 0x7fce8c06b130 2023-11-28T21:41:12.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce81ffb640 1 -- 172.21.15.12:0/674614493 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5718 (secure 0 0 0) 0x7fce88053e70 con 0x7fce8c06b130 2023-11-28T21:41:12.862 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:12.862 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:11.562981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:reconnect","state_seq":48,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2023-11-28T21:41:12.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 -- 172.21.15.12:0/674614493 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fce70066470 msgr2=0x7fce70068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fce70066470 0x7fce70068930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0b2000 tx=0x7fce7c008040 comp rx=0 tx=0).stop 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 -- 172.21.15.12:0/674614493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce8c06b130 msgr2=0x7fce8c14fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce8c06b130 0x7fce8c14fdf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c500 tx=0x7fce8800c9c0 comp rx=0 tx=0).stop 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 -- 172.21.15.12:0/674614493 shutdown_connections 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fce70066470 0x7fce70068930 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fce8c0cb020 0x7fce8c149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce8c0cac40 0x7fce8c152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 --2- 172.21.15.12:0/674614493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce8c06b130 0x7fce8c14fdf0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 -- 172.21.15.12:0/674614493 >> 172.21.15.12:0/674614493 conn(0x7fce8c0720e0 msgr2=0x7fce8c0cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 -- 172.21.15.12:0/674614493 shutdown_connections 2023-11-28T21:41:12.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:12.860+0000 7fce933d1640 1 -- 172.21.15.12:0/674614493 wait complete. 2023-11-28T21:41:12.884 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:41:14.885 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:15.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.036+0000 7f85cce46640 1 Processor -- start 2023-11-28T21:41:15.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.036+0000 7f85cce46640 1 -- start start 2023-11-28T21:41:15.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85cce46640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f85c8151af0 0x7f85c80c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:15.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85cce46640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f85c8151ed0 0x7f85c80c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:15.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85cce46640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85c806e300 0x7f85c80c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:15.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85cce46640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80707f0 con 0x7f85c8151af0 2023-11-28T21:41:15.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85cce46640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80c7b90 con 0x7f85c8151ed0 2023-11-28T21:41:15.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85cce46640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8071c20 con 0x7f85c806e300 2023-11-28T21:41:15.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85c659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f85c8151af0 0x7f85c80c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85c659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f85c8151af0 0x7f85c80c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37130/0 (socket says 172.21.15.12:37130) 2023-11-28T21:41:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85c659c640 1 -- 172.21.15.12:0/1613502837 learned_addr learned my addr 172.21.15.12:0/1613502837 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85bffff640 1 --2- 172.21.15.12:0/1613502837 >> v2:172.21.15.144:3300/0 conn(0x7f85c8151ed0 0x7f85c80c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85c6d9d640 1 --2- 172.21.15.12:0/1613502837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85c806e300 0x7f85c80c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85c659c640 1 -- 172.21.15.12:0/1613502837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85c806e300 msgr2=0x7f85c80c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85c659c640 1 --2- 172.21.15.12:0/1613502837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85c806e300 0x7f85c80c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85c659c640 1 -- 172.21.15.12:0/1613502837 >> v2:172.21.15.144:3300/0 conn(0x7f85c8151ed0 msgr2=0x7f85c80c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85c659c640 1 --2- 172.21.15.12:0/1613502837 >> v2:172.21.15.144:3300/0 conn(0x7f85c8151ed0 0x7f85c80c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.040+0000 7f85c659c640 1 -- 172.21.15.12:0/1613502837 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8157b70 con 0x7f85c8151af0 2023-11-28T21:41:15.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85c659c640 1 --2- 172.21.15.12:0/1613502837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8151af0 0x7f85c80c7650 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f85c80c8e60 tx=0x7f85b801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956dd65f66753aed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:15.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85c5d9b640 1 -- 172.21.15.12:0/1613502837 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b80ac410 con 0x7f85c8151af0 2023-11-28T21:41:15.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85c5d9b640 1 -- 172.21.15.12:0/1613502837 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b80ac570 con 0x7f85c8151af0 2023-11-28T21:41:15.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85c5d9b640 1 -- 172.21.15.12:0/1613502837 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b80ac900 con 0x7f85c8151af0 2023-11-28T21:41:15.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 -- 172.21.15.12:0/1613502837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8151af0 msgr2=0x7f85c80c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 --2- 172.21.15.12:0/1613502837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8151af0 0x7f85c80c7650 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f85c80c8e60 tx=0x7f85b801c500 comp rx=0 tx=0).stop 2023-11-28T21:41:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 -- 172.21.15.12:0/1613502837 shutdown_connections 2023-11-28T21:41:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 --2- 172.21.15.12:0/1613502837 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85c806e300 0x7f85c80c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 --2- 172.21.15.12:0/1613502837 >> v2:172.21.15.144:3300/0 conn(0x7f85c8151ed0 0x7f85c80c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 --2- 172.21.15.12:0/1613502837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8151af0 0x7f85c80c7650 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 -- 172.21.15.12:0/1613502837 >> 172.21.15.12:0/1613502837 conn(0x7f85c8066360 msgr2=0x7f85c8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:15.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 -- 172.21.15.12:0/1613502837 shutdown_connections 2023-11-28T21:41:15.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 -- 172.21.15.12:0/1613502837 wait complete. 2023-11-28T21:41:15.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 Processor -- start 2023-11-28T21:41:15.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.044+0000 7f85cce46640 1 -- start start 2023-11-28T21:41:15.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85cce46640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85c806e300 0x7f85c815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:15.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85cce46640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85c8151af0 0x7f85c81522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:15.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85cce46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8152800 0x7f85c81f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85cce46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81f6920 con 0x7f85c8152800 2023-11-28T21:41:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85cce46640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8154430 con 0x7f85c806e300 2023-11-28T21:41:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85cce46640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8154710 con 0x7f85c8151af0 2023-11-28T21:41:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85c6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8152800 0x7f85c81f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85c6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8152800 0x7f85c81f6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37142/0 (socket says 172.21.15.12:37142) 2023-11-28T21:41:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85c6d9d640 1 -- 172.21.15.12:0/3752017328 learned_addr learned my addr 172.21.15.12:0/3752017328 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85c659c640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85c806e300 0x7f85c815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85bffff640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85c8151af0 0x7f85c81522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85c6d9d640 1 -- 172.21.15.12:0/3752017328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85c8151af0 msgr2=0x7f85c81522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85c6d9d640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85c8151af0 0x7f85c81522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85c6d9d640 1 -- 172.21.15.12:0/3752017328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85c806e300 msgr2=0x7f85c815a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85c6d9d640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85c806e300 0x7f85c815a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85c6d9d640 1 -- 172.21.15.12:0/3752017328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b80094d0 con 0x7f85c8152800 2023-11-28T21:41:15.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85c6d9d640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8152800 0x7f85c81f6db0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007e30 tx=0x7f85c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:15.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85beffd640 1 -- 172.21.15.12:0/3752017328 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0014420 con 0x7f85c8152800 2023-11-28T21:41:15.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85beffd640 1 -- 172.21.15.12:0/3752017328 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c0014580 con 0x7f85c8152800 2023-11-28T21:41:15.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85beffd640 1 -- 172.21.15.12:0/3752017328 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0014930 con 0x7f85c8152800 2023-11-28T21:41:15.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85cce46640 1 -- 172.21.15.12:0/3752017328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c81549f0 con 0x7f85c8152800 2023-11-28T21:41:15.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85cce46640 1 -- 172.21.15.12:0/3752017328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c8154130 con 0x7f85c8152800 2023-11-28T21:41:15.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85cce46640 1 -- 172.21.15.12:0/3752017328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c81f6ad0 con 0x7f85c8152800 2023-11-28T21:41:15.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.048+0000 7f85beffd640 1 -- 172.21.15.12:0/3752017328 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f85c0014a90 con 0x7f85c8152800 2023-11-28T21:41:15.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.052+0000 7f85beffd640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f85a80664a0 0x7f85a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:15.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.052+0000 7f85beffd640 1 -- 172.21.15.12:0/3752017328 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f85c0084bf0 con 0x7f85c8152800 2023-11-28T21:41:15.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.052+0000 7f85c659c640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f85a80664a0 0x7f85a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:15.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.052+0000 7f85c659c640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f85a80664a0 0x7f85a8068960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f85b80ae000 tx=0x7f85b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:15.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.056+0000 7f85beffd640 1 -- 172.21.15.12:0/3752017328 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f85c81f6ad0 con 0x7f85c8152800 2023-11-28T21:41:15.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 -- 172.21.15.12:0/3752017328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85c8074f60 con 0x7f85c8152800 2023-11-28T21:41:15.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85beffd640 1 -- 172.21.15.12:0/3752017328 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5718 (secure 0 0 0) 0x7f85c0051b80 con 0x7f85c8152800 2023-11-28T21:41:15.247 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:15.247 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:11.562981+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:reconnect","state_seq":48,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:15.248 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 29 2023-11-28T21:41:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 -- 172.21.15.12:0/3752017328 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f85a80664a0 msgr2=0x7f85a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f85a80664a0 0x7f85a8068960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f85b80ae000 tx=0x7f85b8009040 comp rx=0 tx=0).stop 2023-11-28T21:41:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 -- 172.21.15.12:0/3752017328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8152800 msgr2=0x7f85c81f6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8152800 0x7f85c81f6db0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007e30 tx=0x7f85c000e4b0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 -- 172.21.15.12:0/3752017328 shutdown_connections 2023-11-28T21:41:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f85a80664a0 0x7f85a8068960 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85c8152800 0x7f85c81f6db0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85c8151af0 0x7f85c81522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 --2- 172.21.15.12:0/3752017328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85c806e300 0x7f85c815a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:15.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 -- 172.21.15.12:0/3752017328 >> 172.21.15.12:0/3752017328 conn(0x7f85c8066360 msgr2=0x7f85c814dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:15.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 -- 172.21.15.12:0/3752017328 shutdown_connections 2023-11-28T21:41:15.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:15.244+0000 7f85cce46640 1 -- 172.21.15.12:0/3752017328 wait complete. 2023-11-28T21:41:15.269 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:41:17.270 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:17.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec95d8640 1 Processor -- start 2023-11-28T21:41:17.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec95d8640 1 -- start start 2023-11-28T21:41:17.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec95d8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faec4150c20 0x7faec4156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:17.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec95d8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406cf40 0x7faec4158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:17.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec95d8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec4067410 con 0x7faec406e300 2023-11-28T21:41:17.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec95d8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec406e090 con 0x7faec4150c20 2023-11-28T21:41:17.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec95d8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec4157f30 con 0x7faec406cf40 2023-11-28T21:41:17.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec2ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faec4150c20 0x7faec4156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:17.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faeba7fc640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7faec406e300 0x7faec40c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44904/0 (socket says 172.21.15.12:44904) 2023-11-28T21:41:17.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faeba7fc640 1 -- 172.21.15.12:0/3314315756 learned_addr learned my addr 172.21.15.12:0/3314315756 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:41:17.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec37fe640 1 --2- 172.21.15.12:0/3314315756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406cf40 0x7faec4158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:17.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec2ffd640 1 -- 172.21.15.12:0/3314315756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406cf40 msgr2=0x7faec4158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:17.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec2ffd640 1 --2- 172.21.15.12:0/3314315756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406cf40 0x7faec4158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:17.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec2ffd640 1 -- 172.21.15.12:0/3314315756 >> v1:172.21.15.12:6789/0 conn(0x7faec406e300 legacy=0x7faec40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:17.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec2ffd640 1 -- 172.21.15.12:0/3314315756 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec4151000 con 0x7faec4150c20 2023-11-28T21:41:17.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.424+0000 7faec27fc640 1 -- 172.21.15.12:0/3314315756 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889494092 0 0) 0x7faec4067410 con 0x7faec406e300 2023-11-28T21:41:17.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec2ffd640 1 --2- 172.21.15.12:0/3314315756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec4150c20 0x7faec4156910 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faeac0097c0 tx=0x7faeac01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f70c9d5cde548f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:17.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec27fc640 1 -- 172.21.15.12:0/3314315756 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0ad640 con 0x7faec4150c20 2023-11-28T21:41:17.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec27fc640 1 -- 172.21.15.12:0/3314315756 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac0ad7a0 con 0x7faec4150c20 2023-11-28T21:41:17.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec27fc640 1 -- 172.21.15.12:0/3314315756 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0adb30 con 0x7faec4150c20 2023-11-28T21:41:17.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec95d8640 1 -- 172.21.15.12:0/3314315756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec4150c20 msgr2=0x7faec4156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:17.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec95d8640 1 --2- 172.21.15.12:0/3314315756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec4150c20 0x7faec4156910 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faeac0097c0 tx=0x7faeac01c480 comp rx=0 tx=0).stop 2023-11-28T21:41:17.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec95d8640 1 -- 172.21.15.12:0/3314315756 shutdown_connections 2023-11-28T21:41:17.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec95d8640 1 --2- 172.21.15.12:0/3314315756 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406cf40 0x7faec4158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:17.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec95d8640 1 --2- 172.21.15.12:0/3314315756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec4150c20 0x7faec4156910 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:17.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec95d8640 1 -- 172.21.15.12:0/3314315756 >> 172.21.15.12:0/3314315756 conn(0x7faec4066360 msgr2=0x7faec4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:17.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec95d8640 1 -- 172.21.15.12:0/3314315756 shutdown_connections 2023-11-28T21:41:17.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.428+0000 7faec95d8640 1 -- 172.21.15.12:0/3314315756 wait complete. 2023-11-28T21:41:17.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec95d8640 1 Processor -- start 2023-11-28T21:41:17.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec95d8640 1 -- start start 2023-11-28T21:41:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec95d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec406cf40 0x7faec41483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec95d8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406e300 0x7faec40c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec95d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faec4150c20 0x7faec41f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec95d8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec40c70e0 con 0x7faec406cf40 2023-11-28T21:41:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec95d8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec4146aa0 con 0x7faec4150c20 2023-11-28T21:41:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec95d8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec41467e0 con 0x7faec406e300 2023-11-28T21:41:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec406cf40 0x7faec41483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec406cf40 0x7faec41483e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37146/0 (socket says 172.21.15.12:37146) 2023-11-28T21:41:17.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec2ffd640 1 -- 172.21.15.12:0/766214847 learned_addr learned my addr 172.21.15.12:0/766214847 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:17.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faeba7fc640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406e300 0x7faec40c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:17.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec37fe640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faec4150c20 0x7faec41f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:17.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec2ffd640 1 -- 172.21.15.12:0/766214847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406e300 msgr2=0x7faec40c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:17.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec2ffd640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406e300 0x7faec40c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:17.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec2ffd640 1 -- 172.21.15.12:0/766214847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faec4150c20 msgr2=0x7faec41f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:17.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec2ffd640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faec4150c20 0x7faec41f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:17.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec2ffd640 1 -- 172.21.15.12:0/766214847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac009420 con 0x7faec406cf40 2023-11-28T21:41:17.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faeba7fc640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406e300 0x7faec40c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:41:17.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec2ffd640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec406cf40 0x7faec41483e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faeac000c10 tx=0x7faeac019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:17.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faebbfff640 1 -- 172.21.15.12:0/766214847 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac004230 con 0x7faec406cf40 2023-11-28T21:41:17.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faebbfff640 1 -- 172.21.15.12:0/766214847 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac0a6040 con 0x7faec406cf40 2023-11-28T21:41:17.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faebbfff640 1 -- 172.21.15.12:0/766214847 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac003620 con 0x7faec406cf40 2023-11-28T21:41:17.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.432+0000 7faec95d8640 1 -- 172.21.15.12:0/766214847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec41500a0 con 0x7faec406cf40 2023-11-28T21:41:17.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.436+0000 7faec95d8640 1 -- 172.21.15.12:0/766214847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec40c46f0 con 0x7faec406cf40 2023-11-28T21:41:17.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.436+0000 7faec95d8640 1 -- 172.21.15.12:0/766214847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec4074f60 con 0x7faec406cf40 2023-11-28T21:41:17.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.436+0000 7faebbfff640 1 -- 172.21.15.12:0/766214847 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faeac0b4040 con 0x7faec406cf40 2023-11-28T21:41:17.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.436+0000 7faebbfff640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea4066470 0x7faea4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:17.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.436+0000 7faebbfff640 1 -- 172.21.15.12:0/766214847 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7faeac11d7e0 con 0x7faec406cf40 2023-11-28T21:41:17.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.436+0000 7faeba7fc640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea4066470 0x7faea4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:17.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.436+0000 7faeba7fc640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea4066470 0x7faea4068930 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faeb4007fd0 tx=0x7faeb4007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:17.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.440+0000 7faebbfff640 1 -- 172.21.15.12:0/766214847 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faeac0ea750 con 0x7faec406cf40 2023-11-28T21:41:17.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.632+0000 7faec95d8640 1 -- 172.21.15.12:0/766214847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faec40c2c30 con 0x7faec406cf40 2023-11-28T21:41:17.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.632+0000 7faebbfff640 1 -- 172.21.15.12:0/766214847 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5715 (secure 0 0 0) 0x7faeac0b2090 con 0x7faec406cf40 2023-11-28T21:41:17.636 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:17.636 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:16.643353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":50,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:17.637 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 30 2023-11-28T21:41:17.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 -- 172.21.15.12:0/766214847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea4066470 msgr2=0x7faea4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:17.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea4066470 0x7faea4068930 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faeb4007fd0 tx=0x7faeb4007c20 comp rx=0 tx=0).stop 2023-11-28T21:41:17.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 -- 172.21.15.12:0/766214847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec406cf40 msgr2=0x7faec41483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:17.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec406cf40 0x7faec41483e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faeac000c10 tx=0x7faeac019220 comp rx=0 tx=0).stop 2023-11-28T21:41:17.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 -- 172.21.15.12:0/766214847 shutdown_connections 2023-11-28T21:41:17.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faea4066470 0x7faea4068930 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:17.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faec4150c20 0x7faec41f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:17.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faec406e300 0x7faec40c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:17.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 --2- 172.21.15.12:0/766214847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faec406cf40 0x7faec41483e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:17.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 -- 172.21.15.12:0/766214847 >> 172.21.15.12:0/766214847 conn(0x7faec4066360 msgr2=0x7faec40c64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:17.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 -- 172.21.15.12:0/766214847 shutdown_connections 2023-11-28T21:41:17.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:17.636+0000 7faec95d8640 1 -- 172.21.15.12:0/766214847 wait complete. 2023-11-28T21:41:17.658 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 30, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 30, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5232": { "addr": "172.21.15.12:6839/3009112102", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3009112102, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3009112102, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5232, "incarnation": 28, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:rejoin", "state_seq": 50 } }, "last_failure": 0, "last_failure_osd_epoch": 44, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:41:16.643353+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5232 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:41:19.659 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.808+0000 7fdb75daa640 1 Processor -- start 2023-11-28T21:41:19.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb75daa640 1 -- start start 2023-11-28T21:41:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb75daa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb70158b20 0x7fdb70156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb75daa640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb70067410 con 0x7fdb70150c20 2023-11-28T21:41:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb75daa640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7006e090 con 0x7fdb7006e300 2023-11-28T21:41:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb75daa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70157f30 con 0x7fdb70158b20 2023-11-28T21:41:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6f7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdb70150c20 0x7fdb700c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44920/0 (socket says 172.21.15.12:44920) 2023-11-28T21:41:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6f7fe640 1 -- 172.21.15.12:0/2399183036 learned_addr learned my addr 172.21.15.12:0/2399183036 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:41:19.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6ffff640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb70158b20 0x7fdb70156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659719468 0 0) 0x7fdb70067410 con 0x7fdb70150c20 2023-11-28T21:41:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb58003530 con 0x7fdb70150c20 2023-11-28T21:41:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3349717142 0 0) 0x7fdb58003530 con 0x7fdb70150c20 2023-11-28T21:41:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb70158b20 msgr2=0x7fdb70156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb70158b20 0x7fdb70156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 >> v1:172.21.15.144:6789/0 conn(0x7fdb7006e300 legacy=0x7fdb70158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70151000 con 0x7fdb70150c20 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fdb60001680 con 0x7fdb70150c20 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 >> v1:172.21.15.12:6789/0 conn(0x7fdb70150c20 legacy=0x7fdb700c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb70158b20 0x7fdb58003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb58003f60 0x7fdb580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58004d10 0x7fdb58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70157f30 con 0x7fdb58003f60 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5800ba70 con 0x7fdb58004d10 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5800de40 con 0x7fdb70158b20 2023-11-28T21:41:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6effd640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb58003f60 0x7fdb580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6f7fe640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb70158b20 0x7fdb58003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6ffff640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58004d10 0x7fdb58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6effd640 1 -- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb70158b20 msgr2=0x7fdb58003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6effd640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb70158b20 0x7fdb58003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6effd640 1 -- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58004d10 msgr2=0x7fdb58005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6effd640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58004d10 0x7fdb58005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.812+0000 7fdb6effd640 1 -- 172.21.15.12:0/2399183036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58003530 con 0x7fdb58003f60 2023-11-28T21:41:19.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb6effd640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb58003f60 0x7fdb580047d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdb7006e750 tx=0x7fdb6401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cecaedfd9e88241 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb640a88b0 con 0x7fdb58003f60 2023-11-28T21:41:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb640a8a30 con 0x7fdb58003f60 2023-11-28T21:41:19.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb6e7fc640 1 -- 172.21.15.12:0/2399183036 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb640a8de0 con 0x7fdb58003f60 2023-11-28T21:41:19.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb75daa640 1 -- 172.21.15.12:0/2399183036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb58003f60 msgr2=0x7fdb580047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb75daa640 1 --2- 172.21.15.12:0/2399183036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb58003f60 0x7fdb580047d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdb7006e750 tx=0x7fdb6401ec90 comp rx=0 tx=0).stop 2023-11-28T21:41:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb6ffff640 1 -- 172.21.15.12:0/2399183036 reap_dead start 2023-11-28T21:41:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb75daa640 1 -- 172.21.15.12:0/2399183036 shutdown_connections 2023-11-28T21:41:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb75daa640 1 -- 172.21.15.12:0/2399183036 >> 172.21.15.12:0/2399183036 conn(0x7fdb70066360 msgr2=0x7fdb70066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb75daa640 1 -- 172.21.15.12:0/2399183036 shutdown_connections 2023-11-28T21:41:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.816+0000 7fdb75daa640 1 -- 172.21.15.12:0/2399183036 wait complete. 2023-11-28T21:41:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb75daa640 1 Processor -- start 2023-11-28T21:41:19.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb75daa640 1 -- start start 2023-11-28T21:41:19.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb75daa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58003f60 0x7fdb700c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb75daa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb700c4950 0x7fdb701493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb75daa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb700c4dd0 0x7fdb70149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb75daa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb700c3eb0 con 0x7fdb700c4950 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb75daa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb70149e50 con 0x7fdb58003f60 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb75daa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7014a3a0 con 0x7fdb700c4dd0 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb700c4950 0x7fdb701493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb700c4950 0x7fdb701493d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37158/0 (socket says 172.21.15.12:37158) 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6effd640 1 -- 172.21.15.12:0/833236702 learned_addr learned my addr 172.21.15.12:0/833236702 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6f7fe640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58003f60 0x7fdb700c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6ffff640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb700c4dd0 0x7fdb70149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6effd640 1 -- 172.21.15.12:0/833236702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb700c4dd0 msgr2=0x7fdb70149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:19.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6effd640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb700c4dd0 0x7fdb70149910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6effd640 1 -- 172.21.15.12:0/833236702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58003f60 msgr2=0x7fdb700c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6effd640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58003f60 0x7fdb700c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6effd640 1 -- 172.21.15.12:0/833236702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7014a680 con 0x7fdb700c4950 2023-11-28T21:41:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb6effd640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb700c4950 0x7fdb701493d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdb640b3040 tx=0x7fdb58005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:19.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb57fff640 1 -- 172.21.15.12:0/833236702 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb640052c0 con 0x7fdb700c4950 2023-11-28T21:41:19.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb57fff640 1 -- 172.21.15.12:0/833236702 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb64005420 con 0x7fdb700c4950 2023-11-28T21:41:19.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb57fff640 1 -- 172.21.15.12:0/833236702 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64006a20 con 0x7fdb700c4950 2023-11-28T21:41:19.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb75daa640 1 -- 172.21.15.12:0/833236702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7014a960 con 0x7fdb700c4950 2023-11-28T21:41:19.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.820+0000 7fdb75daa640 1 -- 172.21.15.12:0/833236702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb700c4430 con 0x7fdb700c4950 2023-11-28T21:41:19.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.824+0000 7fdb75daa640 1 -- 172.21.15.12:0/833236702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7014aaf0 con 0x7fdb700c4950 2023-11-28T21:41:19.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.824+0000 7fdb57fff640 1 -- 172.21.15.12:0/833236702 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdb640b7040 con 0x7fdb700c4950 2023-11-28T21:41:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.824+0000 7fdb57fff640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb5c066470 0x7fdb5c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.824+0000 7fdb57fff640 1 -- 172.21.15.12:0/833236702 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fdb641211f0 con 0x7fdb700c4950 2023-11-28T21:41:19.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.824+0000 7fdb6f7fe640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb5c066470 0x7fdb5c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:19.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.824+0000 7fdb6f7fe640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb5c066470 0x7fdb5c068930 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdb60007c30 tx=0x7fdb60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:19.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:19.828+0000 7fdb57fff640 1 -- 172.21.15.12:0/833236702 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb640ee1f0 con 0x7fdb700c4950 2023-11-28T21:41:20.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 -- 172.21.15.12:0/833236702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb70072210 con 0x7fdb700c4950 2023-11-28T21:41:20.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb57fff640 1 -- 172.21.15.12:0/833236702 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fdb640f1810 con 0x7fdb700c4950 2023-11-28T21:41:20.028 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:20.028 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:16.643353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":50,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:20.028 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 30 2023-11-28T21:41:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 -- 172.21.15.12:0/833236702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb5c066470 msgr2=0x7fdb5c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb5c066470 0x7fdb5c068930 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdb60007c30 tx=0x7fdb60009040 comp rx=0 tx=0).stop 2023-11-28T21:41:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 -- 172.21.15.12:0/833236702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb700c4950 msgr2=0x7fdb701493d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb700c4950 0x7fdb701493d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdb640b3040 tx=0x7fdb58005800 comp rx=0 tx=0).stop 2023-11-28T21:41:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 -- 172.21.15.12:0/833236702 shutdown_connections 2023-11-28T21:41:20.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb5c066470 0x7fdb5c068930 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:20.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb700c4dd0 0x7fdb70149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:20.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb700c4950 0x7fdb701493d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:20.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 --2- 172.21.15.12:0/833236702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb58003f60 0x7fdb700c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:20.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.024+0000 7fdb75daa640 1 -- 172.21.15.12:0/833236702 >> 172.21.15.12:0/833236702 conn(0x7fdb70066360 msgr2=0x7fdb700714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:20.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.028+0000 7fdb75daa640 1 -- 172.21.15.12:0/833236702 shutdown_connections 2023-11-28T21:41:20.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:20.028+0000 7fdb75daa640 1 -- 172.21.15.12:0/833236702 wait complete. 2023-11-28T21:41:20.050 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:41:22.051 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:22.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6edc79640 1 Processor -- start 2023-11-28T21:41:22.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6edc79640 1 -- start start 2023-11-28T21:41:22.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6edc79640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff6e806e300 0x7ff6e8156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:22.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6edc79640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff6e806cf40 0x7ff6e8158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:22.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6edc79640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e8067410 con 0x7ff6e8150c20 2023-11-28T21:41:22.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6edc79640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e806e090 con 0x7ff6e806cf40 2023-11-28T21:41:22.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6edc79640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e8157f30 con 0x7ff6e806e300 2023-11-28T21:41:22.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6e7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff6e806cf40 0x7ff6e8158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:22.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6e7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff6e806cf40 0x7ff6e8158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37174/0 (socket says 172.21.15.12:37174) 2023-11-28T21:41:22.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6e7fff640 1 -- 172.21.15.12:0/3732152809 learned_addr learned my addr 172.21.15.12:0/3732152809 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:22.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6e6ffd640 1 --2- 172.21.15.12:0/3732152809 >> v2:172.21.15.144:3300/0 conn(0x7ff6e806e300 0x7ff6e8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:22.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6e7fff640 1 -- 172.21.15.12:0/3732152809 >> v2:172.21.15.144:3300/0 conn(0x7ff6e806e300 msgr2=0x7ff6e8156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:22.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6e7fff640 1 --2- 172.21.15.12:0/3732152809 >> v2:172.21.15.144:3300/0 conn(0x7ff6e806e300 0x7ff6e8156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:22.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6e7fff640 1 -- 172.21.15.12:0/3732152809 >> v1:172.21.15.144:6789/0 conn(0x7ff6e8150c20 legacy=0x7ff6e80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:22.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.204+0000 7ff6e7fff640 1 -- 172.21.15.12:0/3732152809 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e8151000 con 0x7ff6e806cf40 2023-11-28T21:41:22.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6e7fff640 1 --2- 172.21.15.12:0/3732152809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 0x7ff6e8158410 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80c8e60 tx=0x7ff6e001eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c2b50f84f20882 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6e67fc640 1 -- 172.21.15.12:0/3732152809 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000b2d0 con 0x7ff6e806cf40 2023-11-28T21:41:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6e67fc640 1 -- 172.21.15.12:0/3732152809 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e000b430 con 0x7ff6e806cf40 2023-11-28T21:41:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6e67fc640 1 -- 172.21.15.12:0/3732152809 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00a7a70 con 0x7ff6e806cf40 2023-11-28T21:41:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6edc79640 1 -- 172.21.15.12:0/3732152809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 msgr2=0x7ff6e8158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6edc79640 1 --2- 172.21.15.12:0/3732152809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 0x7ff6e8158410 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80c8e60 tx=0x7ff6e001eb90 comp rx=0 tx=0).stop 2023-11-28T21:41:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6edc79640 1 -- 172.21.15.12:0/3732152809 shutdown_connections 2023-11-28T21:41:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6edc79640 1 --2- 172.21.15.12:0/3732152809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 0x7ff6e8158410 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6edc79640 1 --2- 172.21.15.12:0/3732152809 >> v2:172.21.15.144:3300/0 conn(0x7ff6e806e300 0x7ff6e8156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:22.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6edc79640 1 -- 172.21.15.12:0/3732152809 >> 172.21.15.12:0/3732152809 conn(0x7ff6e8066360 msgr2=0x7ff6e8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:22.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6edc79640 1 -- 172.21.15.12:0/3732152809 shutdown_connections 2023-11-28T21:41:22.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6edc79640 1 -- 172.21.15.12:0/3732152809 wait complete. 2023-11-28T21:41:22.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.208+0000 7ff6edc79640 1 Processor -- start 2023-11-28T21:41:22.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6edc79640 1 -- start start 2023-11-28T21:41:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6edc79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 0x7ff6e80c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6edc79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e806e300 0x7ff6e80c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6edc79640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6e8150c20 0x7ff6e8151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6edc79640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e8156630 con 0x7ff6e806cf40 2023-11-28T21:41:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6edc79640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e8153130 con 0x7ff6e8150c20 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6edc79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e81f37b0 con 0x7ff6e806e300 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 0x7ff6e80c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 0x7ff6e80c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37178/0 (socket says 172.21.15.12:37178) 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e77fe640 1 -- 172.21.15.12:0/2560350102 learned_addr learned my addr 172.21.15.12:0/2560350102 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e6ffd640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e806e300 0x7ff6e80c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e7fff640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6e8150c20 0x7ff6e8151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e77fe640 1 -- 172.21.15.12:0/2560350102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e806e300 msgr2=0x7ff6e80c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e77fe640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e806e300 0x7ff6e80c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e77fe640 1 -- 172.21.15.12:0/2560350102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6e8150c20 msgr2=0x7ff6e8151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e77fe640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6e8150c20 0x7ff6e8151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e77fe640 1 -- 172.21.15.12:0/2560350102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e000bd40 con 0x7ff6e806cf40 2023-11-28T21:41:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e6ffd640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e806e300 0x7ff6e80c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:41:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6e77fe640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 0x7ff6e80c4fe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800c450 tx=0x7ff6d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6cffff640 1 -- 172.21.15.12:0/2560350102 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8012830 con 0x7ff6e806cf40 2023-11-28T21:41:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6cffff640 1 -- 172.21.15.12:0/2560350102 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d80129b0 con 0x7ff6e806cf40 2023-11-28T21:41:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6cffff640 1 -- 172.21.15.12:0/2560350102 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8012d60 con 0x7ff6e806cf40 2023-11-28T21:41:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6edc79640 1 -- 172.21.15.12:0/2560350102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e81568a0 con 0x7ff6e806cf40 2023-11-28T21:41:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6edc79640 1 -- 172.21.15.12:0/2560350102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e8151240 con 0x7ff6e806cf40 2023-11-28T21:41:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.212+0000 7ff6edc79640 1 -- 172.21.15.12:0/2560350102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e8074f60 con 0x7ff6e806cf40 2023-11-28T21:41:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.216+0000 7ff6cffff640 1 -- 172.21.15.12:0/2560350102 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff6d801c460 con 0x7ff6e806cf40 2023-11-28T21:41:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.216+0000 7ff6cffff640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6d4066470 0x7ff6d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.216+0000 7ff6cffff640 1 -- 172.21.15.12:0/2560350102 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7ff6d80840c0 con 0x7ff6e806cf40 2023-11-28T21:41:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.216+0000 7ff6e6ffd640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6d4066470 0x7ff6d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:22.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.216+0000 7ff6e6ffd640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6d4066470 0x7ff6d4068930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80c5520 tx=0x7ff6dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.220+0000 7ff6cffff640 1 -- 172.21.15.12:0/2560350102 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6d8051040 con 0x7ff6e806cf40 2023-11-28T21:41:22.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 -- 172.21.15.12:0/2560350102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6e80c2c30 con 0x7ff6e806cf40 2023-11-28T21:41:22.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6cffff640 1 -- 172.21.15.12:0/2560350102 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5715 (secure 0 0 0) 0x7ff6d8054660 con 0x7ff6e806cf40 2023-11-28T21:41:22.412 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:22.412 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:16.643353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":50,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:22.412 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 30 2023-11-28T21:41:22.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 -- 172.21.15.12:0/2560350102 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6d4066470 msgr2=0x7ff6d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:22.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6d4066470 0x7ff6d4068930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80c5520 tx=0x7ff6dc008040 comp rx=0 tx=0).stop 2023-11-28T21:41:22.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 -- 172.21.15.12:0/2560350102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 msgr2=0x7ff6e80c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:22.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 0x7ff6e80c4fe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800c450 tx=0x7ff6d800c910 comp rx=0 tx=0).stop 2023-11-28T21:41:22.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 -- 172.21.15.12:0/2560350102 shutdown_connections 2023-11-28T21:41:22.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff6d4066470 0x7ff6d4068930 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:22.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6e8150c20 0x7ff6e8151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:22.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e806e300 0x7ff6e80c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:22.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 --2- 172.21.15.12:0/2560350102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6e806cf40 0x7ff6e80c4fe0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:22.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.408+0000 7ff6edc79640 1 -- 172.21.15.12:0/2560350102 >> 172.21.15.12:0/2560350102 conn(0x7ff6e8066360 msgr2=0x7ff6e80c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:22.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.412+0000 7ff6edc79640 1 -- 172.21.15.12:0/2560350102 shutdown_connections 2023-11-28T21:41:22.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:22.412+0000 7ff6edc79640 1 -- 172.21.15.12:0/2560350102 wait complete. 2023-11-28T21:41:22.433 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:41:24.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:24.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.584+0000 7f3d618bb640 1 Processor -- start 2023-11-28T21:41:24.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d618bb640 1 -- start start 2023-11-28T21:41:24.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d618bb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3d5c154080 0x7f3d5c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d618bb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3d5c06d420 0x7f3d5c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d618bb640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5c06bee0 con 0x7f3d5c06f690 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d618bb640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c154460 con 0x7f3d5c06d420 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d618bb640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c0724f0 con 0x7f3d5c154080 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d53fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3d5c06d420 0x7f3d5c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d53fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3d5c06d420 0x7f3d5c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37188/0 (socket says 172.21.15.12:37188) 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d53fff640 1 -- 172.21.15.12:0/726119724 learned_addr learned my addr 172.21.15.12:0/726119724 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d5affd640 1 --2- 172.21.15.12:0/726119724 >> v2:172.21.15.144:3300/0 conn(0x7f3d5c154080 0x7f3d5c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d53fff640 1 -- 172.21.15.12:0/726119724 >> v2:172.21.15.144:3300/0 conn(0x7f3d5c154080 msgr2=0x7f3d5c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d53fff640 1 --2- 172.21.15.12:0/726119724 >> v2:172.21.15.144:3300/0 conn(0x7f3d5c154080 0x7f3d5c15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:24.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d53fff640 1 -- 172.21.15.12:0/726119724 >> v1:172.21.15.12:6789/0 conn(0x7f3d5c06f690 legacy=0x7f3d5c06fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:24.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.588+0000 7f3d53fff640 1 -- 172.21.15.12:0/726119724 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0c8120 con 0x7f3d5c06d420 2023-11-28T21:41:24.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d53fff640 1 --2- 172.21.15.12:0/726119724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c06d420 0x7f3d5c06b9a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c066f90 tx=0x7f3d4401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e450e7136956bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:24.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d5a7fc640 1 -- 172.21.15.12:0/726119724 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d440aa430 con 0x7f3d5c06d420 2023-11-28T21:41:24.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d5a7fc640 1 -- 172.21.15.12:0/726119724 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d440aa590 con 0x7f3d5c06d420 2023-11-28T21:41:24.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d5a7fc640 1 -- 172.21.15.12:0/726119724 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d440aa920 con 0x7f3d5c06d420 2023-11-28T21:41:24.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d618bb640 1 -- 172.21.15.12:0/726119724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c06d420 msgr2=0x7f3d5c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:24.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d618bb640 1 --2- 172.21.15.12:0/726119724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c06d420 0x7f3d5c06b9a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c066f90 tx=0x7f3d4401c4d0 comp rx=0 tx=0).stop 2023-11-28T21:41:24.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d618bb640 1 -- 172.21.15.12:0/726119724 shutdown_connections 2023-11-28T21:41:24.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d618bb640 1 --2- 172.21.15.12:0/726119724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c06d420 0x7f3d5c06b9a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:24.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d618bb640 1 --2- 172.21.15.12:0/726119724 >> v2:172.21.15.144:3300/0 conn(0x7f3d5c154080 0x7f3d5c15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:24.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d618bb640 1 -- 172.21.15.12:0/726119724 >> 172.21.15.12:0/726119724 conn(0x7f3d5c066360 msgr2=0x7f3d5c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:24.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d618bb640 1 -- 172.21.15.12:0/726119724 shutdown_connections 2023-11-28T21:41:24.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d618bb640 1 -- 172.21.15.12:0/726119724 wait complete. 2023-11-28T21:41:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d618bb640 1 Processor -- start 2023-11-28T21:41:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.592+0000 7f3d618bb640 1 -- start start 2023-11-28T21:41:24.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d618bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c06d420 0x7f3d5c158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:24.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d618bb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c06f690 0x7f3d5c1ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d618bb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c154080 0x7f3d5c1ec050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d618bb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0c5b80 con 0x7f3d5c154080 2023-11-28T21:41:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d618bb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c157f50 con 0x7f3d5c06f690 2023-11-28T21:41:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d618bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c15a150 con 0x7f3d5c06d420 2023-11-28T21:41:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c154080 0x7f3d5c1ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c154080 0x7f3d5c1ec050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37198/0 (socket says 172.21.15.12:37198) 2023-11-28T21:41:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d5b7fe640 1 -- 172.21.15.12:0/3160143883 learned_addr learned my addr 172.21.15.12:0/3160143883 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d5affd640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c06d420 0x7f3d5c158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d53fff640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c06f690 0x7f3d5c1ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d5b7fe640 1 -- 172.21.15.12:0/3160143883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c06d420 msgr2=0x7f3d5c158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d5b7fe640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c06d420 0x7f3d5c158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d5b7fe640 1 -- 172.21.15.12:0/3160143883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c06f690 msgr2=0x7f3d5c1ebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d5b7fe640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c06f690 0x7f3d5c1ebb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d5b7fe640 1 -- 172.21.15.12:0/3160143883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d440094d0 con 0x7f3d5c154080 2023-11-28T21:41:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d5b7fe640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c154080 0x7f3d5c1ec050 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c770 tx=0x7f3d5400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d58ff9640 1 -- 172.21.15.12:0/3160143883 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d54003b80 con 0x7f3d5c154080 2023-11-28T21:41:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d58ff9640 1 -- 172.21.15.12:0/3160143883 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d54003ce0 con 0x7f3d5c154080 2023-11-28T21:41:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d618bb640 1 -- 172.21.15.12:0/3160143883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c146870 con 0x7f3d5c154080 2023-11-28T21:41:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d58ff9640 1 -- 172.21.15.12:0/3160143883 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5400aa30 con 0x7f3d5c154080 2023-11-28T21:41:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d618bb640 1 -- 172.21.15.12:0/3160143883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c0c6530 con 0x7f3d5c154080 2023-11-28T21:41:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.596+0000 7f3d618bb640 1 -- 172.21.15.12:0/3160143883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c158090 con 0x7f3d5c154080 2023-11-28T21:41:24.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.600+0000 7f3d58ff9640 1 -- 172.21.15.12:0/3160143883 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3d5400ab90 con 0x7f3d5c154080 2023-11-28T21:41:24.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.600+0000 7f3d58ff9640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d3c066470 0x7f3d3c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:24.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.600+0000 7f3d58ff9640 1 -- 172.21.15.12:0/3160143883 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f3d540840b0 con 0x7f3d5c154080 2023-11-28T21:41:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.600+0000 7f3d5affd640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d3c066470 0x7f3d3c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.600+0000 7f3d5affd640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d3c066470 0x7f3d3c068930 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0079c0 tx=0x7f3d4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.604+0000 7f3d58ff9640 1 -- 172.21.15.12:0/3160143883 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3d5c158090 con 0x7f3d5c154080 2023-11-28T21:41:24.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 -- 172.21.15.12:0/3160143883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d5c158090 con 0x7f3d5c154080 2023-11-28T21:41:24.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d58ff9640 1 -- 172.21.15.12:0/3160143883 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f3d5c158090 con 0x7f3d5c154080 2023-11-28T21:41:24.800 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:24.800 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:24.800 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 31 2023-11-28T21:41:24.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 -- 172.21.15.12:0/3160143883 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d3c066470 msgr2=0x7f3d3c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:24.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d3c066470 0x7f3d3c068930 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0079c0 tx=0x7f3d4c008040 comp rx=0 tx=0).stop 2023-11-28T21:41:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 -- 172.21.15.12:0/3160143883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c154080 msgr2=0x7f3d5c1ec050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c154080 0x7f3d5c1ec050 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c770 tx=0x7f3d5400cc30 comp rx=0 tx=0).stop 2023-11-28T21:41:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 -- 172.21.15.12:0/3160143883 shutdown_connections 2023-11-28T21:41:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d3c066470 0x7f3d3c068930 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d5c154080 0x7f3d5c1ec050 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c06f690 0x7f3d5c1ebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 --2- 172.21.15.12:0/3160143883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c06d420 0x7f3d5c158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 -- 172.21.15.12:0/3160143883 >> 172.21.15.12:0/3160143883 conn(0x7f3d5c066360 msgr2=0x7f3d5c14f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 -- 172.21.15.12:0/3160143883 shutdown_connections 2023-11-28T21:41:24.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:24.796+0000 7f3d618bb640 1 -- 172.21.15.12:0/3160143883 wait complete. 2023-11-28T21:41:24.821 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=0, replacing gid=5307 2023-11-28T21:41:24.821 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.a 2023-11-28T21:41:26.822 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2023-11-28T21:41:26.822 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T21:41:26.822 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T21:41:26.826 INFO:tasks.ceph.mds.a:Started 2023-11-28T21:41:26.826 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:26.925 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-11-28T21:41:26.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.980+0000 7f0b7a51d640 1 Processor -- start 2023-11-28T21:41:26.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.980+0000 7f0b7a51d640 1 -- start start 2023-11-28T21:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b7a51d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b74151af0 0x7f0b740c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b7a51d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0b7406e300 0x7f0b74072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b7a51d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b74157b70 con 0x7f0b74151ed0 2023-11-28T21:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b7a51d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b740707f0 con 0x7f0b7406e300 2023-11-28T21:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b7a51d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b740c7b90 con 0x7f0b74151af0 2023-11-28T21:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b73fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b74151af0 0x7f0b740c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b737fe640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f0b74151ed0 0x7f0b740c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:54802/0 (socket says 172.21.15.12:54802) 2023-11-28T21:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b78aba640 1 --2- 172.21.15.12:0/3939749257 >> v2:172.21.15.144:3300/0 conn(0x7f0b7406e300 0x7f0b74072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b737fe640 1 -- 172.21.15.12:0/3939749257 learned_addr learned my addr 172.21.15.12:0/3939749257 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:41:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b73fff640 1 -- 172.21.15.12:0/3939749257 >> v2:172.21.15.144:3300/0 conn(0x7f0b7406e300 msgr2=0x7f0b74072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:41:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b73fff640 1 --2- 172.21.15.12:0/3939749257 >> v2:172.21.15.144:3300/0 conn(0x7f0b7406e300 0x7f0b74072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b73fff640 1 -- 172.21.15.12:0/3939749257 >> v1:172.21.15.144:6789/0 conn(0x7f0b74151ed0 legacy=0x7f0b740c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.984+0000 7f0b73fff640 1 -- 172.21.15.12:0/3939749257 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b74157e50 con 0x7f0b74151af0 2023-11-28T21:41:26.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b73fff640 1 --2- 172.21.15.12:0/3939749257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b74151af0 0x7f0b740c7650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b640097c0 tx=0x7f0b6401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a77cd26349c0329b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:26.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b72ffd640 1 -- 172.21.15.12:0/3939749257 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640ab600 con 0x7f0b74151af0 2023-11-28T21:41:26.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b72ffd640 1 -- 172.21.15.12:0/3939749257 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b640a7050 con 0x7f0b74151af0 2023-11-28T21:41:26.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b72ffd640 1 -- 172.21.15.12:0/3939749257 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640ab980 con 0x7f0b74151af0 2023-11-28T21:41:26.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b7a51d640 1 -- 172.21.15.12:0/3939749257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b74151af0 msgr2=0x7f0b740c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:26.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b7a51d640 1 --2- 172.21.15.12:0/3939749257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b74151af0 0x7f0b740c7650 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b640097c0 tx=0x7f0b6401c480 comp rx=0 tx=0).stop 2023-11-28T21:41:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b7a51d640 1 -- 172.21.15.12:0/3939749257 shutdown_connections 2023-11-28T21:41:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b7a51d640 1 --2- 172.21.15.12:0/3939749257 >> v2:172.21.15.144:3300/0 conn(0x7f0b7406e300 0x7f0b74072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b7a51d640 1 --2- 172.21.15.12:0/3939749257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b74151af0 0x7f0b740c7650 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b7a51d640 1 -- 172.21.15.12:0/3939749257 >> 172.21.15.12:0/3939749257 conn(0x7f0b74066360 msgr2=0x7f0b74066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b7a51d640 1 -- 172.21.15.12:0/3939749257 shutdown_connections 2023-11-28T21:41:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.988+0000 7f0b7a51d640 1 -- 172.21.15.12:0/3939749257 wait complete. 2023-11-28T21:41:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 Processor -- start 2023-11-28T21:41:26.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 -- start start 2023-11-28T21:41:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7406e300 0x7f0b7413dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b74151af0 0x7f0b741e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b7414b760 0x7f0b74144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b741ea720 con 0x7f0b74151af0 2023-11-28T21:41:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b74139990 con 0x7f0b7414b760 2023-11-28T21:41:26.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b74139e20 con 0x7f0b7406e300 2023-11-28T21:41:26.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b74151af0 0x7f0b741e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:26.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b74151af0 0x7f0b741e9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44264/0 (socket says 172.21.15.12:44264) 2023-11-28T21:41:26.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b73fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7406e300 0x7f0b7413dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:26.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b737fe640 1 -- 172.21.15.12:0/792524402 learned_addr learned my addr 172.21.15.12:0/792524402 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:26.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b78aba640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b7414b760 0x7f0b74144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b737fe640 1 -- 172.21.15.12:0/792524402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7406e300 msgr2=0x7f0b7413dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b737fe640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7406e300 0x7f0b7413dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b737fe640 1 -- 172.21.15.12:0/792524402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b7414b760 msgr2=0x7f0b74144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b737fe640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b7414b760 0x7f0b74144f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b737fe640 1 -- 172.21.15.12:0/792524402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b64009420 con 0x7f0b74151af0 2023-11-28T21:41:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b737fe640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b74151af0 0x7f0b741e9ae0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c4d0 tx=0x7f0b6800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b70ff9640 1 -- 172.21.15.12:0/792524402 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68005cf0 con 0x7f0b74151af0 2023-11-28T21:41:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b70ff9640 1 -- 172.21.15.12:0/792524402 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68005e50 con 0x7f0b74151af0 2023-11-28T21:41:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b70ff9640 1 -- 172.21.15.12:0/792524402 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68007270 con 0x7f0b74151af0 2023-11-28T21:41:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 -- 172.21.15.12:0/792524402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b740c9510 con 0x7f0b74151af0 2023-11-28T21:41:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 -- 172.21.15.12:0/792524402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b74138d80 con 0x7f0b74151af0 2023-11-28T21:41:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.992+0000 7f0b7a51d640 1 -- 172.21.15.12:0/792524402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b740c79c0 con 0x7f0b74151af0 2023-11-28T21:41:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.996+0000 7f0b70ff9640 1 -- 172.21.15.12:0/792524402 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0b680073d0 con 0x7f0b74151af0 2023-11-28T21:41:26.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.996+0000 7f0b70ff9640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0b600664a0 0x7f0b60068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:26.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.996+0000 7f0b70ff9640 1 -- 172.21.15.12:0/792524402 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f0b68083690 con 0x7f0b74151af0 2023-11-28T21:41:26.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.996+0000 7f0b73fff640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0b600664a0 0x7f0b60068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:27.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:26.996+0000 7f0b73fff640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0b600664a0 0x7f0b60068960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0b640ad000 tx=0x7f0b64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:27.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.000+0000 7f0b70ff9640 1 -- 172.21.15.12:0/792524402 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0b68050740 con 0x7f0b74151af0 2023-11-28T21:41:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.188+0000 7f0b7a51d640 1 -- 172.21.15.12:0/792524402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b74139b00 con 0x7f0b74151af0 2023-11-28T21:41:27.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.188+0000 7f0b70ff9640 1 -- 172.21.15.12:0/792524402 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f0b68053d60 con 0x7f0b74151af0 2023-11-28T21:41:27.193 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:27.194 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:27.194 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 31 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 -- 172.21.15.12:0/792524402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0b600664a0 msgr2=0x7f0b60068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0b600664a0 0x7f0b60068960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0b640ad000 tx=0x7f0b64008040 comp rx=0 tx=0).stop 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 -- 172.21.15.12:0/792524402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b74151af0 msgr2=0x7f0b741e9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b74151af0 0x7f0b741e9ae0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c4d0 tx=0x7f0b6800c990 comp rx=0 tx=0).stop 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 -- 172.21.15.12:0/792524402 shutdown_connections 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0b600664a0 0x7f0b60068960 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b7414b760 0x7f0b74144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b74151af0 0x7f0b741e9ae0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 --2- 172.21.15.12:0/792524402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7406e300 0x7f0b7413dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 -- 172.21.15.12:0/792524402 >> 172.21.15.12:0/792524402 conn(0x7f0b74066360 msgr2=0x7f0b7414f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 -- 172.21.15.12:0/792524402 shutdown_connections 2023-11-28T21:41:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:27.192+0000 7f0b7a51d640 1 -- 172.21.15.12:0/792524402 wait complete. 2023-11-28T21:41:27.215 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2023-11-28T21:41:29.216 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:41:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.364+0000 7f8c95980640 1 Processor -- start 2023-11-28T21:41:29.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.364+0000 7f8c95980640 1 -- start start 2023-11-28T21:41:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.368+0000 7f8c95980640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8c90151ed0 0x7f8c900c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.368+0000 7f8c95980640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c90157b70 con 0x7f8c90151af0 2023-11-28T21:41:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.368+0000 7f8c95980640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c90157e50 con 0x7f8c9006e300 2023-11-28T21:41:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.368+0000 7f8c95980640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c900707f0 con 0x7f8c90151ed0 2023-11-28T21:41:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.368+0000 7f8c87fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8c90151ed0 0x7f8c900c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:29.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.368+0000 7f8c8effd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8c90151af0 0x7f8c900c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49912/0 (socket says 172.21.15.12:49912) 2023-11-28T21:41:29.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.368+0000 7f8c8effd640 1 -- 172.21.15.12:0/1171788512 learned_addr learned my addr 172.21.15.12:0/1171788512 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:41:29.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.368+0000 7f8c87fff640 1 -- 172.21.15.12:0/1171788512 >> v1:172.21.15.144:6789/0 conn(0x7f8c9006e300 legacy=0x7f8c90072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:29.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.368+0000 7f8c87fff640 1 -- 172.21.15.12:0/1171788512 >> v1:172.21.15.12:6789/0 conn(0x7f8c90151af0 legacy=0x7f8c900c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:29.372 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.368+0000 7f8c87fff640 1 -- 172.21.15.12:0/1171788512 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c90158130 con 0x7f8c90151ed0 2023-11-28T21:41:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c87fff640 1 --2- 172.21.15.12:0/1171788512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c90151ed0 0x7f8c900c7650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8c780097c0 tx=0x7f8c7801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcd97b9921621f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c8e7fc640 1 -- 172.21.15.12:0/1171788512 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780ab600 con 0x7f8c90151ed0 2023-11-28T21:41:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c8e7fc640 1 -- 172.21.15.12:0/1171788512 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c780a7050 con 0x7f8c90151ed0 2023-11-28T21:41:29.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c8e7fc640 1 -- 172.21.15.12:0/1171788512 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780ab980 con 0x7f8c90151ed0 2023-11-28T21:41:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 -- 172.21.15.12:0/1171788512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c90151ed0 msgr2=0x7f8c900c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 --2- 172.21.15.12:0/1171788512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c90151ed0 0x7f8c900c7650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8c780097c0 tx=0x7f8c7801c480 comp rx=0 tx=0).stop 2023-11-28T21:41:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 -- 172.21.15.12:0/1171788512 shutdown_connections 2023-11-28T21:41:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 --2- 172.21.15.12:0/1171788512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c90151ed0 0x7f8c900c7650 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 -- 172.21.15.12:0/1171788512 >> 172.21.15.12:0/1171788512 conn(0x7f8c90066360 msgr2=0x7f8c90066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 -- 172.21.15.12:0/1171788512 shutdown_connections 2023-11-28T21:41:29.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 -- 172.21.15.12:0/1171788512 wait complete. 2023-11-28T21:41:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 Processor -- start 2023-11-28T21:41:29.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 -- start start 2023-11-28T21:41:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c9006e300 0x7f8c90142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c90151af0 0x7f8c90143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c90155740 0x7f8c9014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90156ea0 con 0x7f8c9006e300 2023-11-28T21:41:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c901462f0 con 0x7f8c90155740 2023-11-28T21:41:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.372+0000 7f8c95980640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90146870 con 0x7f8c90151af0 2023-11-28T21:41:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c9006e300 0x7f8c90142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c9006e300 0x7f8c90142670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44278/0 (socket says 172.21.15.12:44278) 2023-11-28T21:41:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8effd640 1 -- 172.21.15.12:0/1905639609 learned_addr learned my addr 172.21.15.12:0/1905639609 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:41:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8f7fe640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c90155740 0x7f8c9014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c87fff640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c90151af0 0x7f8c90143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8effd640 1 -- 172.21.15.12:0/1905639609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c90151af0 msgr2=0x7f8c90143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8effd640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c90151af0 0x7f8c90143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8effd640 1 -- 172.21.15.12:0/1905639609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c90155740 msgr2=0x7f8c9014bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8effd640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c90155740 0x7f8c9014bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8effd640 1 -- 172.21.15.12:0/1905639609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c78009420 con 0x7f8c9006e300 2023-11-28T21:41:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8f7fe640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c90155740 0x7f8c9014bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:41:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c87fff640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c90151af0 0x7f8c90143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:41:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8effd640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c9006e300 0x7f8c90142670 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c500 tx=0x7f8c8000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:29.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8cff9640 1 -- 172.21.15.12:0/1905639609 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80005cf0 con 0x7f8c9006e300 2023-11-28T21:41:29.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8cff9640 1 -- 172.21.15.12:0/1905639609 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80005e50 con 0x7f8c9006e300 2023-11-28T21:41:29.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8cff9640 1 -- 172.21.15.12:0/1905639609 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c800072f0 con 0x7f8c9006e300 2023-11-28T21:41:29.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c95980640 1 -- 172.21.15.12:0/1905639609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c90146b10 con 0x7f8c9006e300 2023-11-28T21:41:29.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c95980640 1 -- 172.21.15.12:0/1905639609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c901575e0 con 0x7f8c9006e300 2023-11-28T21:41:29.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c95980640 1 -- 172.21.15.12:0/1905639609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c90157160 con 0x7f8c9006e300 2023-11-28T21:41:29.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.376+0000 7f8c8cff9640 1 -- 172.21.15.12:0/1905639609 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8c80007450 con 0x7f8c9006e300 2023-11-28T21:41:29.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.380+0000 7f8c8cff9640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c70066470 0x7f8c70068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:41:29.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.380+0000 7f8c8cff9640 1 -- 172.21.15.12:0/1905639609 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f8c800838e0 con 0x7f8c9006e300 2023-11-28T21:41:29.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.380+0000 7f8c87fff640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c70066470 0x7f8c70068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:41:29.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.380+0000 7f8c87fff640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c70066470 0x7f8c70068930 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c780ad000 tx=0x7f8c78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:41:29.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.384+0000 7f8c8cff9640 1 -- 172.21.15.12:0/1905639609 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8c90157160 con 0x7f8c9006e300 2023-11-28T21:41:29.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 -- 172.21.15.12:0/1905639609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c90157160 con 0x7f8c9006e300 2023-11-28T21:41:29.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c8cff9640 1 -- 172.21.15.12:0/1905639609 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f8c90157160 con 0x7f8c9006e300 2023-11-28T21:41:29.580 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:41:29.580 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:41:29.580 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2023-11-28T21:41:29.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 -- 172.21.15.12:0/1905639609 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c70066470 msgr2=0x7f8c70068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:29.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c70066470 0x7f8c70068930 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c780ad000 tx=0x7f8c78008040 comp rx=0 tx=0).stop 2023-11-28T21:41:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 -- 172.21.15.12:0/1905639609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c9006e300 msgr2=0x7f8c90142670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:41:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c9006e300 0x7f8c90142670 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c500 tx=0x7f8c8000c9c0 comp rx=0 tx=0).stop 2023-11-28T21:41:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 -- 172.21.15.12:0/1905639609 shutdown_connections 2023-11-28T21:41:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8c70066470 0x7f8c70068930 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c90155740 0x7f8c9014bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c90151af0 0x7f8c90143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 --2- 172.21.15.12:0/1905639609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8c9006e300 0x7f8c90142670 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:41:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 -- 172.21.15.12:0/1905639609 >> 172.21.15.12:0/1905639609 conn(0x7f8c90066360 msgr2=0x7f8c90150d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:41:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 -- 172.21.15.12:0/1905639609 shutdown_connections 2023-11-28T21:41:29.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:41:29.576+0000 7f8c95980640 1 -- 172.21.15.12:0/1905639609 wait complete. 2023-11-28T21:41:29.601 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2023-11-28T21:41:29.601 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 52 secs before thrashing 2023-11-28T21:41:30.366 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-11-28T21:41:30.364+0000 7fe52b35d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T21:41:37.420 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:41:37.425 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:42:07.475 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:42:13.496 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:42:21.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:21.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f4980479640 1 Processor -- start 2023-11-28T21:42:21.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f4980479640 1 -- start start 2023-11-28T21:42:21.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f4980479640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4978151af0 0x7f49780c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:21.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f4980479640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f497806e300 0x7f4978072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:21.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f4980479640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4978157b70 con 0x7f4978151ed0 2023-11-28T21:42:21.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f4980479640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49780707f0 con 0x7f497806e300 2023-11-28T21:42:21.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f4980479640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49780c7b90 con 0x7f4978151af0 2023-11-28T21:42:21.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f497ea16640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f497806e300 0x7f4978072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:21.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f497da14640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4978151ed0 0x7f49780c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53128/0 (socket says 172.21.15.12:53128) 2023-11-28T21:42:21.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f497da14640 1 -- 172.21.15.12:0/3215815517 learned_addr learned my addr 172.21.15.12:0/3215815517 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:21.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f497e215640 1 --2- 172.21.15.12:0/3215815517 >> v2:172.21.15.144:3300/0 conn(0x7f4978151af0 0x7f49780c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:21.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f497d213640 1 -- 172.21.15.12:0/3215815517 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320562291 0 0) 0x7f4978157b70 con 0x7f4978151ed0 2023-11-28T21:42:21.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f497d213640 1 -- 172.21.15.12:0/3215815517 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4964003530 con 0x7f4978151ed0 2023-11-28T21:42:21.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f497ea16640 1 -- 172.21.15.12:0/3215815517 >> v2:172.21.15.144:3300/0 conn(0x7f4978151af0 msgr2=0x7f49780c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:21.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f497ea16640 1 --2- 172.21.15.12:0/3215815517 >> v2:172.21.15.144:3300/0 conn(0x7f4978151af0 0x7f49780c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:21.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f497ea16640 1 -- 172.21.15.12:0/3215815517 >> v1:172.21.15.12:6789/0 conn(0x7f4978151ed0 legacy=0x7f49780c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:21.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.752+0000 7f497ea16640 1 -- 172.21.15.12:0/3215815517 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4978157e50 con 0x7f497806e300 2023-11-28T21:42:21.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f497ea16640 1 --2- 172.21.15.12:0/3215815517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 0x7f4978072070 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f49780c7d20 tx=0x7f497001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3953d18276711553 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:21.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f497d213640 1 -- 172.21.15.12:0/3215815517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700a57d0 con 0x7f497806e300 2023-11-28T21:42:21.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f497d213640 1 -- 172.21.15.12:0/3215815517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49700a5930 con 0x7f497806e300 2023-11-28T21:42:21.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f497d213640 1 -- 172.21.15.12:0/3215815517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700a5ca0 con 0x7f497806e300 2023-11-28T21:42:21.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f4980479640 1 -- 172.21.15.12:0/3215815517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 msgr2=0x7f4978072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:21.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f4980479640 1 --2- 172.21.15.12:0/3215815517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 0x7f4978072070 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f49780c7d20 tx=0x7f497001c910 comp rx=0 tx=0).stop 2023-11-28T21:42:21.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f4980479640 1 -- 172.21.15.12:0/3215815517 shutdown_connections 2023-11-28T21:42:21.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f4980479640 1 --2- 172.21.15.12:0/3215815517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 0x7f4978072070 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:21.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f4980479640 1 --2- 172.21.15.12:0/3215815517 >> v2:172.21.15.144:3300/0 conn(0x7f4978151af0 0x7f49780c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:21.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f4980479640 1 -- 172.21.15.12:0/3215815517 >> 172.21.15.12:0/3215815517 conn(0x7f4978066360 msgr2=0x7f4978066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:21.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f4980479640 1 -- 172.21.15.12:0/3215815517 shutdown_connections 2023-11-28T21:42:21.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.756+0000 7f4980479640 1 -- 172.21.15.12:0/3215815517 wait complete. 2023-11-28T21:42:21.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f4980479640 1 Processor -- start 2023-11-28T21:42:21.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f4980479640 1 -- start start 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f4980479640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 0x7f49781f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f4980479640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4978151af0 0x7f4978140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f4980479640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49781f4000 0x7f497814ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f4980479640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781f3b20 con 0x7f497806e300 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f4980479640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781f2b90 con 0x7f49781f4000 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f4980479640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4978156460 con 0x7f4978151af0 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497e215640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 0x7f49781f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497e215640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 0x7f49781f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54736/0 (socket says 172.21.15.12:54736) 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497e215640 1 -- 172.21.15.12:0/341100068 learned_addr learned my addr 172.21.15.12:0/341100068 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497da14640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4978151af0 0x7f4978140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:21.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497ea16640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49781f4000 0x7f497814ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497e215640 1 -- 172.21.15.12:0/341100068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4978151af0 msgr2=0x7f4978140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497e215640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4978151af0 0x7f4978140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497e215640 1 -- 172.21.15.12:0/341100068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49781f4000 msgr2=0x7f497814ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497e215640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49781f4000 0x7f497814ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497e215640 1 -- 172.21.15.12:0/341100068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970009a70 con 0x7f497806e300 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497ea16640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49781f4000 0x7f497814ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.760+0000 7f497e215640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 0x7f49781f35e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f497400e930 tx=0x7f497400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.764+0000 7f496effd640 1 -- 172.21.15.12:0/341100068 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974009690 con 0x7f497806e300 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.764+0000 7f496effd640 1 -- 172.21.15.12:0/341100068 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400ca00 con 0x7f497806e300 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.764+0000 7f496effd640 1 -- 172.21.15.12:0/341100068 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400cd70 con 0x7f497806e300 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.764+0000 7f4980479640 1 -- 172.21.15.12:0/341100068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4978156740 con 0x7f497806e300 2023-11-28T21:42:21.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.764+0000 7f4980479640 1 -- 172.21.15.12:0/341100068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49781f30c0 con 0x7f497806e300 2023-11-28T21:42:21.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.764+0000 7f4980479640 1 -- 172.21.15.12:0/341100068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49781f32e0 con 0x7f497806e300 2023-11-28T21:42:21.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.764+0000 7f496effd640 1 -- 172.21.15.12:0/341100068 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4974013470 con 0x7f497806e300 2023-11-28T21:42:21.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.764+0000 7f496effd640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f495c0664a0 0x7f495c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:21.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.764+0000 7f496effd640 1 -- 172.21.15.12:0/341100068 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f49740843e0 con 0x7f497806e300 2023-11-28T21:42:21.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.768+0000 7f497da14640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f495c0664a0 0x7f495c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:21.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.768+0000 7f497da14640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f495c0664a0 0x7f495c068960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4968007fd0 tx=0x7f4968009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:21.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.772+0000 7f496effd640 1 -- 172.21.15.12:0/341100068 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f49740515f0 con 0x7f497806e300 2023-11-28T21:42:21.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.980+0000 7f4980479640 1 -- 172.21.15.12:0/341100068 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4978074f60 con 0x7f497806e300 2023-11-28T21:42:21.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.980+0000 7f496effd640 1 -- 172.21.15.12:0/341100068 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f4974054c10 con 0x7f497806e300 2023-11-28T21:42:21.984 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:21.984 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:21.984 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2023-11-28T21:42:21.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 -- 172.21.15.12:0/341100068 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f495c0664a0 msgr2=0x7f495c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:21.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f495c0664a0 0x7f495c068960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4968007fd0 tx=0x7f4968009040 comp rx=0 tx=0).stop 2023-11-28T21:42:21.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 -- 172.21.15.12:0/341100068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 msgr2=0x7f49781f35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 0x7f49781f35e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f497400e930 tx=0x7f497400edf0 comp rx=0 tx=0).stop 2023-11-28T21:42:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 -- 172.21.15.12:0/341100068 shutdown_connections 2023-11-28T21:42:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f495c0664a0 0x7f495c068960 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49781f4000 0x7f497814ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4978151af0 0x7f4978140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 --2- 172.21.15.12:0/341100068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f497806e300 0x7f49781f35e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 -- 172.21.15.12:0/341100068 >> 172.21.15.12:0/341100068 conn(0x7f4978066360 msgr2=0x7f497806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 -- 172.21.15.12:0/341100068 shutdown_connections 2023-11-28T21:42:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:21.984+0000 7f4980479640 1 -- 172.21.15.12:0/341100068 wait complete. 2023-11-28T21:42:22.005 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2023-11-28T21:42:22.005 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-11-28T21:42:22.005 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:42:22.025 INFO:tasks.ceph.mds.e:Stopped 2023-11-28T21:42:22.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:22.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.204+0000 7f31be8c8640 1 Processor -- start 2023-11-28T21:42:22.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.204+0000 7f31be8c8640 1 -- start start 2023-11-28T21:42:22.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be8c8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f31b8153fb0 0x7f31b80ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:22.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be8c8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f31b80b41e0 0x7f31b80caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:22.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be8c8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b815d390 con 0x7f31b815a980 2023-11-28T21:42:22.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be8c8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b80b2b60 con 0x7f31b80b41e0 2023-11-28T21:42:22.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be8c8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b815e1b0 con 0x7f31b8153fb0 2023-11-28T21:42:22.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be0c7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f31b80b41e0 0x7f31b80caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:22.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be0c7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f31b80b41e0 0x7f31b80caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54744/0 (socket says 172.21.15.12:54744) 2023-11-28T21:42:22.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be0c7640 1 -- 172.21.15.12:0/1190779089 learned_addr learned my addr 172.21.15.12:0/1190779089 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:22.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31bd8c6640 1 --2- 172.21.15.12:0/1190779089 >> v2:172.21.15.144:3300/0 conn(0x7f31b8153fb0 0x7f31b80ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be0c7640 1 -- 172.21.15.12:0/1190779089 >> v2:172.21.15.144:3300/0 conn(0x7f31b8153fb0 msgr2=0x7f31b80ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be0c7640 1 --2- 172.21.15.12:0/1190779089 >> v2:172.21.15.144:3300/0 conn(0x7f31b8153fb0 0x7f31b80ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be0c7640 1 -- 172.21.15.12:0/1190779089 >> v1:172.21.15.12:6789/0 conn(0x7f31b815a980 legacy=0x7f31b80b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:22.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be0c7640 1 -- 172.21.15.12:0/1190779089 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b815d630 con 0x7f31b80b41e0 2023-11-28T21:42:22.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.208+0000 7f31be0c7640 1 --2- 172.21.15.12:0/1190779089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b80b41e0 0x7f31b80caa30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f31b80d04c0 tx=0x7f31b001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65321fb5e6b8bb02 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:22.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31bc8c4640 1 -- 172.21.15.12:0/1190779089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b00af400 con 0x7f31b80b41e0 2023-11-28T21:42:22.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31bc8c4640 1 -- 172.21.15.12:0/1190779089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b00ab050 con 0x7f31b80b41e0 2023-11-28T21:42:22.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31bc8c4640 1 -- 172.21.15.12:0/1190779089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b00af780 con 0x7f31b80b41e0 2023-11-28T21:42:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 -- 172.21.15.12:0/1190779089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b80b41e0 msgr2=0x7f31b80caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 --2- 172.21.15.12:0/1190779089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b80b41e0 0x7f31b80caa30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f31b80d04c0 tx=0x7f31b001f1c0 comp rx=0 tx=0).stop 2023-11-28T21:42:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 -- 172.21.15.12:0/1190779089 shutdown_connections 2023-11-28T21:42:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 --2- 172.21.15.12:0/1190779089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b80b41e0 0x7f31b80caa30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 --2- 172.21.15.12:0/1190779089 >> v2:172.21.15.144:3300/0 conn(0x7f31b8153fb0 0x7f31b80ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 -- 172.21.15.12:0/1190779089 >> 172.21.15.12:0/1190779089 conn(0x7f31b8071ce0 msgr2=0x7f31b80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 -- 172.21.15.12:0/1190779089 shutdown_connections 2023-11-28T21:42:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 -- 172.21.15.12:0/1190779089 wait complete. 2023-11-28T21:42:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 Processor -- start 2023-11-28T21:42:22.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 -- start start 2023-11-28T21:42:22.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31b80b41e0 0x7f31b80b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:22.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b8153fb0 0x7f31b81e5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:22.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b815a980 0x7f31b80cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:22.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b81e5e00 con 0x7f31b815a980 2023-11-28T21:42:22.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b81525b0 con 0x7f31b8153fb0 2023-11-28T21:42:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be8c8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8152890 con 0x7f31b80b41e0 2023-11-28T21:42:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be0c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b815a980 0x7f31b80cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be0c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b815a980 0x7f31b80cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54746/0 (socket says 172.21.15.12:54746) 2023-11-28T21:42:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31be0c7640 1 -- 172.21.15.12:0/3326828212 learned_addr learned my addr 172.21.15.12:0/3326828212 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31bd8c6640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31b80b41e0 0x7f31b80b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.212+0000 7f31bd0c5640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b8153fb0 0x7f31b81e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31be0c7640 1 -- 172.21.15.12:0/3326828212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31b80b41e0 msgr2=0x7f31b80b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31be0c7640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31b80b41e0 0x7f31b80b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31be0c7640 1 -- 172.21.15.12:0/3326828212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b8153fb0 msgr2=0x7f31b81e5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31be0c7640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b8153fb0 0x7f31b81e5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31be0c7640 1 -- 172.21.15.12:0/3326828212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b0009b30 con 0x7f31b815a980 2023-11-28T21:42:22.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31bd0c5640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b8153fb0 0x7f31b81e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:22.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31be0c7640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b815a980 0x7f31b80cd8d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f31b00b2040 tx=0x7f31b001bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:22.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31ae7fc640 1 -- 172.21.15.12:0/3326828212 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b00afde0 con 0x7f31b815a980 2023-11-28T21:42:22.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31ae7fc640 1 -- 172.21.15.12:0/3326828212 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b0009510 con 0x7f31b815a980 2023-11-28T21:42:22.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31be8c8640 1 -- 172.21.15.12:0/3326828212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b8152b70 con 0x7f31b815a980 2023-11-28T21:42:22.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31ae7fc640 1 -- 172.21.15.12:0/3326828212 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b00ba660 con 0x7f31b815a980 2023-11-28T21:42:22.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31be8c8640 1 -- 172.21.15.12:0/3326828212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b8153130 con 0x7f31b815a980 2023-11-28T21:42:22.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31be8c8640 1 -- 172.21.15.12:0/3326828212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b8153320 con 0x7f31b815a980 2023-11-28T21:42:22.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31ae7fc640 1 -- 172.21.15.12:0/3326828212 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f31b00b6040 con 0x7f31b815a980 2023-11-28T21:42:22.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31ae7fc640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f319c0664a0 0x7f319c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:22.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31ae7fc640 1 -- 172.21.15.12:0/3326828212 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f31b0122430 con 0x7f31b815a980 2023-11-28T21:42:22.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.216+0000 7f31bd8c6640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f319c0664a0 0x7f319c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:22.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.220+0000 7f31bd8c6640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f319c0664a0 0x7f319c068960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f31b4007a60 tx=0x7f31b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:22.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.220+0000 7f31ae7fc640 1 -- 172.21.15.12:0/3326828212 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f31b00ef590 con 0x7f31b815a980 2023-11-28T21:42:22.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.412+0000 7f31be8c8640 1 -- 172.21.15.12:0/3326828212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31b81529d0 con 0x7f31b815a980 2023-11-28T21:42:22.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.412+0000 7f31ae7fc640 1 -- 172.21.15.12:0/3326828212 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f31b81529d0 con 0x7f31b815a980 2023-11-28T21:42:22.417 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:22.417 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:22.417 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2023-11-28T21:42:22.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 -- 172.21.15.12:0/3326828212 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f319c0664a0 msgr2=0x7f319c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:22.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f319c0664a0 0x7f319c068960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f31b4007a60 tx=0x7f31b4008040 comp rx=0 tx=0).stop 2023-11-28T21:42:22.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 -- 172.21.15.12:0/3326828212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b815a980 msgr2=0x7f31b80cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:22.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b815a980 0x7f31b80cd8d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f31b00b2040 tx=0x7f31b001bf60 comp rx=0 tx=0).stop 2023-11-28T21:42:22.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 -- 172.21.15.12:0/3326828212 shutdown_connections 2023-11-28T21:42:22.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f319c0664a0 0x7f319c068960 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:22.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b815a980 0x7f31b80cd8d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:22.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b8153fb0 0x7f31b81e5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:22.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 --2- 172.21.15.12:0/3326828212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31b80b41e0 0x7f31b80b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:22.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 -- 172.21.15.12:0/3326828212 >> 172.21.15.12:0/3326828212 conn(0x7f31b8071ce0 msgr2=0x7f31b80b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:22.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 -- 172.21.15.12:0/3326828212 shutdown_connections 2023-11-28T21:42:22.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:22.416+0000 7f31be8c8640 1 -- 172.21.15.12:0/3326828212 wait complete. 2023-11-28T21:42:22.438 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:42:24.439 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:24.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfbb232640 1 Processor -- start 2023-11-28T21:42:24.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfbb232640 1 -- start start 2023-11-28T21:42:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfbb232640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbfb4153e40 0x7fbfb40ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfbb232640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb40cb180 0x7fbfb40cb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfbb232640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb415acc0 con 0x7fbfb415dc80 2023-11-28T21:42:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfbb232640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb4159cb0 con 0x7fbfb4153e40 2023-11-28T21:42:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfbb232640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4159ea0 con 0x7fbfb40cb180 2023-11-28T21:42:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb3fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbfb415dc80 0x7fbfb40cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53146/0 (socket says 172.21.15.12:53146) 2023-11-28T21:42:24.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb3fff640 1 -- 172.21.15.12:0/767030634 learned_addr learned my addr 172.21.15.12:0/767030634 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb97cf640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb40cb180 0x7fbfb40cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb8fce640 1 --2- 172.21.15.12:0/767030634 >> v2:172.21.15.144:3300/0 conn(0x7fbfb4153e40 0x7fbfb40ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143699918 0 0) 0x7fbfb415acc0 con 0x7fbfb415dc80 2023-11-28T21:42:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfa4003530 con 0x7fbfb415dc80 2023-11-28T21:42:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1293790837 0 0) 0x7fbfa4003530 con 0x7fbfb415dc80 2023-11-28T21:42:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb40cb180 msgr2=0x7fbfb40cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb37fe640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb40cb180 0x7fbfb40cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 >> v2:172.21.15.144:3300/0 conn(0x7fbfb4153e40 msgr2=0x7fbfb40ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:24.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb37fe640 1 --2- 172.21.15.12:0/767030634 >> v2:172.21.15.144:3300/0 conn(0x7fbfb4153e40 0x7fbfb40ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb415a140 con 0x7fbfb415dc80 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fbfa0001680 con 0x7fbfb415dc80 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.592+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 >> v1:172.21.15.12:6789/0 conn(0x7fbfb415dc80 legacy=0x7fbfb40cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb37fe640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 0x7fbfa4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb37fe640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb40cb180 0x7fbfa40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb37fe640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa4004d10 0x7fbfa4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4159ea0 con 0x7fbfa40039f0 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400b990 con 0x7fbfa4004d10 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400bb00 con 0x7fbfb40cb180 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb8fce640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 0x7fbfa4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb3fff640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb40cb180 0x7fbfa40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb97cf640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa4004d10 0x7fbfa4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:24.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb8fce640 1 -- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb40cb180 msgr2=0x7fbfa40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb8fce640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb40cb180 0x7fbfa40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb8fce640 1 -- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa4004d10 msgr2=0x7fbfa4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb8fce640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa4004d10 0x7fbfa4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb8fce640 1 -- 172.21.15.12:0/767030634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa4003530 con 0x7fbfa40039f0 2023-11-28T21:42:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb3fff640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb40cb180 0x7fbfa40047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb97cf640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa4004d10 0x7fbfa4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:24.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.596+0000 7fbfb8fce640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 0x7fbfa4003e10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009ae0 tx=0x7fbfa801ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287cc9af8f47c80f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:24.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80a5630 con 0x7fbfa40039f0 2023-11-28T21:42:24.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa80a5790 con 0x7fbfa40039f0 2023-11-28T21:42:24.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfb37fe640 1 -- 172.21.15.12:0/767030634 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80a5b40 con 0x7fbfa40039f0 2023-11-28T21:42:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 -- 172.21.15.12:0/767030634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 msgr2=0x7fbfa4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 --2- 172.21.15.12:0/767030634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 0x7fbfa4003e10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009ae0 tx=0x7fbfa801ca60 comp rx=0 tx=0).stop 2023-11-28T21:42:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfb97cf640 1 -- 172.21.15.12:0/767030634 reap_dead start 2023-11-28T21:42:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 -- 172.21.15.12:0/767030634 shutdown_connections 2023-11-28T21:42:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 -- 172.21.15.12:0/767030634 >> 172.21.15.12:0/767030634 conn(0x7fbfb406b7b0 msgr2=0x7fbfb4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 -- 172.21.15.12:0/767030634 shutdown_connections 2023-11-28T21:42:24.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 -- 172.21.15.12:0/767030634 wait complete. 2023-11-28T21:42:24.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 Processor -- start 2023-11-28T21:42:24.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 -- start start 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 0x7fbfb40b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb4152ca0 0x7fbfb40b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb40b3580 0x7fbfb415af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb40b3af0 con 0x7fbfa40039f0 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb41f56b0 con 0x7fbfb40b3580 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.600+0000 7fbfbb232640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb41f5990 con 0x7fbfb4152ca0 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb8fce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 0x7fbfb40b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb8fce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 0x7fbfb40b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54764/0 (socket says 172.21.15.12:54764) 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb8fce640 1 -- 172.21.15.12:0/1248601295 learned_addr learned my addr 172.21.15.12:0/1248601295 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb3fff640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb4152ca0 0x7fbfb40b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:24.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb97cf640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb40b3580 0x7fbfb415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb8fce640 1 -- 172.21.15.12:0/1248601295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb4152ca0 msgr2=0x7fbfb40b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb8fce640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb4152ca0 0x7fbfb40b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb8fce640 1 -- 172.21.15.12:0/1248601295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb40b3580 msgr2=0x7fbfb415af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb8fce640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb40b3580 0x7fbfb415af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb8fce640 1 -- 172.21.15.12:0/1248601295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb41f5c70 con 0x7fbfa40039f0 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb8fce640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 0x7fbfb40b48f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009ae0 tx=0x7fbfa4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb17fa640 1 -- 172.21.15.12:0/1248601295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80be070 con 0x7fbfa40039f0 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb17fa640 1 -- 172.21.15.12:0/1248601295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa800f000 con 0x7fbfa40039f0 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfbb232640 1 -- 172.21.15.12:0/1248601295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb41f5f50 con 0x7fbfa40039f0 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb17fa640 1 -- 172.21.15.12:0/1248601295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8004220 con 0x7fbfa40039f0 2023-11-28T21:42:24.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfbb232640 1 -- 172.21.15.12:0/1248601295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb4153100 con 0x7fbfa40039f0 2023-11-28T21:42:24.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfbb232640 1 -- 172.21.15.12:0/1248601295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb4074f60 con 0x7fbfa40039f0 2023-11-28T21:42:24.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.604+0000 7fbfb17fa640 1 -- 172.21.15.12:0/1248601295 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fbfa80b4040 con 0x7fbfa40039f0 2023-11-28T21:42:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.608+0000 7fbfb17fa640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbf98066470 0x7fbf98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.608+0000 7fbfb17fa640 1 -- 172.21.15.12:0/1248601295 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fbfa811dc60 con 0x7fbfa40039f0 2023-11-28T21:42:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.608+0000 7fbfb3fff640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbf98066470 0x7fbf98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:24.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.608+0000 7fbfb3fff640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbf98066470 0x7fbf98068930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbfb41599f0 tx=0x7fbfa0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:24.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.612+0000 7fbfb17fa640 1 -- 172.21.15.12:0/1248601295 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbfa80eac60 con 0x7fbfa40039f0 2023-11-28T21:42:24.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.800+0000 7fbfbb232640 1 -- 172.21.15.12:0/1248601295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfb415d8f0 con 0x7fbfa40039f0 2023-11-28T21:42:24.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.800+0000 7fbfb17fa640 1 -- 172.21.15.12:0/1248601295 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+6503 (secure 0 0 0) 0x7fbfa80ee280 con 0x7fbfa40039f0 2023-11-28T21:42:24.805 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:24.805 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:24.806 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2023-11-28T21:42:24.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 -- 172.21.15.12:0/1248601295 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbf98066470 msgr2=0x7fbf98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:24.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbf98066470 0x7fbf98068930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbfb41599f0 tx=0x7fbfa0009040 comp rx=0 tx=0).stop 2023-11-28T21:42:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 -- 172.21.15.12:0/1248601295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 msgr2=0x7fbfb40b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 0x7fbfb40b48f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009ae0 tx=0x7fbfa4005800 comp rx=0 tx=0).stop 2023-11-28T21:42:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 -- 172.21.15.12:0/1248601295 shutdown_connections 2023-11-28T21:42:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbf98066470 0x7fbf98068930 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb40b3580 0x7fbfb415af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfb4152ca0 0x7fbfb40b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 --2- 172.21.15.12:0/1248601295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbfa40039f0 0x7fbfb40b48f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 -- 172.21.15.12:0/1248601295 >> 172.21.15.12:0/1248601295 conn(0x7fbfb406b7b0 msgr2=0x7fbfb4072570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 -- 172.21.15.12:0/1248601295 shutdown_connections 2023-11-28T21:42:24.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:24.804+0000 7fbfbb232640 1 -- 172.21.15.12:0/1248601295 wait complete. 2023-11-28T21:42:24.827 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:42:26.828 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa014f0f640 1 Processor -- start 2023-11-28T21:42:26.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa014f0f640 1 -- start start 2023-11-28T21:42:26.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa014f0f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0100ca110 0x7fa0100cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:26.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa014f0f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa0100cfba0 0x7fa0100cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:26.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa014f0f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01015acc0 con 0x7fa010153e40 2023-11-28T21:42:26.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa014f0f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa010159cb0 con 0x7fa0100cfba0 2023-11-28T21:42:26.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa014f0f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0100ca4f0 con 0x7fa0100ca110 2023-11-28T21:42:26.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa00e59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa010153e40 0x7fa01015dc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36122/0 (socket says 172.21.15.12:36122) 2023-11-28T21:42:26.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa00e59c640 1 -- 172.21.15.12:0/2140227330 learned_addr learned my addr 172.21.15.12:0/2140227330 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa00ed9d640 1 --2- 172.21.15.12:0/2140227330 >> v2:172.21.15.12:3300/0 conn(0x7fa0100cfba0 0x7fa0100cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa007fff640 1 --2- 172.21.15.12:0/2140227330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0100ca110 0x7fa0100cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa00ed9d640 1 -- 172.21.15.12:0/2140227330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0100ca110 msgr2=0x7fa0100cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa00ed9d640 1 --2- 172.21.15.12:0/2140227330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0100ca110 0x7fa0100cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa00ed9d640 1 -- 172.21.15.12:0/2140227330 >> v1:172.21.15.12:6789/0 conn(0x7fa010153e40 legacy=0x7fa01015dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:26.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.980+0000 7fa00ed9d640 1 -- 172.21.15.12:0/2140227330 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010159ea0 con 0x7fa0100cfba0 2023-11-28T21:42:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa00ed9d640 1 --2- 172.21.15.12:0/2140227330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0100cfba0 0x7fa0100cb180 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa008009d90 tx=0x7fa00801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e366c65953389c1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa00dd9b640 1 -- 172.21.15.12:0/2140227330 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080a59e0 con 0x7fa0100cfba0 2023-11-28T21:42:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa00dd9b640 1 -- 172.21.15.12:0/2140227330 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0080a5b40 con 0x7fa0100cfba0 2023-11-28T21:42:26.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa00dd9b640 1 -- 172.21.15.12:0/2140227330 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080ad440 con 0x7fa0100cfba0 2023-11-28T21:42:26.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa014f0f640 1 -- 172.21.15.12:0/2140227330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0100cfba0 msgr2=0x7fa0100cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:26.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa014f0f640 1 --2- 172.21.15.12:0/2140227330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0100cfba0 0x7fa0100cb180 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa008009d90 tx=0x7fa00801c920 comp rx=0 tx=0).stop 2023-11-28T21:42:26.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa014f0f640 1 -- 172.21.15.12:0/2140227330 shutdown_connections 2023-11-28T21:42:26.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa014f0f640 1 --2- 172.21.15.12:0/2140227330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa0100cfba0 0x7fa0100cb180 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:26.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa014f0f640 1 --2- 172.21.15.12:0/2140227330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0100ca110 0x7fa0100cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:26.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa014f0f640 1 -- 172.21.15.12:0/2140227330 >> 172.21.15.12:0/2140227330 conn(0x7fa01006b7b0 msgr2=0x7fa010066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:26.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa014f0f640 1 -- 172.21.15.12:0/2140227330 shutdown_connections 2023-11-28T21:42:26.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.984+0000 7fa014f0f640 1 -- 172.21.15.12:0/2140227330 wait complete. 2023-11-28T21:42:26.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa014f0f640 1 Processor -- start 2023-11-28T21:42:26.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa014f0f640 1 -- start start 2023-11-28T21:42:26.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa014f0f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0100ca110 0x7fa0100cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:26.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa014f0f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0100cfba0 0x7fa0101f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:26.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa014f0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa010153e40 0x7fa01014c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:26.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa014f0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0100ce910 con 0x7fa010153e40 2023-11-28T21:42:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa014f0f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0100cf780 con 0x7fa0100ca110 2023-11-28T21:42:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa014f0f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0101f5d50 con 0x7fa0100cfba0 2023-11-28T21:42:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa010153e40 0x7fa01014c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa010153e40 0x7fa01014c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43280/0 (socket says 172.21.15.12:43280) 2023-11-28T21:42:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00ed9d640 1 -- 172.21.15.12:0/3745329725 learned_addr learned my addr 172.21.15.12:0/3745329725 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa007fff640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0100cfba0 0x7fa0101f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:26.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00e59c640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0100ca110 0x7fa0100cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:26.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00ed9d640 1 -- 172.21.15.12:0/3745329725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0100cfba0 msgr2=0x7fa0101f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:26.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00ed9d640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0100cfba0 0x7fa0101f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:26.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00ed9d640 1 -- 172.21.15.12:0/3745329725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0100ca110 msgr2=0x7fa0100cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:26.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00ed9d640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0100ca110 0x7fa0100cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:26.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00ed9d640 1 -- 172.21.15.12:0/3745329725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008009a70 con 0x7fa010153e40 2023-11-28T21:42:26.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00e59c640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0100ca110 0x7fa0100cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa00ed9d640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa010153e40 0x7fa01014c860 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa0080b0040 tx=0x7fa0080196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa006ffd640 1 -- 172.21.15.12:0/3745329725 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080ade40 con 0x7fa010153e40 2023-11-28T21:42:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa006ffd640 1 -- 172.21.15.12:0/3745329725 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa008009450 con 0x7fa010153e40 2023-11-28T21:42:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa006ffd640 1 -- 172.21.15.12:0/3745329725 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080b9690 con 0x7fa010153e40 2023-11-28T21:42:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.988+0000 7fa014f0f640 1 -- 172.21.15.12:0/3745329725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0100cf1c0 con 0x7fa010153e40 2023-11-28T21:42:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.992+0000 7fa014f0f640 1 -- 172.21.15.12:0/3745329725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01014a300 con 0x7fa010153e40 2023-11-28T21:42:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.992+0000 7fa014f0f640 1 -- 172.21.15.12:0/3745329725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa010074f60 con 0x7fa010153e40 2023-11-28T21:42:26.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.992+0000 7fa006ffd640 1 -- 172.21.15.12:0/3745329725 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa0080b97f0 con 0x7fa010153e40 2023-11-28T21:42:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.992+0000 7fa006ffd640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ff0066470 0x7f9ff0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.992+0000 7fa006ffd640 1 -- 172.21.15.12:0/3745329725 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fa00811fb80 con 0x7fa010153e40 2023-11-28T21:42:26.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.992+0000 7fa00e59c640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ff0066470 0x7f9ff0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:26.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.992+0000 7fa00e59c640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ff0066470 0x7f9ff0068930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa000005890 tx=0x7fa000009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:27.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:26.996+0000 7fa006ffd640 1 -- 172.21.15.12:0/3745329725 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa0080ecaf0 con 0x7fa010153e40 2023-11-28T21:42:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.192+0000 7fa014f0f640 1 -- 172.21.15.12:0/3745329725 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0100cf980 con 0x7fa010153e40 2023-11-28T21:42:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.192+0000 7fa006ffd640 1 -- 172.21.15.12:0/3745329725 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+6503 (secure 0 0 0) 0x7fa0080f0110 con 0x7fa010153e40 2023-11-28T21:42:27.197 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:27.197 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2023-11-28T21:42:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 -- 172.21.15.12:0/3745329725 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ff0066470 msgr2=0x7f9ff0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ff0066470 0x7f9ff0068930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa000005890 tx=0x7fa000009040 comp rx=0 tx=0).stop 2023-11-28T21:42:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 -- 172.21.15.12:0/3745329725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa010153e40 msgr2=0x7fa01014c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa010153e40 0x7fa01014c860 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa0080b0040 tx=0x7fa0080196c0 comp rx=0 tx=0).stop 2023-11-28T21:42:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 -- 172.21.15.12:0/3745329725 shutdown_connections 2023-11-28T21:42:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ff0066470 0x7f9ff0068930 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa010153e40 0x7fa01014c860 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:27.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0100cfba0 0x7fa0101f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:27.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 --2- 172.21.15.12:0/3745329725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0100ca110 0x7fa0100cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:27.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 -- 172.21.15.12:0/3745329725 >> 172.21.15.12:0/3745329725 conn(0x7fa01006b7b0 msgr2=0x7fa0100cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:27.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 -- 172.21.15.12:0/3745329725 shutdown_connections 2023-11-28T21:42:27.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:27.196+0000 7fa014f0f640 1 -- 172.21.15.12:0/3745329725 wait complete. 2023-11-28T21:42:27.218 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:42:29.219 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.372+0000 7fe10e50f640 1 Processor -- start 2023-11-28T21:42:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.372+0000 7fe10e50f640 1 -- start start 2023-11-28T21:42:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe10e50f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe108151ed0 0x7fe1080c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe10e50f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe10806e300 0x7fe108072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe10e50f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe108157b70 con 0x7fe108151af0 2023-11-28T21:42:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe10e50f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1080707f0 con 0x7fe10806e300 2023-11-28T21:42:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe10e50f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080c7b90 con 0x7fe108151ed0 2023-11-28T21:42:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe107fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe108151af0 0x7fe1080c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36124/0 (socket says 172.21.15.12:36124) 2023-11-28T21:42:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe107fff640 1 -- 172.21.15.12:0/1940293688 learned_addr learned my addr 172.21.15.12:0/1940293688 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe10caac640 1 --2- 172.21.15.12:0/1940293688 >> v2:172.21.15.12:3300/0 conn(0x7fe10806e300 0x7fe108072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe1077fe640 1 --2- 172.21.15.12:0/1940293688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe108151ed0 0x7fe1080c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe10caac640 1 -- 172.21.15.12:0/1940293688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe108151ed0 msgr2=0x7fe1080c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe10caac640 1 --2- 172.21.15.12:0/1940293688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe108151ed0 0x7fe1080c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe10caac640 1 -- 172.21.15.12:0/1940293688 >> v1:172.21.15.12:6789/0 conn(0x7fe108151af0 legacy=0x7fe1080c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:29.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.376+0000 7fe10caac640 1 -- 172.21.15.12:0/1940293688 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe108157e50 con 0x7fe10806e300 2023-11-28T21:42:29.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10caac640 1 --2- 172.21.15.12:0/1940293688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 0x7fe108072070 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe100009d90 tx=0x7fe10001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76b354f014832e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:29.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe106ffd640 1 -- 172.21.15.12:0/1940293688 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000a5810 con 0x7fe10806e300 2023-11-28T21:42:29.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe106ffd640 1 -- 172.21.15.12:0/1940293688 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1000a5970 con 0x7fe10806e300 2023-11-28T21:42:29.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe106ffd640 1 -- 172.21.15.12:0/1940293688 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000a5d00 con 0x7fe10806e300 2023-11-28T21:42:29.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10e50f640 1 -- 172.21.15.12:0/1940293688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 msgr2=0x7fe108072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10e50f640 1 --2- 172.21.15.12:0/1940293688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 0x7fe108072070 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe100009d90 tx=0x7fe10001c950 comp rx=0 tx=0).stop 2023-11-28T21:42:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10e50f640 1 -- 172.21.15.12:0/1940293688 shutdown_connections 2023-11-28T21:42:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10e50f640 1 --2- 172.21.15.12:0/1940293688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 0x7fe108072070 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10e50f640 1 --2- 172.21.15.12:0/1940293688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe108151ed0 0x7fe1080c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10e50f640 1 -- 172.21.15.12:0/1940293688 >> 172.21.15.12:0/1940293688 conn(0x7fe108066360 msgr2=0x7fe108066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10e50f640 1 -- 172.21.15.12:0/1940293688 shutdown_connections 2023-11-28T21:42:29.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10e50f640 1 -- 172.21.15.12:0/1940293688 wait complete. 2023-11-28T21:42:29.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10e50f640 1 Processor -- start 2023-11-28T21:42:29.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.380+0000 7fe10e50f640 1 -- start start 2023-11-28T21:42:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10e50f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 0x7fe10814cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10e50f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe108151af0 0x7fe1080c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10e50f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10814d520 0x7fe1080c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10e50f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe108144b20 con 0x7fe10806e300 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10e50f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe108155f70 con 0x7fe10814d520 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10e50f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1081f3b70 con 0x7fe108151af0 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe107fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 0x7fe10814cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe107fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 0x7fe10814cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43302/0 (socket says 172.21.15.12:43302) 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe107fff640 1 -- 172.21.15.12:0/1494121982 learned_addr learned my addr 172.21.15.12:0/1494121982 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe1077fe640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe108151af0 0x7fe1080c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10caac640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10814d520 0x7fe1080c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe107fff640 1 -- 172.21.15.12:0/1494121982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe108151af0 msgr2=0x7fe1080c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe107fff640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe108151af0 0x7fe1080c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe107fff640 1 -- 172.21.15.12:0/1494121982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10814d520 msgr2=0x7fe1080c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe107fff640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10814d520 0x7fe1080c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe107fff640 1 -- 172.21.15.12:0/1494121982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe100009a70 con 0x7fe10806e300 2023-11-28T21:42:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10caac640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10814d520 0x7fe1080c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:42:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe107fff640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 0x7fe10814cfe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800c420 tx=0x7fe0f800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe104ff9640 1 -- 172.21.15.12:0/1494121982 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8005cf0 con 0x7fe10806e300 2023-11-28T21:42:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe104ff9640 1 -- 172.21.15.12:0/1494121982 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8005e50 con 0x7fe10806e300 2023-11-28T21:42:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10e50f640 1 -- 172.21.15.12:0/1494121982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1081f40a0 con 0x7fe10806e300 2023-11-28T21:42:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe104ff9640 1 -- 172.21.15.12:0/1494121982 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80072f0 con 0x7fe10806e300 2023-11-28T21:42:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10e50f640 1 -- 172.21.15.12:0/1494121982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1080c9280 con 0x7fe10806e300 2023-11-28T21:42:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe10e50f640 1 -- 172.21.15.12:0/1494121982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe108074f60 con 0x7fe10806e300 2023-11-28T21:42:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.384+0000 7fe104ff9640 1 -- 172.21.15.12:0/1494121982 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe0f8007450 con 0x7fe10806e300 2023-11-28T21:42:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.388+0000 7fe104ff9640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe0f40664a0 0x7fe0f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.388+0000 7fe104ff9640 1 -- 172.21.15.12:0/1494121982 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7fe0f80838b0 con 0x7fe10806e300 2023-11-28T21:42:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.388+0000 7fe1077fe640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe0f40664a0 0x7fe0f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.388+0000 7fe1077fe640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe0f40664a0 0x7fe0f4068960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe1080cab30 tx=0x7fe0fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:29.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.392+0000 7fe104ff9640 1 -- 172.21.15.12:0/1494121982 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe0f8050820 con 0x7fe10806e300 2023-11-28T21:42:29.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.588+0000 7fe10e50f640 1 -- 172.21.15.12:0/1494121982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe108144c90 con 0x7fe10806e300 2023-11-28T21:42:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.588+0000 7fe104ff9640 1 -- 172.21.15.12:0/1494121982 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+6503 (secure 0 0 0) 0x7fe0f8017020 con 0x7fe10806e300 2023-11-28T21:42:29.592 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:29.592 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:29.592 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2023-11-28T21:42:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.588+0000 7fe10e50f640 1 -- 172.21.15.12:0/1494121982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe0f40664a0 msgr2=0x7fe0f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.588+0000 7fe10e50f640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe0f40664a0 0x7fe0f4068960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe1080cab30 tx=0x7fe0fc008040 comp rx=0 tx=0).stop 2023-11-28T21:42:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.588+0000 7fe10e50f640 1 -- 172.21.15.12:0/1494121982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 msgr2=0x7fe10814cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.588+0000 7fe10e50f640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 0x7fe10814cfe0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800c420 tx=0x7fe0f800c8e0 comp rx=0 tx=0).stop 2023-11-28T21:42:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.592+0000 7fe10e50f640 1 -- 172.21.15.12:0/1494121982 shutdown_connections 2023-11-28T21:42:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.592+0000 7fe10e50f640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe0f40664a0 0x7fe0f4068960 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:29.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.592+0000 7fe10e50f640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe10814d520 0x7fe1080c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.592+0000 7fe10e50f640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe108151af0 0x7fe1080c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.592+0000 7fe10e50f640 1 --2- 172.21.15.12:0/1494121982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806e300 0x7fe10814cfe0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.592+0000 7fe10e50f640 1 -- 172.21.15.12:0/1494121982 >> 172.21.15.12:0/1494121982 conn(0x7fe108066360 msgr2=0x7fe10806cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.592+0000 7fe10e50f640 1 -- 172.21.15.12:0/1494121982 shutdown_connections 2023-11-28T21:42:29.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:29.592+0000 7fe10e50f640 1 -- 172.21.15.12:0/1494121982 wait complete. 2023-11-28T21:42:29.614 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:42:31.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:31.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f5541a17640 1 Processor -- start 2023-11-28T21:42:31.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f5541a17640 1 -- start start 2023-11-28T21:42:31.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f5541a17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c153a40 0x7f553c0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:31.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f5541a17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f553c15df30 con 0x7f553c0ca7c0 2023-11-28T21:42:31.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f5541a17640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f553c0c45f0 con 0x7f553c0c5da0 2023-11-28T21:42:31.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f5541a17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553c0c5270 con 0x7f553c153a40 2023-11-28T21:42:31.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f553bfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f553c0ca7c0 0x7f553c0c5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36134/0 (socket says 172.21.15.12:36134) 2023-11-28T21:42:31.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f553bfff640 1 -- 172.21.15.12:0/3550433807 learned_addr learned my addr 172.21.15.12:0/3550433807 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:31.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f5540a15640 1 --2- 172.21.15.12:0/3550433807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c153a40 0x7f553c0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:31.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f553b7fe640 1 -- 172.21.15.12:0/3550433807 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249286468 0 0) 0x7f553c15df30 con 0x7f553c0ca7c0 2023-11-28T21:42:31.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f553b7fe640 1 -- 172.21.15.12:0/3550433807 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f552c003530 con 0x7f553c0ca7c0 2023-11-28T21:42:31.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f553b7fe640 1 -- 172.21.15.12:0/3550433807 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693586182 0 0) 0x7f553c0c45f0 con 0x7f553c0c5da0 2023-11-28T21:42:31.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f553b7fe640 1 -- 172.21.15.12:0/3550433807 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f553c15df30 con 0x7f553c0c5da0 2023-11-28T21:42:31.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f553b7fe640 1 -- 172.21.15.12:0/3550433807 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3874951400 0 0) 0x7f552c003530 con 0x7f553c0ca7c0 2023-11-28T21:42:31.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f5540a15640 1 -- 172.21.15.12:0/3550433807 >> v1:172.21.15.144:6789/0 conn(0x7f553c0c5da0 legacy=0x7f553c15d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:31.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f5540a15640 1 -- 172.21.15.12:0/3550433807 >> v1:172.21.15.12:6789/0 conn(0x7f553c0ca7c0 legacy=0x7f553c0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:31.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.768+0000 7f5540a15640 1 -- 172.21.15.12:0/3550433807 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c0c48d0 con 0x7f553c153a40 2023-11-28T21:42:31.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f5540a15640 1 --2- 172.21.15.12:0/3550433807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c153a40 0x7f553c0ca280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f55300097f0 tx=0x7f553001ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5100fd7e6c025bc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:31.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f553b7fe640 1 -- 172.21.15.12:0/3550433807 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528001680 con 0x7f553c153a40 2023-11-28T21:42:31.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f553b7fe640 1 -- 172.21.15.12:0/3550433807 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55300a8650 con 0x7f553c153a40 2023-11-28T21:42:31.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f553b7fe640 1 -- 172.21.15.12:0/3550433807 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55300a89e0 con 0x7f553c153a40 2023-11-28T21:42:31.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f5541a17640 1 -- 172.21.15.12:0/3550433807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c153a40 msgr2=0x7f553c0ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:31.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f5541a17640 1 --2- 172.21.15.12:0/3550433807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c153a40 0x7f553c0ca280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f55300097f0 tx=0x7f553001ebd0 comp rx=0 tx=0).stop 2023-11-28T21:42:31.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f5541a17640 1 -- 172.21.15.12:0/3550433807 shutdown_connections 2023-11-28T21:42:31.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f5541a17640 1 --2- 172.21.15.12:0/3550433807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c153a40 0x7f553c0ca280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:31.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f5541a17640 1 -- 172.21.15.12:0/3550433807 >> 172.21.15.12:0/3550433807 conn(0x7f553c06bda0 msgr2=0x7f553c06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:31.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f5541a17640 1 -- 172.21.15.12:0/3550433807 shutdown_connections 2023-11-28T21:42:31.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.772+0000 7f5541a17640 1 -- 172.21.15.12:0/3550433807 wait complete. 2023-11-28T21:42:31.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541a17640 1 Processor -- start 2023-11-28T21:42:31.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541a17640 1 -- start start 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541a17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c0c5da0 0x7f553c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541a17640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f553c0ca7c0 0x7f553c1ef650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541a17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c153a40 0x7f553c1efb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541a17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c0c6da0 con 0x7f553c153a40 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541a17640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c1f2bc0 con 0x7f553c0ca7c0 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541a17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553c0c7e70 con 0x7f553c0c5da0 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541216640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c153a40 0x7f553c1efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541216640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c153a40 0x7f553c1efb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43306/0 (socket says 172.21.15.12:43306) 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541216640 1 -- 172.21.15.12:0/2344835057 learned_addr learned my addr 172.21.15.12:0/2344835057 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5540a15640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c0c5da0 0x7f553c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:31.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f553bfff640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f553c0ca7c0 0x7f553c1ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:31.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541216640 1 -- 172.21.15.12:0/2344835057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c0c5da0 msgr2=0x7f553c156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:31.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541216640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c0c5da0 0x7f553c156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:31.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541216640 1 -- 172.21.15.12:0/2344835057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f553c0ca7c0 msgr2=0x7f553c1ef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:31.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541216640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f553c0ca7c0 0x7f553c1ef650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:31.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541216640 1 -- 172.21.15.12:0/2344835057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530009420 con 0x7f553c153a40 2023-11-28T21:42:31.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541216640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c153a40 0x7f553c1efb90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f553400c790 tx=0x7f553400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:31.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f55397fa640 1 -- 172.21.15.12:0/2344835057 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553400cea0 con 0x7f553c153a40 2023-11-28T21:42:31.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f55397fa640 1 -- 172.21.15.12:0/2344835057 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5534003b80 con 0x7f553c153a40 2023-11-28T21:42:31.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f55397fa640 1 -- 172.21.15.12:0/2344835057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553400a850 con 0x7f553c153a40 2023-11-28T21:42:31.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541a17640 1 -- 172.21.15.12:0/2344835057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553c0c6800 con 0x7f553c153a40 2023-11-28T21:42:31.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.776+0000 7f5541a17640 1 -- 172.21.15.12:0/2344835057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553c1f2e10 con 0x7f553c153a40 2023-11-28T21:42:31.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.780+0000 7f5541a17640 1 -- 172.21.15.12:0/2344835057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f553c074f60 con 0x7f553c153a40 2023-11-28T21:42:31.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.780+0000 7f55397fa640 1 -- 172.21.15.12:0/2344835057 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f553400a9b0 con 0x7f553c153a40 2023-11-28T21:42:31.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.780+0000 7f55397fa640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5520066470 0x7f5520068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:31.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.780+0000 7f55397fa640 1 -- 172.21.15.12:0/2344835057 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f5534084090 con 0x7f553c153a40 2023-11-28T21:42:31.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.780+0000 7f5540a15640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5520066470 0x7f5520068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:31.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.780+0000 7f5540a15640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5520066470 0x7f5520068930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f55300b2000 tx=0x7f55300aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:31.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.784+0000 7f55397fa640 1 -- 172.21.15.12:0/2344835057 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5534051140 con 0x7f553c153a40 2023-11-28T21:42:31.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.972+0000 7f5541a17640 1 -- 172.21.15.12:0/2344835057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f553c0cad60 con 0x7f553c153a40 2023-11-28T21:42:31.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.972+0000 7f55397fa640 1 -- 172.21.15.12:0/2344835057 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f5534054760 con 0x7f553c153a40 2023-11-28T21:42:31.977 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:31.977 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:31.977 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2023-11-28T21:42:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 -- 172.21.15.12:0/2344835057 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5520066470 msgr2=0x7f5520068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5520066470 0x7f5520068930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f55300b2000 tx=0x7f55300aa070 comp rx=0 tx=0).stop 2023-11-28T21:42:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 -- 172.21.15.12:0/2344835057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c153a40 msgr2=0x7f553c1efb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c153a40 0x7f553c1efb90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f553400c790 tx=0x7f553400cc50 comp rx=0 tx=0).stop 2023-11-28T21:42:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 -- 172.21.15.12:0/2344835057 shutdown_connections 2023-11-28T21:42:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5520066470 0x7f5520068930 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:31.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f553c153a40 0x7f553c1efb90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:31.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f553c0ca7c0 0x7f553c1ef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:31.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 --2- 172.21.15.12:0/2344835057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f553c0c5da0 0x7f553c156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:31.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 -- 172.21.15.12:0/2344835057 >> 172.21.15.12:0/2344835057 conn(0x7f553c06bda0 msgr2=0x7f553c0cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:31.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 -- 172.21.15.12:0/2344835057 shutdown_connections 2023-11-28T21:42:31.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:31.976+0000 7f5541a17640 1 -- 172.21.15.12:0/2344835057 wait complete. 2023-11-28T21:42:31.999 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:42:34.000 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:34.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.152+0000 7faf5fc9c640 1 Processor -- start 2023-11-28T21:42:34.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.152+0000 7faf5fc9c640 1 -- start start 2023-11-28T21:42:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.152+0000 7faf5fc9c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf580c27c0 0x7faf58156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.152+0000 7faf5fc9c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faf5806d7d0 0x7faf5806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.152+0000 7faf5fc9c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf580c6ce0 con 0x7faf581500d0 2023-11-28T21:42:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.152+0000 7faf5fc9c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5806e120 con 0x7faf580c27c0 2023-11-28T21:42:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.152+0000 7faf5fc9c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf58157030 con 0x7faf5806d7d0 2023-11-28T21:42:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5d237640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf580c27c0 0x7faf58156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5d237640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf580c27c0 0x7faf58156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43322/0 (socket says 172.21.15.12:43322) 2023-11-28T21:42:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5d237640 1 -- 172.21.15.12:0/2890066038 learned_addr learned my addr 172.21.15.12:0/2890066038 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5e239640 1 --2- 172.21.15.12:0/2890066038 >> v2:172.21.15.144:3300/0 conn(0x7faf5806d7d0 0x7faf5806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5d237640 1 -- 172.21.15.12:0/2890066038 >> v2:172.21.15.144:3300/0 conn(0x7faf5806d7d0 msgr2=0x7faf5806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5d237640 1 --2- 172.21.15.12:0/2890066038 >> v2:172.21.15.144:3300/0 conn(0x7faf5806d7d0 0x7faf5806dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5d237640 1 -- 172.21.15.12:0/2890066038 >> v1:172.21.15.12:6789/0 conn(0x7faf581500d0 legacy=0x7faf580c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:34.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5d237640 1 -- 172.21.15.12:0/2890066038 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580c5df0 con 0x7faf580c27c0 2023-11-28T21:42:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5d237640 1 --2- 172.21.15.12:0/2890066038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf580c27c0 0x7faf58156af0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faf581508e0 tx=0x7faf4801ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d44d87950e286a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:34.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5ca36640 1 -- 172.21.15.12:0/2890066038 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480a7a20 con 0x7faf580c27c0 2023-11-28T21:42:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5ca36640 1 -- 172.21.15.12:0/2890066038 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf480a7b80 con 0x7faf580c27c0 2023-11-28T21:42:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.156+0000 7faf5ca36640 1 -- 172.21.15.12:0/2890066038 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480af440 con 0x7faf580c27c0 2023-11-28T21:42:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 -- 172.21.15.12:0/2890066038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf580c27c0 msgr2=0x7faf58156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:34.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 --2- 172.21.15.12:0/2890066038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf580c27c0 0x7faf58156af0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faf581508e0 tx=0x7faf4801ebd0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 -- 172.21.15.12:0/2890066038 shutdown_connections 2023-11-28T21:42:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 --2- 172.21.15.12:0/2890066038 >> v2:172.21.15.144:3300/0 conn(0x7faf5806d7d0 0x7faf5806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 --2- 172.21.15.12:0/2890066038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf580c27c0 0x7faf58156af0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 -- 172.21.15.12:0/2890066038 >> 172.21.15.12:0/2890066038 conn(0x7faf58066360 msgr2=0x7faf58066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 -- 172.21.15.12:0/2890066038 shutdown_connections 2023-11-28T21:42:34.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 -- 172.21.15.12:0/2890066038 wait complete. 2023-11-28T21:42:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 Processor -- start 2023-11-28T21:42:34.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 -- start start 2023-11-28T21:42:34.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5806d7d0 0x7faf580ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:34.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf580c27c0 0x7faf58070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:34.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf581500d0 0x7faf580713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf58070a10 con 0x7faf5806d7d0 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf58154140 con 0x7faf580c27c0 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5fc9c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf58070400 con 0x7faf581500d0 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5da38640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5806d7d0 0x7faf580ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5da38640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5806d7d0 0x7faf580ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43324/0 (socket says 172.21.15.12:43324) 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5da38640 1 -- 172.21.15.12:0/3213182296 learned_addr learned my addr 172.21.15.12:0/3213182296 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5e239640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf581500d0 0x7faf580713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.160+0000 7faf5d237640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf580c27c0 0x7faf58070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5da38640 1 -- 172.21.15.12:0/3213182296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf581500d0 msgr2=0x7faf580713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5da38640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf581500d0 0x7faf580713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5da38640 1 -- 172.21.15.12:0/3213182296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf580c27c0 msgr2=0x7faf58070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:34.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5da38640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf580c27c0 0x7faf58070ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5da38640 1 -- 172.21.15.12:0/3213182296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4800b770 con 0x7faf5806d7d0 2023-11-28T21:42:34.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5d237640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf580c27c0 0x7faf58070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:34.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5da38640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5806d7d0 0x7faf580ca370 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c530 tx=0x7faf5400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:34.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf4e7fc640 1 -- 172.21.15.12:0/3213182296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54005220 con 0x7faf5806d7d0 2023-11-28T21:42:34.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf4e7fc640 1 -- 172.21.15.12:0/3213182296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf54005380 con 0x7faf5806d7d0 2023-11-28T21:42:34.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5fc9c640 1 -- 172.21.15.12:0/3213182296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf581f34e0 con 0x7faf5806d7d0 2023-11-28T21:42:34.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf4e7fc640 1 -- 172.21.15.12:0/3213182296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54007270 con 0x7faf5806d7d0 2023-11-28T21:42:34.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5fc9c640 1 -- 172.21.15.12:0/3213182296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf581f37c0 con 0x7faf5806d7d0 2023-11-28T21:42:34.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5fc9c640 1 -- 172.21.15.12:0/3213182296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf58067ff0 con 0x7faf5806d7d0 2023-11-28T21:42:34.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf4e7fc640 1 -- 172.21.15.12:0/3213182296 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faf540073d0 con 0x7faf5806d7d0 2023-11-28T21:42:34.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf4e7fc640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf3c066470 0x7faf3c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:34.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf4e7fc640 1 -- 172.21.15.12:0/3213182296 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7faf54083ae0 con 0x7faf5806d7d0 2023-11-28T21:42:34.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.164+0000 7faf5d237640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf3c066470 0x7faf3c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:34.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.168+0000 7faf5d237640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf3c066470 0x7faf3c068930 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf480b1000 tx=0x7faf48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:34.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.172+0000 7faf4e7fc640 1 -- 172.21.15.12:0/3213182296 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faf54050b80 con 0x7faf5806d7d0 2023-11-28T21:42:34.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.456+0000 7faf5fc9c640 1 -- 172.21.15.12:0/3213182296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf58074f60 con 0x7faf5806d7d0 2023-11-28T21:42:34.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.456+0000 7faf4e7fc640 1 -- 172.21.15.12:0/3213182296 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+6503 (secure 0 0 0) 0x7faf540541a0 con 0x7faf5806d7d0 2023-11-28T21:42:34.467 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:34.467 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:34.467 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2023-11-28T21:42:34.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.472+0000 7faf5fc9c640 1 -- 172.21.15.12:0/3213182296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf3c066470 msgr2=0x7faf3c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:34.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.472+0000 7faf5fc9c640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf3c066470 0x7faf3c068930 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf480b1000 tx=0x7faf48008040 comp rx=0 tx=0).stop 2023-11-28T21:42:34.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.472+0000 7faf5fc9c640 1 -- 172.21.15.12:0/3213182296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5806d7d0 msgr2=0x7faf580ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:34.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.472+0000 7faf5fc9c640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5806d7d0 0x7faf580ca370 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c530 tx=0x7faf5400c9f0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.476+0000 7faf5fc9c640 1 -- 172.21.15.12:0/3213182296 shutdown_connections 2023-11-28T21:42:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.476+0000 7faf5fc9c640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf3c066470 0x7faf3c068930 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.476+0000 7faf5fc9c640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf581500d0 0x7faf580713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.476+0000 7faf5fc9c640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf580c27c0 0x7faf58070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.476+0000 7faf5fc9c640 1 --2- 172.21.15.12:0/3213182296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5806d7d0 0x7faf580ca370 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:34.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.476+0000 7faf5fc9c640 1 -- 172.21.15.12:0/3213182296 >> 172.21.15.12:0/3213182296 conn(0x7faf58066360 msgr2=0x7faf581551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.484+0000 7faf5fc9c640 1 -- 172.21.15.12:0/3213182296 shutdown_connections 2023-11-28T21:42:34.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:34.484+0000 7faf5fc9c640 1 -- 172.21.15.12:0/3213182296 wait complete. 2023-11-28T21:42:34.563 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:42:36.564 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:36.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.716+0000 7f591c6d6640 1 Processor -- start 2023-11-28T21:42:36.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.716+0000 7f591c6d6640 1 -- start start 2023-11-28T21:42:36.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.716+0000 7f591c6d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5914150c20 0x7f5914156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:36.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.716+0000 7f591c6d6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f591406cf40 0x7f5914158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:36.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591c6d6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5914067410 con 0x7f591406e300 2023-11-28T21:42:36.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591c6d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591406e090 con 0x7f5914150c20 2023-11-28T21:42:36.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591c6d6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5914157f30 con 0x7f591406cf40 2023-11-28T21:42:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591a472640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5914150c20 0x7f5914156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591a472640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5914150c20 0x7f5914156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42742/0 (socket says 172.21.15.12:42742) 2023-11-28T21:42:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591a472640 1 -- 172.21.15.12:0/1477153142 learned_addr learned my addr 172.21.15.12:0/1477153142 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591ac73640 1 --2- 172.21.15.12:0/1477153142 >> v2:172.21.15.144:3300/0 conn(0x7f591406cf40 0x7f5914158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591a472640 1 -- 172.21.15.12:0/1477153142 >> v2:172.21.15.144:3300/0 conn(0x7f591406cf40 msgr2=0x7f5914158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591a472640 1 --2- 172.21.15.12:0/1477153142 >> v2:172.21.15.144:3300/0 conn(0x7f591406cf40 0x7f5914158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591a472640 1 -- 172.21.15.12:0/1477153142 >> v1:172.21.15.144:6789/0 conn(0x7f591406e300 legacy=0x7f59140c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591a472640 1 -- 172.21.15.12:0/1477153142 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5914151000 con 0x7f5914150c20 2023-11-28T21:42:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f591a472640 1 --2- 172.21.15.12:0/1477153142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5914150c20 0x7f5914156910 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f591406b9d0 tx=0x7f591001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fe3b6d9be8e14d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:36.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f5919470640 1 -- 172.21.15.12:0/1477153142 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100ab610 con 0x7f5914150c20 2023-11-28T21:42:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f5919470640 1 -- 172.21.15.12:0/1477153142 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59100ab770 con 0x7f5914150c20 2023-11-28T21:42:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.720+0000 7f5919470640 1 -- 172.21.15.12:0/1477153142 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100abb00 con 0x7f5914150c20 2023-11-28T21:42:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 -- 172.21.15.12:0/1477153142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5914150c20 msgr2=0x7f5914156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 --2- 172.21.15.12:0/1477153142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5914150c20 0x7f5914156910 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f591406b9d0 tx=0x7f591001c500 comp rx=0 tx=0).stop 2023-11-28T21:42:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 -- 172.21.15.12:0/1477153142 shutdown_connections 2023-11-28T21:42:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 --2- 172.21.15.12:0/1477153142 >> v2:172.21.15.144:3300/0 conn(0x7f591406cf40 0x7f5914158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 --2- 172.21.15.12:0/1477153142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5914150c20 0x7f5914156910 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 -- 172.21.15.12:0/1477153142 >> 172.21.15.12:0/1477153142 conn(0x7f5914066360 msgr2=0x7f5914066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:36.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 -- 172.21.15.12:0/1477153142 shutdown_connections 2023-11-28T21:42:36.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 -- 172.21.15.12:0/1477153142 wait complete. 2023-11-28T21:42:36.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 Processor -- start 2023-11-28T21:42:36.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 -- start start 2023-11-28T21:42:36.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f591406cf40 0x7f5914153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:36.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f591406e300 0x7f5914160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:36.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5914150c20 0x7f5914151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:36.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59141577b0 con 0x7f591406cf40 2023-11-28T21:42:36.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5914157280 con 0x7f5914150c20 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591c6d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59141574d0 con 0x7f591406e300 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591a472640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f591406cf40 0x7f5914153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591a472640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f591406cf40 0x7f5914153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42758/0 (socket says 172.21.15.12:42758) 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591a472640 1 -- 172.21.15.12:0/603668443 learned_addr learned my addr 172.21.15.12:0/603668443 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591ac73640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5914150c20 0x7f5914151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f5919c71640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f591406e300 0x7f5914160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591a472640 1 -- 172.21.15.12:0/603668443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f591406e300 msgr2=0x7f5914160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591a472640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f591406e300 0x7f5914160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591a472640 1 -- 172.21.15.12:0/603668443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5914150c20 msgr2=0x7f5914151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.724+0000 7f591a472640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5914150c20 0x7f5914151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f591a472640 1 -- 172.21.15.12:0/603668443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59100094d0 con 0x7f591406cf40 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f591a472640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f591406cf40 0x7f5914153a30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5910019110 tx=0x7f591001ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:36.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f590affd640 1 -- 172.21.15.12:0/603668443 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100b4800 con 0x7f591406cf40 2023-11-28T21:42:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f590affd640 1 -- 172.21.15.12:0/603668443 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59100b4980 con 0x7f591406cf40 2023-11-28T21:42:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f590affd640 1 -- 172.21.15.12:0/603668443 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100b4d30 con 0x7f591406cf40 2023-11-28T21:42:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f591c6d6640 1 -- 172.21.15.12:0/603668443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59141513e0 con 0x7f591406cf40 2023-11-28T21:42:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f591c6d6640 1 -- 172.21.15.12:0/603668443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59141519a0 con 0x7f591406cf40 2023-11-28T21:42:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f591c6d6640 1 -- 172.21.15.12:0/603668443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5914151240 con 0x7f591406cf40 2023-11-28T21:42:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f590affd640 1 -- 172.21.15.12:0/603668443 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f59100b3380 con 0x7f591406cf40 2023-11-28T21:42:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f590affd640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58f8066470 0x7f58f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:36.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f590affd640 1 -- 172.21.15.12:0/603668443 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6835+0+0 (secure 0 0 0) 0x7f591011e400 con 0x7f591406cf40 2023-11-28T21:42:36.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.728+0000 7f5919c71640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58f8066470 0x7f58f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:36.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.732+0000 7f5919c71640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58f8066470 0x7f58f8068930 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5914159540 tx=0x7f5904009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:36.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.732+0000 7f590affd640 1 -- 172.21.15.12:0/603668443 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5914151240 con 0x7f591406cf40 2023-11-28T21:42:36.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.924+0000 7f591c6d6640 1 -- 172.21.15.12:0/603668443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5914151240 con 0x7f591406cf40 2023-11-28T21:42:36.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.924+0000 7f590affd640 1 -- 172.21.15.12:0/603668443 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f5914151240 con 0x7f591406cf40 2023-11-28T21:42:36.928 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:36.929 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:41:21.708127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":44,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5232":{"gid":5232,"name":"e","rank":0,"incarnation":28,"state":"up:active","state_seq":52,"addr":"172.21.15.12:6839/3009112102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3009112102},{"type":"v1","addr":"172.21.15.12:6839","nonce":3009112102}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:36.929 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 32 2023-11-28T21:42:36.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 -- 172.21.15.12:0/603668443 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58f8066470 msgr2=0x7f58f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:36.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58f8066470 0x7f58f8068930 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5914159540 tx=0x7f5904009040 comp rx=0 tx=0).stop 2023-11-28T21:42:36.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 -- 172.21.15.12:0/603668443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f591406cf40 msgr2=0x7f5914153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:36.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f591406cf40 0x7f5914153a30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5910019110 tx=0x7f591001ad40 comp rx=0 tx=0).stop 2023-11-28T21:42:36.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 -- 172.21.15.12:0/603668443 shutdown_connections 2023-11-28T21:42:36.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58f8066470 0x7f58f8068930 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:36.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5914150c20 0x7f5914151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:36.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f591406e300 0x7f5914160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:36.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 --2- 172.21.15.12:0/603668443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f591406cf40 0x7f5914153a30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:36.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 -- 172.21.15.12:0/603668443 >> 172.21.15.12:0/603668443 conn(0x7f5914066360 msgr2=0x7f59140726b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:36.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 -- 172.21.15.12:0/603668443 shutdown_connections 2023-11-28T21:42:36.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:36.928+0000 7f591c6d6640 1 -- 172.21.15.12:0/603668443 wait complete. 2023-11-28T21:42:36.950 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:42:38.951 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.103+0000 7f544252a640 1 Processor -- start 2023-11-28T21:42:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.103+0000 7f544252a640 1 -- start start 2023-11-28T21:42:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f544252a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c150c20 0x7f543c156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f544252a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f543c06e300 0x7f543c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f544252a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c067410 con 0x7f543c06cf40 2023-11-28T21:42:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f544252a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543c06e090 con 0x7f543c06e300 2023-11-28T21:42:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f544252a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c157f30 con 0x7f543c150c20 2023-11-28T21:42:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f5440ac7640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f543c06cf40 0x7f543c158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41360/0 (socket says 172.21.15.12:41360) 2023-11-28T21:42:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f5440ac7640 1 -- 172.21.15.12:0/2397984539 learned_addr learned my addr 172.21.15.12:0/2397984539 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543b7fe640 1 --2- 172.21.15.12:0/2397984539 >> v2:172.21.15.144:3300/0 conn(0x7f543c06e300 0x7f543c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543bfff640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c150c20 0x7f543c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739878807 0 0) 0x7f543c067410 con 0x7f543c06cf40 2023-11-28T21:42:39.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5424005740 con 0x7f543c06cf40 2023-11-28T21:42:39.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1263797904 0 0) 0x7f5424005740 con 0x7f543c06cf40 2023-11-28T21:42:39.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c150c20 msgr2=0x7f543c156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:39.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c150c20 0x7f543c156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 >> v2:172.21.15.144:3300/0 conn(0x7f543c06e300 msgr2=0x7f543c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 --2- 172.21.15.12:0/2397984539 >> v2:172.21.15.144:3300/0 conn(0x7f543c06e300 0x7f543c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c151000 con 0x7f543c06cf40 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f5434001a00 con 0x7f543c06cf40 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 >> v1:172.21.15.12:6789/0 conn(0x7f543c06cf40 legacy=0x7f543c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5424005c00 0x7f5424006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54240069e0 0x7f5424006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c150c20 0x7f54240073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c06e090 con 0x7f5424005c00 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c157f30 con 0x7f54240069e0 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542400dbd0 con 0x7f543c150c20 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543bfff640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5424005c00 0x7f5424006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543b7fe640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54240069e0 0x7f5424006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f5440ac7640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c150c20 0x7f54240073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543bfff640 1 -- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c150c20 msgr2=0x7f54240073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543bfff640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c150c20 0x7f54240073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543bfff640 1 -- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54240069e0 msgr2=0x7f5424006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543bfff640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54240069e0 0x7f5424006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543bfff640 1 -- 172.21.15.12:0/2397984539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424005740 con 0x7f5424005c00 2023-11-28T21:42:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f5440ac7640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c150c20 0x7f54240073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:42:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.107+0000 7f543b7fe640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54240069e0 0x7f5424006e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:39.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f543bfff640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5424005c00 0x7f5424006020 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f542c009ae0 tx=0x7f542c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a312249a6bfc16f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:39.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c0a9460 con 0x7f5424005c00 2023-11-28T21:42:39.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c0a95c0 con 0x7f5424005c00 2023-11-28T21:42:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f543affd640 1 -- 172.21.15.12:0/2397984539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c0a9970 con 0x7f5424005c00 2023-11-28T21:42:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f544252a640 1 -- 172.21.15.12:0/2397984539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5424005c00 msgr2=0x7f5424006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f544252a640 1 --2- 172.21.15.12:0/2397984539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5424005c00 0x7f5424006020 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f542c009ae0 tx=0x7f542c01ca50 comp rx=0 tx=0).stop 2023-11-28T21:42:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f5440ac7640 1 -- 172.21.15.12:0/2397984539 reap_dead start 2023-11-28T21:42:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f544252a640 1 -- 172.21.15.12:0/2397984539 shutdown_connections 2023-11-28T21:42:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f544252a640 1 -- 172.21.15.12:0/2397984539 >> 172.21.15.12:0/2397984539 conn(0x7f543c066360 msgr2=0x7f543c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f544252a640 1 -- 172.21.15.12:0/2397984539 shutdown_connections 2023-11-28T21:42:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.111+0000 7f544252a640 1 -- 172.21.15.12:0/2397984539 wait complete. 2023-11-28T21:42:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 Processor -- start 2023-11-28T21:42:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 -- start start 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424005c00 0x7f543c0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f543c14ddd0 0x7f543c0c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c0c3ce0 0x7f543c14b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0c3260 con 0x7f543c14ddd0 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0c34e0 con 0x7f5424005c00 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0c37c0 con 0x7f543c0c3ce0 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f543c14ddd0 0x7f543c0c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f543c14ddd0 0x7f543c0c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42786/0 (socket says 172.21.15.12:42786) 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543b7fe640 1 -- 172.21.15.12:0/4028085314 learned_addr learned my addr 172.21.15.12:0/4028085314 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f5440ac7640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c0c3ce0 0x7f543c14b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543bfff640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424005c00 0x7f543c0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543b7fe640 1 -- 172.21.15.12:0/4028085314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c0c3ce0 msgr2=0x7f543c14b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543b7fe640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c0c3ce0 0x7f543c14b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543b7fe640 1 -- 172.21.15.12:0/4028085314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424005c00 msgr2=0x7f543c0c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543b7fe640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424005c00 0x7f543c0c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543b7fe640 1 -- 172.21.15.12:0/4028085314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c1f3370 con 0x7f543c14ddd0 2023-11-28T21:42:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f5440ac7640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c0c3ce0 0x7f543c14b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:42:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543bfff640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424005c00 0x7f543c0c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:42:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f543b7fe640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f543c14ddd0 0x7f543c0c41a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f54300118d0 tx=0x7f5430011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f5438ff9640 1 -- 172.21.15.12:0/4028085314 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430004220 con 0x7f543c14ddd0 2023-11-28T21:42:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f5438ff9640 1 -- 172.21.15.12:0/4028085314 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430004380 con 0x7f543c14ddd0 2023-11-28T21:42:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f5438ff9640 1 -- 172.21.15.12:0/4028085314 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543000fbd0 con 0x7f543c14ddd0 2023-11-28T21:42:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 -- 172.21.15.12:0/4028085314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543c1f3650 con 0x7f543c14ddd0 2023-11-28T21:42:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 -- 172.21.15.12:0/4028085314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c0c2ec0 con 0x7f543c14ddd0 2023-11-28T21:42:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.115+0000 7f544252a640 1 -- 172.21.15.12:0/4028085314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543c074f60 con 0x7f543c14ddd0 2023-11-28T21:42:39.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.119+0000 7f5438ff9640 1 -- 172.21.15.12:0/4028085314 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f543000fd30 con 0x7f543c14ddd0 2023-11-28T21:42:39.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.119+0000 7f5438ff9640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54280664a0 0x7f5428068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.119+0000 7f5438ff9640 1 -- 172.21.15.12:0/4028085314 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f5430084920 con 0x7f543c14ddd0 2023-11-28T21:42:39.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.119+0000 7f543bfff640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54280664a0 0x7f5428068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.119+0000 7f543bfff640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54280664a0 0x7f5428068960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f542c0ae000 tx=0x7f542c0b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:39.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.123+0000 7f5438ff9640 1 -- 172.21.15.12:0/4028085314 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5430051850 con 0x7f543c14ddd0 2023-11-28T21:42:39.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.311+0000 7f544252a640 1 -- 172.21.15.12:0/4028085314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f543c0c2c30 con 0x7f543c14ddd0 2023-11-28T21:42:39.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.311+0000 7f5438ff9640 1 -- 172.21.15.12:0/4028085314 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5711 (secure 0 0 0) 0x7f5430022020 con 0x7f543c14ddd0 2023-11-28T21:42:39.317 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:39.317 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:38.424253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:39.317 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2023-11-28T21:42:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 -- 172.21.15.12:0/4028085314 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54280664a0 msgr2=0x7f5428068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54280664a0 0x7f5428068960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f542c0ae000 tx=0x7f542c0b3040 comp rx=0 tx=0).stop 2023-11-28T21:42:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 -- 172.21.15.12:0/4028085314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f543c14ddd0 msgr2=0x7f543c0c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f543c14ddd0 0x7f543c0c41a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f54300118d0 tx=0x7f5430011d90 comp rx=0 tx=0).stop 2023-11-28T21:42:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 -- 172.21.15.12:0/4028085314 shutdown_connections 2023-11-28T21:42:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54280664a0 0x7f5428068960 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f543c0c3ce0 0x7f543c14b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f543c14ddd0 0x7f543c0c41a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 --2- 172.21.15.12:0/4028085314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5424005c00 0x7f543c0c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 -- 172.21.15.12:0/4028085314 >> 172.21.15.12:0/4028085314 conn(0x7f543c066360 msgr2=0x7f543c0c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:39.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 -- 172.21.15.12:0/4028085314 shutdown_connections 2023-11-28T21:42:39.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.315+0000 7f544252a640 1 -- 172.21.15.12:0/4028085314 wait complete. 2023-11-28T21:42:39.338 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-11-28T21:42:39.339 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:42:39.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:39.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.555+0000 7f8a87b0f640 1 Processor -- start 2023-11-28T21:42:39.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.555+0000 7f8a87b0f640 1 -- start start 2023-11-28T21:42:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.559+0000 7f8a87b0f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8a8015d320 0x7f8a8006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.559+0000 7f8a87b0f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8015e270 con 0x7f8a8006aaf0 2023-11-28T21:42:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.559+0000 7f8a87b0f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a800c4e60 con 0x7f8a80154100 2023-11-28T21:42:39.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.559+0000 7f8a87b0f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8015d740 con 0x7f8a8015d320 2023-11-28T21:42:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.559+0000 7f8a850aa640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8a8015d320 0x7f8a8006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.559+0000 7f8a858ab640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8a80154100 0x7f8a800c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:48692/0 (socket says 172.21.15.12:48692) 2023-11-28T21:42:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.559+0000 7f8a858ab640 1 -- 172.21.15.12:0/3466992505 learned_addr learned my addr 172.21.15.12:0/3466992505 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.559+0000 7f8a850aa640 1 -- 172.21.15.12:0/3466992505 >> v1:172.21.15.144:6789/0 conn(0x7f8a80154100 legacy=0x7f8a800c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.559+0000 7f8a850aa640 1 -- 172.21.15.12:0/3466992505 >> v1:172.21.15.12:6789/0 conn(0x7f8a8006aaf0 legacy=0x7f8a800c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.559+0000 7f8a850aa640 1 -- 172.21.15.12:0/3466992505 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a801544e0 con 0x7f8a8015d320 2023-11-28T21:42:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.567+0000 7f8a850aa640 1 --2- 172.21.15.12:0/3466992505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8015d320 0x7f8a8006a5b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8a700097c0 tx=0x7f8a7001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c48f432020c6215c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.567+0000 7f8a848a9640 1 -- 172.21.15.12:0/3466992505 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700ab680 con 0x7f8a8015d320 2023-11-28T21:42:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.567+0000 7f8a848a9640 1 -- 172.21.15.12:0/3466992505 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a700ab7e0 con 0x7f8a8015d320 2023-11-28T21:42:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.567+0000 7f8a848a9640 1 -- 172.21.15.12:0/3466992505 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700abb70 con 0x7f8a8015d320 2023-11-28T21:42:39.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.567+0000 7f8a87b0f640 1 -- 172.21.15.12:0/3466992505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8015d320 msgr2=0x7f8a8006a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.567+0000 7f8a87b0f640 1 --2- 172.21.15.12:0/3466992505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8015d320 0x7f8a8006a5b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8a700097c0 tx=0x7f8a7001c480 comp rx=0 tx=0).stop 2023-11-28T21:42:39.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.567+0000 7f8a87b0f640 1 -- 172.21.15.12:0/3466992505 shutdown_connections 2023-11-28T21:42:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.571+0000 7f8a87b0f640 1 --2- 172.21.15.12:0/3466992505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8015d320 0x7f8a8006a5b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.571+0000 7f8a87b0f640 1 -- 172.21.15.12:0/3466992505 >> 172.21.15.12:0/3466992505 conn(0x7f8a8006c690 msgr2=0x7f8a80066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.571+0000 7f8a87b0f640 1 -- 172.21.15.12:0/3466992505 shutdown_connections 2023-11-28T21:42:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.571+0000 7f8a87b0f640 1 -- 172.21.15.12:0/3466992505 wait complete. 2023-11-28T21:42:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.571+0000 7f8a87b0f640 1 Processor -- start 2023-11-28T21:42:39.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.571+0000 7f8a87b0f640 1 -- start start 2023-11-28T21:42:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a87b0f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8006aaf0 0x7f8a80141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a87b0f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a80154100 0x7f8a801f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a87b0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a8015d320 0x7f8a8013bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a87b0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a801457b0 con 0x7f8a8015d320 2023-11-28T21:42:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a87b0f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80140ba0 con 0x7f8a8006aaf0 2023-11-28T21:42:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a87b0f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80146980 con 0x7f8a80154100 2023-11-28T21:42:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a858ab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8006aaf0 0x7f8a80141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a858ab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8006aaf0 0x7f8a80141fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:47682/0 (socket says 172.21.15.12:47682) 2023-11-28T21:42:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a858ab640 1 -- 172.21.15.12:0/2694358438 learned_addr learned my addr 172.21.15.12:0/2694358438 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:39.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a860ac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a8015d320 0x7f8a8013bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a858ab640 1 -- 172.21.15.12:0/2694358438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a80154100 msgr2=0x7f8a801f63e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T21:42:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a858ab640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a80154100 0x7f8a801f63e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a858ab640 1 -- 172.21.15.12:0/2694358438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a8015d320 msgr2=0x7f8a8013bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a858ab640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a8015d320 0x7f8a8013bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.575+0000 7f8a858ab640 1 -- 172.21.15.12:0/2694358438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a70009420 con 0x7f8a8006aaf0 2023-11-28T21:42:39.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.579+0000 7f8a858ab640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8006aaf0 0x7f8a80141fb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00c3d0 tx=0x7f8a7c00c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:39.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.579+0000 7f8a767fc640 1 -- 172.21.15.12:0/2694358438 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c005cf0 con 0x7f8a8006aaf0 2023-11-28T21:42:39.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.579+0000 7f8a767fc640 1 -- 172.21.15.12:0/2694358438 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c005e50 con 0x7f8a8006aaf0 2023-11-28T21:42:39.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.579+0000 7f8a767fc640 1 -- 172.21.15.12:0/2694358438 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c0072f0 con 0x7f8a8006aaf0 2023-11-28T21:42:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.579+0000 7f8a87b0f640 1 -- 172.21.15.12:0/2694358438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a801f6d20 con 0x7f8a8006aaf0 2023-11-28T21:42:39.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.579+0000 7f8a87b0f640 1 -- 172.21.15.12:0/2694358438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a80153350 con 0x7f8a8006aaf0 2023-11-28T21:42:39.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.583+0000 7f8a87b0f640 1 -- 172.21.15.12:0/2694358438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a80067c40 con 0x7f8a8006aaf0 2023-11-28T21:42:39.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.583+0000 7f8a767fc640 1 -- 172.21.15.12:0/2694358438 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8a7c007450 con 0x7f8a8006aaf0 2023-11-28T21:42:39.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.583+0000 7f8a767fc640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a640664a0 0x7f8a64068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:39.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.583+0000 7f8a767fc640 1 -- 172.21.15.12:0/2694358438 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f8a7c0837c0 con 0x7f8a8006aaf0 2023-11-28T21:42:39.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.587+0000 7f8a850aa640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a640664a0 0x7f8a64068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:39.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.591+0000 7f8a767fc640 1 -- 172.21.15.12:0/2694358438 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a7c0508c0 con 0x7f8a8006aaf0 2023-11-28T21:42:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.591+0000 7f8a850aa640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a640664a0 0x7f8a64068960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8a700af000 tx=0x7f8a70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:39.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.783+0000 7f8a87b0f640 1 -- 172.21.15.12:0/2694358438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a80074f60 con 0x7f8a8006aaf0 2023-11-28T21:42:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.783+0000 7f8a767fc640 1 -- 172.21.15.12:0/2694358438 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5711 (secure 0 0 0) 0x7f8a7c053ee0 con 0x7f8a8006aaf0 2023-11-28T21:42:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:39.790 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:38.424253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:39.790 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2023-11-28T21:42:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 -- 172.21.15.12:0/2694358438 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a640664a0 msgr2=0x7f8a64068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a640664a0 0x7f8a64068960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8a700af000 tx=0x7f8a70008040 comp rx=0 tx=0).stop 2023-11-28T21:42:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 -- 172.21.15.12:0/2694358438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8006aaf0 msgr2=0x7f8a80141fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8006aaf0 0x7f8a80141fb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00c3d0 tx=0x7f8a7c00c890 comp rx=0 tx=0).stop 2023-11-28T21:42:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 -- 172.21.15.12:0/2694358438 shutdown_connections 2023-11-28T21:42:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8a640664a0 0x7f8a64068960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a8015d320 0x7f8a8013bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a80154100 0x7f8a801f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 --2- 172.21.15.12:0/2694358438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a8006aaf0 0x7f8a80141fb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 -- 172.21.15.12:0/2694358438 >> 172.21.15.12:0/2694358438 conn(0x7f8a8006c690 msgr2=0x7f8a800cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 -- 172.21.15.12:0/2694358438 shutdown_connections 2023-11-28T21:42:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:39.787+0000 7f8a87b0f640 1 -- 172.21.15.12:0/2694358438 wait complete. 2023-11-28T21:42:39.813 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 33, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 33, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5568": { "addr": "172.21.15.12:6838/291731973", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 291731973, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 291731973, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5568, "incarnation": 33, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 45, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:42:38.424253+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5568 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:42:41.814 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.963+0000 7fc947c46640 1 Processor -- start 2023-11-28T21:42:41.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.963+0000 7fc947c46640 1 -- start start 2023-11-28T21:42:41.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.967+0000 7fc947c46640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc9401500d0 0x7fc9400c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:41.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.967+0000 7fc947c46640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9400c6ce0 con 0x7fc940156af0 2023-11-28T21:42:41.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.967+0000 7fc947c46640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc940156f60 con 0x7fc94006d7d0 2023-11-28T21:42:41.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.967+0000 7fc947c46640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9400c7170 con 0x7fc9401500d0 2023-11-28T21:42:41.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.967+0000 7fc9459e2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc9401500d0 0x7fc9400c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:41.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.967+0000 7fc9459e2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc9401500d0 0x7fc9400c27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42806/0 (socket says 172.21.15.12:42806) 2023-11-28T21:42:41.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.967+0000 7fc9459e2640 1 -- 172.21.15.12:0/621772400 learned_addr learned my addr 172.21.15.12:0/621772400 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:41.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.967+0000 7fc9459e2640 1 -- 172.21.15.12:0/621772400 >> v1:172.21.15.144:6789/0 conn(0x7fc94006d7d0 legacy=0x7fc9401609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:41.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.967+0000 7fc9459e2640 1 -- 172.21.15.12:0/621772400 >> v1:172.21.15.12:6789/0 conn(0x7fc940156af0 legacy=0x7fc9400c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:41.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.967+0000 7fc9459e2640 1 -- 172.21.15.12:0/621772400 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400c5df0 con 0x7fc9401500d0 2023-11-28T21:42:41.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc9459e2640 1 --2- 172.21.15.12:0/621772400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9401500d0 0x7fc9400c27c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009870 tx=0x7fc93c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42510d0025160c9f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:41.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc9449e0640 1 -- 172.21.15.12:0/621772400 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0aa5f0 con 0x7fc9401500d0 2023-11-28T21:42:41.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc9449e0640 1 -- 172.21.15.12:0/621772400 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c0aa750 con 0x7fc9401500d0 2023-11-28T21:42:41.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc9449e0640 1 -- 172.21.15.12:0/621772400 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0aaae0 con 0x7fc9401500d0 2023-11-28T21:42:41.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc947c46640 1 -- 172.21.15.12:0/621772400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9401500d0 msgr2=0x7fc9400c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:41.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc947c46640 1 --2- 172.21.15.12:0/621772400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9401500d0 0x7fc9400c27c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009870 tx=0x7fc93c01c510 comp rx=0 tx=0).stop 2023-11-28T21:42:41.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc947c46640 1 -- 172.21.15.12:0/621772400 shutdown_connections 2023-11-28T21:42:41.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc947c46640 1 --2- 172.21.15.12:0/621772400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9401500d0 0x7fc9400c27c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:41.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc947c46640 1 -- 172.21.15.12:0/621772400 >> 172.21.15.12:0/621772400 conn(0x7fc940066360 msgr2=0x7fc940066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:41.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc947c46640 1 -- 172.21.15.12:0/621772400 shutdown_connections 2023-11-28T21:42:41.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc947c46640 1 -- 172.21.15.12:0/621772400 wait complete. 2023-11-28T21:42:41.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc947c46640 1 Processor -- start 2023-11-28T21:42:41.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.971+0000 7fc947c46640 1 -- start start 2023-11-28T21:42:41.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc947c46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94006d7d0 0x7fc94015fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc947c46640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9401500d0 0x7fc940071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc947c46640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc940156af0 0x7fc9401591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc947c46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401f34b0 con 0x7fc94006d7d0 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc947c46640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401f3940 con 0x7fc9401500d0 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc947c46640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc940161cd0 con 0x7fc940156af0 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9459e2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94006d7d0 0x7fc94015fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9459e2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94006d7d0 0x7fc94015fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42822/0 (socket says 172.21.15.12:42822) 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9459e2640 1 -- 172.21.15.12:0/1493654062 learned_addr learned my addr 172.21.15.12:0/1493654062 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9461e3640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc940156af0 0x7fc9401591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9451e1640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9401500d0 0x7fc940071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9459e2640 1 -- 172.21.15.12:0/1493654062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc940156af0 msgr2=0x7fc9401591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:41.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9459e2640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc940156af0 0x7fc9401591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:41.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9459e2640 1 -- 172.21.15.12:0/1493654062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9401500d0 msgr2=0x7fc940071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:41.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9459e2640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9401500d0 0x7fc940071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:41.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9459e2640 1 -- 172.21.15.12:0/1493654062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c0094d0 con 0x7fc94006d7d0 2023-11-28T21:42:41.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9451e1640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9401500d0 0x7fc940071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:42:41.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9461e3640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc940156af0 0x7fc9401591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:42:41.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9459e2640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94006d7d0 0x7fc94015fd80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0ac040 tx=0x7fc93c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:41.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9367fc640 1 -- 172.21.15.12:0/1493654062 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0b3800 con 0x7fc94006d7d0 2023-11-28T21:42:41.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9367fc640 1 -- 172.21.15.12:0/1493654062 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c0b3980 con 0x7fc94006d7d0 2023-11-28T21:42:41.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9367fc640 1 -- 172.21.15.12:0/1493654062 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0b3d30 con 0x7fc94006d7d0 2023-11-28T21:42:41.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc947c46640 1 -- 172.21.15.12:0/1493654062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc940161730 con 0x7fc94006d7d0 2023-11-28T21:42:41.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc947c46640 1 -- 172.21.15.12:0/1493654062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9401f2b80 con 0x7fc94006d7d0 2023-11-28T21:42:41.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc947c46640 1 -- 172.21.15.12:0/1493654062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc940074f60 con 0x7fc94006d7d0 2023-11-28T21:42:41.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.975+0000 7fc9367fc640 1 -- 172.21.15.12:0/1493654062 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc93c0b2380 con 0x7fc94006d7d0 2023-11-28T21:42:41.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.979+0000 7fc9367fc640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9240664a0 0x7fc924068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:41.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.979+0000 7fc9367fc640 1 -- 172.21.15.12:0/1493654062 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fc93c0a7070 con 0x7fc94006d7d0 2023-11-28T21:42:41.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.979+0000 7fc9451e1640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9240664a0 0x7fc924068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:41.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.979+0000 7fc9451e1640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9240664a0 0x7fc924068960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc940158f60 tx=0x7fc930009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:41.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:41.983+0000 7fc9367fc640 1 -- 172.21.15.12:0/1493654062 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc93c0eb900 con 0x7fc94006d7d0 2023-11-28T21:42:42.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 -- 172.21.15.12:0/1493654062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9400c7ba0 con 0x7fc94006d7d0 2023-11-28T21:42:42.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc9367fc640 1 -- 172.21.15.12:0/1493654062 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5711 (secure 0 0 0) 0x7fc93c0eef20 con 0x7fc94006d7d0 2023-11-28T21:42:42.179 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:42.179 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:38.424253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:42.179 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2023-11-28T21:42:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 -- 172.21.15.12:0/1493654062 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9240664a0 msgr2=0x7fc924068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9240664a0 0x7fc924068960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc940158f60 tx=0x7fc930009040 comp rx=0 tx=0).stop 2023-11-28T21:42:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 -- 172.21.15.12:0/1493654062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94006d7d0 msgr2=0x7fc94015fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94006d7d0 0x7fc94015fd80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0ac040 tx=0x7fc93c01ad50 comp rx=0 tx=0).stop 2023-11-28T21:42:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 -- 172.21.15.12:0/1493654062 shutdown_connections 2023-11-28T21:42:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc9240664a0 0x7fc924068960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc940156af0 0x7fc9401591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:42.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9401500d0 0x7fc940071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:42.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 --2- 172.21.15.12:0/1493654062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc94006d7d0 0x7fc94015fd80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:42.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 -- 172.21.15.12:0/1493654062 >> 172.21.15.12:0/1493654062 conn(0x7fc940066360 msgr2=0x7fc94014d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:42.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 -- 172.21.15.12:0/1493654062 shutdown_connections 2023-11-28T21:42:42.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:42.175+0000 7fc947c46640 1 -- 172.21.15.12:0/1493654062 wait complete. 2023-11-28T21:42:42.200 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:42:43.560 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:42:43.566 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:42:44.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:44.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.347+0000 7ff940417640 1 Processor -- start 2023-11-28T21:42:44.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.347+0000 7ff940417640 1 -- start start 2023-11-28T21:42:44.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.347+0000 7ff940417640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff938159d70 0x7ff93815a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:44.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.347+0000 7ff940417640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93815d360 con 0x7ff938153fc0 2023-11-28T21:42:44.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.347+0000 7ff940417640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93806d6d0 con 0x7ff93806fa10 2023-11-28T21:42:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.347+0000 7ff940417640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff938070170 con 0x7ff938159d70 2023-11-28T21:42:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.347+0000 7ff93e1b3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff938153fc0 0x7ff9380c7e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41394/0 (socket says 172.21.15.12:41394) 2023-11-28T21:42:44.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.347+0000 7ff93e1b3640 1 -- 172.21.15.12:0/2182111645 learned_addr learned my addr 172.21.15.12:0/2182111645 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93e9b4640 1 --2- 172.21.15.12:0/2182111645 >> v2:172.21.15.144:3300/0 conn(0x7ff938159d70 0x7ff93815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:44.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911667334 0 0) 0x7ff93815d360 con 0x7ff938153fc0 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff924003530 con 0x7ff938153fc0 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2476785495 0 0) 0x7ff924003530 con 0x7ff938153fc0 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 >> v2:172.21.15.144:3300/0 conn(0x7ff938159d70 msgr2=0x7ff93815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 --2- 172.21.15.12:0/2182111645 >> v2:172.21.15.144:3300/0 conn(0x7ff938159d70 0x7ff93815a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 >> v1:172.21.15.144:6789/0 conn(0x7ff93806fa10 legacy=0x7ff9380ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93815d5b0 con 0x7ff938153fc0 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7ff934001680 con 0x7ff938153fc0 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 >> v1:172.21.15.12:6789/0 conn(0x7ff938153fc0 legacy=0x7ff9380c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff924003a40 0x7ff924003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9240047d0 0x7ff924004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240051e0 0x7ff92400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:44.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938070170 con 0x7ff9240047d0 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92400c110 con 0x7ff9240051e0 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92400c2d0 con 0x7ff924003a40 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d9b2640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9240047d0 0x7ff924004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93e1b3640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff924003a40 0x7ff924003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93e9b4640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240051e0 0x7ff92400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d9b2640 1 -- 172.21.15.12:0/2182111645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff924003a40 msgr2=0x7ff924003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d9b2640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff924003a40 0x7ff924003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d9b2640 1 -- 172.21.15.12:0/2182111645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240051e0 msgr2=0x7ff92400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d9b2640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240051e0 0x7ff92400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93d9b2640 1 -- 172.21.15.12:0/2182111645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff924003530 con 0x7ff9240047d0 2023-11-28T21:42:44.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.351+0000 7ff93e9b4640 1 -- 172.21.15.12:0/2182111645 reap_dead start 2023-11-28T21:42:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff93d9b2640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9240047d0 0x7ff924004c50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff928001780 tx=0x7ff92801c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8740db0fc8da9742 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9280052d0 con 0x7ff9240047d0 2023-11-28T21:42:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9280a7050 con 0x7ff9240047d0 2023-11-28T21:42:44.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff93d1b1640 1 -- 172.21.15.12:0/2182111645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928003600 con 0x7ff9240047d0 2023-11-28T21:42:44.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff940417640 1 -- 172.21.15.12:0/2182111645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9240047d0 msgr2=0x7ff924004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:44.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff940417640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9240047d0 0x7ff924004c50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff928001780 tx=0x7ff92801c200 comp rx=0 tx=0).stop 2023-11-28T21:42:44.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff940417640 1 -- 172.21.15.12:0/2182111645 shutdown_connections 2023-11-28T21:42:44.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff940417640 1 --2- 172.21.15.12:0/2182111645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9240047d0 0x7ff924004c50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff940417640 1 -- 172.21.15.12:0/2182111645 >> 172.21.15.12:0/2182111645 conn(0x7ff938066530 msgr2=0x7ff938066940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:44.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff940417640 1 -- 172.21.15.12:0/2182111645 shutdown_connections 2023-11-28T21:42:44.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff940417640 1 -- 172.21.15.12:0/2182111645 wait complete. 2023-11-28T21:42:44.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff940417640 1 Processor -- start 2023-11-28T21:42:44.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.355+0000 7ff940417640 1 -- start start 2023-11-28T21:42:44.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff940417640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9240047d0 0x7ff93814b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff940417640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff93815af30 0x7ff93814aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff940417640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9380c6890 0x7ff9381f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff940417640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938157e90 con 0x7ff9380c6890 2023-11-28T21:42:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff940417640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93814a5b0 con 0x7ff93815af30 2023-11-28T21:42:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff940417640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380c6310 con 0x7ff9240047d0 2023-11-28T21:42:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93e9b4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9380c6890 0x7ff9381f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93e9b4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9380c6890 0x7ff9381f2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42838/0 (socket says 172.21.15.12:42838) 2023-11-28T21:42:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93e9b4640 1 -- 172.21.15.12:0/540476830 learned_addr learned my addr 172.21.15.12:0/540476830 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:44.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93e1b3640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9240047d0 0x7ff93814b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93d9b2640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff93815af30 0x7ff93814aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93e9b4640 1 -- 172.21.15.12:0/540476830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9240047d0 msgr2=0x7ff93814b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93e9b4640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9240047d0 0x7ff93814b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93e9b4640 1 -- 172.21.15.12:0/540476830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff93815af30 msgr2=0x7ff93814aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93e9b4640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff93815af30 0x7ff93814aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93e9b4640 1 -- 172.21.15.12:0/540476830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938151050 con 0x7ff9380c6890 2023-11-28T21:42:44.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff93e9b4640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9380c6890 0x7ff9381f2c20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff9380c7270 tx=0x7ff9300115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff92effd640 1 -- 172.21.15.12:0/540476830 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000e070 con 0x7ff9380c6890 2023-11-28T21:42:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff92effd640 1 -- 172.21.15.12:0/540476830 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff930007eb0 con 0x7ff9380c6890 2023-11-28T21:42:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff92effd640 1 -- 172.21.15.12:0/540476830 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930011de0 con 0x7ff9380c6890 2023-11-28T21:42:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff940417640 1 -- 172.21.15.12:0/540476830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93815b390 con 0x7ff9380c6890 2023-11-28T21:42:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff940417640 1 -- 172.21.15.12:0/540476830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9381515b0 con 0x7ff9380c6890 2023-11-28T21:42:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff940417640 1 -- 172.21.15.12:0/540476830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93814a760 con 0x7ff9380c6890 2023-11-28T21:42:44.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.359+0000 7ff92effd640 1 -- 172.21.15.12:0/540476830 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff9300039a0 con 0x7ff9380c6890 2023-11-28T21:42:44.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.363+0000 7ff92effd640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff91c066470 0x7ff91c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:44.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.363+0000 7ff92effd640 1 -- 172.21.15.12:0/540476830 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7ff930084c30 con 0x7ff9380c6890 2023-11-28T21:42:44.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.363+0000 7ff93e1b3640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff91c066470 0x7ff91c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:44.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.363+0000 7ff93e1b3640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff91c066470 0x7ff91c068930 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff934007be0 tx=0x7ff934000ee0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:44.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.367+0000 7ff92effd640 1 -- 172.21.15.12:0/540476830 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff93814a760 con 0x7ff9380c6890 2023-11-28T21:42:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.571+0000 7ff940417640 1 -- 172.21.15.12:0/540476830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff93814a760 con 0x7ff9380c6890 2023-11-28T21:42:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff92effd640 1 -- 172.21.15.12:0/540476830 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5711 (secure 0 0 0) 0x7ff93814a760 con 0x7ff9380c6890 2023-11-28T21:42:44.578 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:44.578 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:38.424253+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 33 2023-11-28T21:42:44.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 -- 172.21.15.12:0/540476830 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff91c066470 msgr2=0x7ff91c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:44.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff91c066470 0x7ff91c068930 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff934007be0 tx=0x7ff934000ee0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 -- 172.21.15.12:0/540476830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9380c6890 msgr2=0x7ff9381f2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:44.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9380c6890 0x7ff9381f2c20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff9380c7270 tx=0x7ff9300115a0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 -- 172.21.15.12:0/540476830 shutdown_connections 2023-11-28T21:42:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff91c066470 0x7ff91c068930 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9380c6890 0x7ff9381f2c20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff93815af30 0x7ff93814aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 --2- 172.21.15.12:0/540476830 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9240047d0 0x7ff93814b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 -- 172.21.15.12:0/540476830 >> 172.21.15.12:0/540476830 conn(0x7ff938066530 msgr2=0x7ff9381592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 -- 172.21.15.12:0/540476830 shutdown_connections 2023-11-28T21:42:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:44.575+0000 7ff940417640 1 -- 172.21.15.12:0/540476830 wait complete. 2023-11-28T21:42:44.602 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:42:46.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:46.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64bdd8640 1 Processor -- start 2023-11-28T21:42:46.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64bdd8640 1 -- start start 2023-11-28T21:42:46.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64bdd8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa644150c20 0x7fa644156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:46.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64bdd8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64406e300 0x7fa6440c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:46.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64bdd8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa64406cf40 0x7fa6440c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:46.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64bdd8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa644067410 con 0x7fa64406cf40 2023-11-28T21:42:46.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64bdd8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64406e090 con 0x7fa644150c20 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64bdd8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa644157f30 con 0x7fa64406e300 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64a375640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa64406cf40 0x7fa6440c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64a375640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa64406cf40 0x7fa6440c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53108/0 (socket says 172.21.15.12:53108) 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64a375640 1 -- 172.21.15.12:0/3172803180 learned_addr learned my addr 172.21.15.12:0/3172803180 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa649373640 1 --2- 172.21.15.12:0/3172803180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64406e300 0x7fa6440c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa649b74640 1 --2- 172.21.15.12:0/3172803180 >> v2:172.21.15.144:3300/0 conn(0x7fa644150c20 0x7fa644156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64a375640 1 -- 172.21.15.12:0/3172803180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64406e300 msgr2=0x7fa6440c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64a375640 1 --2- 172.21.15.12:0/3172803180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64406e300 0x7fa6440c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64a375640 1 -- 172.21.15.12:0/3172803180 >> v2:172.21.15.144:3300/0 conn(0x7fa644150c20 msgr2=0x7fa644156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64a375640 1 --2- 172.21.15.12:0/3172803180 >> v2:172.21.15.144:3300/0 conn(0x7fa644150c20 0x7fa644156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.755+0000 7fa64a375640 1 -- 172.21.15.12:0/3172803180 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644151000 con 0x7fa64406cf40 2023-11-28T21:42:46.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa64a375640 1 --2- 172.21.15.12:0/3172803180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 0x7fa6440c7310 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa64406c6a0 tx=0x7fa63c01ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98ae60915e0db4b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:46.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa648b72640 1 -- 172.21.15.12:0/3172803180 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0a7960 con 0x7fa64406cf40 2023-11-28T21:42:46.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa648b72640 1 -- 172.21.15.12:0/3172803180 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c0a7ac0 con 0x7fa64406cf40 2023-11-28T21:42:46.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa648b72640 1 -- 172.21.15.12:0/3172803180 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0a7e50 con 0x7fa64406cf40 2023-11-28T21:42:46.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa64bdd8640 1 -- 172.21.15.12:0/3172803180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 msgr2=0x7fa6440c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:46.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa64bdd8640 1 --2- 172.21.15.12:0/3172803180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 0x7fa6440c7310 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa64406c6a0 tx=0x7fa63c01ebe0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa64bdd8640 1 -- 172.21.15.12:0/3172803180 shutdown_connections 2023-11-28T21:42:46.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa64bdd8640 1 --2- 172.21.15.12:0/3172803180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 0x7fa6440c7310 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa64bdd8640 1 --2- 172.21.15.12:0/3172803180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64406e300 0x7fa6440c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa64bdd8640 1 --2- 172.21.15.12:0/3172803180 >> v2:172.21.15.144:3300/0 conn(0x7fa644150c20 0x7fa644156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa64bdd8640 1 -- 172.21.15.12:0/3172803180 >> 172.21.15.12:0/3172803180 conn(0x7fa644066360 msgr2=0x7fa644066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:46.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa64bdd8640 1 -- 172.21.15.12:0/3172803180 shutdown_connections 2023-11-28T21:42:46.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.759+0000 7fa64bdd8640 1 -- 172.21.15.12:0/3172803180 wait complete. 2023-11-28T21:42:46.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64bdd8640 1 Processor -- start 2023-11-28T21:42:46.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64bdd8640 1 -- start start 2023-11-28T21:42:46.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64bdd8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 0x7fa644155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:46.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64bdd8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa64406e300 0x7fa6441f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64bdd8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa644150c20 0x7fa64414eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64bdd8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644151520 con 0x7fa64406cf40 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64bdd8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6441f6880 con 0x7fa64406e300 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64bdd8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa644151320 con 0x7fa644150c20 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa649b74640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 0x7fa644155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa649b74640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 0x7fa644155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53114/0 (socket says 172.21.15.12:53114) 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa649b74640 1 -- 172.21.15.12:0/993949422 learned_addr learned my addr 172.21.15.12:0/993949422 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64a375640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa644150c20 0x7fa64414eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa649373640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa64406e300 0x7fa6441f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa649b74640 1 -- 172.21.15.12:0/993949422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa644150c20 msgr2=0x7fa64414eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:46.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa649b74640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa644150c20 0x7fa64414eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa649b74640 1 -- 172.21.15.12:0/993949422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa64406e300 msgr2=0x7fa6441f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:46.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa649b74640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa64406e300 0x7fa6441f28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa649b74640 1 -- 172.21.15.12:0/993949422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c009b30 con 0x7fa64406cf40 2023-11-28T21:42:46.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa649b74640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 0x7fa644155f50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa64000e8a0 tx=0x7fa64000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:46.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa63a7fc640 1 -- 172.21.15.12:0/993949422 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640009510 con 0x7fa64406cf40 2023-11-28T21:42:46.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa63a7fc640 1 -- 172.21.15.12:0/993949422 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64000c940 con 0x7fa64406cf40 2023-11-28T21:42:46.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64bdd8640 1 -- 172.21.15.12:0/993949422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6441f5f50 con 0x7fa64406cf40 2023-11-28T21:42:46.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa63a7fc640 1 -- 172.21.15.12:0/993949422 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64000ccb0 con 0x7fa64406cf40 2023-11-28T21:42:46.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.763+0000 7fa64bdd8640 1 -- 172.21.15.12:0/993949422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6441f6510 con 0x7fa64406cf40 2023-11-28T21:42:46.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.767+0000 7fa64bdd8640 1 -- 172.21.15.12:0/993949422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa644074f60 con 0x7fa64406cf40 2023-11-28T21:42:46.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.767+0000 7fa63a7fc640 1 -- 172.21.15.12:0/993949422 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa640013470 con 0x7fa64406cf40 2023-11-28T21:42:46.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.767+0000 7fa63a7fc640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa628066470 0x7fa628068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:46.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.767+0000 7fa63a7fc640 1 -- 172.21.15.12:0/993949422 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fa640084870 con 0x7fa64406cf40 2023-11-28T21:42:46.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.767+0000 7fa649373640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa628066470 0x7fa628068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:46.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.767+0000 7fa649373640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa628066470 0x7fa628068930 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa6441f6fd0 tx=0x7fa634008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:46.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.771+0000 7fa63a7fc640 1 -- 172.21.15.12:0/993949422 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa640051790 con 0x7fa64406cf40 2023-11-28T21:42:46.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.959+0000 7fa64bdd8640 1 -- 172.21.15.12:0/993949422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa64406bae0 con 0x7fa64406cf40 2023-11-28T21:42:46.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.959+0000 7fa63a7fc640 1 -- 172.21.15.12:0/993949422 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fa640054db0 con 0x7fa64406cf40 2023-11-28T21:42:46.990 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:46.990 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:45.734917+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:46.992 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-11-28T21:42:46.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 -- 172.21.15.12:0/993949422 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa628066470 msgr2=0x7fa628068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:46.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa628066470 0x7fa628068930 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa6441f6fd0 tx=0x7fa634008040 comp rx=0 tx=0).stop 2023-11-28T21:42:46.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 -- 172.21.15.12:0/993949422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 msgr2=0x7fa644155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:46.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 0x7fa644155f50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa64000e8a0 tx=0x7fa64000eca0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 -- 172.21.15.12:0/993949422 shutdown_connections 2023-11-28T21:42:46.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa628066470 0x7fa628068930 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa644150c20 0x7fa64414eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa64406e300 0x7fa6441f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 --2- 172.21.15.12:0/993949422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64406cf40 0x7fa644155f50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:46.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 -- 172.21.15.12:0/993949422 >> 172.21.15.12:0/993949422 conn(0x7fa644066360 msgr2=0x7fa64415a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:46.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 -- 172.21.15.12:0/993949422 shutdown_connections 2023-11-28T21:42:46.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:46.963+0000 7fa64bdd8640 1 -- 172.21.15.12:0/993949422 wait complete. 2023-11-28T21:42:46.997 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:42:48.998 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:49.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.147+0000 7f6a5c396640 1 Processor -- start 2023-11-28T21:42:49.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.147+0000 7f6a5c396640 1 -- start start 2023-11-28T21:42:49.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5c396640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a54150c20 0x7f6a54156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:49.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5c396640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6a5406e300 0x7f6a540c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:49.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5c396640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a54067410 con 0x7f6a5406cf40 2023-11-28T21:42:49.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5c396640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5406e090 con 0x7f6a5406e300 2023-11-28T21:42:49.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5c396640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a54157f30 con 0x7f6a54150c20 2023-11-28T21:42:49.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5a933640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6a5406cf40 0x7f6a54158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:46270/0 (socket says 172.21.15.12:46270) 2023-11-28T21:42:49.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5a933640 1 -- 172.21.15.12:0/628117135 learned_addr learned my addr 172.21.15.12:0/628117135 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:49.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59931640 1 --2- 172.21.15.12:0/628117135 >> v2:172.21.15.144:3300/0 conn(0x7f6a5406e300 0x7f6a540c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:49.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5a132640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a54150c20 0x7f6a54156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:49.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396506901 0 0) 0x7f6a54067410 con 0x7f6a5406cf40 2023-11-28T21:42:49.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a40003530 con 0x7f6a5406cf40 2023-11-28T21:42:49.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1663730010 0 0) 0x7f6a40003530 con 0x7f6a5406cf40 2023-11-28T21:42:49.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a54150c20 msgr2=0x7f6a54156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:49.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a54150c20 0x7f6a54156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 >> v2:172.21.15.144:3300/0 conn(0x7f6a5406e300 msgr2=0x7f6a540c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:49.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 --2- 172.21.15.12:0/628117135 >> v2:172.21.15.144:3300/0 conn(0x7f6a5406e300 0x7f6a540c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54151000 con 0x7f6a5406cf40 2023-11-28T21:42:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f6a4c001a00 con 0x7f6a5406cf40 2023-11-28T21:42:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 >> v1:172.21.15.12:6789/0 conn(0x7f6a5406cf40 legacy=0x7f6a54158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a54150c20 0x7f6a400039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a40003fc0 0x7f6a400047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:49.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a40004d10 0x7f6a40005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5406e090 con 0x7f6a40003fc0 2023-11-28T21:42:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a54157f30 con 0x7f6a40004d10 2023-11-28T21:42:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4000dc20 con 0x7f6a54150c20 2023-11-28T21:42:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59931640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a40003fc0 0x7f6a400047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5a132640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a54150c20 0x7f6a400039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:49.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5a933640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a40004d10 0x7f6a40005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59931640 1 -- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a54150c20 msgr2=0x7f6a400039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59931640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a54150c20 0x7f6a400039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59931640 1 -- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a40004d10 msgr2=0x7f6a40005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59931640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a40004d10 0x7f6a40005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a59931640 1 -- 172.21.15.12:0/628117135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a40003530 con 0x7f6a40003fc0 2023-11-28T21:42:49.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.151+0000 7f6a5a933640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a40004d10 0x7f6a40005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:42:49.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a59931640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a40003fc0 0x7f6a400047d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009ae0 tx=0x7f6a4401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df91889c19026b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:49.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a440a64b0 con 0x7f6a40003fc0 2023-11-28T21:42:49.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a440a6610 con 0x7f6a40003fc0 2023-11-28T21:42:49.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a59130640 1 -- 172.21.15.12:0/628117135 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a440a69c0 con 0x7f6a40003fc0 2023-11-28T21:42:49.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a5c396640 1 -- 172.21.15.12:0/628117135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a40003fc0 msgr2=0x7f6a400047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:49.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a5c396640 1 --2- 172.21.15.12:0/628117135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a40003fc0 0x7f6a400047d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009ae0 tx=0x7f6a4401c8b0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a5a933640 1 -- 172.21.15.12:0/628117135 reap_dead start 2023-11-28T21:42:49.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a5c396640 1 -- 172.21.15.12:0/628117135 shutdown_connections 2023-11-28T21:42:49.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a5c396640 1 -- 172.21.15.12:0/628117135 >> 172.21.15.12:0/628117135 conn(0x7f6a54066360 msgr2=0x7f6a54066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:49.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a5c396640 1 -- 172.21.15.12:0/628117135 shutdown_connections 2023-11-28T21:42:49.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.155+0000 7f6a5c396640 1 -- 172.21.15.12:0/628117135 wait complete. 2023-11-28T21:42:49.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5c396640 1 Processor -- start 2023-11-28T21:42:49.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5c396640 1 -- start start 2023-11-28T21:42:49.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5c396640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a40003fc0 0x7f6a54071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:49.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5c396640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a54151240 0x7f6a5414fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:49.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5c396640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a540c2090 0x7f6a54150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:49.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5c396640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a541f4310 con 0x7f6a540c2090 2023-11-28T21:42:49.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5c396640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a541f0cb0 con 0x7f6a54151240 2023-11-28T21:42:49.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5c396640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a541f33b0 con 0x7f6a40003fc0 2023-11-28T21:42:49.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5a933640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a540c2090 0x7f6a54150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:49.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a59931640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a54151240 0x7f6a5414fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:49.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5a933640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a540c2090 0x7f6a54150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53130/0 (socket says 172.21.15.12:53130) 2023-11-28T21:42:49.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5a132640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a40003fc0 0x7f6a54071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:49.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5a933640 1 -- 172.21.15.12:0/885387884 learned_addr learned my addr 172.21.15.12:0/885387884 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:49.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5a933640 1 -- 172.21.15.12:0/885387884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a40003fc0 msgr2=0x7f6a54071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5a933640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a40003fc0 0x7f6a54071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5a933640 1 -- 172.21.15.12:0/885387884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a54151240 msgr2=0x7f6a5414fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5a933640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a54151240 0x7f6a5414fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5a933640 1 -- 172.21.15.12:0/885387884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a541f3690 con 0x7f6a540c2090 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5a933640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a540c2090 0x7f6a54150180 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6a540c79a0 tx=0x7f6a4c010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a4affd640 1 -- 172.21.15.12:0/885387884 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00e8b0 con 0x7f6a540c2090 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a4affd640 1 -- 172.21.15.12:0/885387884 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c00ea50 con 0x7f6a540c2090 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a4affd640 1 -- 172.21.15.12:0/885387884 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00ee00 con 0x7f6a540c2090 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5c396640 1 -- 172.21.15.12:0/885387884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a541f3970 con 0x7f6a540c2090 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.159+0000 7f6a5c396640 1 -- 172.21.15.12:0/885387884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a541f3f30 con 0x7f6a540c2090 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.163+0000 7f6a5c396640 1 -- 172.21.15.12:0/885387884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a541f40f0 con 0x7f6a540c2090 2023-11-28T21:42:49.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.163+0000 7f6a4affd640 1 -- 172.21.15.12:0/885387884 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6a4c015400 con 0x7f6a540c2090 2023-11-28T21:42:49.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.163+0000 7f6a4affd640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6a38066470 0x7f6a38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:49.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.163+0000 7f6a4affd640 1 -- 172.21.15.12:0/885387884 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f6a4c086d80 con 0x7f6a540c2090 2023-11-28T21:42:49.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.163+0000 7f6a5a132640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6a38066470 0x7f6a38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:49.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.163+0000 7f6a5a132640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6a38066470 0x7f6a38068930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009b00 tx=0x7f6a50009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:49.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.167+0000 7f6a4affd640 1 -- 172.21.15.12:0/885387884 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6a4c053d50 con 0x7f6a540c2090 2023-11-28T21:42:49.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 -- 172.21.15.12:0/885387884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a54074f60 con 0x7f6a540c2090 2023-11-28T21:42:49.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a4affd640 1 -- 172.21.15.12:0/885387884 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f6a4c01a070 con 0x7f6a540c2090 2023-11-28T21:42:49.363 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:49.363 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:45.734917+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:49.363 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 -- 172.21.15.12:0/885387884 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6a38066470 msgr2=0x7f6a38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6a38066470 0x7f6a38068930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009b00 tx=0x7f6a50009190 comp rx=0 tx=0).stop 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 -- 172.21.15.12:0/885387884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a540c2090 msgr2=0x7f6a54150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a540c2090 0x7f6a54150180 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6a540c79a0 tx=0x7f6a4c010c40 comp rx=0 tx=0).stop 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 -- 172.21.15.12:0/885387884 shutdown_connections 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6a38066470 0x7f6a38068930 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6a540c2090 0x7f6a54150180 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a54151240 0x7f6a5414fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 --2- 172.21.15.12:0/885387884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a40003fc0 0x7f6a54071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 -- 172.21.15.12:0/885387884 >> 172.21.15.12:0/885387884 conn(0x7f6a54066360 msgr2=0x7f6a541559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:49.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 -- 172.21.15.12:0/885387884 shutdown_connections 2023-11-28T21:42:49.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:49.359+0000 7f6a5c396640 1 -- 172.21.15.12:0/885387884 wait complete. 2023-11-28T21:42:49.384 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:42:51.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:51.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.535+0000 7fc1bd2bb640 1 Processor -- start 2023-11-28T21:42:51.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.535+0000 7fc1bd2bb640 1 -- start start 2023-11-28T21:42:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1bd2bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b806e300 0x7fc1b8156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1bd2bb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc1b806cf40 0x7fc1b8158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1bd2bb640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b8067410 con 0x7fc1b8150c20 2023-11-28T21:42:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1bd2bb640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b806e090 con 0x7fc1b806cf40 2023-11-28T21:42:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1bd2bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8157f30 con 0x7fc1b806e300 2023-11-28T21:42:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b6ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc1b8150c20 0x7fc1b80c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:46274/0 (socket says 172.21.15.12:46274) 2023-11-28T21:42:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b6ffd640 1 -- 172.21.15.12:0/4136857438 learned_addr learned my addr 172.21.15.12:0/4136857438 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1affff640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b806e300 0x7fc1b8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b77fe640 1 --2- 172.21.15.12:0/4136857438 >> v2:172.21.15.144:3300/0 conn(0x7fc1b806cf40 0x7fc1b8158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408000252 0 0) 0x7fc1b8067410 con 0x7fc1b8150c20 2023-11-28T21:42:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a4003530 con 0x7fc1b8150c20 2023-11-28T21:42:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 768456462 0 0) 0x7fc1a4003530 con 0x7fc1b8150c20 2023-11-28T21:42:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b806e300 msgr2=0x7fc1b8156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b806e300 0x7fc1b8156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 >> v2:172.21.15.144:3300/0 conn(0x7fc1b806cf40 msgr2=0x7fc1b8158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 --2- 172.21.15.12:0/4136857438 >> v2:172.21.15.144:3300/0 conn(0x7fc1b806cf40 0x7fc1b8158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b8151000 con 0x7fc1b8150c20 2023-11-28T21:42:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fc1a8001680 con 0x7fc1b8150c20 2023-11-28T21:42:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 >> v1:172.21.15.12:6789/0 conn(0x7fc1b8150c20 legacy=0x7fc1b80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1a40039f0 0x7fc1a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1a40047d0 0x7fc1a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b806e300 0x7fc1a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b806e090 con 0x7fc1a40047d0 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8157f30 con 0x7fc1a40039f0 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a400b990 con 0x7fc1b806e300 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1affff640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1a40047d0 0x7fc1a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b6ffd640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1a40039f0 0x7fc1a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b77fe640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b806e300 0x7fc1a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1affff640 1 -- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b806e300 msgr2=0x7fc1a4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1affff640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1b806e300 0x7fc1a4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1affff640 1 -- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1a40039f0 msgr2=0x7fc1a4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:51.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1affff640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1a40039f0 0x7fc1a4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1affff640 1 -- 172.21.15.12:0/4136857438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a4003530 con 0x7fc1a40047d0 2023-11-28T21:42:51.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.539+0000 7fc1b6ffd640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1a40039f0 0x7fc1a4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:51.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1affff640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1a40047d0 0x7fc1a4004c50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009b10 tx=0x7fc1a001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146fcc061db86259 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:51.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00a5560 con 0x7fc1a40047d0 2023-11-28T21:42:51.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a00a56c0 con 0x7fc1a40047d0 2023-11-28T21:42:51.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1b67fc640 1 -- 172.21.15.12:0/4136857438 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00a5a70 con 0x7fc1a40047d0 2023-11-28T21:42:51.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/4136857438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1a40047d0 msgr2=0x7fc1a4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:51.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1bd2bb640 1 --2- 172.21.15.12:0/4136857438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1a40047d0 0x7fc1a4004c50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009b10 tx=0x7fc1a001ca50 comp rx=0 tx=0).stop 2023-11-28T21:42:51.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1b77fe640 1 -- 172.21.15.12:0/4136857438 reap_dead start 2023-11-28T21:42:51.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/4136857438 shutdown_connections 2023-11-28T21:42:51.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/4136857438 >> 172.21.15.12:0/4136857438 conn(0x7fc1b8066360 msgr2=0x7fc1b8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:51.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/4136857438 shutdown_connections 2023-11-28T21:42:51.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/4136857438 wait complete. 2023-11-28T21:42:51.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1bd2bb640 1 Processor -- start 2023-11-28T21:42:51.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.543+0000 7fc1bd2bb640 1 -- start start 2023-11-28T21:42:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1bd2bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1a40047d0 0x7fc1b81f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1bd2bb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b80c4300 0x7fc1b80ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1bd2bb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b8152e50 0x7fc1b8148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1bd2bb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b8153550 con 0x7fc1b8152e50 2023-11-28T21:42:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1bd2bb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b81f7a60 con 0x7fc1b80c4300 2023-11-28T21:42:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1bd2bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b81539b0 con 0x7fc1a40047d0 2023-11-28T21:42:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b8152e50 0x7fc1b8148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1b6ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1a40047d0 0x7fc1b81f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b8152e50 0x7fc1b8148a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53152/0 (socket says 172.21.15.12:53152) 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1b77fe640 1 -- 172.21.15.12:0/628114065 learned_addr learned my addr 172.21.15.12:0/628114065 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1affff640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b80c4300 0x7fc1b80ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1b77fe640 1 -- 172.21.15.12:0/628114065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1a40047d0 msgr2=0x7fc1b81f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1b77fe640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1a40047d0 0x7fc1b81f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1b77fe640 1 -- 172.21.15.12:0/628114065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b80c4300 msgr2=0x7fc1b80ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1b77fe640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b80c4300 0x7fc1b80ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1b77fe640 1 -- 172.21.15.12:0/628114065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b8153c90 con 0x7fc1b8152e50 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1b77fe640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b8152e50 0x7fc1b8148a60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8157a00 tx=0x7fc1b00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1af7fe640 1 -- 172.21.15.12:0/628114065 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b000e070 con 0x7fc1b8152e50 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1af7fe640 1 -- 172.21.15.12:0/628114065 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b0011ed0 con 0x7fc1b8152e50 2023-11-28T21:42:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1af7fe640 1 -- 172.21.15.12:0/628114065 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00039a0 con 0x7fc1b8152e50 2023-11-28T21:42:51.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/628114065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b80c2ef0 con 0x7fc1b8152e50 2023-11-28T21:42:51.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/628114065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b80c34b0 con 0x7fc1b8152e50 2023-11-28T21:42:51.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.547+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/628114065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b8074f60 con 0x7fc1b8152e50 2023-11-28T21:42:51.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.551+0000 7fc1af7fe640 1 -- 172.21.15.12:0/628114065 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc1b0003b20 con 0x7fc1b8152e50 2023-11-28T21:42:51.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.551+0000 7fc1af7fe640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1980664a0 0x7fc198068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:51.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.551+0000 7fc1af7fe640 1 -- 172.21.15.12:0/628114065 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fc1b00849b0 con 0x7fc1b8152e50 2023-11-28T21:42:51.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.551+0000 7fc1b6ffd640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1980664a0 0x7fc198068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:51.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.551+0000 7fc1b6ffd640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1980664a0 0x7fc198068960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8001000 tx=0x7fc1a8001a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:51.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.555+0000 7fc1af7fe640 1 -- 172.21.15.12:0/628114065 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc1b0051960 con 0x7fc1b8152e50 2023-11-28T21:42:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.751+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/628114065 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1b806bae0 con 0x7fc1b8152e50 2023-11-28T21:42:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.751+0000 7fc1af7fe640 1 -- 172.21.15.12:0/628114065 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fc1b0054f80 con 0x7fc1b8152e50 2023-11-28T21:42:51.756 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:51.757 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:45.734917+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 34 2023-11-28T21:42:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.751+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/628114065 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1980664a0 msgr2=0x7fc198068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.751+0000 7fc1bd2bb640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1980664a0 0x7fc198068960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8001000 tx=0x7fc1a8001a80 comp rx=0 tx=0).stop 2023-11-28T21:42:51.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.751+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/628114065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b8152e50 msgr2=0x7fc1b8148a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.751+0000 7fc1bd2bb640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b8152e50 0x7fc1b8148a60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8157a00 tx=0x7fc1b00115a0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.755+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/628114065 shutdown_connections 2023-11-28T21:42:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.755+0000 7fc1bd2bb640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc1980664a0 0x7fc198068960 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.755+0000 7fc1bd2bb640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1b8152e50 0x7fc1b8148a60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.755+0000 7fc1bd2bb640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1b80c4300 0x7fc1b80ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.755+0000 7fc1bd2bb640 1 --2- 172.21.15.12:0/628114065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1a40047d0 0x7fc1b81f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.755+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/628114065 >> 172.21.15.12:0/628114065 conn(0x7fc1b8066360 msgr2=0x7fc1b8154d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.755+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/628114065 shutdown_connections 2023-11-28T21:42:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:51.755+0000 7fc1bd2bb640 1 -- 172.21.15.12:0/628114065 wait complete. 2023-11-28T21:42:51.779 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 34, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 34, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5568": { "addr": "172.21.15.12:6838/291731973", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 291731973, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 291731973, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5568, "incarnation": 33, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 21 } }, "last_failure": 0, "last_failure_osd_epoch": 45, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:42:45.734917+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5568 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:42:53.780 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd53965640 1 Processor -- start 2023-11-28T21:42:53.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd53965640 1 -- start start 2023-11-28T21:42:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd53965640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdd4c15d320 0x7fdd4c06a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd53965640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 0x7fdd4c0c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd53965640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4c15e270 con 0x7fdd4c154100 2023-11-28T21:42:53.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd53965640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c0c4e60 con 0x7fdd4c15d320 2023-11-28T21:42:53.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd53965640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c15d740 con 0x7fdd4c06aaf0 2023-11-28T21:42:53.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd51f02640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 0x7fdd4c0c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:53.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd50f00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdd4c15d320 0x7fdd4c06a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:53.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd51f02640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 0x7fdd4c0c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:58932/0 (socket says 172.21.15.12:58932) 2023-11-28T21:42:53.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd51f02640 1 -- 172.21.15.12:0/1883090666 learned_addr learned my addr 172.21.15.12:0/1883090666 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:53.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd50f00640 1 -- 172.21.15.12:0/1883090666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 msgr2=0x7fdd4c0c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:53.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd50f00640 1 --2- 172.21.15.12:0/1883090666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 0x7fdd4c0c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:53.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd50f00640 1 -- 172.21.15.12:0/1883090666 >> v1:172.21.15.144:6789/0 conn(0x7fdd4c154100 legacy=0x7fdd4c0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:53.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.931+0000 7fdd50f00640 1 -- 172.21.15.12:0/1883090666 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c1544e0 con 0x7fdd4c15d320 2023-11-28T21:42:53.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd50f00640 1 --2- 172.21.15.12:0/1883090666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd4c15d320 0x7fdd4c06a5b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0097c0 tx=0x7fdd3c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db0a9a73be38a17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd43fff640 1 -- 172.21.15.12:0/1883090666 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0ad660 con 0x7fdd4c15d320 2023-11-28T21:42:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd43fff640 1 -- 172.21.15.12:0/1883090666 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c0ad7c0 con 0x7fdd4c15d320 2023-11-28T21:42:53.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd43fff640 1 -- 172.21.15.12:0/1883090666 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0adb50 con 0x7fdd4c15d320 2023-11-28T21:42:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd53965640 1 -- 172.21.15.12:0/1883090666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd4c15d320 msgr2=0x7fdd4c06a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd53965640 1 --2- 172.21.15.12:0/1883090666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd4c15d320 0x7fdd4c06a5b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0097c0 tx=0x7fdd3c01c450 comp rx=0 tx=0).stop 2023-11-28T21:42:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd53965640 1 -- 172.21.15.12:0/1883090666 shutdown_connections 2023-11-28T21:42:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd53965640 1 --2- 172.21.15.12:0/1883090666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 0x7fdd4c0c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd53965640 1 --2- 172.21.15.12:0/1883090666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd4c15d320 0x7fdd4c06a5b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd53965640 1 -- 172.21.15.12:0/1883090666 >> 172.21.15.12:0/1883090666 conn(0x7fdd4c06c690 msgr2=0x7fdd4c066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd53965640 1 -- 172.21.15.12:0/1883090666 shutdown_connections 2023-11-28T21:42:53.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.935+0000 7fdd53965640 1 -- 172.21.15.12:0/1883090666 wait complete. 2023-11-28T21:42:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd53965640 1 Processor -- start 2023-11-28T21:42:53.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd53965640 1 -- start start 2023-11-28T21:42:53.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd53965640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 0x7fdd4c146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:53.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd53965640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd4c154100 0x7fdd4c145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:53.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd53965640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd4c15d320 0x7fdd4c15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:53.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd53965640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c149bc0 con 0x7fdd4c154100 2023-11-28T21:42:53.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd53965640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c15b3a0 con 0x7fdd4c15d320 2023-11-28T21:42:53.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd53965640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c145a00 con 0x7fdd4c06aaf0 2023-11-28T21:42:53.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd50f00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd4c154100 0x7fdd4c145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:53.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd50f00640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd4c154100 0x7fdd4c145e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53166/0 (socket says 172.21.15.12:53166) 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd50f00640 1 -- 172.21.15.12:0/3971419860 learned_addr learned my addr 172.21.15.12:0/3971419860 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd51701640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 0x7fdd4c146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd51f02640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd4c15d320 0x7fdd4c15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd50f00640 1 -- 172.21.15.12:0/3971419860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 msgr2=0x7fdd4c146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd50f00640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 0x7fdd4c146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd50f00640 1 -- 172.21.15.12:0/3971419860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd4c15d320 msgr2=0x7fdd4c15ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd50f00640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd4c15d320 0x7fdd4c15ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd50f00640 1 -- 172.21.15.12:0/3971419860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c009420 con 0x7fdd4c154100 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd51701640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 0x7fdd4c146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd50f00640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd4c154100 0x7fdd4c145e90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c000c10 tx=0x7fdd3c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd41ffb640 1 -- 172.21.15.12:0/3971419860 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0b6600 con 0x7fdd4c154100 2023-11-28T21:42:53.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd41ffb640 1 -- 172.21.15.12:0/3971419860 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c0b6780 con 0x7fdd4c154100 2023-11-28T21:42:53.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd41ffb640 1 -- 172.21.15.12:0/3971419860 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0b6b30 con 0x7fdd4c154100 2023-11-28T21:42:53.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.939+0000 7fdd53965640 1 -- 172.21.15.12:0/3971419860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c15cb00 con 0x7fdd4c154100 2023-11-28T21:42:53.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.943+0000 7fdd53965640 1 -- 172.21.15.12:0/3971419860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c15d080 con 0x7fdd4c154100 2023-11-28T21:42:53.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.943+0000 7fdd53965640 1 -- 172.21.15.12:0/3971419860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c067c40 con 0x7fdd4c154100 2023-11-28T21:42:53.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.943+0000 7fdd41ffb640 1 -- 172.21.15.12:0/3971419860 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdd3c0b4040 con 0x7fdd4c154100 2023-11-28T21:42:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.943+0000 7fdd41ffb640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdd300664a0 0x7fdd30068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.943+0000 7fdd41ffb640 1 -- 172.21.15.12:0/3971419860 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fdd3c11d9b0 con 0x7fdd4c154100 2023-11-28T21:42:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.943+0000 7fdd51701640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdd300664a0 0x7fdd30068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:53.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.947+0000 7fdd51701640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdd300664a0 0x7fdd30068960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd48001670 tx=0x7fdd48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:53.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:53.947+0000 7fdd41ffb640 1 -- 172.21.15.12:0/3971419860 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdd3c0ea960 con 0x7fdd4c154100 2023-11-28T21:42:54.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.139+0000 7fdd53965640 1 -- 172.21.15.12:0/3971419860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd4c0c4820 con 0x7fdd4c154100 2023-11-28T21:42:54.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd41ffb640 1 -- 172.21.15.12:0/3971419860 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5712 (secure 0 0 0) 0x7fdd4c0c4820 con 0x7fdd4c154100 2023-11-28T21:42:54.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:54.146 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:51.678300+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:54.146 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 35 2023-11-28T21:42:54.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 -- 172.21.15.12:0/3971419860 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdd300664a0 msgr2=0x7fdd30068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:54.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdd300664a0 0x7fdd30068960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd48001670 tx=0x7fdd48009040 comp rx=0 tx=0).stop 2023-11-28T21:42:54.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 -- 172.21.15.12:0/3971419860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd4c154100 msgr2=0x7fdd4c145e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:54.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd4c154100 0x7fdd4c145e90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c000c10 tx=0x7fdd3c0191f0 comp rx=0 tx=0).stop 2023-11-28T21:42:54.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 -- 172.21.15.12:0/3971419860 shutdown_connections 2023-11-28T21:42:54.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdd300664a0 0x7fdd30068960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:54.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd4c15d320 0x7fdd4c15ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:54.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd4c154100 0x7fdd4c145e90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:54.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 --2- 172.21.15.12:0/3971419860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd4c06aaf0 0x7fdd4c146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:54.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 -- 172.21.15.12:0/3971419860 >> 172.21.15.12:0/3971419860 conn(0x7fdd4c06c690 msgr2=0x7fdd4c1442c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:54.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 -- 172.21.15.12:0/3971419860 shutdown_connections 2023-11-28T21:42:54.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:54.143+0000 7fdd53965640 1 -- 172.21.15.12:0/3971419860 wait complete. 2023-11-28T21:42:54.167 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:42:56.169 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:56.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.315+0000 7ff9b2a79640 1 Processor -- start 2023-11-28T21:42:56.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9b2a79640 1 -- start start 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9b2a79640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff9ac150c20 0x7ff9ac156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9b2a79640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff9ac06e300 0x7ff9ac0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9b2a79640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac067410 con 0x7ff9ac06cf40 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9b2a79640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac06e090 con 0x7ff9ac06e300 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9b2a79640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac157f30 con 0x7ff9ac150c20 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9abfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff9ac06e300 0x7ff9ac0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9abfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff9ac06e300 0x7ff9ac0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44024/0 (socket says 172.21.15.12:44024) 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9abfff640 1 -- 172.21.15.12:0/1379094490 learned_addr learned my addr 172.21.15.12:0/1379094490 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9b0815640 1 --2- 172.21.15.12:0/1379094490 >> v2:172.21.15.144:3300/0 conn(0x7ff9ac150c20 0x7ff9ac156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9abfff640 1 -- 172.21.15.12:0/1379094490 >> v2:172.21.15.144:3300/0 conn(0x7ff9ac150c20 msgr2=0x7ff9ac156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:56.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9abfff640 1 --2- 172.21.15.12:0/1379094490 >> v2:172.21.15.144:3300/0 conn(0x7ff9ac150c20 0x7ff9ac156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9abfff640 1 -- 172.21.15.12:0/1379094490 >> v1:172.21.15.144:6789/0 conn(0x7ff9ac06cf40 legacy=0x7ff9ac158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:56.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.319+0000 7ff9abfff640 1 -- 172.21.15.12:0/1379094490 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac151000 con 0x7ff9ac06e300 2023-11-28T21:42:56.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9abfff640 1 --2- 172.21.15.12:0/1379094490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06e300 0x7ff9ac0c6dd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac06c6a0 tx=0x7ff99801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fce121e44d8933f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:56.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9ab7fe640 1 -- 172.21.15.12:0/1379094490 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980ac410 con 0x7ff9ac06e300 2023-11-28T21:42:56.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9ab7fe640 1 -- 172.21.15.12:0/1379094490 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9980ac570 con 0x7ff9ac06e300 2023-11-28T21:42:56.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9ab7fe640 1 -- 172.21.15.12:0/1379094490 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980ac900 con 0x7ff9ac06e300 2023-11-28T21:42:56.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9b2a79640 1 -- 172.21.15.12:0/1379094490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06e300 msgr2=0x7ff9ac0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:56.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9b2a79640 1 --2- 172.21.15.12:0/1379094490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06e300 0x7ff9ac0c6dd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac06c6a0 tx=0x7ff99801c4c0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9b2a79640 1 -- 172.21.15.12:0/1379094490 shutdown_connections 2023-11-28T21:42:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9b2a79640 1 --2- 172.21.15.12:0/1379094490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06e300 0x7ff9ac0c6dd0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9b2a79640 1 --2- 172.21.15.12:0/1379094490 >> v2:172.21.15.144:3300/0 conn(0x7ff9ac150c20 0x7ff9ac156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9b2a79640 1 -- 172.21.15.12:0/1379094490 >> 172.21.15.12:0/1379094490 conn(0x7ff9ac066360 msgr2=0x7ff9ac066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9b2a79640 1 -- 172.21.15.12:0/1379094490 shutdown_connections 2023-11-28T21:42:56.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9b2a79640 1 -- 172.21.15.12:0/1379094490 wait complete. 2023-11-28T21:42:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9b2a79640 1 Processor -- start 2023-11-28T21:42:56.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.323+0000 7ff9b2a79640 1 -- start start 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b2a79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06cf40 0x7ff9ac154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b2a79640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9ac06e300 0x7ff9ac0725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b2a79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9ac150c20 0x7ff9ac0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b2a79640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac1f3fd0 con 0x7ff9ac06cf40 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b2a79640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac1f2fc0 con 0x7ff9ac06e300 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b2a79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac1f3b40 con 0x7ff9ac150c20 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b0815640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06cf40 0x7ff9ac154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b0815640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06cf40 0x7ff9ac154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44038/0 (socket says 172.21.15.12:44038) 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b0815640 1 -- 172.21.15.12:0/2500976156 learned_addr learned my addr 172.21.15.12:0/2500976156 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b1016640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9ac150c20 0x7ff9ac0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9abfff640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9ac06e300 0x7ff9ac0725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:56.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b0815640 1 -- 172.21.15.12:0/2500976156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9ac150c20 msgr2=0x7ff9ac0c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b0815640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9ac150c20 0x7ff9ac0c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b0815640 1 -- 172.21.15.12:0/2500976156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9ac06e300 msgr2=0x7ff9ac0725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b0815640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9ac06e300 0x7ff9ac0725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b0815640 1 -- 172.21.15.12:0/2500976156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9980094d0 con 0x7ff9ac06cf40 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9abfff640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9ac06e300 0x7ff9ac0725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b1016640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9ac150c20 0x7ff9ac0c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b0815640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06cf40 0x7ff9ac154550 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000d620 tx=0x7ff9a000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9a97fa640 1 -- 172.21.15.12:0/2500976156 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000b620 con 0x7ff9ac06cf40 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9a97fa640 1 -- 172.21.15.12:0/2500976156 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a000b780 con 0x7ff9ac06cf40 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9a97fa640 1 -- 172.21.15.12:0/2500976156 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000bb10 con 0x7ff9ac06cf40 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b2a79640 1 -- 172.21.15.12:0/2500976156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac1f6660 con 0x7ff9ac06cf40 2023-11-28T21:42:56.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b2a79640 1 -- 172.21.15.12:0/2500976156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac1f34f0 con 0x7ff9ac06cf40 2023-11-28T21:42:56.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.327+0000 7ff9b2a79640 1 -- 172.21.15.12:0/2500976156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac074f60 con 0x7ff9ac06cf40 2023-11-28T21:42:56.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.331+0000 7ff9a97fa640 1 -- 172.21.15.12:0/2500976156 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff9a000bc70 con 0x7ff9ac06cf40 2023-11-28T21:42:56.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.331+0000 7ff9a97fa640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff9900664a0 0x7ff990068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:56.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.331+0000 7ff9a97fa640 1 -- 172.21.15.12:0/2500976156 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7ff9a00847e0 con 0x7ff9ac06cf40 2023-11-28T21:42:56.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.331+0000 7ff9abfff640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff9900664a0 0x7ff990068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:56.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.331+0000 7ff9abfff640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff9900664a0 0x7ff990068960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff9980ae000 tx=0x7ff998008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:56.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.335+0000 7ff9a97fa640 1 -- 172.21.15.12:0/2500976156 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff9a0051790 con 0x7ff9ac06cf40 2023-11-28T21:42:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 -- 172.21.15.12:0/2500976156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9ac06bae0 con 0x7ff9ac06cf40 2023-11-28T21:42:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9a97fa640 1 -- 172.21.15.12:0/2500976156 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5712 (secure 0 0 0) 0x7ff9a0054db0 con 0x7ff9ac06cf40 2023-11-28T21:42:56.531 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:56.531 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:51.678300+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 35 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 -- 172.21.15.12:0/2500976156 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff9900664a0 msgr2=0x7ff990068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff9900664a0 0x7ff990068960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff9980ae000 tx=0x7ff998008040 comp rx=0 tx=0).stop 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 -- 172.21.15.12:0/2500976156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06cf40 msgr2=0x7ff9ac154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06cf40 0x7ff9ac154550 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000d620 tx=0x7ff9a000dae0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 -- 172.21.15.12:0/2500976156 shutdown_connections 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff9900664a0 0x7ff990068960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9ac150c20 0x7ff9ac0c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9ac06e300 0x7ff9ac0725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 --2- 172.21.15.12:0/2500976156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff9ac06cf40 0x7ff9ac154550 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 -- 172.21.15.12:0/2500976156 >> 172.21.15.12:0/2500976156 conn(0x7ff9ac066360 msgr2=0x7ff9ac0c3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 -- 172.21.15.12:0/2500976156 shutdown_connections 2023-11-28T21:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:56.527+0000 7ff9b2a79640 1 -- 172.21.15.12:0/2500976156 wait complete. 2023-11-28T21:42:56.552 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:42:58.553 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:42:58.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e5f10640 1 Processor -- start 2023-11-28T21:42:58.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e5f10640 1 -- start start 2023-11-28T21:42:58.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e5f10640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e00b41e0 0x7f44e00cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:58.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e5f10640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e015d390 con 0x7f44e0153fb0 2023-11-28T21:42:58.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e5f10640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e015d630 con 0x7f44e015a980 2023-11-28T21:42:58.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e5f10640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e00b2b60 con 0x7f44e00b41e0 2023-11-28T21:42:58.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e4f0e640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f44e0153fb0 0x7f44e00ca4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60822/0 (socket says 172.21.15.12:60822) 2023-11-28T21:42:58.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e4f0e640 1 -- 172.21.15.12:0/2798849957 learned_addr learned my addr 172.21.15.12:0/2798849957 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:42:58.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e570f640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e00b41e0 0x7f44e00cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:58.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113000854 0 0) 0x7f44e015d390 con 0x7f44e0153fb0 2023-11-28T21:42:58.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d0003530 con 0x7f44e0153fb0 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3658394492 0 0) 0x7f44d0003530 con 0x7f44e0153fb0 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e00b41e0 msgr2=0x7f44e00cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e00b41e0 0x7f44e00cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 >> v1:172.21.15.144:6789/0 conn(0x7f44e015a980 legacy=0x7f44e00b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e015d910 con 0x7f44e0153fb0 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f44d4001680 con 0x7f44e0153fb0 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 >> v1:172.21.15.12:6789/0 conn(0x7f44e0153fb0 legacy=0x7f44e00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 0x7f44d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e00b41e0 0x7f44d00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d0004d10 0x7f44d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:58.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e00b2b60 con 0x7f44d00039b0 2023-11-28T21:42:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d000ba40 con 0x7f44d0004d10 2023-11-28T21:42:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d000bc00 con 0x7f44e00b41e0 2023-11-28T21:42:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e4f0e640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 0x7f44d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44dffff640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e00b41e0 0x7f44d00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e570f640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d0004d10 0x7f44d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e4f0e640 1 -- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e00b41e0 msgr2=0x7f44d00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e4f0e640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e00b41e0 0x7f44d00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e4f0e640 1 -- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d0004d10 msgr2=0x7f44d0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:42:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e4f0e640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44d0004d10 0x7f44d0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:58.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.707+0000 7f44e4f0e640 1 -- 172.21.15.12:0/2798849957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d0003530 con 0x7f44d00039b0 2023-11-28T21:42:58.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44e4f0e640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 0x7f44d0003dd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c190 tx=0x7f44d401cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f7af46984adb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:58.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4005ae0 con 0x7f44d00039b0 2023-11-28T21:42:58.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d4005c40 con 0x7f44d00039b0 2023-11-28T21:42:58.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44df7fe640 1 -- 172.21.15.12:0/2798849957 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40a58b0 con 0x7f44d00039b0 2023-11-28T21:42:58.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44e5f10640 1 -- 172.21.15.12:0/2798849957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 msgr2=0x7f44d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:58.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44e5f10640 1 --2- 172.21.15.12:0/2798849957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 0x7f44d0003dd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c190 tx=0x7f44d401cbd0 comp rx=0 tx=0).stop 2023-11-28T21:42:58.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44e570f640 1 -- 172.21.15.12:0/2798849957 reap_dead start 2023-11-28T21:42:58.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44e5f10640 1 -- 172.21.15.12:0/2798849957 shutdown_connections 2023-11-28T21:42:58.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44e5f10640 1 -- 172.21.15.12:0/2798849957 >> 172.21.15.12:0/2798849957 conn(0x7f44e0071ce0 msgr2=0x7f44e00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:58.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44e5f10640 1 -- 172.21.15.12:0/2798849957 shutdown_connections 2023-11-28T21:42:58.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.711+0000 7f44e5f10640 1 -- 172.21.15.12:0/2798849957 wait complete. 2023-11-28T21:42:58.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e5f10640 1 Processor -- start 2023-11-28T21:42:58.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e5f10640 1 -- start start 2023-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e5f10640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 0x7f44e00c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e5f10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e01f36b0 0x7f44e01f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e5f10640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e0152700 0x7f44e015c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e5f10640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e015e1b0 con 0x7f44d00039b0 2023-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e5f10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e00cd970 con 0x7f44e01f36b0 2023-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e5f10640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e01f29c0 con 0x7f44e0152700 2023-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e4f0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 0x7f44e00c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e4f0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 0x7f44e00c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44064/0 (socket says 172.21.15.12:44064) 2023-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e4f0e640 1 -- 172.21.15.12:0/2626117710 learned_addr learned my addr 172.21.15.12:0/2626117710 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:42:58.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44dffff640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e01f36b0 0x7f44e01f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:58.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e570f640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e0152700 0x7f44e015c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:58.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e4f0e640 1 -- 172.21.15.12:0/2626117710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e0152700 msgr2=0x7f44e015c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:58.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e4f0e640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e0152700 0x7f44e015c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:58.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e4f0e640 1 -- 172.21.15.12:0/2626117710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e01f36b0 msgr2=0x7f44e01f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:58.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e4f0e640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e01f36b0 0x7f44e01f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:58.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e4f0e640 1 -- 172.21.15.12:0/2626117710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e01f3220 con 0x7f44d00039b0 2023-11-28T21:42:58.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e4f0e640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 0x7f44e00c8f70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f44d401cc10 tx=0x7f44d0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:58.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44dd7fa640 1 -- 172.21.15.12:0/2626117710 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40b4070 con 0x7f44d00039b0 2023-11-28T21:42:58.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44dd7fa640 1 -- 172.21.15.12:0/2626117710 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d4005db0 con 0x7f44d00039b0 2023-11-28T21:42:58.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44dd7fa640 1 -- 172.21.15.12:0/2626117710 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4001640 con 0x7f44d00039b0 2023-11-28T21:42:58.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e5f10640 1 -- 172.21.15.12:0/2626117710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e0157620 con 0x7f44d00039b0 2023-11-28T21:42:58.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.715+0000 7f44e5f10640 1 -- 172.21.15.12:0/2626117710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e01f2bd0 con 0x7f44d00039b0 2023-11-28T21:42:58.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.719+0000 7f44e5f10640 1 -- 172.21.15.12:0/2626117710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e0074f60 con 0x7f44d00039b0 2023-11-28T21:42:58.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.719+0000 7f44dd7fa640 1 -- 172.21.15.12:0/2626117710 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f44d40a58b0 con 0x7f44d00039b0 2023-11-28T21:42:58.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.719+0000 7f44dd7fa640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44c4066470 0x7f44c4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:42:58.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.719+0000 7f44dffff640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44c4066470 0x7f44c4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:42:58.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.719+0000 7f44dd7fa640 1 -- 172.21.15.12:0/2626117710 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f44d40ee780 con 0x7f44d00039b0 2023-11-28T21:42:58.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.719+0000 7f44dffff640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44c4066470 0x7f44c4068930 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f44e00cabd0 tx=0x7f44cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:42:58.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.727+0000 7f44dd7fa640 1 -- 172.21.15.12:0/2626117710 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f44d40ea3c0 con 0x7f44d00039b0 2023-11-28T21:42:58.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.915+0000 7f44e5f10640 1 -- 172.21.15.12:0/2626117710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44e00cdb80 con 0x7f44d00039b0 2023-11-28T21:42:58.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44dd7fa640 1 -- 172.21.15.12:0/2626117710 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5718 (secure 0 0 0) 0x7f44d40ed9e0 con 0x7f44d00039b0 2023-11-28T21:42:58.922 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:42:58.922 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:57.753166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:clientreplay","state_seq":25,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:42:58.923 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 36 2023-11-28T21:42:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 -- 172.21.15.12:0/2626117710 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44c4066470 msgr2=0x7f44c4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44c4066470 0x7f44c4068930 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f44e00cabd0 tx=0x7f44cc009040 comp rx=0 tx=0).stop 2023-11-28T21:42:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 -- 172.21.15.12:0/2626117710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 msgr2=0x7f44e00c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:42:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 0x7f44e00c8f70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f44d401cc10 tx=0x7f44d0005800 comp rx=0 tx=0).stop 2023-11-28T21:42:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 -- 172.21.15.12:0/2626117710 shutdown_connections 2023-11-28T21:42:58.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44c4066470 0x7f44c4068930 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e0152700 0x7f44e015c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e01f36b0 0x7f44e01f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 --2- 172.21.15.12:0/2626117710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44d00039b0 0x7f44e00c8f70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:42:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 -- 172.21.15.12:0/2626117710 >> 172.21.15.12:0/2626117710 conn(0x7f44e0071ce0 msgr2=0x7f44e0151ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:42:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 -- 172.21.15.12:0/2626117710 shutdown_connections 2023-11-28T21:42:58.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:42:58.919+0000 7f44e5f10640 1 -- 172.21.15.12:0/2626117710 wait complete. 2023-11-28T21:42:58.945 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:43:00.947 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:43:01.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.095+0000 7ff304679640 1 Processor -- start 2023-11-28T21:43:01.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.095+0000 7ff304679640 1 -- start start 2023-11-28T21:43:01.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff304679640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff2fc151af0 0x7ff2fc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:01.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff304679640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc06e300 0x7ff2fc072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:01.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff304679640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2fc157b70 con 0x7ff2fc151ed0 2023-11-28T21:43:01.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff304679640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2fc0707f0 con 0x7ff2fc151af0 2023-11-28T21:43:01.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff304679640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0c7b90 con 0x7ff2fc06e300 2023-11-28T21:43:01.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff302415640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff2fc151af0 0x7ff2fc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:01.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff302415640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff2fc151af0 0x7ff2fc0c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44066/0 (socket says 172.21.15.12:44066) 2023-11-28T21:43:01.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff302415640 1 -- 172.21.15.12:0/2645015645 learned_addr learned my addr 172.21.15.12:0/2645015645 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:01.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff302c16640 1 --2- 172.21.15.12:0/2645015645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc06e300 0x7ff2fc072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:01.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff302415640 1 -- 172.21.15.12:0/2645015645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc06e300 msgr2=0x7ff2fc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:43:01.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff302415640 1 --2- 172.21.15.12:0/2645015645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc06e300 0x7ff2fc072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:01.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff302415640 1 -- 172.21.15.12:0/2645015645 >> v1:172.21.15.12:6789/0 conn(0x7ff2fc151ed0 legacy=0x7ff2fc0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:01.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.099+0000 7ff302415640 1 -- 172.21.15.12:0/2645015645 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc157e50 con 0x7ff2fc151af0 2023-11-28T21:43:01.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff302415640 1 --2- 172.21.15.12:0/2645015645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 0x7ff2fc0c7650 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009870 tx=0x7ff2f801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35eac75539df19d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:01.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff301413640 1 -- 172.21.15.12:0/2645015645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f80ab600 con 0x7ff2fc151af0 2023-11-28T21:43:01.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff301413640 1 -- 172.21.15.12:0/2645015645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f80a7050 con 0x7ff2fc151af0 2023-11-28T21:43:01.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff301413640 1 -- 172.21.15.12:0/2645015645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f80ab980 con 0x7ff2fc151af0 2023-11-28T21:43:01.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff304679640 1 -- 172.21.15.12:0/2645015645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 msgr2=0x7ff2fc0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:01.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff304679640 1 --2- 172.21.15.12:0/2645015645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 0x7ff2fc0c7650 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009870 tx=0x7ff2f801c540 comp rx=0 tx=0).stop 2023-11-28T21:43:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff304679640 1 -- 172.21.15.12:0/2645015645 shutdown_connections 2023-11-28T21:43:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff304679640 1 --2- 172.21.15.12:0/2645015645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc06e300 0x7ff2fc072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff304679640 1 --2- 172.21.15.12:0/2645015645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 0x7ff2fc0c7650 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff304679640 1 -- 172.21.15.12:0/2645015645 >> 172.21.15.12:0/2645015645 conn(0x7ff2fc066360 msgr2=0x7ff2fc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:01.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff304679640 1 -- 172.21.15.12:0/2645015645 shutdown_connections 2023-11-28T21:43:01.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff304679640 1 -- 172.21.15.12:0/2645015645 wait complete. 2023-11-28T21:43:01.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff304679640 1 Processor -- start 2023-11-28T21:43:01.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.103+0000 7ff304679640 1 -- start start 2023-11-28T21:43:01.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff304679640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2fc06e300 0x7ff2fc0c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:01.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff304679640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 0x7ff2fc1f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:01.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff304679640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc13f700 0x7ff2fc13e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:01.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff304679640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0c7030 con 0x7ff2fc151af0 2023-11-28T21:43:01.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff304679640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0c9160 con 0x7ff2fc06e300 2023-11-28T21:43:01.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff304679640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc0c9440 con 0x7ff2fc13f700 2023-11-28T21:43:01.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff301c14640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 0x7ff2fc1f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:01.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff301c14640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 0x7ff2fc1f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44078/0 (socket says 172.21.15.12:44078) 2023-11-28T21:43:01.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff301c14640 1 -- 172.21.15.12:0/4225277059 learned_addr learned my addr 172.21.15.12:0/4225277059 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:01.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff302415640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2fc06e300 0x7ff2fc0c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:01.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff302c16640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc13f700 0x7ff2fc13e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:01.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff301c14640 1 -- 172.21.15.12:0/4225277059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc13f700 msgr2=0x7ff2fc13e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:01.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff301c14640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc13f700 0x7ff2fc13e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:01.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff301c14640 1 -- 172.21.15.12:0/4225277059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2fc06e300 msgr2=0x7ff2fc0c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:01.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff301c14640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2fc06e300 0x7ff2fc0c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:01.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff301c14640 1 -- 172.21.15.12:0/4225277059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f80094d0 con 0x7ff2fc151af0 2023-11-28T21:43:01.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff302415640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2fc06e300 0x7ff2fc0c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:43:01.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff302c16640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc13f700 0x7ff2fc13e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:43:01.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff301c14640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 0x7ff2fc1f6d40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c3c0 tx=0x7ff2ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:01.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff2f2ffd640 1 -- 172.21.15.12:0/4225277059 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0052d0 con 0x7ff2fc151af0 2023-11-28T21:43:01.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff2f2ffd640 1 -- 172.21.15.12:0/4225277059 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec005430 con 0x7ff2fc151af0 2023-11-28T21:43:01.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff2f2ffd640 1 -- 172.21.15.12:0/4225277059 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec003840 con 0x7ff2fc151af0 2023-11-28T21:43:01.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff304679640 1 -- 172.21.15.12:0/4225277059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc0c9720 con 0x7ff2fc151af0 2023-11-28T21:43:01.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff304679640 1 -- 172.21.15.12:0/4225277059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc0c6c50 con 0x7ff2fc151af0 2023-11-28T21:43:01.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.107+0000 7ff304679640 1 -- 172.21.15.12:0/4225277059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc0c6e10 con 0x7ff2fc151af0 2023-11-28T21:43:01.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.111+0000 7ff2f2ffd640 1 -- 172.21.15.12:0/4225277059 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff2ec0039a0 con 0x7ff2fc151af0 2023-11-28T21:43:01.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.111+0000 7ff2f2ffd640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2e0066470 0x7ff2e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:01.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.111+0000 7ff2f2ffd640 1 -- 172.21.15.12:0/4225277059 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7ff2ec083820 con 0x7ff2fc151af0 2023-11-28T21:43:01.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.111+0000 7ff302415640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2e0066470 0x7ff2e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:01.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.111+0000 7ff302415640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2e0066470 0x7ff2e0068930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0c8e60 tx=0x7ff2f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:01.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.115+0000 7ff2f2ffd640 1 -- 172.21.15.12:0/4225277059 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff2ec0507d0 con 0x7ff2fc151af0 2023-11-28T21:43:01.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 -- 172.21.15.12:0/4225277059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2fc074f60 con 0x7ff2fc151af0 2023-11-28T21:43:01.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff2f2ffd640 1 -- 172.21.15.12:0/4225277059 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5718 (secure 0 0 0) 0x7ff2ec053df0 con 0x7ff2fc151af0 2023-11-28T21:43:01.311 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:43:01.311 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:57.753166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:clientreplay","state_seq":25,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:43:01.312 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 36 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 -- 172.21.15.12:0/4225277059 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2e0066470 msgr2=0x7ff2e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2e0066470 0x7ff2e0068930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0c8e60 tx=0x7ff2f8008040 comp rx=0 tx=0).stop 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 -- 172.21.15.12:0/4225277059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 msgr2=0x7ff2fc1f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 0x7ff2fc1f6d40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c3c0 tx=0x7ff2ec00c880 comp rx=0 tx=0).stop 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 -- 172.21.15.12:0/4225277059 shutdown_connections 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff2e0066470 0x7ff2e0068930 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2fc13f700 0x7ff2fc13e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff2fc151af0 0x7ff2fc1f6d40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 --2- 172.21.15.12:0/4225277059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2fc06e300 0x7ff2fc0c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 -- 172.21.15.12:0/4225277059 >> 172.21.15.12:0/4225277059 conn(0x7ff2fc066360 msgr2=0x7ff2fc06faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:01.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 -- 172.21.15.12:0/4225277059 shutdown_connections 2023-11-28T21:43:01.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:01.307+0000 7ff304679640 1 -- 172.21.15.12:0/4225277059 wait complete. 2023-11-28T21:43:01.333 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:43:03.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:43:03.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.487+0000 7fdca046b640 1 Processor -- start 2023-11-28T21:43:03.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.487+0000 7fdca046b640 1 -- start start 2023-11-28T21:43:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdca046b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdc9806e300 0x7fdc98156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdca046b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdc9806cf40 0x7fdc98158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdca046b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc98067410 con 0x7fdc98150c20 2023-11-28T21:43:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdca046b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9806e090 con 0x7fdc9806cf40 2023-11-28T21:43:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdca046b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc98157f30 con 0x7fdc9806e300 2023-11-28T21:43:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9ea08640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdc9806cf40 0x7fdc98158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9ea08640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdc9806cf40 0x7fdc98158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44084/0 (socket says 172.21.15.12:44084) 2023-11-28T21:43:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9ea08640 1 -- 172.21.15.12:0/362375430 learned_addr learned my addr 172.21.15.12:0/362375430 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9da06640 1 --2- 172.21.15.12:0/362375430 >> v2:172.21.15.144:3300/0 conn(0x7fdc9806e300 0x7fdc98156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:03.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9ea08640 1 -- 172.21.15.12:0/362375430 >> v2:172.21.15.144:3300/0 conn(0x7fdc9806e300 msgr2=0x7fdc98156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:43:03.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9ea08640 1 --2- 172.21.15.12:0/362375430 >> v2:172.21.15.144:3300/0 conn(0x7fdc9806e300 0x7fdc98156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:03.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9ea08640 1 -- 172.21.15.12:0/362375430 >> v1:172.21.15.144:6789/0 conn(0x7fdc98150c20 legacy=0x7fdc980c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:43:03.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9ea08640 1 -- 172.21.15.12:0/362375430 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98151000 con 0x7fdc9806cf40 2023-11-28T21:43:03.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9da06640 1 --2- 172.21.15.12:0/362375430 >> v2:172.21.15.144:3300/0 conn(0x7fdc9806e300 0x7fdc98156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:43:03.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9ea08640 1 --2- 172.21.15.12:0/362375430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 0x7fdc98158410 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fdc980c8e60 tx=0x7fdc9001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdd5e4eb540c29b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:03.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.491+0000 7fdc9d205640 1 -- 172.21.15.12:0/362375430 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900a5640 con 0x7fdc9806cf40 2023-11-28T21:43:03.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdc9d205640 1 -- 172.21.15.12:0/362375430 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc900a57a0 con 0x7fdc9806cf40 2023-11-28T21:43:03.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdc9d205640 1 -- 172.21.15.12:0/362375430 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900a5b30 con 0x7fdc9806cf40 2023-11-28T21:43:03.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 -- 172.21.15.12:0/362375430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 msgr2=0x7fdc98158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:03.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 --2- 172.21.15.12:0/362375430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 0x7fdc98158410 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fdc980c8e60 tx=0x7fdc9001c990 comp rx=0 tx=0).stop 2023-11-28T21:43:03.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 -- 172.21.15.12:0/362375430 shutdown_connections 2023-11-28T21:43:03.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 --2- 172.21.15.12:0/362375430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 0x7fdc98158410 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:03.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 --2- 172.21.15.12:0/362375430 >> v2:172.21.15.144:3300/0 conn(0x7fdc9806e300 0x7fdc98156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:03.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 -- 172.21.15.12:0/362375430 >> 172.21.15.12:0/362375430 conn(0x7fdc98066360 msgr2=0x7fdc98066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:03.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 -- 172.21.15.12:0/362375430 shutdown_connections 2023-11-28T21:43:03.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 -- 172.21.15.12:0/362375430 wait complete. 2023-11-28T21:43:03.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 Processor -- start 2023-11-28T21:43:03.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 -- start start 2023-11-28T21:43:03.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 0x7fdc98153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:03.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc9806e300 0x7fdc98156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:03.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98150c20 0x7fdc98154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:03.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc98154410 con 0x7fdc9806cf40 2023-11-28T21:43:03.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981f2b90 con 0x7fdc98150c20 2023-11-28T21:43:03.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdca046b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981f2e50 con 0x7fdc9806e300 2023-11-28T21:43:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdc9e207640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 0x7fdc98153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdc9e207640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 0x7fdc98153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44086/0 (socket says 172.21.15.12:44086) 2023-11-28T21:43:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.495+0000 7fdc9e207640 1 -- 172.21.15.12:0/1589708726 learned_addr learned my addr 172.21.15.12:0/1589708726 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc9ea08640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98150c20 0x7fdc98154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc9da06640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc9806e300 0x7fdc98156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc9e207640 1 -- 172.21.15.12:0/1589708726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc9806e300 msgr2=0x7fdc98156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc9e207640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc9806e300 0x7fdc98156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:03.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc9e207640 1 -- 172.21.15.12:0/1589708726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98150c20 msgr2=0x7fdc98154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:03.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc9e207640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98150c20 0x7fdc98154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:03.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc9e207640 1 -- 172.21.15.12:0/1589708726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc90009b30 con 0x7fdc9806cf40 2023-11-28T21:43:03.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc9da06640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc9806e300 0x7fdc98156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:43:03.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc9e207640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 0x7fdc98153dd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c450 tx=0x7fdc9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc8effd640 1 -- 172.21.15.12:0/1589708726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94012830 con 0x7fdc9806cf40 2023-11-28T21:43:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc8effd640 1 -- 172.21.15.12:0/1589708726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc940129b0 con 0x7fdc9806cf40 2023-11-28T21:43:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc8effd640 1 -- 172.21.15.12:0/1589708726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94012d60 con 0x7fdc9806cf40 2023-11-28T21:43:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdca046b640 1 -- 172.21.15.12:0/1589708726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc981f3130 con 0x7fdc9806cf40 2023-11-28T21:43:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdca046b640 1 -- 172.21.15.12:0/1589708726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc981f36f0 con 0x7fdc9806cf40 2023-11-28T21:43:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdca046b640 1 -- 172.21.15.12:0/1589708726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc981f3910 con 0x7fdc9806cf40 2023-11-28T21:43:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc8effd640 1 -- 172.21.15.12:0/1589708726 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdc9401c460 con 0x7fdc9806cf40 2023-11-28T21:43:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc8effd640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc7c066470 0x7fdc7c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.499+0000 7fdc8effd640 1 -- 172.21.15.12:0/1589708726 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fdc94084220 con 0x7fdc9806cf40 2023-11-28T21:43:03.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.503+0000 7fdc9da06640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc7c066470 0x7fdc7c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.503+0000 7fdc9da06640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc7c066470 0x7fdc7c068930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdc9814f810 tx=0x7fdc88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:03.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.503+0000 7fdc8effd640 1 -- 172.21.15.12:0/1589708726 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdc94051140 con 0x7fdc9806cf40 2023-11-28T21:43:03.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.695+0000 7fdca046b640 1 -- 172.21.15.12:0/1589708726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc98074f60 con 0x7fdc9806cf40 2023-11-28T21:43:03.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.695+0000 7fdc8effd640 1 -- 172.21.15.12:0/1589708726 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5718 (secure 0 0 0) 0x7fdc94054760 con 0x7fdc9806cf40 2023-11-28T21:43:03.700 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:43:03.700 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:42:57.753166+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:clientreplay","state_seq":25,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:43:03.700 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 36 2023-11-28T21:43:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.695+0000 7fdca046b640 1 -- 172.21.15.12:0/1589708726 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc7c066470 msgr2=0x7fdc7c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.695+0000 7fdca046b640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc7c066470 0x7fdc7c068930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdc9814f810 tx=0x7fdc88008040 comp rx=0 tx=0).stop 2023-11-28T21:43:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.699+0000 7fdca046b640 1 -- 172.21.15.12:0/1589708726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 msgr2=0x7fdc98153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.699+0000 7fdca046b640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 0x7fdc98153dd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c450 tx=0x7fdc9400c910 comp rx=0 tx=0).stop 2023-11-28T21:43:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.699+0000 7fdca046b640 1 -- 172.21.15.12:0/1589708726 shutdown_connections 2023-11-28T21:43:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.699+0000 7fdca046b640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdc7c066470 0x7fdc7c068930 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:03.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.699+0000 7fdca046b640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98150c20 0x7fdc98154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.699+0000 7fdca046b640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc9806e300 0x7fdc98156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.699+0000 7fdca046b640 1 --2- 172.21.15.12:0/1589708726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc9806cf40 0x7fdc98153dd0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.699+0000 7fdca046b640 1 -- 172.21.15.12:0/1589708726 >> 172.21.15.12:0/1589708726 conn(0x7fdc98066360 msgr2=0x7fdc9806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.699+0000 7fdca046b640 1 -- 172.21.15.12:0/1589708726 shutdown_connections 2023-11-28T21:43:03.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:03.699+0000 7fdca046b640 1 -- 172.21.15.12:0/1589708726 wait complete. 2023-11-28T21:43:03.722 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 36, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 36, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5568": { "addr": "172.21.15.12:6838/291731973", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 291731973, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 291731973, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5568, "incarnation": 33, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:clientreplay", "state_seq": 25 } }, "last_failure": 0, "last_failure_osd_epoch": 45, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:42:57.753166+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5568 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:43:05.723 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:43:05.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c6d47b640 1 Processor -- start 2023-11-28T21:43:05.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c6d47b640 1 -- start start 2023-11-28T21:43:05.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c6d47b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5c680c6f20 0x7f5c68156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:05.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c6d47b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c7300 0x7f5c68150840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:05.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c6d47b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6806b7c0 con 0x7f5c680c26b0 2023-11-28T21:43:05.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c6d47b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6806a8f0 con 0x7f5c680c6f20 2023-11-28T21:43:05.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c6d47b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6806abb0 con 0x7f5c680c7300 2023-11-28T21:43:05.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c67fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5c680c6f20 0x7f5c68156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:05.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c67fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5c680c6f20 0x7f5c68156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52484/0 (socket says 172.21.15.12:52484) 2023-11-28T21:43:05.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c67fff640 1 -- 172.21.15.12:0/3423589738 learned_addr learned my addr 172.21.15.12:0/3423589738 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:05.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c6cc7a640 1 --2- 172.21.15.12:0/3423589738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c7300 0x7f5c68150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:05.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c67fff640 1 -- 172.21.15.12:0/3423589738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c7300 msgr2=0x7f5c68150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:43:05.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c67fff640 1 --2- 172.21.15.12:0/3423589738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c7300 0x7f5c68150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:05.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c67fff640 1 -- 172.21.15.12:0/3423589738 >> v1:172.21.15.144:6789/0 conn(0x7f5c680c26b0 legacy=0x7f5c68158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:05.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c67fff640 1 -- 172.21.15.12:0/3423589738 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6806ae90 con 0x7f5c680c6f20 2023-11-28T21:43:05.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.875+0000 7f5c6cc7a640 1 --2- 172.21.15.12:0/3423589738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c7300 0x7f5c68150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:43:05.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c67fff640 1 --2- 172.21.15.12:0/3423589738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 0x7f5c68156ac0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009870 tx=0x7f5c5801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1151662fa90eb0b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:05.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c66ffd640 1 -- 172.21.15.12:0/3423589738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c580a9400 con 0x7f5c680c6f20 2023-11-28T21:43:05.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c66ffd640 1 -- 172.21.15.12:0/3423589738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c580a9560 con 0x7f5c680c6f20 2023-11-28T21:43:05.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c66ffd640 1 -- 172.21.15.12:0/3423589738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c580a98f0 con 0x7f5c680c6f20 2023-11-28T21:43:05.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c6d47b640 1 -- 172.21.15.12:0/3423589738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 msgr2=0x7f5c68156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:05.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c6d47b640 1 --2- 172.21.15.12:0/3423589738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 0x7f5c68156ac0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009870 tx=0x7f5c5801c540 comp rx=0 tx=0).stop 2023-11-28T21:43:05.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c6d47b640 1 -- 172.21.15.12:0/3423589738 shutdown_connections 2023-11-28T21:43:05.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c6d47b640 1 --2- 172.21.15.12:0/3423589738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c7300 0x7f5c68150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:05.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c6d47b640 1 --2- 172.21.15.12:0/3423589738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 0x7f5c68156ac0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:05.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c6d47b640 1 -- 172.21.15.12:0/3423589738 >> 172.21.15.12:0/3423589738 conn(0x7f5c68066360 msgr2=0x7f5c68066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:05.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c6d47b640 1 -- 172.21.15.12:0/3423589738 shutdown_connections 2023-11-28T21:43:05.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.879+0000 7f5c6d47b640 1 -- 172.21.15.12:0/3423589738 wait complete. 2023-11-28T21:43:05.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c6d47b640 1 Processor -- start 2023-11-28T21:43:05.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c6d47b640 1 -- start start 2023-11-28T21:43:05.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c6d47b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c26b0 0x7f5c68155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c6d47b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 0x7f5c681f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c6d47b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c680c7300 0x7f5c681f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c6d47b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68156d50 con 0x7f5c680c6f20 2023-11-28T21:43:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c6d47b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680c4be0 con 0x7f5c680c7300 2023-11-28T21:43:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c6d47b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6814b110 con 0x7f5c680c26b0 2023-11-28T21:43:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 0x7f5c681f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 0x7f5c681f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52488/0 (socket says 172.21.15.12:52488) 2023-11-28T21:43:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c677fe640 1 -- 172.21.15.12:0/2051736458 learned_addr learned my addr 172.21.15.12:0/2051736458 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c6cc7a640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c680c7300 0x7f5c681f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:05.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c67fff640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c26b0 0x7f5c68155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:05.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c677fe640 1 -- 172.21.15.12:0/2051736458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c26b0 msgr2=0x7f5c68155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:05.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c677fe640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c26b0 0x7f5c68155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:05.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c677fe640 1 -- 172.21.15.12:0/2051736458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c680c7300 msgr2=0x7f5c681f0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:05.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c677fe640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c680c7300 0x7f5c681f0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:05.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c677fe640 1 -- 172.21.15.12:0/2051736458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580094d0 con 0x7f5c680c6f20 2023-11-28T21:43:05.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c677fe640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 0x7f5c681f3640 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00c500 tx=0x7f5c5c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:05.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c64ff9640 1 -- 172.21.15.12:0/2051736458 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c005220 con 0x7f5c680c6f20 2023-11-28T21:43:05.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c64ff9640 1 -- 172.21.15.12:0/2051736458 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c005380 con 0x7f5c680c6f20 2023-11-28T21:43:05.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c64ff9640 1 -- 172.21.15.12:0/2051736458 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c007270 con 0x7f5c680c6f20 2023-11-28T21:43:05.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.883+0000 7f5c6d47b640 1 -- 172.21.15.12:0/2051736458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c681f31b0 con 0x7f5c680c6f20 2023-11-28T21:43:05.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.887+0000 7f5c6d47b640 1 -- 172.21.15.12:0/2051736458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c680c4df0 con 0x7f5c680c6f20 2023-11-28T21:43:05.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.887+0000 7f5c6d47b640 1 -- 172.21.15.12:0/2051736458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c68067c70 con 0x7f5c680c6f20 2023-11-28T21:43:05.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.887+0000 7f5c64ff9640 1 -- 172.21.15.12:0/2051736458 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5c5c0073d0 con 0x7f5c680c6f20 2023-11-28T21:43:05.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.887+0000 7f5c64ff9640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c54066470 0x7f5c54068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:05.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.887+0000 7f5c64ff9640 1 -- 172.21.15.12:0/2051736458 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f5c5c083bd0 con 0x7f5c680c6f20 2023-11-28T21:43:05.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.887+0000 7f5c67fff640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c54066470 0x7f5c54068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:05.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.887+0000 7f5c67fff640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c54066470 0x7f5c54068930 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5c580ae000 tx=0x7f5c580b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:05.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:05.891+0000 7f5c64ff9640 1 -- 172.21.15.12:0/2051736458 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c5c050bb0 con 0x7f5c680c6f20 2023-11-28T21:43:06.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 -- 172.21.15.12:0/2051736458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c681f3340 con 0x7f5c680c6f20 2023-11-28T21:43:06.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c64ff9640 1 -- 172.21.15.12:0/2051736458 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5712 (secure 0 0 0) 0x7f5c5c0541d0 con 0x7f5c680c6f20 2023-11-28T21:43:06.087 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:43:06.088 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:43:04.550912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":27,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:43:06.088 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 37 2023-11-28T21:43:06.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 -- 172.21.15.12:0/2051736458 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c54066470 msgr2=0x7f5c54068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:06.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c54066470 0x7f5c54068930 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5c580ae000 tx=0x7f5c580b3040 comp rx=0 tx=0).stop 2023-11-28T21:43:06.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 -- 172.21.15.12:0/2051736458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 msgr2=0x7f5c681f3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:06.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 0x7f5c681f3640 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00c500 tx=0x7f5c5c00c9c0 comp rx=0 tx=0).stop 2023-11-28T21:43:06.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 -- 172.21.15.12:0/2051736458 shutdown_connections 2023-11-28T21:43:06.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c54066470 0x7f5c54068930 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:06.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c680c7300 0x7f5c681f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:06.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c680c6f20 0x7f5c681f3640 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:06.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 --2- 172.21.15.12:0/2051736458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c680c26b0 0x7f5c68155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:06.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 -- 172.21.15.12:0/2051736458 >> 172.21.15.12:0/2051736458 conn(0x7f5c68066360 msgr2=0x7f5c680ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:06.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 -- 172.21.15.12:0/2051736458 shutdown_connections 2023-11-28T21:43:06.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:06.083+0000 7f5c6d47b640 1 -- 172.21.15.12:0/2051736458 wait complete. 2023-11-28T21:43:06.109 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=5232 2023-11-28T21:43:06.109 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.e 2023-11-28T21:43:10.109 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-11-28T21:43:10.110 INFO:tasks.ceph.mds.e:Restarting daemon 2023-11-28T21:43:10.110 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-11-28T21:43:10.114 INFO:tasks.ceph.mds.e:Started 2023-11-28T21:43:10.114 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:43:10.203 INFO:tasks.ceph.mds.e.smithi012.stdout:starting mds.e at 2023-11-28T21:43:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe359ebb640 1 Processor -- start 2023-11-28T21:43:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe359ebb640 1 -- start start 2023-11-28T21:43:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe359ebb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe35406e300 0x7fe3540c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe359ebb640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe354157b70 con 0x7fe354151af0 2023-11-28T21:43:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe359ebb640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe354157e50 con 0x7fe354151ed0 2023-11-28T21:43:10.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe359ebb640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3540707f0 con 0x7fe35406e300 2023-11-28T21:43:10.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3537fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe354151af0 0x7fe3540c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:54888/0 (socket says 172.21.15.12:54888) 2023-11-28T21:43:10.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3537fe640 1 -- 172.21.15.12:0/1923309742 learned_addr learned my addr 172.21.15.12:0/1923309742 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:43:10.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe353fff640 1 --2- 172.21.15.12:0/1923309742 >> v2:172.21.15.144:3300/0 conn(0x7fe35406e300 0x7fe3540c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:10.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269415467 0 0) 0x7fe354157b70 con 0x7fe354151af0 2023-11-28T21:43:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe33c003530 con 0x7fe354151af0 2023-11-28T21:43:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 849034525 0 0) 0x7fe33c003530 con 0x7fe354151af0 2023-11-28T21:43:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 >> v2:172.21.15.144:3300/0 conn(0x7fe35406e300 msgr2=0x7fe3540c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:43:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 --2- 172.21.15.12:0/1923309742 >> v2:172.21.15.144:3300/0 conn(0x7fe35406e300 0x7fe3540c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 >> v1:172.21.15.144:6789/0 conn(0x7fe354151ed0 legacy=0x7fe354072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:10.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe354158130 con 0x7fe354151af0 2023-11-28T21:43:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fe344001680 con 0x7fe354151af0 2023-11-28T21:43:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 >> v1:172.21.15.12:6789/0 conn(0x7fe354151af0 legacy=0x7fe3540c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe33c0039f0 0x7fe33c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe33c0047d0 0x7fe33c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe33c005190 0x7fe33c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3540707f0 con 0x7fe33c005190 2023-11-28T21:43:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c00c0c0 con 0x7fe33c0047d0 2023-11-28T21:43:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c00c280 con 0x7fe33c0039f0 2023-11-28T21:43:10.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe353fff640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe33c005190 0x7fe33c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:10.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe3537fe640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe33c0039f0 0x7fe33c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:10.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe352ffd640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe33c0047d0 0x7fe33c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:10.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe353fff640 1 -- 172.21.15.12:0/1923309742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe33c0039f0 msgr2=0x7fe33c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:43:10.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe353fff640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe33c0039f0 0x7fe33c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe353fff640 1 -- 172.21.15.12:0/1923309742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe33c0047d0 msgr2=0x7fe33c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:43:10.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe353fff640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe33c0047d0 0x7fe33c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.255+0000 7fe353fff640 1 -- 172.21.15.12:0/1923309742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c003530 con 0x7fe33c005190 2023-11-28T21:43:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe353fff640 1 -- 172.21.15.12:0/1923309742 reap_dead start 2023-11-28T21:43:10.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe353fff640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe33c005190 0x7fe33c00b990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009f90 tx=0x7fe34c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8e11abea1ff3e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0a5c70 con 0x7fe33c005190 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c00f040 con 0x7fe33c005190 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe3527fc640 1 -- 172.21.15.12:0/1923309742 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0039a0 con 0x7fe33c005190 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe359ebb640 1 -- 172.21.15.12:0/1923309742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe33c005190 msgr2=0x7fe33c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe359ebb640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe33c005190 0x7fe33c00b990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009f90 tx=0x7fe34c01cc60 comp rx=0 tx=0).stop 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe359ebb640 1 -- 172.21.15.12:0/1923309742 shutdown_connections 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe359ebb640 1 --2- 172.21.15.12:0/1923309742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe33c005190 0x7fe33c00b990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe359ebb640 1 -- 172.21.15.12:0/1923309742 >> 172.21.15.12:0/1923309742 conn(0x7fe354066360 msgr2=0x7fe354066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe359ebb640 1 -- 172.21.15.12:0/1923309742 shutdown_connections 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.259+0000 7fe359ebb640 1 -- 172.21.15.12:0/1923309742 wait complete. 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe359ebb640 1 Processor -- start 2023-11-28T21:43:10.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe359ebb640 1 -- start start 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe359ebb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe33c005190 0x7fe35414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe359ebb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe35414f300 0x7fe354131d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe359ebb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe354134250 0x7fe3541346d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe359ebb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe354150510 con 0x7fe35414f300 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe359ebb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3540c6f90 con 0x7fe354134250 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe359ebb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35413f6c0 con 0x7fe33c005190 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe352ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe35414f300 0x7fe354131d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe352ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe35414f300 0x7fe354131d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52534/0 (socket says 172.21.15.12:52534) 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe352ffd640 1 -- 172.21.15.12:0/156363276 learned_addr learned my addr 172.21.15.12:0/156363276 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe3537fe640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe33c005190 0x7fe35414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe353fff640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe354134250 0x7fe3541346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:10.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe352ffd640 1 -- 172.21.15.12:0/156363276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe33c005190 msgr2=0x7fe35414ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe352ffd640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe33c005190 0x7fe35414ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe352ffd640 1 -- 172.21.15.12:0/156363276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe354134250 msgr2=0x7fe3541346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe352ffd640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe354134250 0x7fe3541346d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe352ffd640 1 -- 172.21.15.12:0/156363276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35413f980 con 0x7fe35414f300 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe353fff640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe354134250 0x7fe3541346d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe352ffd640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe35414f300 0x7fe354131d60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe348001ec0 tx=0x7fe34800e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe33bfff640 1 -- 172.21.15.12:0/156363276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3480052e0 con 0x7fe35414f300 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe33bfff640 1 -- 172.21.15.12:0/156363276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe348005440 con 0x7fe35414f300 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe33bfff640 1 -- 172.21.15.12:0/156363276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348003840 con 0x7fe35414f300 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe359ebb640 1 -- 172.21.15.12:0/156363276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe354150710 con 0x7fe35414f300 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.263+0000 7fe359ebb640 1 -- 172.21.15.12:0/156363276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe354150cd0 con 0x7fe35414f300 2023-11-28T21:43:10.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.267+0000 7fe359ebb640 1 -- 172.21.15.12:0/156363276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe354074f60 con 0x7fe35414f300 2023-11-28T21:43:10.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.267+0000 7fe33bfff640 1 -- 172.21.15.12:0/156363276 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe3480039a0 con 0x7fe35414f300 2023-11-28T21:43:10.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.267+0000 7fe33bfff640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3400664c0 0x7fe340068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:10.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.267+0000 7fe33bfff640 1 -- 172.21.15.12:0/156363276 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fe348083a60 con 0x7fe35414f300 2023-11-28T21:43:10.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.267+0000 7fe3537fe640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3400664c0 0x7fe340068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:10.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.267+0000 7fe3537fe640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3400664c0 0x7fe340068980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe3440018f0 tx=0x7fe344007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:10.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.271+0000 7fe33bfff640 1 -- 172.21.15.12:0/156363276 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe348050b40 con 0x7fe35414f300 2023-11-28T21:43:10.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.471+0000 7fe359ebb640 1 -- 172.21.15.12:0/156363276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe35406a010 con 0x7fe35414f300 2023-11-28T21:43:10.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.471+0000 7fe33bfff640 1 -- 172.21.15.12:0/156363276 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6503 (secure 0 0 0) 0x7fe348019090 con 0x7fe35414f300 2023-11-28T21:43:10.476 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:43:10.476 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5727,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:43:04.550912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":27,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:43:10.476 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2023-11-28T21:43:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.471+0000 7fe359ebb640 1 -- 172.21.15.12:0/156363276 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3400664c0 msgr2=0x7fe340068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.471+0000 7fe359ebb640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3400664c0 0x7fe340068980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe3440018f0 tx=0x7fe344007040 comp rx=0 tx=0).stop 2023-11-28T21:43:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.471+0000 7fe359ebb640 1 -- 172.21.15.12:0/156363276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe35414f300 msgr2=0x7fe354131d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.471+0000 7fe359ebb640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe35414f300 0x7fe354131d60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe348001ec0 tx=0x7fe34800e8a0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.475+0000 7fe359ebb640 1 -- 172.21.15.12:0/156363276 shutdown_connections 2023-11-28T21:43:10.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.475+0000 7fe359ebb640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3400664c0 0x7fe340068980 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.475+0000 7fe359ebb640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe354134250 0x7fe3541346d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.475+0000 7fe359ebb640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe35414f300 0x7fe354131d60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.475+0000 7fe359ebb640 1 --2- 172.21.15.12:0/156363276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe33c005190 0x7fe35414ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:10.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.475+0000 7fe359ebb640 1 -- 172.21.15.12:0/156363276 >> 172.21.15.12:0/156363276 conn(0x7fe354066360 msgr2=0x7fe35406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:10.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.475+0000 7fe359ebb640 1 -- 172.21.15.12:0/156363276 shutdown_connections 2023-11-28T21:43:10.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:10.475+0000 7fe359ebb640 1 -- 172.21.15.12:0/156363276 wait complete. 2023-11-28T21:43:10.497 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-11-28T21:43:10.497 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 45 secs before thrashing 2023-11-28T21:43:12.724 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-11-28T21:43:12.719+0000 7f989f30c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T21:43:13.702 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:43:13.706 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:43:43.920 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:43:43.924 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:43:55.499 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:43:55.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f9655417640 1 Processor -- start 2023-11-28T21:43:55.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f9655417640 1 -- start start 2023-11-28T21:43:55.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f9655417640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9650151af0 0x7f96500c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:55.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f9655417640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f965006e300 0x7f9650072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:55.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f9655417640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9650157b70 con 0x7f9650151ed0 2023-11-28T21:43:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f9655417640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96500707f0 con 0x7f9650151af0 2023-11-28T21:43:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f9655417640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96500c7b90 con 0x7f965006e300 2023-11-28T21:43:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f964effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9650151af0 0x7f96500c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f964effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9650151af0 0x7f96500c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59764/0 (socket says 172.21.15.12:59764) 2023-11-28T21:43:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f964effd640 1 -- 172.21.15.12:0/2862929462 learned_addr learned my addr 172.21.15.12:0/2862929462 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f964f7fe640 1 --2- 172.21.15.12:0/2862929462 >> v2:172.21.15.144:3300/0 conn(0x7f965006e300 0x7f9650072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f964effd640 1 -- 172.21.15.12:0/2862929462 >> v2:172.21.15.144:3300/0 conn(0x7f965006e300 msgr2=0x7f9650072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:43:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f964effd640 1 --2- 172.21.15.12:0/2862929462 >> v2:172.21.15.144:3300/0 conn(0x7f965006e300 0x7f9650072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f964effd640 1 -- 172.21.15.12:0/2862929462 >> v1:172.21.15.144:6789/0 conn(0x7f9650151ed0 legacy=0x7f96500c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:43:55.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.647+0000 7f964effd640 1 -- 172.21.15.12:0/2862929462 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650157e50 con 0x7f9650151af0 2023-11-28T21:43:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f964effd640 1 --2- 172.21.15.12:0/2862929462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9650151af0 0x7f96500c7650 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f96500716f0 tx=0x7f964001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf6d12318610fdb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f964e7fc640 1 -- 172.21.15.12:0/2862929462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96400ab600 con 0x7f9650151af0 2023-11-28T21:43:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f964e7fc640 1 -- 172.21.15.12:0/2862929462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96400a7050 con 0x7f9650151af0 2023-11-28T21:43:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f964e7fc640 1 -- 172.21.15.12:0/2862929462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96400ab980 con 0x7f9650151af0 2023-11-28T21:43:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f9655417640 1 -- 172.21.15.12:0/2862929462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9650151af0 msgr2=0x7f96500c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f9655417640 1 --2- 172.21.15.12:0/2862929462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9650151af0 0x7f96500c7650 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f96500716f0 tx=0x7f964001c4c0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f9655417640 1 -- 172.21.15.12:0/2862929462 shutdown_connections 2023-11-28T21:43:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f9655417640 1 --2- 172.21.15.12:0/2862929462 >> v2:172.21.15.144:3300/0 conn(0x7f965006e300 0x7f9650072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f9655417640 1 --2- 172.21.15.12:0/2862929462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9650151af0 0x7f96500c7650 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f9655417640 1 -- 172.21.15.12:0/2862929462 >> 172.21.15.12:0/2862929462 conn(0x7f9650066360 msgr2=0x7f9650066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f9655417640 1 -- 172.21.15.12:0/2862929462 shutdown_connections 2023-11-28T21:43:55.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.651+0000 7f9655417640 1 -- 172.21.15.12:0/2862929462 wait complete. 2023-11-28T21:43:55.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9655417640 1 Processor -- start 2023-11-28T21:43:55.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9655417640 1 -- start start 2023-11-28T21:43:55.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9655417640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f965006e300 0x7f96501f15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:55.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9655417640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9650151af0 0x7f9650154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9655417640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965014bf00 0x7f96501532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9655417640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965014b080 con 0x7f965014bf00 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9655417640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501f1b30 con 0x7f965006e300 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9655417640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96501522c0 con 0x7f9650151af0 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f964f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965014bf00 0x7f96501532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f964f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965014bf00 0x7f96501532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59772/0 (socket says 172.21.15.12:59772) 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f964f7fe640 1 -- 172.21.15.12:0/621368325 learned_addr learned my addr 172.21.15.12:0/621368325 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f964effd640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f965006e300 0x7f96501f15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9647fff640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9650151af0 0x7f9650154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f964f7fe640 1 -- 172.21.15.12:0/621368325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9650151af0 msgr2=0x7f9650154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f964f7fe640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9650151af0 0x7f9650154c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f964f7fe640 1 -- 172.21.15.12:0/621368325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f965006e300 msgr2=0x7f96501f15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:55.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f964f7fe640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f965006e300 0x7f96501f15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f964f7fe640 1 -- 172.21.15.12:0/621368325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400094d0 con 0x7f965014bf00 2023-11-28T21:43:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f964f7fe640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965014bf00 0x7f96501532d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9648007e30 tx=0x7f964800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f96477fe640 1 -- 172.21.15.12:0/621368325 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648014620 con 0x7f965014bf00 2023-11-28T21:43:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f96477fe640 1 -- 172.21.15.12:0/621368325 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964800ee90 con 0x7f965014bf00 2023-11-28T21:43:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f96477fe640 1 -- 172.21.15.12:0/621368325 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96480149c0 con 0x7f965014bf00 2023-11-28T21:43:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9655417640 1 -- 172.21.15.12:0/621368325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965014c3d0 con 0x7f965014bf00 2023-11-28T21:43:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.655+0000 7f9655417640 1 -- 172.21.15.12:0/621368325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96501527f0 con 0x7f965014bf00 2023-11-28T21:43:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.659+0000 7f9655417640 1 -- 172.21.15.12:0/621368325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9650074f60 con 0x7f965014bf00 2023-11-28T21:43:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.659+0000 7f96477fe640 1 -- 172.21.15.12:0/621368325 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9648014b20 con 0x7f965014bf00 2023-11-28T21:43:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.659+0000 7f96477fe640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9630066470 0x7f9630068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.659+0000 7f96477fe640 1 -- 172.21.15.12:0/621368325 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f9648084d50 con 0x7f965014bf00 2023-11-28T21:43:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.659+0000 7f964effd640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9630066470 0x7f9630068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.659+0000 7f964effd640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9630066470 0x7f9630068930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96400ad000 tx=0x7f96400a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:55.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.663+0000 7f96477fe640 1 -- 172.21.15.12:0/621368325 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9648051c70 con 0x7f965014bf00 2023-11-28T21:43:55.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.855+0000 7f9655417640 1 -- 172.21.15.12:0/621368325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9650072c10 con 0x7f965014bf00 2023-11-28T21:43:55.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.855+0000 7f96477fe640 1 -- 172.21.15.12:0/621368325 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f9648055290 con 0x7f965014bf00 2023-11-28T21:43:55.860 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:43:55.860 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5727,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:43:04.550912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":27,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:43:55.861 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2023-11-28T21:43:55.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.855+0000 7f9655417640 1 -- 172.21.15.12:0/621368325 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9630066470 msgr2=0x7f9630068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:55.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.855+0000 7f9655417640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9630066470 0x7f9630068930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96400ad000 tx=0x7f96400a6000 comp rx=0 tx=0).stop 2023-11-28T21:43:55.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.855+0000 7f9655417640 1 -- 172.21.15.12:0/621368325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965014bf00 msgr2=0x7f96501532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:55.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.855+0000 7f9655417640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965014bf00 0x7f96501532d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9648007e30 tx=0x7f964800e4b0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.859+0000 7f9655417640 1 -- 172.21.15.12:0/621368325 shutdown_connections 2023-11-28T21:43:55.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.859+0000 7f9655417640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9630066470 0x7f9630068930 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.859+0000 7f9655417640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f965014bf00 0x7f96501532d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.859+0000 7f9655417640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9650151af0 0x7f9650154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.859+0000 7f9655417640 1 --2- 172.21.15.12:0/621368325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f965006e300 0x7f96501f15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:55.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.859+0000 7f9655417640 1 -- 172.21.15.12:0/621368325 >> 172.21.15.12:0/621368325 conn(0x7f9650066360 msgr2=0x7f96501501c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:55.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.859+0000 7f9655417640 1 -- 172.21.15.12:0/621368325 shutdown_connections 2023-11-28T21:43:55.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:55.859+0000 7f9655417640 1 -- 172.21.15.12:0/621368325 wait complete. 2023-11-28T21:43:55.882 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2023-11-28T21:43:55.883 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T21:43:55.883 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:43:55.905 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T21:43:55.905 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:43:56.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae92d6640 1 Processor -- start 2023-11-28T21:43:56.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae92d6640 1 -- start start 2023-11-28T21:43:56.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae92d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2ae40cac40 0x7f2ae406b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:56.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae92d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae40c4da0 con 0x7f2ae40cb020 2023-11-28T21:43:56.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae92d6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae415a5e0 con 0x7f2ae40c4980 2023-11-28T21:43:56.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae92d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae415a8c0 con 0x7f2ae40cac40 2023-11-28T21:43:56.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae2ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2ae40cac40 0x7f2ae406b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:56.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae37fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2ae40cb020 0x7f2ae4159d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44040/0 (socket says 172.21.15.12:44040) 2023-11-28T21:43:56.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae2ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2ae40cac40 0x7f2ae406b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59774/0 (socket says 172.21.15.12:59774) 2023-11-28T21:43:56.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae37fe640 1 -- 172.21.15.12:0/1355975888 learned_addr learned my addr 172.21.15.12:0/1355975888 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:43:56.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae2ffd640 1 -- 172.21.15.12:0/1355975888 >> v1:172.21.15.144:6789/0 conn(0x7f2ae40c4980 legacy=0x7f2ae40ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:56.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae2ffd640 1 -- 172.21.15.12:0/1355975888 >> v1:172.21.15.12:6789/0 conn(0x7f2ae40cb020 legacy=0x7f2ae4159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:56.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.095+0000 7f2ae2ffd640 1 -- 172.21.15.12:0/1355975888 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae406a650 con 0x7f2ae40cac40 2023-11-28T21:43:56.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae2ffd640 1 --2- 172.21.15.12:0/1355975888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 0x7f2ae406b130 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40097c0 tx=0x7f2ad401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3754bf6d5c751a35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:56.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae1ffb640 1 -- 172.21.15.12:0/1355975888 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40a8970 con 0x7f2ae40cac40 2023-11-28T21:43:56.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae1ffb640 1 -- 172.21.15.12:0/1355975888 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad40a8ad0 con 0x7f2ae40cac40 2023-11-28T21:43:56.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae1ffb640 1 -- 172.21.15.12:0/1355975888 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40a8e60 con 0x7f2ae40cac40 2023-11-28T21:43:56.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae92d6640 1 -- 172.21.15.12:0/1355975888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 msgr2=0x7f2ae406b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:56.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae92d6640 1 --2- 172.21.15.12:0/1355975888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 0x7f2ae406b130 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40097c0 tx=0x7f2ad401ec70 comp rx=0 tx=0).stop 2023-11-28T21:43:56.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae92d6640 1 -- 172.21.15.12:0/1355975888 shutdown_connections 2023-11-28T21:43:56.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae92d6640 1 --2- 172.21.15.12:0/1355975888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 0x7f2ae406b130 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:56.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae92d6640 1 -- 172.21.15.12:0/1355975888 >> 172.21.15.12:0/1355975888 conn(0x7f2ae40720e0 msgr2=0x7f2ae40724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:56.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae92d6640 1 -- 172.21.15.12:0/1355975888 shutdown_connections 2023-11-28T21:43:56.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.099+0000 7f2ae92d6640 1 -- 172.21.15.12:0/1355975888 wait complete. 2023-11-28T21:43:56.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 Processor -- start 2023-11-28T21:43:56.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 -- start start 2023-11-28T21:43:56.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ae40c4980 0x7f2ae4157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:56.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 0x7f2ae415af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae40cb020 0x7f2ae415b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae415e070 con 0x7f2ae40cac40 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae415d4e0 con 0x7f2ae40cb020 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae41efb80 con 0x7f2ae40c4980 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 0x7f2ae415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 0x7f2ae415af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59786/0 (socket says 172.21.15.12:59786) 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae27fc640 1 -- 172.21.15.12:0/4119017737 learned_addr learned my addr 172.21.15.12:0/4119017737 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae37fe640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae40cb020 0x7f2ae415b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae2ffd640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ae40c4980 0x7f2ae4157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae27fc640 1 -- 172.21.15.12:0/4119017737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ae40c4980 msgr2=0x7f2ae4157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae27fc640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ae40c4980 0x7f2ae4157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:56.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae27fc640 1 -- 172.21.15.12:0/4119017737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae40cb020 msgr2=0x7f2ae415b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae27fc640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae40cb020 0x7f2ae415b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae27fc640 1 -- 172.21.15.12:0/4119017737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad4009420 con 0x7f2ae40cac40 2023-11-28T21:43:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae27fc640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 0x7f2ae415af10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800c500 tx=0x7f2ad800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2acb7fe640 1 -- 172.21.15.12:0/4119017737 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8005220 con 0x7f2ae40cac40 2023-11-28T21:43:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2acb7fe640 1 -- 172.21.15.12:0/4119017737 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad8005380 con 0x7f2ae40cac40 2023-11-28T21:43:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 -- 172.21.15.12:0/4119017737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae415d730 con 0x7f2ae40cac40 2023-11-28T21:43:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2acb7fe640 1 -- 172.21.15.12:0/4119017737 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8007270 con 0x7f2ae40cac40 2023-11-28T21:43:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 -- 172.21.15.12:0/4119017737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae41f0120 con 0x7f2ae40cac40 2023-11-28T21:43:56.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.103+0000 7f2ae92d6640 1 -- 172.21.15.12:0/4119017737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae4067c70 con 0x7f2ae40cac40 2023-11-28T21:43:56.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.107+0000 7f2acb7fe640 1 -- 172.21.15.12:0/4119017737 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2ad80073d0 con 0x7f2ae40cac40 2023-11-28T21:43:56.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.107+0000 7f2acb7fe640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ad0066470 0x7f2ad0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:56.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.107+0000 7f2acb7fe640 1 -- 172.21.15.12:0/4119017737 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f2ad8083bc0 con 0x7f2ae40cac40 2023-11-28T21:43:56.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.107+0000 7f2ae2ffd640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ad0066470 0x7f2ad0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:56.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.107+0000 7f2ae2ffd640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ad0066470 0x7f2ad0068930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40b2000 tx=0x7f2ad4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:56.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.111+0000 7f2acb7fe640 1 -- 172.21.15.12:0/4119017737 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2ad8050b90 con 0x7f2ae40cac40 2023-11-28T21:43:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.299+0000 7f2ae92d6640 1 -- 172.21.15.12:0/4119017737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ae4074f60 con 0x7f2ae40cac40 2023-11-28T21:43:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.299+0000 7f2acb7fe640 1 -- 172.21.15.12:0/4119017737 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f2ad80541b0 con 0x7f2ae40cac40 2023-11-28T21:43:56.304 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:43:56.304 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5727,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:43:04.550912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":27,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:43:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2023-11-28T21:43:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.299+0000 7f2ae92d6640 1 -- 172.21.15.12:0/4119017737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ad0066470 msgr2=0x7f2ad0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.299+0000 7f2ae92d6640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ad0066470 0x7f2ad0068930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40b2000 tx=0x7f2ad4008040 comp rx=0 tx=0).stop 2023-11-28T21:43:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.299+0000 7f2ae92d6640 1 -- 172.21.15.12:0/4119017737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 msgr2=0x7f2ae415af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.299+0000 7f2ae92d6640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 0x7f2ae415af10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800c500 tx=0x7f2ad800c9c0 comp rx=0 tx=0).stop 2023-11-28T21:43:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.299+0000 7f2ae92d6640 1 -- 172.21.15.12:0/4119017737 shutdown_connections 2023-11-28T21:43:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.299+0000 7f2ae92d6640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ad0066470 0x7f2ad0068930 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.303+0000 7f2ae92d6640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae40cb020 0x7f2ae415b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.303+0000 7f2ae92d6640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ae40cac40 0x7f2ae415af10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.303+0000 7f2ae92d6640 1 --2- 172.21.15.12:0/4119017737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ae40c4980 0x7f2ae4157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.303+0000 7f2ae92d6640 1 -- 172.21.15.12:0/4119017737 >> 172.21.15.12:0/4119017737 conn(0x7f2ae40720e0 msgr2=0x7f2ae4157bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.303+0000 7f2ae92d6640 1 -- 172.21.15.12:0/4119017737 shutdown_connections 2023-11-28T21:43:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:56.303+0000 7f2ae92d6640 1 -- 172.21.15.12:0/4119017737 wait complete. 2023-11-28T21:43:56.325 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:43:58.326 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:43:58.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc49e8c9640 1 Processor -- start 2023-11-28T21:43:58.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc49e8c9640 1 -- start start 2023-11-28T21:43:58.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc49e8c9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc498150c20 0x7fc498156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc49e8c9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc49806cf40 0x7fc498158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc49e8c9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc498067410 con 0x7fc49806e300 2023-11-28T21:43:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc49e8c9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49806e090 con 0x7fc498150c20 2023-11-28T21:43:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc49e8c9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc498157f30 con 0x7fc49806cf40 2023-11-28T21:43:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc497fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc498150c20 0x7fc498156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc497fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc498150c20 0x7fc498156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59794/0 (socket says 172.21.15.12:59794) 2023-11-28T21:43:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc497fff640 1 -- 172.21.15.12:0/2218724966 learned_addr learned my addr 172.21.15.12:0/2218724966 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc49ce66640 1 --2- 172.21.15.12:0/2218724966 >> v2:172.21.15.144:3300/0 conn(0x7fc49806cf40 0x7fc498158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc497fff640 1 -- 172.21.15.12:0/2218724966 >> v2:172.21.15.144:3300/0 conn(0x7fc49806cf40 msgr2=0x7fc498158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:43:58.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc497fff640 1 --2- 172.21.15.12:0/2218724966 >> v2:172.21.15.144:3300/0 conn(0x7fc49806cf40 0x7fc498158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:58.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc497fff640 1 -- 172.21.15.12:0/2218724966 >> v1:172.21.15.12:6789/0 conn(0x7fc49806e300 legacy=0x7fc4980c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:58.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc497fff640 1 -- 172.21.15.12:0/2218724966 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc498151000 con 0x7fc498150c20 2023-11-28T21:43:58.486 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.479+0000 7fc496ffd640 1 -- 172.21.15.12:0/2218724966 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257212511 0 0) 0x7fc498067410 con 0x7fc49806e300 2023-11-28T21:43:58.487 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc497fff640 1 --2- 172.21.15.12:0/2218724966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 0x7fc498156910 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc49806b9d0 tx=0x7fc48801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f387dd5c6b901e2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:58.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc496ffd640 1 -- 172.21.15.12:0/2218724966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4880ab610 con 0x7fc498150c20 2023-11-28T21:43:58.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc496ffd640 1 -- 172.21.15.12:0/2218724966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4880ab770 con 0x7fc498150c20 2023-11-28T21:43:58.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc496ffd640 1 -- 172.21.15.12:0/2218724966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4880abb00 con 0x7fc498150c20 2023-11-28T21:43:58.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc49e8c9640 1 -- 172.21.15.12:0/2218724966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 msgr2=0x7fc498156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:58.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc49e8c9640 1 --2- 172.21.15.12:0/2218724966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 0x7fc498156910 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc49806b9d0 tx=0x7fc48801c500 comp rx=0 tx=0).stop 2023-11-28T21:43:58.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc49e8c9640 1 -- 172.21.15.12:0/2218724966 shutdown_connections 2023-11-28T21:43:58.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc49e8c9640 1 --2- 172.21.15.12:0/2218724966 >> v2:172.21.15.144:3300/0 conn(0x7fc49806cf40 0x7fc498158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:58.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc49e8c9640 1 --2- 172.21.15.12:0/2218724966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 0x7fc498156910 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:58.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc49e8c9640 1 -- 172.21.15.12:0/2218724966 >> 172.21.15.12:0/2218724966 conn(0x7fc498066360 msgr2=0x7fc498066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:58.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc49e8c9640 1 -- 172.21.15.12:0/2218724966 shutdown_connections 2023-11-28T21:43:58.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.483+0000 7fc49e8c9640 1 -- 172.21.15.12:0/2218724966 wait complete. 2023-11-28T21:43:58.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49e8c9640 1 Processor -- start 2023-11-28T21:43:58.490 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49e8c9640 1 -- start start 2023-11-28T21:43:58.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49e8c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc49806cf40 0x7fc4981f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:58.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49e8c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc49806e300 0x7fc49814dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:58.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49e8c9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 0x7fc49814e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:58.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49e8c9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc498150330 con 0x7fc498150c20 2023-11-28T21:43:58.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49e8c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4981f4c50 con 0x7fc49806cf40 2023-11-28T21:43:58.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49e8c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980c34f0 con 0x7fc49806e300 2023-11-28T21:43:58.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49ce66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 0x7fc49814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:58.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49ce66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 0x7fc49814e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59800/0 (socket says 172.21.15.12:59800) 2023-11-28T21:43:58.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49ce66640 1 -- 172.21.15.12:0/3538397230 learned_addr learned my addr 172.21.15.12:0/3538397230 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:43:58.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc497fff640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc49806cf40 0x7fc4981f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:58.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc4977fe640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc49806e300 0x7fc49814dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:58.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49ce66640 1 -- 172.21.15.12:0/3538397230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc49806e300 msgr2=0x7fc49814dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:58.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49ce66640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc49806e300 0x7fc49814dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:58.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49ce66640 1 -- 172.21.15.12:0/3538397230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc49806cf40 msgr2=0x7fc4981f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:58.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49ce66640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc49806cf40 0x7fc4981f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:58.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49ce66640 1 -- 172.21.15.12:0/3538397230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4880094d0 con 0x7fc498150c20 2023-11-28T21:43:58.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc497fff640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc49806cf40 0x7fc4981f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:43:58.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49ce66640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 0x7fc49814e270 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc490007e40 tx=0x7fc49000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:58.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc494ff9640 1 -- 172.21.15.12:0/3538397230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490014620 con 0x7fc498150c20 2023-11-28T21:43:58.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc494ff9640 1 -- 172.21.15.12:0/3538397230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49000ee90 con 0x7fc498150c20 2023-11-28T21:43:58.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc494ff9640 1 -- 172.21.15.12:0/3538397230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4900149e0 con 0x7fc498150c20 2023-11-28T21:43:58.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49e8c9640 1 -- 172.21.15.12:0/3538397230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4980c37d0 con 0x7fc498150c20 2023-11-28T21:43:58.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.487+0000 7fc49e8c9640 1 -- 172.21.15.12:0/3538397230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49814f8f0 con 0x7fc498150c20 2023-11-28T21:43:58.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.491+0000 7fc49e8c9640 1 -- 172.21.15.12:0/3538397230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49814fae0 con 0x7fc498150c20 2023-11-28T21:43:58.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.491+0000 7fc494ff9640 1 -- 172.21.15.12:0/3538397230 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc490014b40 con 0x7fc498150c20 2023-11-28T21:43:58.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.491+0000 7fc494ff9640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc484066470 0x7fc484068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:43:58.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.491+0000 7fc494ff9640 1 -- 172.21.15.12:0/3538397230 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fc490084d80 con 0x7fc498150c20 2023-11-28T21:43:58.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.491+0000 7fc497fff640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc484066470 0x7fc484068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:43:58.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.491+0000 7fc497fff640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc484066470 0x7fc484068930 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4880af000 tx=0x7fc4880a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:43:58.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.495+0000 7fc494ff9640 1 -- 172.21.15.12:0/3538397230 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc490051ca0 con 0x7fc498150c20 2023-11-28T21:43:58.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.687+0000 7fc49e8c9640 1 -- 172.21.15.12:0/3538397230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc498074f60 con 0x7fc498150c20 2023-11-28T21:43:58.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.687+0000 7fc494ff9640 1 -- 172.21.15.12:0/3538397230 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6503 (secure 0 0 0) 0x7fc490084020 con 0x7fc498150c20 2023-11-28T21:43:58.692 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:43:58.693 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5727,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:43:04.550912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":27,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:43:58.693 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2023-11-28T21:43:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 -- 172.21.15.12:0/3538397230 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc484066470 msgr2=0x7fc484068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc484066470 0x7fc484068930 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4880af000 tx=0x7fc4880a7000 comp rx=0 tx=0).stop 2023-11-28T21:43:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 -- 172.21.15.12:0/3538397230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 msgr2=0x7fc49814e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:43:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 0x7fc49814e270 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc490007e40 tx=0x7fc49000e4b0 comp rx=0 tx=0).stop 2023-11-28T21:43:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 -- 172.21.15.12:0/3538397230 shutdown_connections 2023-11-28T21:43:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc484066470 0x7fc484068930 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc498150c20 0x7fc49814e270 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc49806e300 0x7fc49814dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:58.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 --2- 172.21.15.12:0/3538397230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc49806cf40 0x7fc4981f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:43:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 -- 172.21.15.12:0/3538397230 >> 172.21.15.12:0/3538397230 conn(0x7fc498066360 msgr2=0x7fc4980c9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:43:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 -- 172.21.15.12:0/3538397230 shutdown_connections 2023-11-28T21:43:58.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:43:58.691+0000 7fc49e8c9640 1 -- 172.21.15.12:0/3538397230 wait complete. 2023-11-28T21:43:58.715 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:44:00.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:00.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2bd432640 1 Processor -- start 2023-11-28T21:44:00.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2bd432640 1 -- start start 2023-11-28T21:44:00.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2bd432640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2b80c9e60 0x7fb2b80cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2bd432640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb2b80cf9b0 0x7fb2b806ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2bd432640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b80b2660 con 0x7fb2b8154070 2023-11-28T21:44:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2bd432640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b80cfdb0 con 0x7fb2b80cf9b0 2023-11-28T21:44:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2bd432640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b80d0430 con 0x7fb2b80c9e60 2023-11-28T21:44:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2b6ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb2b8154070 0x7fb2b815a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44058/0 (socket says 172.21.15.12:44058) 2023-11-28T21:44:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2b6ffd640 1 -- 172.21.15.12:0/922649230 learned_addr learned my addr 172.21.15.12:0/922649230 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:44:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2b77fe640 1 --2- 172.21.15.12:0/922649230 >> v2:172.21.15.12:3300/0 conn(0x7fb2b80cf9b0 0x7fb2b806ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2ae7fc640 1 --2- 172.21.15.12:0/922649230 >> v2:172.21.15.144:3300/0 conn(0x7fb2b80c9e60 0x7fb2b80cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2b77fe640 1 -- 172.21.15.12:0/922649230 >> v2:172.21.15.144:3300/0 conn(0x7fb2b80c9e60 msgr2=0x7fb2b80cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:00.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2b77fe640 1 --2- 172.21.15.12:0/922649230 >> v2:172.21.15.144:3300/0 conn(0x7fb2b80c9e60 0x7fb2b80cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:00.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2b77fe640 1 -- 172.21.15.12:0/922649230 >> v1:172.21.15.12:6789/0 conn(0x7fb2b8154070 legacy=0x7fb2b815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:00.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2b77fe640 1 -- 172.21.15.12:0/922649230 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b8154450 con 0x7fb2b80cf9b0 2023-11-28T21:44:00.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.867+0000 7fb2b67fc640 1 -- 172.21.15.12:0/922649230 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723492051 0 0) 0x7fb2a80014a0 con 0x7fb2b8154070 2023-11-28T21:44:00.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2b77fe640 1 --2- 172.21.15.12:0/922649230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b80cf9b0 0x7fb2b806ab70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb2b81578c0 tx=0x7fb2b001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d879fb56fef7180 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:00.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2b67fc640 1 -- 172.21.15.12:0/922649230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b00ab660 con 0x7fb2b80cf9b0 2023-11-28T21:44:00.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2b67fc640 1 -- 172.21.15.12:0/922649230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b00ab7c0 con 0x7fb2b80cf9b0 2023-11-28T21:44:00.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2b67fc640 1 -- 172.21.15.12:0/922649230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b00abb50 con 0x7fb2b80cf9b0 2023-11-28T21:44:00.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2bd432640 1 -- 172.21.15.12:0/922649230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b80cf9b0 msgr2=0x7fb2b806ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:00.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2bd432640 1 --2- 172.21.15.12:0/922649230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b80cf9b0 0x7fb2b806ab70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb2b81578c0 tx=0x7fb2b001c460 comp rx=0 tx=0).stop 2023-11-28T21:44:00.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2bd432640 1 -- 172.21.15.12:0/922649230 shutdown_connections 2023-11-28T21:44:00.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2bd432640 1 --2- 172.21.15.12:0/922649230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b80cf9b0 0x7fb2b806ab70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:00.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2bd432640 1 --2- 172.21.15.12:0/922649230 >> v2:172.21.15.144:3300/0 conn(0x7fb2b80c9e60 0x7fb2b80cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:00.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2bd432640 1 -- 172.21.15.12:0/922649230 >> 172.21.15.12:0/922649230 conn(0x7fb2b806bd60 msgr2=0x7fb2b806c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:00.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2bd432640 1 -- 172.21.15.12:0/922649230 shutdown_connections 2023-11-28T21:44:00.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.871+0000 7fb2bd432640 1 -- 172.21.15.12:0/922649230 wait complete. 2023-11-28T21:44:00.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2bd432640 1 Processor -- start 2023-11-28T21:44:00.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2bd432640 1 -- start start 2023-11-28T21:44:00.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2bd432640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2b80c9e60 0x7fb2b81ec720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:00.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2bd432640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2b80cf9b0 0x7fb2b815c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:00.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2bd432640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b8154070 0x7fb2b81474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:00.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2bd432640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b815bc20 con 0x7fb2b8154070 2023-11-28T21:44:00.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2bd432640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b81ecc60 con 0x7fb2b80c9e60 2023-11-28T21:44:00.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2bd432640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8146aa0 con 0x7fb2b80cf9b0 2023-11-28T21:44:00.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b8154070 0x7fb2b81474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:00.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b8154070 0x7fb2b81474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59820/0 (socket says 172.21.15.12:59820) 2023-11-28T21:44:00.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2b77fe640 1 -- 172.21.15.12:0/3093092322 learned_addr learned my addr 172.21.15.12:0/3093092322 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:00.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2b6ffd640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2b80c9e60 0x7fb2b81ec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:00.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2ae7fc640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2b80cf9b0 0x7fb2b815c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:00.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2b77fe640 1 -- 172.21.15.12:0/3093092322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2b80cf9b0 msgr2=0x7fb2b815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2b77fe640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2b80cf9b0 0x7fb2b815c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2b77fe640 1 -- 172.21.15.12:0/3093092322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2b80c9e60 msgr2=0x7fb2b81ec720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2b77fe640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2b80c9e60 0x7fb2b81ec720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2b77fe640 1 -- 172.21.15.12:0/3093092322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0009420 con 0x7fb2b8154070 2023-11-28T21:44:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2b77fe640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b8154070 0x7fb2b81474e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00ad040 tx=0x7fb2b001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2affff640 1 -- 172.21.15.12:0/3093092322 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0004230 con 0x7fb2b8154070 2023-11-28T21:44:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2affff640 1 -- 172.21.15.12:0/3093092322 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b00b9070 con 0x7fb2b8154070 2023-11-28T21:44:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2affff640 1 -- 172.21.15.12:0/3093092322 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0003620 con 0x7fb2b8154070 2023-11-28T21:44:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2bd432640 1 -- 172.21.15.12:0/3093092322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b8146060 con 0x7fb2b8154070 2023-11-28T21:44:00.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.875+0000 7fb2bd432640 1 -- 172.21.15.12:0/3093092322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b81f33e0 con 0x7fb2b8154070 2023-11-28T21:44:00.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.879+0000 7fb2bd432640 1 -- 172.21.15.12:0/3093092322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b8074f60 con 0x7fb2b8154070 2023-11-28T21:44:00.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.879+0000 7fb2affff640 1 -- 172.21.15.12:0/3093092322 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb2b00a9070 con 0x7fb2b8154070 2023-11-28T21:44:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.879+0000 7fb2affff640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb2980664a0 0x7fb298068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.879+0000 7fb2affff640 1 -- 172.21.15.12:0/3093092322 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fb2b00bb070 con 0x7fb2b8154070 2023-11-28T21:44:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.879+0000 7fb2b6ffd640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb2980664a0 0x7fb298068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:00.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.879+0000 7fb2b6ffd640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb2980664a0 0x7fb298068960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80012f0 tx=0x7fb2a8001690 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:00.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:00.883+0000 7fb2affff640 1 -- 172.21.15.12:0/3093092322 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb2b0120e60 con 0x7fb2b8154070 2023-11-28T21:44:01.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.071+0000 7fb2bd432640 1 -- 172.21.15.12:0/3093092322 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2b80ca3d0 con 0x7fb2b8154070 2023-11-28T21:44:01.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.071+0000 7fb2affff640 1 -- 172.21.15.12:0/3093092322 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6503 (secure 0 0 0) 0x7fb2b00ebb10 con 0x7fb2b8154070 2023-11-28T21:44:01.077 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:01.077 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5727,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:43:04.550912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":27,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:01.077 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2023-11-28T21:44:01.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 -- 172.21.15.12:0/3093092322 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb2980664a0 msgr2=0x7fb298068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:01.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb2980664a0 0x7fb298068960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80012f0 tx=0x7fb2a8001690 comp rx=0 tx=0).stop 2023-11-28T21:44:01.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 -- 172.21.15.12:0/3093092322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b8154070 msgr2=0x7fb2b81474e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:01.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b8154070 0x7fb2b81474e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00ad040 tx=0x7fb2b001aca0 comp rx=0 tx=0).stop 2023-11-28T21:44:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 -- 172.21.15.12:0/3093092322 shutdown_connections 2023-11-28T21:44:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb2980664a0 0x7fb298068960 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb2b8154070 0x7fb2b81474e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2b80cf9b0 0x7fb2b815c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 --2- 172.21.15.12:0/3093092322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2b80c9e60 0x7fb2b81ec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 -- 172.21.15.12:0/3093092322 >> 172.21.15.12:0/3093092322 conn(0x7fb2b806bd60 msgr2=0x7fb2b80b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 -- 172.21.15.12:0/3093092322 shutdown_connections 2023-11-28T21:44:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:01.075+0000 7fb2bd432640 1 -- 172.21.15.12:0/3093092322 wait complete. 2023-11-28T21:44:01.098 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:44:03.100 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:03.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37ecd96640 1 Processor -- start 2023-11-28T21:44:03.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37ecd96640 1 -- start start 2023-11-28T21:44:03.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37ecd96640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f37e806bc20 0x7f37e8066360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:03.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37ecd96640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e8159b90 con 0x7f37e8153a20 2023-11-28T21:44:03.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37ecd96640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e806c260 con 0x7f37e80caea0 2023-11-28T21:44:03.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37ecd96640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e815a4d0 con 0x7f37e806bc20 2023-11-28T21:44:03.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37e77fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f37e8153a20 0x7f37e80c4690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44060/0 (socket says 172.21.15.12:44060) 2023-11-28T21:44:03.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37e77fe640 1 -- 172.21.15.12:0/3394491236 learned_addr learned my addr 172.21.15.12:0/3394491236 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:44:03.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37e6ffd640 1 --2- 172.21.15.12:0/3394491236 >> v2:172.21.15.12:3300/0 conn(0x7f37e806bc20 0x7f37e8066360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:03.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37e6ffd640 1 -- 172.21.15.12:0/3394491236 >> v1:172.21.15.144:6789/0 conn(0x7f37e80caea0 legacy=0x7f37e8164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:03.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37e6ffd640 1 -- 172.21.15.12:0/3394491236 >> v1:172.21.15.12:6789/0 conn(0x7f37e8153a20 legacy=0x7f37e80c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:03.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37e67fc640 1 -- 172.21.15.12:0/3394491236 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534207759 0 0) 0x7f37e8159b90 con 0x7f37e8153a20 2023-11-28T21:44:03.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.251+0000 7f37e6ffd640 1 -- 172.21.15.12:0/3394491236 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e806c540 con 0x7f37e806bc20 2023-11-28T21:44:03.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37e6ffd640 1 --2- 172.21.15.12:0/3394491236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e806bc20 0x7f37e8066360 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0097c0 tx=0x7f37dc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963a9be2eab89e42 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:03.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37e67fc640 1 -- 172.21.15.12:0/3394491236 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0ab600 con 0x7f37e806bc20 2023-11-28T21:44:03.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37e67fc640 1 -- 172.21.15.12:0/3394491236 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc0a7050 con 0x7f37e806bc20 2023-11-28T21:44:03.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37e67fc640 1 -- 172.21.15.12:0/3394491236 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0ab980 con 0x7f37e806bc20 2023-11-28T21:44:03.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37ecd96640 1 -- 172.21.15.12:0/3394491236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e806bc20 msgr2=0x7f37e8066360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:03.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37ecd96640 1 --2- 172.21.15.12:0/3394491236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e806bc20 0x7f37e8066360 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0097c0 tx=0x7f37dc01c480 comp rx=0 tx=0).stop 2023-11-28T21:44:03.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37ecd96640 1 -- 172.21.15.12:0/3394491236 shutdown_connections 2023-11-28T21:44:03.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37ecd96640 1 --2- 172.21.15.12:0/3394491236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e806bc20 0x7f37e8066360 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:03.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37ecd96640 1 -- 172.21.15.12:0/3394491236 >> 172.21.15.12:0/3394491236 conn(0x7f37e80cf550 msgr2=0x7f37e80cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:03.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37ecd96640 1 -- 172.21.15.12:0/3394491236 shutdown_connections 2023-11-28T21:44:03.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37ecd96640 1 -- 172.21.15.12:0/3394491236 wait complete. 2023-11-28T21:44:03.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.255+0000 7f37ecd96640 1 Processor -- start 2023-11-28T21:44:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37ecd96640 1 -- start start 2023-11-28T21:44:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37ecd96640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e806bc20 0x7f37e815c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37ecd96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e80caea0 0x7f37e815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37ecd96640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e8153a20 0x7f37e80b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37ecd96640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e815c5f0 con 0x7f37e80caea0 2023-11-28T21:44:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37ecd96640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80c9490 con 0x7f37e806bc20 2023-11-28T21:44:03.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37ecd96640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80c9730 con 0x7f37e8153a20 2023-11-28T21:44:03.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e80caea0 0x7f37e815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:03.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e80caea0 0x7f37e815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59840/0 (socket says 172.21.15.12:59840) 2023-11-28T21:44:03.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e6ffd640 1 -- 172.21.15.12:0/3767412241 learned_addr learned my addr 172.21.15.12:0/3767412241 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:03.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e77fe640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e806bc20 0x7f37e815c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:03.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e7fff640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e8153a20 0x7f37e80b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:03.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e6ffd640 1 -- 172.21.15.12:0/3767412241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e8153a20 msgr2=0x7f37e80b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:03.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e6ffd640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e8153a20 0x7f37e80b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:03.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e6ffd640 1 -- 172.21.15.12:0/3767412241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e806bc20 msgr2=0x7f37e815c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:03.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e6ffd640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e806bc20 0x7f37e815c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:03.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e6ffd640 1 -- 172.21.15.12:0/3767412241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc009420 con 0x7f37e80caea0 2023-11-28T21:44:03.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e7fff640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e8153a20 0x7f37e80b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:44:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37e6ffd640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e80caea0 0x7f37e815afb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0ae040 tx=0x7f37dc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37cffff640 1 -- 172.21.15.12:0/3767412241 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0b7650 con 0x7f37e80caea0 2023-11-28T21:44:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37cffff640 1 -- 172.21.15.12:0/3767412241 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc0b77b0 con 0x7f37e80caea0 2023-11-28T21:44:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37cffff640 1 -- 172.21.15.12:0/3767412241 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0b7b60 con 0x7f37e80caea0 2023-11-28T21:44:03.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37ecd96640 1 -- 172.21.15.12:0/3767412241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e815cf40 con 0x7f37e80caea0 2023-11-28T21:44:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37ecd96640 1 -- 172.21.15.12:0/3767412241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e815d500 con 0x7f37e80caea0 2023-11-28T21:44:03.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.259+0000 7f37ecd96640 1 -- 172.21.15.12:0/3767412241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e80b3530 con 0x7f37e80caea0 2023-11-28T21:44:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.263+0000 7f37cffff640 1 -- 172.21.15.12:0/3767412241 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f37dc0b7cc0 con 0x7f37e80caea0 2023-11-28T21:44:03.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.263+0000 7f37cffff640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f37d4066470 0x7f37d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.263+0000 7f37cffff640 1 -- 172.21.15.12:0/3767412241 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f37dc11ec80 con 0x7f37e80caea0 2023-11-28T21:44:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.263+0000 7f37e77fe640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f37d4066470 0x7f37d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.263+0000 7f37e77fe640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f37d4066470 0x7f37d4068930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f37d8000f70 tx=0x7f37d8007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:03.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.267+0000 7f37cffff640 1 -- 172.21.15.12:0/3767412241 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f37dc0ebd50 con 0x7f37e80caea0 2023-11-28T21:44:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.459+0000 7f37ecd96640 1 -- 172.21.15.12:0/3767412241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37e8074f60 con 0x7f37e80caea0 2023-11-28T21:44:03.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.459+0000 7f37cffff640 1 -- 172.21.15.12:0/3767412241 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f37dc0b0090 con 0x7f37e80caea0 2023-11-28T21:44:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:03.465 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5727,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:43:04.550912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":27,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:03.465 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2023-11-28T21:44:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 -- 172.21.15.12:0/3767412241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f37d4066470 msgr2=0x7f37d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f37d4066470 0x7f37d4068930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f37d8000f70 tx=0x7f37d8007240 comp rx=0 tx=0).stop 2023-11-28T21:44:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 -- 172.21.15.12:0/3767412241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e80caea0 msgr2=0x7f37e815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e80caea0 0x7f37e815afb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0ae040 tx=0x7f37dc01acc0 comp rx=0 tx=0).stop 2023-11-28T21:44:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 -- 172.21.15.12:0/3767412241 shutdown_connections 2023-11-28T21:44:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f37d4066470 0x7f37d4068930 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37e8153a20 0x7f37e80b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37e80caea0 0x7f37e815afb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 --2- 172.21.15.12:0/3767412241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37e806bc20 0x7f37e815c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 -- 172.21.15.12:0/3767412241 >> 172.21.15.12:0/3767412241 conn(0x7f37e80cf550 msgr2=0x7f37e8150f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 -- 172.21.15.12:0/3767412241 shutdown_connections 2023-11-28T21:44:03.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:03.463+0000 7f37ecd96640 1 -- 172.21.15.12:0/3767412241 wait complete. 2023-11-28T21:44:03.487 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:44:05.488 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.639+0000 7f66a5334640 1 Processor -- start 2023-11-28T21:44:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.639+0000 7f66a5334640 1 -- start start 2023-11-28T21:44:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.639+0000 7f66a5334640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f66a0150420 0x7f66a00c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.639+0000 7f66a5334640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a00c3070 con 0x7f66a006b0a0 2023-11-28T21:44:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.639+0000 7f66a5334640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a00c3330 con 0x7f66a006b480 2023-11-28T21:44:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.639+0000 7f66a5334640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a00c4290 con 0x7f66a0150420 2023-11-28T21:44:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f66a006b0a0 0x7f66a014fd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43514/0 (socket says 172.21.15.12:43514) 2023-11-28T21:44:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669ffff640 1 -- 172.21.15.12:0/2773550444 learned_addr learned my addr 172.21.15.12:0/2773550444 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:44:05.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669f7fe640 1 --2- 172.21.15.12:0/2773550444 >> v2:172.21.15.144:3300/0 conn(0x7f66a0150420 0x7f66a00c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508277805 0 0) 0x7f66a00c3070 con 0x7f66a006b0a0 2023-11-28T21:44:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6688003450 con 0x7f66a006b0a0 2023-11-28T21:44:05.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4080000194 0 0) 0x7f6688003450 con 0x7f66a006b0a0 2023-11-28T21:44:05.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 >> v2:172.21.15.144:3300/0 conn(0x7f66a0150420 msgr2=0x7f66a00c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:05.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 --2- 172.21.15.12:0/2773550444 >> v2:172.21.15.144:3300/0 conn(0x7f66a0150420 0x7f66a00c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 >> v1:172.21.15.144:6789/0 conn(0x7f66a006b480 legacy=0x7f66a0160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:05.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a00c3610 con 0x7f66a006b0a0 2023-11-28T21:44:05.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f66900016a0 con 0x7f66a006b0a0 2023-11-28T21:44:05.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 >> v1:172.21.15.12:6789/0 conn(0x7f66a006b0a0 legacy=0x7f66a014fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:05.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6688003a70 0x7f6688003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f668800ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668800b490 0x7f668800d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a00c4290 con 0x7f6688008860 2023-11-28T21:44:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668800e230 con 0x7f668800b490 2023-11-28T21:44:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668800e3f0 con 0x7f6688003a70 2023-11-28T21:44:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669f7fe640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f668800ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669ffff640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6688003a70 0x7f6688003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f66a4b33640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668800b490 0x7f668800d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669f7fe640 1 -- 172.21.15.12:0/2773550444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6688003a70 msgr2=0x7f6688003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669f7fe640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6688003a70 0x7f6688003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669f7fe640 1 -- 172.21.15.12:0/2773550444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668800b490 msgr2=0x7f668800d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669f7fe640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668800b490 0x7f668800d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669f7fe640 1 -- 172.21.15.12:0/2773550444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6688003450 con 0x7f6688008860 2023-11-28T21:44:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f66a4b33640 1 -- 172.21.15.12:0/2773550444 reap_dead start 2023-11-28T21:44:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.643+0000 7f669ffff640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6688003a70 0x7f6688003e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:44:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f669f7fe640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f668800ac90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6694009ae0 tx=0x7f669401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef54d37596e16ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66940a5640 con 0x7f6688008860 2023-11-28T21:44:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66940a57a0 con 0x7f6688008860 2023-11-28T21:44:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f669effd640 1 -- 172.21.15.12:0/2773550444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66940a5b50 con 0x7f6688008860 2023-11-28T21:44:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f66a5334640 1 -- 172.21.15.12:0/2773550444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 msgr2=0x7f668800ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f66a5334640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f668800ac90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6694009ae0 tx=0x7f669401ca50 comp rx=0 tx=0).stop 2023-11-28T21:44:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f66a5334640 1 -- 172.21.15.12:0/2773550444 shutdown_connections 2023-11-28T21:44:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f66a5334640 1 --2- 172.21.15.12:0/2773550444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f668800ac90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f66a5334640 1 -- 172.21.15.12:0/2773550444 >> 172.21.15.12:0/2773550444 conn(0x7f66a0066360 msgr2=0x7f66a0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f66a5334640 1 -- 172.21.15.12:0/2773550444 shutdown_connections 2023-11-28T21:44:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f66a5334640 1 -- 172.21.15.12:0/2773550444 wait complete. 2023-11-28T21:44:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f66a5334640 1 Processor -- start 2023-11-28T21:44:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.647+0000 7f66a5334640 1 -- start start 2023-11-28T21:44:05.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f66a5334640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f66a0071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:05.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f66a5334640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66a014bb00 0x7f66a00c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:05.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f66a5334640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a00c1c30 0x7f66a01f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:05.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f66a5334640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a0071f40 con 0x7f6688008860 2023-11-28T21:44:05.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f66a5334640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a0148e50 con 0x7f66a00c1c30 2023-11-28T21:44:05.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f66a5334640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a0149130 con 0x7f66a014bb00 2023-11-28T21:44:05.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f66a0071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:05.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f66a0071890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35504/0 (socket says 172.21.15.12:35504) 2023-11-28T21:44:05.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669ffff640 1 -- 172.21.15.12:0/3037473958 learned_addr learned my addr 172.21.15.12:0/3037473958 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:05.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669f7fe640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66a014bb00 0x7f66a00c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:05.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f66a4b33640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a00c1c30 0x7f66a01f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:05.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669ffff640 1 -- 172.21.15.12:0/3037473958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66a014bb00 msgr2=0x7f66a00c16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:05.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669ffff640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66a014bb00 0x7f66a00c16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669ffff640 1 -- 172.21.15.12:0/3037473958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a00c1c30 msgr2=0x7f66a01f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:05.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669ffff640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a00c1c30 0x7f66a01f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669ffff640 1 -- 172.21.15.12:0/3037473958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a0149410 con 0x7f6688008860 2023-11-28T21:44:05.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669f7fe640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66a014bb00 0x7f66a00c16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:44:05.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669ffff640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f66a0071890 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6690011810 tx=0x7f6690011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:05.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669cff9640 1 -- 172.21.15.12:0/3037473958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669001a030 con 0x7f6688008860 2023-11-28T21:44:05.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669cff9640 1 -- 172.21.15.12:0/3037473958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669000f960 con 0x7f6688008860 2023-11-28T21:44:05.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669cff9640 1 -- 172.21.15.12:0/3037473958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669000fd10 con 0x7f6688008860 2023-11-28T21:44:05.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f66a5334640 1 -- 172.21.15.12:0/3037473958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a01496f0 con 0x7f6688008860 2023-11-28T21:44:05.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f66a5334640 1 -- 172.21.15.12:0/3037473958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a01f7720 con 0x7f6688008860 2023-11-28T21:44:05.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f66a5334640 1 -- 172.21.15.12:0/3037473958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a01f6fc0 con 0x7f6688008860 2023-11-28T21:44:05.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.651+0000 7f669cff9640 1 -- 172.21.15.12:0/3037473958 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f669001d070 con 0x7f6688008860 2023-11-28T21:44:05.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.655+0000 7f669cff9640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f668c0664a0 0x7f668c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:05.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.655+0000 7f669cff9640 1 -- 172.21.15.12:0/3037473958 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f669001b070 con 0x7f6688008860 2023-11-28T21:44:05.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.655+0000 7f669f7fe640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f668c0664a0 0x7f668c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:05.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.655+0000 7f669f7fe640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f668c0664a0 0x7f668c068960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f66940af000 tx=0x7f6694008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:05.663 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.659+0000 7f669cff9640 1 -- 172.21.15.12:0/3037473958 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6690054110 con 0x7f6688008860 2023-11-28T21:44:05.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 -- 172.21.15.12:0/3037473958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66a0067c70 con 0x7f6688008860 2023-11-28T21:44:05.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f669cff9640 1 -- 172.21.15.12:0/3037473958 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f6690057730 con 0x7f6688008860 2023-11-28T21:44:05.856 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:05.856 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5727,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:43:04.550912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":27,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:05.856 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2023-11-28T21:44:05.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 -- 172.21.15.12:0/3037473958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f668c0664a0 msgr2=0x7f668c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:05.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f668c0664a0 0x7f668c068960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f66940af000 tx=0x7f6694008040 comp rx=0 tx=0).stop 2023-11-28T21:44:05.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 -- 172.21.15.12:0/3037473958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 msgr2=0x7f66a0071890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:05.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f66a0071890 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6690011810 tx=0x7f6690011cd0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 -- 172.21.15.12:0/3037473958 shutdown_connections 2023-11-28T21:44:05.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f668c0664a0 0x7f668c068960 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66a00c1c30 0x7f66a01f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66a014bb00 0x7f66a00c16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 --2- 172.21.15.12:0/3037473958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6688008860 0x7f66a0071890 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:05.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.851+0000 7f66a5334640 1 -- 172.21.15.12:0/3037473958 >> 172.21.15.12:0/3037473958 conn(0x7f66a0066360 msgr2=0x7f66a00c4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:05.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.855+0000 7f66a5334640 1 -- 172.21.15.12:0/3037473958 shutdown_connections 2023-11-28T21:44:05.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:05.855+0000 7f66a5334640 1 -- 172.21.15.12:0/3037473958 wait complete. 2023-11-28T21:44:05.877 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:44:07.878 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:08.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.027+0000 7fabf95b8640 1 Processor -- start 2023-11-28T21:44:08.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.027+0000 7fabf95b8640 1 -- start start 2023-11-28T21:44:08.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf95b8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fabf415b180 0x7fabf415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:08.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf95b8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf406d2a0 con 0x7fabf41540f0 2023-11-28T21:44:08.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf95b8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf406d580 con 0x7fabf415d210 2023-11-28T21:44:08.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf95b8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf406e200 con 0x7fabf415b180 2023-11-28T21:44:08.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf2ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fabf41540f0 0x7fabf40c3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43524/0 (socket says 172.21.15.12:43524) 2023-11-28T21:44:08.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf2ffd640 1 -- 172.21.15.12:0/1876793388 learned_addr learned my addr 172.21.15.12:0/1876793388 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:44:08.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabebfff640 1 --2- 172.21.15.12:0/1876793388 >> v2:172.21.15.144:3300/0 conn(0x7fabf415b180 0x7fabf415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:08.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717919081 0 0) 0x7fabf406d2a0 con 0x7fabf41540f0 2023-11-28T21:44:08.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe0003530 con 0x7fabf41540f0 2023-11-28T21:44:08.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3654472110 0 0) 0x7fabe0003530 con 0x7fabf41540f0 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 >> v2:172.21.15.144:3300/0 conn(0x7fabf415b180 msgr2=0x7fabf415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 --2- 172.21.15.12:0/1876793388 >> v2:172.21.15.144:3300/0 conn(0x7fabf415b180 0x7fabf415b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 >> v1:172.21.15.144:6789/0 conn(0x7fabf415d210 legacy=0x7fabf415d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf406d860 con 0x7fabf41540f0 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fabe4001680 con 0x7fabf41540f0 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 >> v1:172.21.15.12:6789/0 conn(0x7fabf41540f0 legacy=0x7fabf40c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabe0003a40 0x7fabe0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe00047d0 0x7fabe0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe00051e0 0x7fabe000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf406e200 con 0x7fabe00047d0 2023-11-28T21:44:08.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe000c110 con 0x7fabe00051e0 2023-11-28T21:44:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe000c2d0 con 0x7fabe0003a40 2023-11-28T21:44:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabebfff640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe00047d0 0x7fabe0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf2ffd640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabe0003a40 0x7fabe0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf37fe640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe00051e0 0x7fabe000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabebfff640 1 -- 172.21.15.12:0/1876793388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabe0003a40 msgr2=0x7fabe0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabebfff640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabe0003a40 0x7fabe0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabebfff640 1 -- 172.21.15.12:0/1876793388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe00051e0 msgr2=0x7fabe000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabebfff640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe00051e0 0x7fabe000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabebfff640 1 -- 172.21.15.12:0/1876793388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe0003530 con 0x7fabe00047d0 2023-11-28T21:44:08.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.031+0000 7fabf37fe640 1 -- 172.21.15.12:0/1876793388 reap_dead start 2023-11-28T21:44:08.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabebfff640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe00047d0 0x7fabe0004c50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009ae0 tx=0x7fabdc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916e3c9572e0646f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:08.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0a55e0 con 0x7fabe00047d0 2023-11-28T21:44:08.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc0a5740 con 0x7fabe00047d0 2023-11-28T21:44:08.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabf27fc640 1 -- 172.21.15.12:0/1876793388 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0a5af0 con 0x7fabe00047d0 2023-11-28T21:44:08.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabf95b8640 1 -- 172.21.15.12:0/1876793388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe00047d0 msgr2=0x7fabe0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:08.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabf95b8640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe00047d0 0x7fabe0004c50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009ae0 tx=0x7fabdc01c9f0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabf95b8640 1 -- 172.21.15.12:0/1876793388 shutdown_connections 2023-11-28T21:44:08.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabf95b8640 1 --2- 172.21.15.12:0/1876793388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabe00047d0 0x7fabe0004c50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabf95b8640 1 -- 172.21.15.12:0/1876793388 >> 172.21.15.12:0/1876793388 conn(0x7fabf4066360 msgr2=0x7fabf4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:08.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabf95b8640 1 -- 172.21.15.12:0/1876793388 shutdown_connections 2023-11-28T21:44:08.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.035+0000 7fabf95b8640 1 -- 172.21.15.12:0/1876793388 wait complete. 2023-11-28T21:44:08.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 Processor -- start 2023-11-28T21:44:08.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 -- start start 2023-11-28T21:44:08.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe00047d0 0x7fabf41452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf4146690 0x7fabf40c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf41f1670 0x7fabf41f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf40c7890 con 0x7fabf4146690 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4145820 con 0x7fabe00047d0 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4154630 con 0x7fabf41f1670 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf4146690 0x7fabf40c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf4146690 0x7fabf40c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35520/0 (socket says 172.21.15.12:35520) 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabebfff640 1 -- 172.21.15.12:0/738757465 learned_addr learned my addr 172.21.15.12:0/738757465 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf2ffd640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe00047d0 0x7fabf41452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf37fe640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf41f1670 0x7fabf41f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabebfff640 1 -- 172.21.15.12:0/738757465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf41f1670 msgr2=0x7fabf41f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabebfff640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf41f1670 0x7fabf41f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabebfff640 1 -- 172.21.15.12:0/738757465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe00047d0 msgr2=0x7fabf41452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabebfff640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe00047d0 0x7fabf41452e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabebfff640 1 -- 172.21.15.12:0/738757465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf41f2030 con 0x7fabf4146690 2023-11-28T21:44:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf37fe640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf41f1670 0x7fabf41f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:44:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabebfff640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf4146690 0x7fabf40c68b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b0040 tx=0x7fabdc0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabeb7fe640 1 -- 172.21.15.12:0/738757465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc004220 con 0x7fabf4146690 2023-11-28T21:44:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabeb7fe640 1 -- 172.21.15.12:0/738757465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc0043c0 con 0x7fabf4146690 2023-11-28T21:44:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 -- 172.21.15.12:0/738757465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf4143f30 con 0x7fabf4146690 2023-11-28T21:44:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabeb7fe640 1 -- 172.21.15.12:0/738757465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc003860 con 0x7fabf4146690 2023-11-28T21:44:08.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 -- 172.21.15.12:0/738757465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf41444f0 con 0x7fabf4146690 2023-11-28T21:44:08.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.039+0000 7fabf95b8640 1 -- 172.21.15.12:0/738757465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf4074f60 con 0x7fabf4146690 2023-11-28T21:44:08.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.043+0000 7fabeb7fe640 1 -- 172.21.15.12:0/738757465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fabdc0b4070 con 0x7fabf4146690 2023-11-28T21:44:08.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.043+0000 7fabeb7fe640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fabd4066470 0x7fabd4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.043+0000 7fabeb7fe640 1 -- 172.21.15.12:0/738757465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fabdc11e250 con 0x7fabf4146690 2023-11-28T21:44:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.043+0000 7fabf2ffd640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fabd4066470 0x7fabd4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:08.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.043+0000 7fabf2ffd640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fabd4066470 0x7fabd4068930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fabf40501f0 tx=0x7fabe4004840 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:08.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.047+0000 7fabeb7fe640 1 -- 172.21.15.12:0/738757465 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fabdc0eb190 con 0x7fabf4146690 2023-11-28T21:44:08.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.231+0000 7fabf95b8640 1 -- 172.21.15.12:0/738757465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabf4146c20 con 0x7fabf4146690 2023-11-28T21:44:08.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabeb7fe640 1 -- 172.21.15.12:0/738757465 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6503 (secure 0 0 0) 0x7fabdc0ee7b0 con 0x7fabf4146690 2023-11-28T21:44:08.239 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:08.239 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5727,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:43:04.550912+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5568":{"gid":5568,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":27,"addr":"172.21.15.12:6838/291731973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":291731973},{"type":"v1","addr":"172.21.15.12:6838","nonce":291731973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:08.239 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 38 2023-11-28T21:44:08.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 -- 172.21.15.12:0/738757465 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fabd4066470 msgr2=0x7fabd4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:08.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fabd4066470 0x7fabd4068930 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fabf40501f0 tx=0x7fabe4004840 comp rx=0 tx=0).stop 2023-11-28T21:44:08.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 -- 172.21.15.12:0/738757465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf4146690 msgr2=0x7fabf40c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:08.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf4146690 0x7fabf40c68b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b0040 tx=0x7fabdc0197f0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 -- 172.21.15.12:0/738757465 shutdown_connections 2023-11-28T21:44:08.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fabd4066470 0x7fabd4068930 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf41f1670 0x7fabf41f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fabf4146690 0x7fabf40c68b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 --2- 172.21.15.12:0/738757465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe00047d0 0x7fabf41452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:08.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 -- 172.21.15.12:0/738757465 >> 172.21.15.12:0/738757465 conn(0x7fabf4066360 msgr2=0x7fabf4152310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:08.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 -- 172.21.15.12:0/738757465 shutdown_connections 2023-11-28T21:44:08.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:08.235+0000 7fabf95b8640 1 -- 172.21.15.12:0/738757465 wait complete. 2023-11-28T21:44:08.259 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:44:10.260 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:10.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f744c65f640 1 Processor -- start 2023-11-28T21:44:10.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f744c65f640 1 -- start start 2023-11-28T21:44:10.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f744c65f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f744406e300 0x7f7444156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f744c65f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7444067410 con 0x7f744406cf40 2023-11-28T21:44:10.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f744c65f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744406e090 con 0x7f7444150c20 2023-11-28T21:44:10.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f744c65f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7444157f30 con 0x7f744406e300 2023-11-28T21:44:10.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f7449bfa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f744406e300 0x7f7444156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f7449bfa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f744406e300 0x7f7444156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35522/0 (socket says 172.21.15.12:35522) 2023-11-28T21:44:10.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f7449bfa640 1 -- 172.21.15.12:0/3126365373 learned_addr learned my addr 172.21.15.12:0/3126365373 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:10.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f744abfc640 1 --1- 172.21.15.12:0/3126365373 >> v1:172.21.15.12:6789/0 conn(0x7f744406cf40 0x7f7444158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43540/0 (socket says 172.21.15.12:43540) 2023-11-28T21:44:10.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f7449bfa640 1 -- 172.21.15.12:0/3126365373 >> v1:172.21.15.144:6789/0 conn(0x7f7444150c20 legacy=0x7f74440c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f7449bfa640 1 -- 172.21.15.12:0/3126365373 >> v1:172.21.15.12:6789/0 conn(0x7f744406cf40 legacy=0x7f7444158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f74493f9640 1 -- 172.21.15.12:0/3126365373 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825229591 0 0) 0x7f7444067410 con 0x7f744406cf40 2023-11-28T21:44:10.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.411+0000 7f7449bfa640 1 -- 172.21.15.12:0/3126365373 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7444151000 con 0x7f744406e300 2023-11-28T21:44:10.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f7449bfa640 1 --2- 172.21.15.12:0/3126365373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406e300 0x7f7444156910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7434009870 tx=0x7f743401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d6359d415bd49a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:10.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f74493f9640 1 -- 172.21.15.12:0/3126365373 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340ab650 con 0x7f744406e300 2023-11-28T21:44:10.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f74493f9640 1 -- 172.21.15.12:0/3126365373 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74340ab7b0 con 0x7f744406e300 2023-11-28T21:44:10.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f74493f9640 1 -- 172.21.15.12:0/3126365373 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74340abb40 con 0x7f744406e300 2023-11-28T21:44:10.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f744c65f640 1 -- 172.21.15.12:0/3126365373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406e300 msgr2=0x7f7444156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f744c65f640 1 --2- 172.21.15.12:0/3126365373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406e300 0x7f7444156910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7434009870 tx=0x7f743401c540 comp rx=0 tx=0).stop 2023-11-28T21:44:10.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f744c65f640 1 -- 172.21.15.12:0/3126365373 shutdown_connections 2023-11-28T21:44:10.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f744c65f640 1 --2- 172.21.15.12:0/3126365373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406e300 0x7f7444156910 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f744c65f640 1 -- 172.21.15.12:0/3126365373 >> 172.21.15.12:0/3126365373 conn(0x7f7444066360 msgr2=0x7f7444066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:10.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f744c65f640 1 -- 172.21.15.12:0/3126365373 shutdown_connections 2023-11-28T21:44:10.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.415+0000 7f744c65f640 1 -- 172.21.15.12:0/3126365373 wait complete. 2023-11-28T21:44:10.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 Processor -- start 2023-11-28T21:44:10.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 -- start start 2023-11-28T21:44:10.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406cf40 0x7f74440c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f744406e300 0x7f74440c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7444150c20 0x7f7444148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744414db40 con 0x7f744406cf40 2023-11-28T21:44:10.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74441556b0 con 0x7f7444150c20 2023-11-28T21:44:10.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7444155c30 con 0x7f744406e300 2023-11-28T21:44:10.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744a3fb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406cf40 0x7f74440c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744a3fb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406cf40 0x7f74440c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35532/0 (socket says 172.21.15.12:35532) 2023-11-28T21:44:10.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744a3fb640 1 -- 172.21.15.12:0/926485223 learned_addr learned my addr 172.21.15.12:0/926485223 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f7449bfa640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f744406e300 0x7f74440c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744abfc640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7444150c20 0x7f7444148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744a3fb640 1 -- 172.21.15.12:0/926485223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f744406e300 msgr2=0x7f74440c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744a3fb640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f744406e300 0x7f74440c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744a3fb640 1 -- 172.21.15.12:0/926485223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7444150c20 msgr2=0x7f7444148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744a3fb640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7444150c20 0x7f7444148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744a3fb640 1 -- 172.21.15.12:0/926485223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74340094d0 con 0x7f744406cf40 2023-11-28T21:44:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744a3fb640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406cf40 0x7f74440c30b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f744000c3c0 tx=0x7f744000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f743affd640 1 -- 172.21.15.12:0/926485223 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400052d0 con 0x7f744406cf40 2023-11-28T21:44:10.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f743affd640 1 -- 172.21.15.12:0/926485223 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7440005430 con 0x7f744406cf40 2023-11-28T21:44:10.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f743affd640 1 -- 172.21.15.12:0/926485223 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440003840 con 0x7f744406cf40 2023-11-28T21:44:10.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 -- 172.21.15.12:0/926485223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744414cfc0 con 0x7f744406cf40 2023-11-28T21:44:10.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 -- 172.21.15.12:0/926485223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74440712c0 con 0x7f744406cf40 2023-11-28T21:44:10.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.419+0000 7f744c65f640 1 -- 172.21.15.12:0/926485223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7444074f60 con 0x7f744406cf40 2023-11-28T21:44:10.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.423+0000 7f743affd640 1 -- 172.21.15.12:0/926485223 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f74400039a0 con 0x7f744406cf40 2023-11-28T21:44:10.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.423+0000 7f743affd640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f74280664a0 0x7f7428068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.423+0000 7f743affd640 1 -- 172.21.15.12:0/926485223 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f74400838f0 con 0x7f744406cf40 2023-11-28T21:44:10.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.423+0000 7f7449bfa640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f74280664a0 0x7f7428068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.423+0000 7f7449bfa640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f74280664a0 0x7f7428068960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f74340af000 tx=0x7f74340b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:10.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.427+0000 7f743affd640 1 -- 172.21.15.12:0/926485223 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7440050850 con 0x7f744406cf40 2023-11-28T21:44:10.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.615+0000 7f744c65f640 1 -- 172.21.15.12:0/926485223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74440c2c30 con 0x7f744406cf40 2023-11-28T21:44:10.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.615+0000 7f743affd640 1 -- 172.21.15.12:0/926485223 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f7440053e70 con 0x7f744406cf40 2023-11-28T21:44:10.622 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:10.622 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:08.460495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:10.622 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2023-11-28T21:44:10.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 -- 172.21.15.12:0/926485223 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f74280664a0 msgr2=0x7f7428068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f74280664a0 0x7f7428068960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f74340af000 tx=0x7f74340b6040 comp rx=0 tx=0).stop 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 -- 172.21.15.12:0/926485223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406cf40 msgr2=0x7f74440c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406cf40 0x7f74440c30b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f744000c3c0 tx=0x7f744000c880 comp rx=0 tx=0).stop 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 -- 172.21.15.12:0/926485223 shutdown_connections 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f74280664a0 0x7f7428068960 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7444150c20 0x7f7444148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f744406e300 0x7f74440c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 --2- 172.21.15.12:0/926485223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f744406cf40 0x7f74440c30b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 -- 172.21.15.12:0/926485223 >> 172.21.15.12:0/926485223 conn(0x7f7444066360 msgr2=0x7f744406b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 -- 172.21.15.12:0/926485223 shutdown_connections 2023-11-28T21:44:10.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.619+0000 7f744c65f640 1 -- 172.21.15.12:0/926485223 wait complete. 2023-11-28T21:44:10.643 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2023-11-28T21:44:10.643 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:44:10.643 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:10.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab5b84640 1 Processor -- start 2023-11-28T21:44:10.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab5b84640 1 -- start start 2023-11-28T21:44:10.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab5b84640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feab00c6f20 0x7feab0156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab5b84640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 0x7feab0158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab5b84640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab006b7c0 con 0x7feab00c7300 2023-11-28T21:44:10.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab5b84640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab006a8f0 con 0x7feab00c6f20 2023-11-28T21:44:10.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab5b84640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab006abb0 con 0x7feab00c26b0 2023-11-28T21:44:10.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab5383640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7feab00c7300 0x7feab0150840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43548/0 (socket says 172.21.15.12:43548) 2023-11-28T21:44:10.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab5383640 1 -- 172.21.15.12:0/1036922540 learned_addr learned my addr 172.21.15.12:0/1036922540 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:44:10.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feaaffff640 1 --2- 172.21.15.12:0/1036922540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 0x7feab0158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab4b82640 1 --2- 172.21.15.12:0/1036922540 >> v2:172.21.15.144:3300/0 conn(0x7feab00c6f20 0x7feab0156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feaaf7fe640 1 -- 172.21.15.12:0/1036922540 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880754608 0 0) 0x7feab006b7c0 con 0x7feab00c7300 2023-11-28T21:44:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feaaf7fe640 1 -- 172.21.15.12:0/1036922540 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa0003530 con 0x7feab00c7300 2023-11-28T21:44:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feaaffff640 1 -- 172.21.15.12:0/1036922540 >> v2:172.21.15.144:3300/0 conn(0x7feab00c6f20 msgr2=0x7feab0156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feaaffff640 1 --2- 172.21.15.12:0/1036922540 >> v2:172.21.15.144:3300/0 conn(0x7feab00c6f20 0x7feab0156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feaaf7fe640 1 -- 172.21.15.12:0/1036922540 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1721090674 0 0) 0x7feaa0003530 con 0x7feab00c7300 2023-11-28T21:44:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feaaffff640 1 -- 172.21.15.12:0/1036922540 >> v1:172.21.15.12:6789/0 conn(0x7feab00c7300 legacy=0x7feab0150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feaaffff640 1 -- 172.21.15.12:0/1036922540 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab006ae90 con 0x7feab00c26b0 2023-11-28T21:44:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.783+0000 7feab4b82640 1 --2- 172.21.15.12:0/1036922540 >> v2:172.21.15.144:3300/0 conn(0x7feab00c6f20 0x7feab0156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:44:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feaaffff640 1 --2- 172.21.15.12:0/1036922540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 0x7feab0158170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0097c0 tx=0x7fea9c01c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea452d891e675d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feaaf7fe640 1 -- 172.21.15.12:0/1036922540 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8001a00 con 0x7feab00c26b0 2023-11-28T21:44:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feaaf7fe640 1 -- 172.21.15.12:0/1036922540 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9c0a7050 con 0x7feab00c26b0 2023-11-28T21:44:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feaaf7fe640 1 -- 172.21.15.12:0/1036922540 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0ab600 con 0x7feab00c26b0 2023-11-28T21:44:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feab5b84640 1 -- 172.21.15.12:0/1036922540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 msgr2=0x7feab0158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feab5b84640 1 --2- 172.21.15.12:0/1036922540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 0x7feab0158170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0097c0 tx=0x7fea9c01c3a0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feab5b84640 1 -- 172.21.15.12:0/1036922540 shutdown_connections 2023-11-28T21:44:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feab5b84640 1 --2- 172.21.15.12:0/1036922540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 0x7feab0158170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feab5b84640 1 --2- 172.21.15.12:0/1036922540 >> v2:172.21.15.144:3300/0 conn(0x7feab00c6f20 0x7feab0156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feab5b84640 1 -- 172.21.15.12:0/1036922540 >> 172.21.15.12:0/1036922540 conn(0x7feab0066360 msgr2=0x7feab0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feab5b84640 1 -- 172.21.15.12:0/1036922540 shutdown_connections 2023-11-28T21:44:10.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.787+0000 7feab5b84640 1 -- 172.21.15.12:0/1036922540 wait complete. 2023-11-28T21:44:10.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 Processor -- start 2023-11-28T21:44:10.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 -- start start 2023-11-28T21:44:10.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 0x7feab0072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab00c6f20 0x7feab01f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab00c7300 0x7feab0146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0071010 con 0x7feab00c7300 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0163640 con 0x7feab00c6f20 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab0163920 con 0x7feab00c26b0 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5383640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab00c7300 0x7feab0146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5383640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab00c7300 0x7feab0146970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35540/0 (socket says 172.21.15.12:35540) 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5383640 1 -- 172.21.15.12:0/91957484 learned_addr learned my addr 172.21.15.12:0/91957484 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feaaffff640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab00c6f20 0x7feab01f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab4b82640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 0x7feab0072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5383640 1 -- 172.21.15.12:0/91957484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 msgr2=0x7feab0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5383640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 0x7feab0072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5383640 1 -- 172.21.15.12:0/91957484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab00c6f20 msgr2=0x7feab01f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5383640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab00c6f20 0x7feab01f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5383640 1 -- 172.21.15.12:0/91957484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa4009420 con 0x7feab00c7300 2023-11-28T21:44:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5383640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab00c7300 0x7feab0146970 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7feaa800cb40 tx=0x7feaa800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feaad7fa640 1 -- 172.21.15.12:0/91957484 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa800aca0 con 0x7feab00c7300 2023-11-28T21:44:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feaad7fa640 1 -- 172.21.15.12:0/91957484 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa800ae20 con 0x7feab00c7300 2023-11-28T21:44:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 -- 172.21.15.12:0/91957484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c009420 con 0x7feab00c7300 2023-11-28T21:44:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feaad7fa640 1 -- 172.21.15.12:0/91957484 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8012640 con 0x7feab00c7300 2023-11-28T21:44:10.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 -- 172.21.15.12:0/91957484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab014b120 con 0x7feab00c7300 2023-11-28T21:44:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.791+0000 7feab5b84640 1 -- 172.21.15.12:0/91957484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab0067c70 con 0x7feab00c7300 2023-11-28T21:44:10.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.795+0000 7feaad7fa640 1 -- 172.21.15.12:0/91957484 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7feaa8007070 con 0x7feab00c7300 2023-11-28T21:44:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.795+0000 7feaad7fa640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea94066470 0x7fea94068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.795+0000 7feaad7fa640 1 -- 172.21.15.12:0/91957484 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7feaa8084500 con 0x7feab00c7300 2023-11-28T21:44:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.795+0000 7feab4b82640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea94066470 0x7fea94068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:10.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.795+0000 7feab4b82640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea94066470 0x7fea94068930 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009c80 tx=0x7feaa4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:10.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.799+0000 7feaad7fa640 1 -- 172.21.15.12:0/91957484 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feaa8051340 con 0x7feab00c7300 2023-11-28T21:44:10.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.991+0000 7feab5b84640 1 -- 172.21.15.12:0/91957484 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feab0074f60 con 0x7feab00c7300 2023-11-28T21:44:10.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feaad7fa640 1 -- 172.21.15.12:0/91957484 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5714 (secure 0 0 0) 0x7feaa8017090 con 0x7feab00c7300 2023-11-28T21:44:10.998 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:10.998 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:08.460495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:10.999 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2023-11-28T21:44:11.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 -- 172.21.15.12:0/91957484 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea94066470 msgr2=0x7fea94068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:11.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea94066470 0x7fea94068930 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009c80 tx=0x7feaa4008040 comp rx=0 tx=0).stop 2023-11-28T21:44:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 -- 172.21.15.12:0/91957484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab00c7300 msgr2=0x7feab0146970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab00c7300 0x7feab0146970 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7feaa800cb40 tx=0x7feaa800a060 comp rx=0 tx=0).stop 2023-11-28T21:44:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 -- 172.21.15.12:0/91957484 shutdown_connections 2023-11-28T21:44:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea94066470 0x7fea94068930 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:11.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feab00c7300 0x7feab0146970 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:11.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab00c6f20 0x7feab01f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:11.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 --2- 172.21.15.12:0/91957484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00c26b0 0x7feab0072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:11.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 -- 172.21.15.12:0/91957484 >> 172.21.15.12:0/91957484 conn(0x7feab0066360 msgr2=0x7feab014e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:11.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 -- 172.21.15.12:0/91957484 shutdown_connections 2023-11-28T21:44:11.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:10.995+0000 7feab5b84640 1 -- 172.21.15.12:0/91957484 wait complete. 2023-11-28T21:44:11.020 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 39, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 39, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5727": { "addr": "172.21.15.12:6839/3299030262", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3299030262, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3299030262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5727, "incarnation": 39, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 46, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:44:08.460495+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5727 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:44:13.022 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:13.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.171+0000 7f4a07432640 1 Processor -- start 2023-11-28T21:44:13.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a07432640 1 -- start start 2023-11-28T21:44:13.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a07432640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a00151af0 0x7f4a000c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:13.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a07432640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0006e300 0x7f4a00072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:13.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a07432640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a00157b70 con 0x7f4a00151ed0 2023-11-28T21:44:13.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a07432640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a000707f0 con 0x7f4a00151af0 2023-11-28T21:44:13.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a07432640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c7b90 con 0x7f4a0006e300 2023-11-28T21:44:13.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a051ce640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4a00151af0 0x7f4a000c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:13.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a049cd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4a00151ed0 0x7f4a000c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43556/0 (socket says 172.21.15.12:43556) 2023-11-28T21:44:13.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a049cd640 1 -- 172.21.15.12:0/1566119707 learned_addr learned my addr 172.21.15.12:0/1566119707 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:44:13.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a059cf640 1 --2- 172.21.15.12:0/1566119707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0006e300 0x7f4a00072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:13.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a051ce640 1 -- 172.21.15.12:0/1566119707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0006e300 msgr2=0x7f4a00072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:13.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a051ce640 1 --2- 172.21.15.12:0/1566119707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0006e300 0x7f4a00072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:13.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a051ce640 1 -- 172.21.15.12:0/1566119707 >> v1:172.21.15.12:6789/0 conn(0x7f4a00151ed0 legacy=0x7f4a000c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:13.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f49f7fff640 1 -- 172.21.15.12:0/1566119707 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689904186 0 0) 0x7f4a00157b70 con 0x7f4a00151ed0 2023-11-28T21:44:13.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.175+0000 7f4a051ce640 1 -- 172.21.15.12:0/1566119707 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00157e50 con 0x7f4a00151af0 2023-11-28T21:44:13.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a051ce640 1 --2- 172.21.15.12:0/1566119707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a00151af0 0x7f4a000c7650 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0097c0 tx=0x7f49fc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85cc45dc34fa779 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:13.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f49f7fff640 1 -- 172.21.15.12:0/1566119707 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0ab600 con 0x7f4a00151af0 2023-11-28T21:44:13.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f49f7fff640 1 -- 172.21.15.12:0/1566119707 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc0a7050 con 0x7f4a00151af0 2023-11-28T21:44:13.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f49f7fff640 1 -- 172.21.15.12:0/1566119707 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0ab980 con 0x7f4a00151af0 2023-11-28T21:44:13.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a07432640 1 -- 172.21.15.12:0/1566119707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a00151af0 msgr2=0x7f4a000c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:13.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a07432640 1 --2- 172.21.15.12:0/1566119707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a00151af0 0x7f4a000c7650 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0097c0 tx=0x7f49fc01c480 comp rx=0 tx=0).stop 2023-11-28T21:44:13.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a07432640 1 -- 172.21.15.12:0/1566119707 shutdown_connections 2023-11-28T21:44:13.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a07432640 1 --2- 172.21.15.12:0/1566119707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0006e300 0x7f4a00072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:13.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a07432640 1 --2- 172.21.15.12:0/1566119707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a00151af0 0x7f4a000c7650 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a07432640 1 -- 172.21.15.12:0/1566119707 >> 172.21.15.12:0/1566119707 conn(0x7f4a00066360 msgr2=0x7f4a00066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a07432640 1 -- 172.21.15.12:0/1566119707 shutdown_connections 2023-11-28T21:44:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a07432640 1 -- 172.21.15.12:0/1566119707 wait complete. 2023-11-28T21:44:13.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a07432640 1 Processor -- start 2023-11-28T21:44:13.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.179+0000 7f4a07432640 1 -- start start 2023-11-28T21:44:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a07432640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0006e300 0x7f4a00139440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a07432640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a00151af0 0x7f4a00139980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a07432640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a0014eae0 0x7f4a001475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a07432640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0013fda0 con 0x7f4a0014eae0 2023-11-28T21:44:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a07432640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00140320 con 0x7f4a00151af0 2023-11-28T21:44:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a07432640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001f7290 con 0x7f4a0006e300 2023-11-28T21:44:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a059cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a0014eae0 0x7f4a001475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a059cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a0014eae0 0x7f4a001475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35552/0 (socket says 172.21.15.12:35552) 2023-11-28T21:44:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a059cf640 1 -- 172.21.15.12:0/3300489250 learned_addr learned my addr 172.21.15.12:0/3300489250 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:13.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a051ce640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0006e300 0x7f4a00139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a049cd640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a00151af0 0x7f4a00139980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a059cf640 1 -- 172.21.15.12:0/3300489250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0006e300 msgr2=0x7f4a00139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a059cf640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0006e300 0x7f4a00139440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a059cf640 1 -- 172.21.15.12:0/3300489250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a00151af0 msgr2=0x7f4a00139980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a059cf640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a00151af0 0x7f4a00139980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a059cf640 1 -- 172.21.15.12:0/3300489250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc009420 con 0x7f4a0014eae0 2023-11-28T21:44:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a059cf640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a0014eae0 0x7f4a001475a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f49f8007e30 tx=0x7f49f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:13.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f49f5ffb640 1 -- 172.21.15.12:0/3300489250 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8014620 con 0x7f4a0014eae0 2023-11-28T21:44:13.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f49f5ffb640 1 -- 172.21.15.12:0/3300489250 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f8014780 con 0x7f4a0014eae0 2023-11-28T21:44:13.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f49f5ffb640 1 -- 172.21.15.12:0/3300489250 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8014b30 con 0x7f4a0014eae0 2023-11-28T21:44:13.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a07432640 1 -- 172.21.15.12:0/3300489250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0013e460 con 0x7f4a0014eae0 2023-11-28T21:44:13.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a07432640 1 -- 172.21.15.12:0/3300489250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0013ffb0 con 0x7f4a0014eae0 2023-11-28T21:44:13.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.183+0000 7f4a07432640 1 -- 172.21.15.12:0/3300489250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a00140170 con 0x7f4a0014eae0 2023-11-28T21:44:13.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.187+0000 7f49f5ffb640 1 -- 172.21.15.12:0/3300489250 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f49f8014c90 con 0x7f4a0014eae0 2023-11-28T21:44:13.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.187+0000 7f49f5ffb640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f49e40664a0 0x7f49e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:13.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.187+0000 7f49f5ffb640 1 -- 172.21.15.12:0/3300489250 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f49f8084c60 con 0x7f4a0014eae0 2023-11-28T21:44:13.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.187+0000 7f4a051ce640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f49e40664a0 0x7f49e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:13.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.187+0000 7f4a051ce640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f49e40664a0 0x7f49e4068960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ad000 tx=0x7f49fc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:13.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.191+0000 7f49f5ffb640 1 -- 172.21.15.12:0/3300489250 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f49f8051b30 con 0x7f4a0014eae0 2023-11-28T21:44:13.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.383+0000 7f4a07432640 1 -- 172.21.15.12:0/3300489250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a00139160 con 0x7f4a0014eae0 2023-11-28T21:44:13.388 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.383+0000 7f49f5ffb640 1 -- 172.21.15.12:0/3300489250 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f4a00139160 con 0x7f4a0014eae0 2023-11-28T21:44:13.388 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:13.389 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:08.460495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:13.389 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2023-11-28T21:44:13.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 -- 172.21.15.12:0/3300489250 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f49e40664a0 msgr2=0x7f49e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:13.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f49e40664a0 0x7f49e4068960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ad000 tx=0x7f49fc0a6000 comp rx=0 tx=0).stop 2023-11-28T21:44:13.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 -- 172.21.15.12:0/3300489250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a0014eae0 msgr2=0x7f4a001475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:13.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a0014eae0 0x7f4a001475a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f49f8007e30 tx=0x7f49f800e4b0 comp rx=0 tx=0).stop 2023-11-28T21:44:13.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 -- 172.21.15.12:0/3300489250 shutdown_connections 2023-11-28T21:44:13.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f49e40664a0 0x7f49e4068960 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:13.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a0014eae0 0x7f4a001475a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:13.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a00151af0 0x7f4a00139980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:13.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 --2- 172.21.15.12:0/3300489250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a0006e300 0x7f4a00139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:13.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 -- 172.21.15.12:0/3300489250 >> 172.21.15.12:0/3300489250 conn(0x7f4a00066360 msgr2=0x7f4a000c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:13.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 -- 172.21.15.12:0/3300489250 shutdown_connections 2023-11-28T21:44:13.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:13.387+0000 7f4a07432640 1 -- 172.21.15.12:0/3300489250 wait complete. 2023-11-28T21:44:13.410 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:44:13.965 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:44:13.969 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:44:15.411 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:15.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.551+0000 7fd52aff8640 1 Processor -- start 2023-11-28T21:44:15.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.551+0000 7fd52aff8640 1 -- start start 2023-11-28T21:44:15.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd52aff8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd524143070 0x7fd52415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:15.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd52aff8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd52415b760 0x7fd52406acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:15.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd52aff8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52406f050 con 0x7fd52415b340 2023-11-28T21:44:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd52aff8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52415bb70 con 0x7fd52415b760 2023-11-28T21:44:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd52aff8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52415ae50 con 0x7fd524143070 2023-11-28T21:44:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd529595640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd52415b760 0x7fd52406acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd529595640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd52415b760 0x7fd52406acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45642/0 (socket says 172.21.15.12:45642) 2023-11-28T21:44:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd529595640 1 -- 172.21.15.12:0/3885409185 learned_addr learned my addr 172.21.15.12:0/3885409185 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd528d94640 1 --2- 172.21.15.12:0/3885409185 >> v2:172.21.15.144:3300/0 conn(0x7fd524143070 0x7fd52415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd529595640 1 -- 172.21.15.12:0/3885409185 >> v2:172.21.15.144:3300/0 conn(0x7fd524143070 msgr2=0x7fd52415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd529595640 1 --2- 172.21.15.12:0/3885409185 >> v2:172.21.15.144:3300/0 conn(0x7fd524143070 0x7fd52415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd529595640 1 -- 172.21.15.12:0/3885409185 >> v1:172.21.15.144:6789/0 conn(0x7fd52415b340 legacy=0x7fd52406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.555+0000 7fd529595640 1 -- 172.21.15.12:0/3885409185 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52406f580 con 0x7fd52415b760 2023-11-28T21:44:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.559+0000 7fd529595640 1 --2- 172.21.15.12:0/3885409185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 0x7fd52406acc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd5240688a0 tx=0x7fd51c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7800399ecdc981d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.559+0000 7fd5237fe640 1 -- 172.21.15.12:0/3885409185 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0a5870 con 0x7fd52415b760 2023-11-28T21:44:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.559+0000 7fd5237fe640 1 -- 172.21.15.12:0/3885409185 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c0a59d0 con 0x7fd52415b760 2023-11-28T21:44:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.559+0000 7fd5237fe640 1 -- 172.21.15.12:0/3885409185 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0a5d60 con 0x7fd52415b760 2023-11-28T21:44:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.559+0000 7fd52aff8640 1 -- 172.21.15.12:0/3885409185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 msgr2=0x7fd52406acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.559+0000 7fd52aff8640 1 --2- 172.21.15.12:0/3885409185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 0x7fd52406acc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd5240688a0 tx=0x7fd51c01c9b0 comp rx=0 tx=0).stop 2023-11-28T21:44:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 -- 172.21.15.12:0/3885409185 shutdown_connections 2023-11-28T21:44:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 --2- 172.21.15.12:0/3885409185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 0x7fd52406acc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 --2- 172.21.15.12:0/3885409185 >> v2:172.21.15.144:3300/0 conn(0x7fd524143070 0x7fd52415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 -- 172.21.15.12:0/3885409185 >> 172.21.15.12:0/3885409185 conn(0x7fd52406b6c0 msgr2=0x7fd52406bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 -- 172.21.15.12:0/3885409185 shutdown_connections 2023-11-28T21:44:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 -- 172.21.15.12:0/3885409185 wait complete. 2023-11-28T21:44:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 Processor -- start 2023-11-28T21:44:15.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 -- start start 2023-11-28T21:44:15.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd524143070 0x7fd524133aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:15.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415b340 0x7fd52413c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 0x7fd52413c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd524156200 con 0x7fd52415b760 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52413cec0 con 0x7fd52415b340 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd52aff8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd524154700 con 0x7fd524143070 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd529595640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 0x7fd52413c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd529595640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 0x7fd52413c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45644/0 (socket says 172.21.15.12:45644) 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd529595640 1 -- 172.21.15.12:0/3822108406 learned_addr learned my addr 172.21.15.12:0/3822108406 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd528d94640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd524143070 0x7fd524133aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd523fff640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415b340 0x7fd52413c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd529595640 1 -- 172.21.15.12:0/3822108406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd524143070 msgr2=0x7fd524133aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:15.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd529595640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd524143070 0x7fd524133aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:15.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd529595640 1 -- 172.21.15.12:0/3822108406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415b340 msgr2=0x7fd52413c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:15.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd529595640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415b340 0x7fd52413c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:15.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd529595640 1 -- 172.21.15.12:0/3822108406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c009b30 con 0x7fd52415b760 2023-11-28T21:44:15.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.563+0000 7fd523fff640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415b340 0x7fd52413c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:44:15.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.567+0000 7fd529595640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 0x7fd52413c790 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0af040 tx=0x7fd51c019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:15.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.567+0000 7fd5217fa640 1 -- 172.21.15.12:0/3822108406 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0b8670 con 0x7fd52415b760 2023-11-28T21:44:15.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.567+0000 7fd5217fa640 1 -- 172.21.15.12:0/3822108406 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c0b87d0 con 0x7fd52415b760 2023-11-28T21:44:15.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.567+0000 7fd52aff8640 1 -- 172.21.15.12:0/3822108406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd524133fe0 con 0x7fd52415b760 2023-11-28T21:44:15.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.567+0000 7fd5217fa640 1 -- 172.21.15.12:0/3822108406 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0b8b60 con 0x7fd52415b760 2023-11-28T21:44:15.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.567+0000 7fd52aff8640 1 -- 172.21.15.12:0/3822108406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd524154910 con 0x7fd52415b760 2023-11-28T21:44:15.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.567+0000 7fd52aff8640 1 -- 172.21.15.12:0/3822108406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5241555d0 con 0x7fd52415b760 2023-11-28T21:44:15.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.567+0000 7fd5217fa640 1 -- 172.21.15.12:0/3822108406 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd51c0b8cc0 con 0x7fd52415b760 2023-11-28T21:44:15.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.571+0000 7fd5217fa640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5080664a0 0x7fd508068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:15.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.571+0000 7fd5217fa640 1 -- 172.21.15.12:0/3822108406 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7fd51c11f780 con 0x7fd52415b760 2023-11-28T21:44:15.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.571+0000 7fd528d94640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5080664a0 0x7fd508068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:15.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.571+0000 7fd528d94640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5080664a0 0x7fd508068960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd518009b60 tx=0x7fd518009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:15.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.575+0000 7fd5217fa640 1 -- 172.21.15.12:0/3822108406 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd51c0ec8a0 con 0x7fd52415b760 2023-11-28T21:44:15.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 -- 172.21.15.12:0/3822108406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5241553f0 con 0x7fd52415b760 2023-11-28T21:44:15.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd5217fa640 1 -- 172.21.15.12:0/3822108406 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5714 (secure 0 0 0) 0x7fd5241553f0 con 0x7fd52415b760 2023-11-28T21:44:15.776 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:15.776 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:08.460495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:15.778 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 39 2023-11-28T21:44:15.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 -- 172.21.15.12:0/3822108406 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5080664a0 msgr2=0x7fd508068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:15.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5080664a0 0x7fd508068960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd518009b60 tx=0x7fd518009190 comp rx=0 tx=0).stop 2023-11-28T21:44:15.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 -- 172.21.15.12:0/3822108406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 msgr2=0x7fd52413c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:15.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 0x7fd52413c790 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0af040 tx=0x7fd51c019750 comp rx=0 tx=0).stop 2023-11-28T21:44:15.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 -- 172.21.15.12:0/3822108406 shutdown_connections 2023-11-28T21:44:15.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5080664a0 0x7fd508068960 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd52415b760 0x7fd52413c790 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd52415b340 0x7fd52413c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 --2- 172.21.15.12:0/3822108406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd524143070 0x7fd524133aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 -- 172.21.15.12:0/3822108406 >> 172.21.15.12:0/3822108406 conn(0x7fd52406b6c0 msgr2=0x7fd5240ca8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:15.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 -- 172.21.15.12:0/3822108406 shutdown_connections 2023-11-28T21:44:15.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:15.771+0000 7fd52aff8640 1 -- 172.21.15.12:0/3822108406 wait complete. 2023-11-28T21:44:15.797 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:44:17.799 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:17.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab21716640 1 Processor -- start 2023-11-28T21:44:17.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab21716640 1 -- start start 2023-11-28T21:44:17.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab21716640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fab1c0cd410 0x7fab1c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:17.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab21716640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1c0c7710 con 0x7fab1c153e00 2023-11-28T21:44:17.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab21716640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1c1541e0 con 0x7fab1c15d7f0 2023-11-28T21:44:17.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab21716640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1c15a010 con 0x7fab1c0cd410 2023-11-28T21:44:17.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab1affd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fab1c153e00 0x7fab1c06b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:43232/0 (socket says 172.21.15.12:43232) 2023-11-28T21:44:17.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab1affd640 1 -- 172.21.15.12:0/3716664410 learned_addr learned my addr 172.21.15.12:0/3716664410 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:44:17.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab13fff640 1 --2- 172.21.15.12:0/3716664410 >> v2:172.21.15.12:3300/0 conn(0x7fab1c0cd410 0x7fab1c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:17.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab1a7fc640 1 -- 172.21.15.12:0/3716664410 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000649141 0 0) 0x7fab1c0c7710 con 0x7fab1c153e00 2023-11-28T21:44:17.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab1a7fc640 1 -- 172.21.15.12:0/3716664410 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab08003530 con 0x7fab1c153e00 2023-11-28T21:44:17.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab13fff640 1 -- 172.21.15.12:0/3716664410 >> v1:172.21.15.144:6789/0 conn(0x7fab1c15d7f0 legacy=0x7fab1c15dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:17.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab13fff640 1 -- 172.21.15.12:0/3716664410 >> v1:172.21.15.12:6789/0 conn(0x7fab1c153e00 legacy=0x7fab1c06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:17.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.951+0000 7fab13fff640 1 -- 172.21.15.12:0/3716664410 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c15a2f0 con 0x7fab1c0cd410 2023-11-28T21:44:17.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab13fff640 1 --2- 172.21.15.12:0/3716664410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 0x7fab1c0cd8a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fab040097c0 tx=0x7fab0401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb5cdd0d4dfd084 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:17.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab1a7fc640 1 -- 172.21.15.12:0/3716664410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040a8970 con 0x7fab1c0cd410 2023-11-28T21:44:17.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab1a7fc640 1 -- 172.21.15.12:0/3716664410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab040a8ad0 con 0x7fab1c0cd410 2023-11-28T21:44:17.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab1a7fc640 1 -- 172.21.15.12:0/3716664410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040a8e40 con 0x7fab1c0cd410 2023-11-28T21:44:17.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab21716640 1 -- 172.21.15.12:0/3716664410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 msgr2=0x7fab1c0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:17.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab21716640 1 --2- 172.21.15.12:0/3716664410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 0x7fab1c0cd8a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fab040097c0 tx=0x7fab0401ec70 comp rx=0 tx=0).stop 2023-11-28T21:44:17.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab21716640 1 -- 172.21.15.12:0/3716664410 shutdown_connections 2023-11-28T21:44:17.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab21716640 1 --2- 172.21.15.12:0/3716664410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 0x7fab1c0cd8a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:17.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab21716640 1 -- 172.21.15.12:0/3716664410 >> 172.21.15.12:0/3716664410 conn(0x7fab1c066360 msgr2=0x7fab1c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:17.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab21716640 1 -- 172.21.15.12:0/3716664410 shutdown_connections 2023-11-28T21:44:17.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.955+0000 7fab21716640 1 -- 172.21.15.12:0/3716664410 wait complete. 2023-11-28T21:44:17.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 Processor -- start 2023-11-28T21:44:17.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 -- start start 2023-11-28T21:44:17.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 0x7fab1c0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:17.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab1c153e00 0x7fab1c148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c15d7f0 0x7fab1c149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c15d0f0 con 0x7fab1c0cd410 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c157d10 con 0x7fab1c15d7f0 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0cb390 con 0x7fab1c153e00 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab1affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 0x7fab1c0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab1affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 0x7fab1c0c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45664/0 (socket says 172.21.15.12:45664) 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab1affd640 1 -- 172.21.15.12:0/1497235337 learned_addr learned my addr 172.21.15.12:0/1497235337 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab13fff640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab1c153e00 0x7fab1c148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab1b7fe640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c15d7f0 0x7fab1c149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab1affd640 1 -- 172.21.15.12:0/1497235337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab1c153e00 msgr2=0x7fab1c148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab1affd640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab1c153e00 0x7fab1c148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:17.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab1affd640 1 -- 172.21.15.12:0/1497235337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c15d7f0 msgr2=0x7fab1c149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:17.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab1affd640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c15d7f0 0x7fab1c149130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:17.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab1affd640 1 -- 172.21.15.12:0/1497235337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab04009420 con 0x7fab1c0cd410 2023-11-28T21:44:17.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab1affd640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 0x7fab1c0c7e10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c500 tx=0x7fab0c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:17.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab137fe640 1 -- 172.21.15.12:0/1497235337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c005cf0 con 0x7fab1c0cd410 2023-11-28T21:44:17.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab137fe640 1 -- 172.21.15.12:0/1497235337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c005e50 con 0x7fab1c0cd410 2023-11-28T21:44:17.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 -- 172.21.15.12:0/1497235337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c15cb70 con 0x7fab1c0cd410 2023-11-28T21:44:17.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab137fe640 1 -- 172.21.15.12:0/1497235337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0072f0 con 0x7fab1c0cd410 2023-11-28T21:44:17.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 -- 172.21.15.12:0/1497235337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c15ce10 con 0x7fab1c0cd410 2023-11-28T21:44:17.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.959+0000 7fab21716640 1 -- 172.21.15.12:0/1497235337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c074f60 con 0x7fab1c0cd410 2023-11-28T21:44:17.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.963+0000 7fab137fe640 1 -- 172.21.15.12:0/1497235337 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fab0c007450 con 0x7fab1c0cd410 2023-11-28T21:44:17.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.963+0000 7fab137fe640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faafc0664a0 0x7faafc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:17.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.963+0000 7fab137fe640 1 -- 172.21.15.12:0/1497235337 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7fab0c083920 con 0x7fab1c0cd410 2023-11-28T21:44:17.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.963+0000 7fab13fff640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faafc0664a0 0x7faafc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:17.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.963+0000 7fab13fff640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faafc0664a0 0x7faafc068960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab040b2000 tx=0x7fab040aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:17.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:17.967+0000 7fab137fe640 1 -- 172.21.15.12:0/1497235337 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fab0c050930 con 0x7fab1c0cd410 2023-11-28T21:44:18.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.155+0000 7fab21716640 1 -- 172.21.15.12:0/1497235337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab1c073000 con 0x7fab1c0cd410 2023-11-28T21:44:18.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.155+0000 7fab137fe640 1 -- 172.21.15.12:0/1497235337 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5718 (secure 0 0 0) 0x7fab0c017090 con 0x7fab1c0cd410 2023-11-28T21:44:18.161 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:18.162 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:16.555726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:18.162 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 40 2023-11-28T21:44:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 -- 172.21.15.12:0/1497235337 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faafc0664a0 msgr2=0x7faafc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faafc0664a0 0x7faafc068960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab040b2000 tx=0x7fab040aa070 comp rx=0 tx=0).stop 2023-11-28T21:44:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 -- 172.21.15.12:0/1497235337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 msgr2=0x7fab1c0c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 0x7fab1c0c7e10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c500 tx=0x7fab0c00c9c0 comp rx=0 tx=0).stop 2023-11-28T21:44:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 -- 172.21.15.12:0/1497235337 shutdown_connections 2023-11-28T21:44:18.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faafc0664a0 0x7faafc068960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1c15d7f0 0x7fab1c149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab1c153e00 0x7fab1c148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 --2- 172.21.15.12:0/1497235337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab1c0cd410 0x7fab1c0c7e10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 -- 172.21.15.12:0/1497235337 >> 172.21.15.12:0/1497235337 conn(0x7fab1c066360 msgr2=0x7fab1c0c6270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 -- 172.21.15.12:0/1497235337 shutdown_connections 2023-11-28T21:44:18.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:18.159+0000 7fab21716640 1 -- 172.21.15.12:0/1497235337 wait complete. 2023-11-28T21:44:18.184 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:44:20.185 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:20.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68d4e5a640 1 Processor -- start 2023-11-28T21:44:20.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68d4e5a640 1 -- start start 2023-11-28T21:44:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68d4e5a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68d0151af0 0x7f68d00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68d4e5a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f68d0151ed0 0x7f68d00c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68d4e5a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d006e300 0x7f68d00c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68d4e5a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d00707f0 con 0x7f68d0151af0 2023-11-28T21:44:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68d4e5a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d00c7b90 con 0x7f68d0151ed0 2023-11-28T21:44:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68d4e5a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d0071c20 con 0x7f68d006e300 2023-11-28T21:44:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68ce59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68d0151af0 0x7f68d00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68ce59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f68d0151af0 0x7f68d00c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45674/0 (socket says 172.21.15.12:45674) 2023-11-28T21:44:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.335+0000 7f68ce59c640 1 -- 172.21.15.12:0/2441302726 learned_addr learned my addr 172.21.15.12:0/2441302726 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68ced9d640 1 --2- 172.21.15.12:0/2441302726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d006e300 0x7f68d00c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68c7fff640 1 --2- 172.21.15.12:0/2441302726 >> v2:172.21.15.144:3300/0 conn(0x7f68d0151ed0 0x7f68d00c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68ce59c640 1 -- 172.21.15.12:0/2441302726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d006e300 msgr2=0x7f68d00c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68ce59c640 1 --2- 172.21.15.12:0/2441302726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d006e300 0x7f68d00c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68ce59c640 1 -- 172.21.15.12:0/2441302726 >> v2:172.21.15.144:3300/0 conn(0x7f68d0151ed0 msgr2=0x7f68d00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68ce59c640 1 --2- 172.21.15.12:0/2441302726 >> v2:172.21.15.144:3300/0 conn(0x7f68d0151ed0 0x7f68d00c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68ce59c640 1 -- 172.21.15.12:0/2441302726 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d0157b70 con 0x7f68d0151af0 2023-11-28T21:44:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68ce59c640 1 --2- 172.21.15.12:0/2441302726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 0x7f68d00c7650 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f68d00c8e60 tx=0x7f68c001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda5e65476d545c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68cdd9b640 1 -- 172.21.15.12:0/2441302726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c00ac410 con 0x7f68d0151af0 2023-11-28T21:44:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68cdd9b640 1 -- 172.21.15.12:0/2441302726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c00ac570 con 0x7f68d0151af0 2023-11-28T21:44:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.339+0000 7f68cdd9b640 1 -- 172.21.15.12:0/2441302726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c00ac900 con 0x7f68d0151af0 2023-11-28T21:44:20.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 -- 172.21.15.12:0/2441302726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 msgr2=0x7f68d00c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:20.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 --2- 172.21.15.12:0/2441302726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 0x7f68d00c7650 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f68d00c8e60 tx=0x7f68c001c500 comp rx=0 tx=0).stop 2023-11-28T21:44:20.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 -- 172.21.15.12:0/2441302726 shutdown_connections 2023-11-28T21:44:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 --2- 172.21.15.12:0/2441302726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d006e300 0x7f68d00c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 --2- 172.21.15.12:0/2441302726 >> v2:172.21.15.144:3300/0 conn(0x7f68d0151ed0 0x7f68d00c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 --2- 172.21.15.12:0/2441302726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 0x7f68d00c7650 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 -- 172.21.15.12:0/2441302726 >> 172.21.15.12:0/2441302726 conn(0x7f68d0066360 msgr2=0x7f68d0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 -- 172.21.15.12:0/2441302726 shutdown_connections 2023-11-28T21:44:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 -- 172.21.15.12:0/2441302726 wait complete. 2023-11-28T21:44:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 Processor -- start 2023-11-28T21:44:20.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 -- start start 2023-11-28T21:44:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d006e300 0x7f68d00ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 0x7f68d00ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d0155040 0x7f68d01f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d0153040 con 0x7f68d0151af0 2023-11-28T21:44:20.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d00caef0 con 0x7f68d006e300 2023-11-28T21:44:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68d4e5a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d00c9160 con 0x7f68d0155040 2023-11-28T21:44:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 0x7f68d00ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 0x7f68d00ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45678/0 (socket says 172.21.15.12:45678) 2023-11-28T21:44:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68c7fff640 1 -- 172.21.15.12:0/689795350 learned_addr learned my addr 172.21.15.12:0/689795350 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68ced9d640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d0155040 0x7f68d01f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:20.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68ce59c640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d006e300 0x7f68d00ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68c7fff640 1 -- 172.21.15.12:0/689795350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d0155040 msgr2=0x7f68d01f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68c7fff640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d0155040 0x7f68d01f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68c7fff640 1 -- 172.21.15.12:0/689795350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d006e300 msgr2=0x7f68d00ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:20.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68c7fff640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d006e300 0x7f68d00ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.343+0000 7f68c7fff640 1 -- 172.21.15.12:0/689795350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c00094d0 con 0x7f68d0151af0 2023-11-28T21:44:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68c7fff640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 0x7f68d00ca6e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f68b800d5c0 tx=0x7f68b800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68c6ffd640 1 -- 172.21.15.12:0/689795350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8004240 con 0x7f68d0151af0 2023-11-28T21:44:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68c6ffd640 1 -- 172.21.15.12:0/689795350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b80043a0 con 0x7f68d0151af0 2023-11-28T21:44:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68c6ffd640 1 -- 172.21.15.12:0/689795350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8003840 con 0x7f68d0151af0 2023-11-28T21:44:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68d4e5a640 1 -- 172.21.15.12:0/689795350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d01577a0 con 0x7f68d0151af0 2023-11-28T21:44:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68d4e5a640 1 -- 172.21.15.12:0/689795350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d00c96c0 con 0x7f68d0151af0 2023-11-28T21:44:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68d4e5a640 1 -- 172.21.15.12:0/689795350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d0074f60 con 0x7f68d0151af0 2023-11-28T21:44:20.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68c6ffd640 1 -- 172.21.15.12:0/689795350 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f68b80039a0 con 0x7f68d0151af0 2023-11-28T21:44:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68c6ffd640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f68b0066470 0x7f68b0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68c6ffd640 1 -- 172.21.15.12:0/689795350 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f68b80844b0 con 0x7f68d0151af0 2023-11-28T21:44:20.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.347+0000 7f68ce59c640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f68b0066470 0x7f68b0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:20.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.351+0000 7f68ce59c640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f68b0066470 0x7f68b0068930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f68c00ae000 tx=0x7f68c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:20.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.351+0000 7f68c6ffd640 1 -- 172.21.15.12:0/689795350 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f68b80513a0 con 0x7f68d0151af0 2023-11-28T21:44:20.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.543+0000 7f68d4e5a640 1 -- 172.21.15.12:0/689795350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68d0072c10 con 0x7f68d0151af0 2023-11-28T21:44:20.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.543+0000 7f68c6ffd640 1 -- 172.21.15.12:0/689795350 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5718 (secure 0 0 0) 0x7f68b80549c0 con 0x7f68d0151af0 2023-11-28T21:44:20.548 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:20.549 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:16.555726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:20.549 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 40 2023-11-28T21:44:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 -- 172.21.15.12:0/689795350 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f68b0066470 msgr2=0x7f68b0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f68b0066470 0x7f68b0068930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f68c00ae000 tx=0x7f68c0009040 comp rx=0 tx=0).stop 2023-11-28T21:44:20.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 -- 172.21.15.12:0/689795350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 msgr2=0x7f68d00ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 0x7f68d00ca6e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f68b800d5c0 tx=0x7f68b800da80 comp rx=0 tx=0).stop 2023-11-28T21:44:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 -- 172.21.15.12:0/689795350 shutdown_connections 2023-11-28T21:44:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f68b0066470 0x7f68b0068930 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68d0155040 0x7f68d01f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68d0151af0 0x7f68d00ca6e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 --2- 172.21.15.12:0/689795350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68d006e300 0x7f68d00ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 -- 172.21.15.12:0/689795350 >> 172.21.15.12:0/689795350 conn(0x7f68d0066360 msgr2=0x7f68d014f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 -- 172.21.15.12:0/689795350 shutdown_connections 2023-11-28T21:44:20.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:20.547+0000 7f68d4e5a640 1 -- 172.21.15.12:0/689795350 wait complete. 2023-11-28T21:44:20.570 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:44:22.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:22.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.723+0000 7fc66632a640 1 Processor -- start 2023-11-28T21:44:22.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.723+0000 7fc66632a640 1 -- start start 2023-11-28T21:44:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.723+0000 7fc66632a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc660154120 0x7fc6600ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.723+0000 7fc66632a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc6600cade0 0x7fc6600cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.723+0000 7fc66632a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66015d370 con 0x7fc6600cbba0 2023-11-28T21:44:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.723+0000 7fc66632a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6600b3040 con 0x7fc6600cade0 2023-11-28T21:44:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.723+0000 7fc66632a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc660154500 con 0x7fc660154120 2023-11-28T21:44:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.723+0000 7fc65f7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc6600cade0 0x7fc6600cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65f7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc6600cade0 0x7fc6600cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45694/0 (socket says 172.21.15.12:45694) 2023-11-28T21:44:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65f7fe640 1 -- 172.21.15.12:0/4113279372 learned_addr learned my addr 172.21.15.12:0/4113279372 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65ffff640 1 --2- 172.21.15.12:0/4113279372 >> v2:172.21.15.144:3300/0 conn(0x7fc660154120 0x7fc6600ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65f7fe640 1 -- 172.21.15.12:0/4113279372 >> v2:172.21.15.144:3300/0 conn(0x7fc660154120 msgr2=0x7fc6600ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65f7fe640 1 --2- 172.21.15.12:0/4113279372 >> v2:172.21.15.144:3300/0 conn(0x7fc660154120 0x7fc6600ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65f7fe640 1 -- 172.21.15.12:0/4113279372 >> v1:172.21.15.144:6789/0 conn(0x7fc6600cbba0 legacy=0x7fc66015a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65f7fe640 1 -- 172.21.15.12:0/4113279372 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660159d80 con 0x7fc6600cade0 2023-11-28T21:44:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65f7fe640 1 --2- 172.21.15.12:0/4113279372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6600cade0 0x7fc6600cc0c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc6600ca5c0 tx=0x7fc65401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464e21bacdd97e1f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65effd640 1 -- 172.21.15.12:0/4113279372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6540ab640 con 0x7fc6600cade0 2023-11-28T21:44:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65effd640 1 -- 172.21.15.12:0/4113279372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6540a7070 con 0x7fc6600cade0 2023-11-28T21:44:22.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.727+0000 7fc65effd640 1 -- 172.21.15.12:0/4113279372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6540ab9c0 con 0x7fc6600cade0 2023-11-28T21:44:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 -- 172.21.15.12:0/4113279372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6600cade0 msgr2=0x7fc6600cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 --2- 172.21.15.12:0/4113279372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6600cade0 0x7fc6600cc0c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc6600ca5c0 tx=0x7fc65401c440 comp rx=0 tx=0).stop 2023-11-28T21:44:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 -- 172.21.15.12:0/4113279372 shutdown_connections 2023-11-28T21:44:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 --2- 172.21.15.12:0/4113279372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6600cade0 0x7fc6600cc0c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 --2- 172.21.15.12:0/4113279372 >> v2:172.21.15.144:3300/0 conn(0x7fc660154120 0x7fc6600ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 -- 172.21.15.12:0/4113279372 >> 172.21.15.12:0/4113279372 conn(0x7fc660071ce0 msgr2=0x7fc6600720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 -- 172.21.15.12:0/4113279372 shutdown_connections 2023-11-28T21:44:22.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 -- 172.21.15.12:0/4113279372 wait complete. 2023-11-28T21:44:22.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 Processor -- start 2023-11-28T21:44:22.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 -- start start 2023-11-28T21:44:22.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6600cade0 0x7fc660149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:22.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6600cbba0 0x7fc66014f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:22.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc660154120 0x7fc660144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:22.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66014a460 con 0x7fc660154120 2023-11-28T21:44:22.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6601484e0 con 0x7fc6600cade0 2023-11-28T21:44:22.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc66632a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc660148cb0 con 0x7fc6600cbba0 2023-11-28T21:44:22.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc6648c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc660154120 0x7fc660144f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6600cade0 0x7fc660149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc65ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6600cade0 0x7fc660149570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:51984/0 (socket says 172.21.15.12:51984) 2023-11-28T21:44:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc65ffff640 1 -- 172.21.15.12:0/4061810633 learned_addr learned my addr 172.21.15.12:0/4061810633 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc6648c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc660154120 0x7fc660144f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45704/0 (socket says 172.21.15.12:45704) 2023-11-28T21:44:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.731+0000 7fc65f7fe640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6600cbba0 0x7fc66014f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc6648c7640 1 -- 172.21.15.12:0/4061810633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6600cbba0 msgr2=0x7fc66014f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc6648c7640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6600cbba0 0x7fc66014f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc6648c7640 1 -- 172.21.15.12:0/4061810633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6600cade0 msgr2=0x7fc660149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc6648c7640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6600cade0 0x7fc660149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:22.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc6648c7640 1 -- 172.21.15.12:0/4061810633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540094d0 con 0x7fc660154120 2023-11-28T21:44:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc6648c7640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc660154120 0x7fc660144f80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc65800c700 tx=0x7fc65800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc65cff9640 1 -- 172.21.15.12:0/4061810633 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658003b80 con 0x7fc660154120 2023-11-28T21:44:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc65cff9640 1 -- 172.21.15.12:0/4061810633 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc658003ce0 con 0x7fc660154120 2023-11-28T21:44:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc65cff9640 1 -- 172.21.15.12:0/4061810633 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65800aa30 con 0x7fc660154120 2023-11-28T21:44:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc66632a640 1 -- 172.21.15.12:0/4061810633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc660149ab0 con 0x7fc660154120 2023-11-28T21:44:22.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc66632a640 1 -- 172.21.15.12:0/4061810633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6601486f0 con 0x7fc660154120 2023-11-28T21:44:22.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc66632a640 1 -- 172.21.15.12:0/4061810633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66014a1c0 con 0x7fc660154120 2023-11-28T21:44:22.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc65cff9640 1 -- 172.21.15.12:0/4061810633 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc65800ab90 con 0x7fc660154120 2023-11-28T21:44:22.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.735+0000 7fc65cff9640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc64c0664a0 0x7fc64c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:22.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.739+0000 7fc65cff9640 1 -- 172.21.15.12:0/4061810633 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7fc658083f40 con 0x7fc660154120 2023-11-28T21:44:22.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.739+0000 7fc65ffff640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc64c0664a0 0x7fc64c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:22.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.739+0000 7fc65ffff640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc64c0664a0 0x7fc64c068960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc650007a70 tx=0x7fc650008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:22.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.739+0000 7fc65cff9640 1 -- 172.21.15.12:0/4061810633 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc66014a1c0 con 0x7fc660154120 2023-11-28T21:44:22.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.931+0000 7fc66632a640 1 -- 172.21.15.12:0/4061810633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc66014a5a0 con 0x7fc660154120 2023-11-28T21:44:22.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.931+0000 7fc65cff9640 1 -- 172.21.15.12:0/4061810633 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fc66014a5a0 con 0x7fc660154120 2023-11-28T21:44:22.937 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:22.938 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:21.684309+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:22.938 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2023-11-28T21:44:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 -- 172.21.15.12:0/4061810633 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc64c0664a0 msgr2=0x7fc64c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc64c0664a0 0x7fc64c068960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc650007a70 tx=0x7fc650008040 comp rx=0 tx=0).stop 2023-11-28T21:44:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 -- 172.21.15.12:0/4061810633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc660154120 msgr2=0x7fc660144f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc660154120 0x7fc660144f80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc65800c700 tx=0x7fc65800cbc0 comp rx=0 tx=0).stop 2023-11-28T21:44:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 -- 172.21.15.12:0/4061810633 shutdown_connections 2023-11-28T21:44:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc64c0664a0 0x7fc64c068960 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc660154120 0x7fc660144f80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6600cbba0 0x7fc66014f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 --2- 172.21.15.12:0/4061810633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6600cade0 0x7fc660149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 -- 172.21.15.12:0/4061810633 >> 172.21.15.12:0/4061810633 conn(0x7fc660071ce0 msgr2=0x7fc660158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 -- 172.21.15.12:0/4061810633 shutdown_connections 2023-11-28T21:44:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:22.935+0000 7fc66632a640 1 -- 172.21.15.12:0/4061810633 wait complete. 2023-11-28T21:44:22.959 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 41, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 41, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5727": { "addr": "172.21.15.12:6839/3299030262", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 3299030262, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 3299030262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5727, "incarnation": 39, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:rejoin", "state_seq": 20 } }, "last_failure": 0, "last_failure_osd_epoch": 46, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:44:21.684309+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5727 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:44:24.960 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:25.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2eb52a640 1 Processor -- start 2023-11-28T21:44:25.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2eb52a640 1 -- start start 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2eb52a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2e4158420 0x7fc2e406d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2eb52a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc2e40c38f0 0x7fc2e40c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2eb52a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e415e010 0x7fc2e415e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2eb52a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e4158800 con 0x7fc2e40c38f0 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2eb52a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e40c52c0 con 0x7fc2e4158420 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2eb52a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e406de90 con 0x7fc2e415e010 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2e8ac5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc2e40c38f0 0x7fc2e40c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2e8ac5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc2e40c38f0 0x7fc2e40c3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47134/0 (socket says 172.21.15.12:47134) 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2e8ac5640 1 -- 172.21.15.12:0/1208936334 learned_addr learned my addr 172.21.15.12:0/1208936334 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2e92c6640 1 --2- 172.21.15.12:0/1208936334 >> v2:172.21.15.144:3300/0 conn(0x7fc2e4158420 0x7fc2e406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:25.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2e9ac7640 1 --2- 172.21.15.12:0/1208936334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e415e010 0x7fc2e415e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2e8ac5640 1 -- 172.21.15.12:0/1208936334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e415e010 msgr2=0x7fc2e415e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2e8ac5640 1 --2- 172.21.15.12:0/1208936334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e415e010 0x7fc2e415e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2e8ac5640 1 -- 172.21.15.12:0/1208936334 >> v2:172.21.15.144:3300/0 conn(0x7fc2e4158420 msgr2=0x7fc2e406d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2e8ac5640 1 --2- 172.21.15.12:0/1208936334 >> v2:172.21.15.144:3300/0 conn(0x7fc2e4158420 0x7fc2e406d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.111+0000 7fc2e8ac5640 1 -- 172.21.15.12:0/1208936334 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e40c3460 con 0x7fc2e40c38f0 2023-11-28T21:44:25.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2e8ac5640 1 --2- 172.21.15.12:0/1208936334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e40c38f0 0x7fc2e40c3d40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc2e406c8b0 tx=0x7fc2d401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f349bde10f85be4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2dbfff640 1 -- 172.21.15.12:0/1208936334 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40ab600 con 0x7fc2e40c38f0 2023-11-28T21:44:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2dbfff640 1 -- 172.21.15.12:0/1208936334 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d40a7050 con 0x7fc2e40c38f0 2023-11-28T21:44:25.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2dbfff640 1 -- 172.21.15.12:0/1208936334 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40ab980 con 0x7fc2e40c38f0 2023-11-28T21:44:25.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2eb52a640 1 -- 172.21.15.12:0/1208936334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e40c38f0 msgr2=0x7fc2e40c3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:25.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2eb52a640 1 --2- 172.21.15.12:0/1208936334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e40c38f0 0x7fc2e40c3d40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc2e406c8b0 tx=0x7fc2d401c500 comp rx=0 tx=0).stop 2023-11-28T21:44:25.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2eb52a640 1 -- 172.21.15.12:0/1208936334 shutdown_connections 2023-11-28T21:44:25.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2eb52a640 1 --2- 172.21.15.12:0/1208936334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e415e010 0x7fc2e415e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2eb52a640 1 --2- 172.21.15.12:0/1208936334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e40c38f0 0x7fc2e40c3d40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2eb52a640 1 --2- 172.21.15.12:0/1208936334 >> v2:172.21.15.144:3300/0 conn(0x7fc2e4158420 0x7fc2e406d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2eb52a640 1 -- 172.21.15.12:0/1208936334 >> 172.21.15.12:0/1208936334 conn(0x7fc2e4066360 msgr2=0x7fc2e4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2eb52a640 1 -- 172.21.15.12:0/1208936334 shutdown_connections 2023-11-28T21:44:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.115+0000 7fc2eb52a640 1 -- 172.21.15.12:0/1208936334 wait complete. 2023-11-28T21:44:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2eb52a640 1 Processor -- start 2023-11-28T21:44:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2eb52a640 1 -- start start 2023-11-28T21:44:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2eb52a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e40c38f0 0x7fc2e40ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2eb52a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e4158420 0x7fc2e41f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2eb52a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e415e010 0x7fc2e41f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2eb52a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e41f6bf0 con 0x7fc2e4158420 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2eb52a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e41fa300 con 0x7fc2e415e010 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2eb52a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e41fa5e0 con 0x7fc2e40c38f0 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e8ac5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e4158420 0x7fc2e41f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e8ac5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e4158420 0x7fc2e41f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47144/0 (socket says 172.21.15.12:47144) 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e8ac5640 1 -- 172.21.15.12:0/2931109895 learned_addr learned my addr 172.21.15.12:0/2931109895 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e9ac7640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e415e010 0x7fc2e41f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e92c6640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e40c38f0 0x7fc2e40ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e8ac5640 1 -- 172.21.15.12:0/2931109895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e40c38f0 msgr2=0x7fc2e40ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e8ac5640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e40c38f0 0x7fc2e40ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e8ac5640 1 -- 172.21.15.12:0/2931109895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e415e010 msgr2=0x7fc2e41f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e8ac5640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e415e010 0x7fc2e41f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e8ac5640 1 -- 172.21.15.12:0/2931109895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d40094d0 con 0x7fc2e4158420 2023-11-28T21:44:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e9ac7640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e415e010 0x7fc2e41f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:44:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2e8ac5640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e4158420 0x7fc2e41f70f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc2d401abb0 tx=0x7fc2d40192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2d9ffb640 1 -- 172.21.15.12:0/2931109895 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40b4420 con 0x7fc2e4158420 2023-11-28T21:44:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2d9ffb640 1 -- 172.21.15.12:0/2931109895 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d40b4580 con 0x7fc2e4158420 2023-11-28T21:44:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2d9ffb640 1 -- 172.21.15.12:0/2931109895 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d40b48f0 con 0x7fc2e4158420 2023-11-28T21:44:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2eb52a640 1 -- 172.21.15.12:0/2931109895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e41fa8c0 con 0x7fc2e4158420 2023-11-28T21:44:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.119+0000 7fc2eb52a640 1 -- 172.21.15.12:0/2931109895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e41fae80 con 0x7fc2e4158420 2023-11-28T21:44:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.123+0000 7fc2eb52a640 1 -- 172.21.15.12:0/2931109895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e4074f60 con 0x7fc2e4158420 2023-11-28T21:44:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.123+0000 7fc2d9ffb640 1 -- 172.21.15.12:0/2931109895 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc2d40b2070 con 0x7fc2e4158420 2023-11-28T21:44:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.123+0000 7fc2d9ffb640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2c80664a0 0x7fc2c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.123+0000 7fc2d9ffb640 1 -- 172.21.15.12:0/2931109895 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7fc2d411d660 con 0x7fc2e4158420 2023-11-28T21:44:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.123+0000 7fc2e92c6640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2c80664a0 0x7fc2c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.123+0000 7fc2e92c6640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2c80664a0 0x7fc2c8068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000ba30 tx=0x7fc2e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:25.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.127+0000 7fc2d9ffb640 1 -- 172.21.15.12:0/2931109895 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc2d40ea4a0 con 0x7fc2e4158420 2023-11-28T21:44:25.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.315+0000 7fc2eb52a640 1 -- 172.21.15.12:0/2931109895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2e406e4f0 con 0x7fc2e4158420 2023-11-28T21:44:25.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.315+0000 7fc2d9ffb640 1 -- 172.21.15.12:0/2931109895 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fc2d40b0090 con 0x7fc2e4158420 2023-11-28T21:44:25.321 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:25.321 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:21.684309+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:25.321 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2023-11-28T21:44:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 -- 172.21.15.12:0/2931109895 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2c80664a0 msgr2=0x7fc2c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2c80664a0 0x7fc2c8068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000ba30 tx=0x7fc2e0008040 comp rx=0 tx=0).stop 2023-11-28T21:44:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 -- 172.21.15.12:0/2931109895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e4158420 msgr2=0x7fc2e41f70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e4158420 0x7fc2e41f70f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc2d401abb0 tx=0x7fc2d40192a0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 -- 172.21.15.12:0/2931109895 shutdown_connections 2023-11-28T21:44:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc2c80664a0 0x7fc2c8068960 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e415e010 0x7fc2e41f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2e4158420 0x7fc2e41f70f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 --2- 172.21.15.12:0/2931109895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e40c38f0 0x7fc2e40ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:25.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 -- 172.21.15.12:0/2931109895 >> 172.21.15.12:0/2931109895 conn(0x7fc2e4066360 msgr2=0x7fc2e4070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:25.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 -- 172.21.15.12:0/2931109895 shutdown_connections 2023-11-28T21:44:25.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:25.319+0000 7fc2eb52a640 1 -- 172.21.15.12:0/2931109895 wait complete. 2023-11-28T21:44:25.342 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:44:27.344 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b70d6640 1 Processor -- start 2023-11-28T21:44:27.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b70d6640 1 -- start start 2023-11-28T21:44:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b70d6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f14b0151af0 0x7f14b00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b70d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14b0151ed0 0x7f14b00c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b70d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f14b006e300 0x7f14b00c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b70d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b00707f0 con 0x7f14b006e300 2023-11-28T21:44:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b70d6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b00c7b90 con 0x7f14b0151af0 2023-11-28T21:44:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b70d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b0071c20 con 0x7f14b0151ed0 2023-11-28T21:44:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b5673640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f14b006e300 0x7f14b00c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b5673640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f14b006e300 0x7f14b00c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47156/0 (socket says 172.21.15.12:47156) 2023-11-28T21:44:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b5673640 1 -- 172.21.15.12:0/93024309 learned_addr learned my addr 172.21.15.12:0/93024309 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14affff640 1 --2- 172.21.15.12:0/93024309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14b0151ed0 0x7f14b00c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.491+0000 7f14b4e72640 1 --2- 172.21.15.12:0/93024309 >> v2:172.21.15.144:3300/0 conn(0x7f14b0151af0 0x7f14b00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b5673640 1 -- 172.21.15.12:0/93024309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14b0151ed0 msgr2=0x7f14b00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b5673640 1 --2- 172.21.15.12:0/93024309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14b0151ed0 0x7f14b00c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b5673640 1 -- 172.21.15.12:0/93024309 >> v2:172.21.15.144:3300/0 conn(0x7f14b0151af0 msgr2=0x7f14b00c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b5673640 1 --2- 172.21.15.12:0/93024309 >> v2:172.21.15.144:3300/0 conn(0x7f14b0151af0 0x7f14b00c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b5673640 1 -- 172.21.15.12:0/93024309 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b0157b70 con 0x7f14b006e300 2023-11-28T21:44:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b5673640 1 --2- 172.21.15.12:0/93024309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b006e300 0x7f14b00c4f10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f14b00c7d20 tx=0x7f14a801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4a36261b2c7d13 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14af7fe640 1 -- 172.21.15.12:0/93024309 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80a5740 con 0x7f14b006e300 2023-11-28T21:44:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14af7fe640 1 -- 172.21.15.12:0/93024309 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a80a58a0 con 0x7f14b006e300 2023-11-28T21:44:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14af7fe640 1 -- 172.21.15.12:0/93024309 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80a5c30 con 0x7f14b006e300 2023-11-28T21:44:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b70d6640 1 -- 172.21.15.12:0/93024309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b006e300 msgr2=0x7f14b00c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b70d6640 1 --2- 172.21.15.12:0/93024309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b006e300 0x7f14b00c4f10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f14b00c7d20 tx=0x7f14a801c9d0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b70d6640 1 -- 172.21.15.12:0/93024309 shutdown_connections 2023-11-28T21:44:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b70d6640 1 --2- 172.21.15.12:0/93024309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b006e300 0x7f14b00c4f10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b70d6640 1 --2- 172.21.15.12:0/93024309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14b0151ed0 0x7f14b00c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b70d6640 1 --2- 172.21.15.12:0/93024309 >> v2:172.21.15.144:3300/0 conn(0x7f14b0151af0 0x7f14b00c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b70d6640 1 -- 172.21.15.12:0/93024309 >> 172.21.15.12:0/93024309 conn(0x7f14b0066360 msgr2=0x7f14b0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b70d6640 1 -- 172.21.15.12:0/93024309 shutdown_connections 2023-11-28T21:44:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.495+0000 7f14b70d6640 1 -- 172.21.15.12:0/93024309 wait complete. 2023-11-28T21:44:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14b70d6640 1 Processor -- start 2023-11-28T21:44:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14b70d6640 1 -- start start 2023-11-28T21:44:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14b70d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14b006e300 0x7f14b01f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14b70d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b0151af0 0x7f14b01f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:27.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14affff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b0151af0 0x7f14b01f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:27.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14affff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b0151af0 0x7f14b01f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47160/0 (socket says 172.21.15.12:47160) 2023-11-28T21:44:27.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14b4e72640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14b006e300 0x7f14b01f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:27.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14b4e72640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14b006e300 0x7f14b01f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:48208/0 (socket says 172.21.15.12:48208) 2023-11-28T21:44:27.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14b70d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14b0158c00 0x7f14b00c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14affff640 1 -- 172.21.15.12:0/2996686341 learned_addr learned my addr 172.21.15.12:0/2996686341 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b01f6680 con 0x7f14b0151af0 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00c9600 con 0x7f14b006e300 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b01f6000 con 0x7f14b0158c00 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14affff640 1 -- 172.21.15.12:0/2996686341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14b0158c00 msgr2=0x7f14b00c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14affff640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14b0158c00 0x7f14b00c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14affff640 1 -- 172.21.15.12:0/2996686341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14b006e300 msgr2=0x7f14b01f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14affff640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14b006e300 0x7f14b01f6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14affff640 1 -- 172.21.15.12:0/2996686341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a8009b30 con 0x7f14b0151af0 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.499+0000 7f14affff640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b0151af0 0x7f14b01f29f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f149c00d600 tx=0x7f149c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.503+0000 7f14ad7fa640 1 -- 172.21.15.12:0/2996686341 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c004240 con 0x7f14b0151af0 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.503+0000 7f14ad7fa640 1 -- 172.21.15.12:0/2996686341 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c0043a0 con 0x7f14b0151af0 2023-11-28T21:44:27.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.503+0000 7f14ad7fa640 1 -- 172.21.15.12:0/2996686341 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c003840 con 0x7f14b0151af0 2023-11-28T21:44:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.503+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b00c9820 con 0x7f14b0151af0 2023-11-28T21:44:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.503+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b01f62a0 con 0x7f14b0151af0 2023-11-28T21:44:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.503+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b0074f60 con 0x7f14b0151af0 2023-11-28T21:44:27.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.503+0000 7f14ad7fa640 1 -- 172.21.15.12:0/2996686341 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f149c0039a0 con 0x7f14b0151af0 2023-11-28T21:44:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.503+0000 7f14ad7fa640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1494066470 0x7f1494068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.503+0000 7f14ad7fa640 1 -- 172.21.15.12:0/2996686341 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f149c084400 con 0x7f14b0151af0 2023-11-28T21:44:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.503+0000 7f14b4e72640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1494066470 0x7f1494068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:27.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.507+0000 7f14b4e72640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1494066470 0x7f1494068930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f14a40045f0 tx=0x7f14a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.507+0000 7f14ad7fa640 1 -- 172.21.15.12:0/2996686341 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f149c051360 con 0x7f14b0151af0 2023-11-28T21:44:27.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.699+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14b0072c10 con 0x7f14b0151af0 2023-11-28T21:44:27.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.699+0000 7f14ad7fa640 1 -- 172.21.15.12:0/2996686341 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f149c054980 con 0x7f14b0151af0 2023-11-28T21:44:27.705 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:27.705 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:21.684309+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:27.705 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 41 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1494066470 msgr2=0x7f1494068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1494066470 0x7f1494068930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f14a40045f0 tx=0x7f14a4008040 comp rx=0 tx=0).stop 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b0151af0 msgr2=0x7f14b01f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b0151af0 0x7f14b01f29f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f149c00d600 tx=0x7f149c00dac0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 shutdown_connections 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1494066470 0x7f1494068930 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14b0158c00 0x7f14b00c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14b0151af0 0x7f14b01f29f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 --2- 172.21.15.12:0/2996686341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14b006e300 0x7f14b01f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 >> 172.21.15.12:0/2996686341 conn(0x7f14b0066360 msgr2=0x7f14b0150560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 shutdown_connections 2023-11-28T21:44:27.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:27.703+0000 7f14b70d6640 1 -- 172.21.15.12:0/2996686341 wait complete. 2023-11-28T21:44:27.725 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:44:29.726 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:29.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe6cea640 1 Processor -- start 2023-11-28T21:44:29.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe6cea640 1 -- start start 2023-11-28T21:44:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe6cea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe0151af0 0x7effe00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe6cea640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7effe006e300 0x7effe0072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe6cea640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe0157b70 con 0x7effe0151ed0 2023-11-28T21:44:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe6cea640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe00707f0 con 0x7effe006e300 2023-11-28T21:44:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe6cea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe00c7b90 con 0x7effe0151af0 2023-11-28T21:44:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe5287640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7effe006e300 0x7effe0072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe5287640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7effe006e300 0x7effe0072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47172/0 (socket says 172.21.15.12:47172) 2023-11-28T21:44:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe5287640 1 -- 172.21.15.12:0/1116058634 learned_addr learned my addr 172.21.15.12:0/1116058634 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:29.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effdffff640 1 --1- 172.21.15.12:0/1116058634 >> v1:172.21.15.12:6789/0 conn(0x7effe0151ed0 0x7effe00c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38260/0 (socket says 172.21.15.12:38260) 2023-11-28T21:44:29.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe4a86640 1 --2- 172.21.15.12:0/1116058634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe0151af0 0x7effe00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:29.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe5287640 1 -- 172.21.15.12:0/1116058634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe0151af0 msgr2=0x7effe00c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:29.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe5287640 1 --2- 172.21.15.12:0/1116058634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe0151af0 0x7effe00c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:29.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe5287640 1 -- 172.21.15.12:0/1116058634 >> v1:172.21.15.12:6789/0 conn(0x7effe0151ed0 legacy=0x7effe00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:29.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe5287640 1 -- 172.21.15.12:0/1116058634 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe0157e50 con 0x7effe006e300 2023-11-28T21:44:29.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.878+0000 7effe4a86640 1 --2- 172.21.15.12:0/1116058634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe0151af0 0x7effe00c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:44:29.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effe5287640 1 --2- 172.21.15.12:0/1116058634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe006e300 0x7effe0072070 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7effd8009e50 tx=0x7effd801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e060089d002e8cac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:29.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effdf7fe640 1 -- 172.21.15.12:0/1116058634 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd80a56e0 con 0x7effe006e300 2023-11-28T21:44:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effdf7fe640 1 -- 172.21.15.12:0/1116058634 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd80a5840 con 0x7effe006e300 2023-11-28T21:44:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effdf7fe640 1 -- 172.21.15.12:0/1116058634 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd80a5bd0 con 0x7effe006e300 2023-11-28T21:44:29.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effe6cea640 1 -- 172.21.15.12:0/1116058634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe006e300 msgr2=0x7effe0072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effe6cea640 1 --2- 172.21.15.12:0/1116058634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe006e300 0x7effe0072070 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7effd8009e50 tx=0x7effd801ca10 comp rx=0 tx=0).stop 2023-11-28T21:44:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effe6cea640 1 -- 172.21.15.12:0/1116058634 shutdown_connections 2023-11-28T21:44:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effe6cea640 1 --2- 172.21.15.12:0/1116058634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe006e300 0x7effe0072070 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effe6cea640 1 --2- 172.21.15.12:0/1116058634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe0151af0 0x7effe00c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effe6cea640 1 -- 172.21.15.12:0/1116058634 >> 172.21.15.12:0/1116058634 conn(0x7effe0066360 msgr2=0x7effe0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effe6cea640 1 -- 172.21.15.12:0/1116058634 shutdown_connections 2023-11-28T21:44:29.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.882+0000 7effe6cea640 1 -- 172.21.15.12:0/1116058634 wait complete. 2023-11-28T21:44:29.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 Processor -- start 2023-11-28T21:44:29.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 -- start start 2023-11-28T21:44:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe006e300 0x7effe0142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe0151af0 0x7effe013fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe00c9ad0 0x7effe0141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe014d7e0 con 0x7effe00c9ad0 2023-11-28T21:44:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe01f3720 con 0x7effe0151af0 2023-11-28T21:44:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe01f3bb0 con 0x7effe006e300 2023-11-28T21:44:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe5287640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe00c9ad0 0x7effe0141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe5287640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe00c9ad0 0x7effe0141960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47176/0 (socket says 172.21.15.12:47176) 2023-11-28T21:44:29.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe5287640 1 -- 172.21.15.12:0/4148231392 learned_addr learned my addr 172.21.15.12:0/4148231392 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe4a86640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe006e300 0x7effe0142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effdffff640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe0151af0 0x7effe013fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe5287640 1 -- 172.21.15.12:0/4148231392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe006e300 msgr2=0x7effe0142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe5287640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe006e300 0x7effe0142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe5287640 1 -- 172.21.15.12:0/4148231392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe0151af0 msgr2=0x7effe013fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe5287640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe0151af0 0x7effe013fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe5287640 1 -- 172.21.15.12:0/4148231392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8009b30 con 0x7effe00c9ad0 2023-11-28T21:44:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe4a86640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe006e300 0x7effe0142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:44:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe5287640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe00c9ad0 0x7effe0141960 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7effd801a640 tx=0x7effd80197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:29.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effdd7fa640 1 -- 172.21.15.12:0/4148231392 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd80b8670 con 0x7effe00c9ad0 2023-11-28T21:44:29.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effdd7fa640 1 -- 172.21.15.12:0/4148231392 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd80b87d0 con 0x7effe00c9ad0 2023-11-28T21:44:29.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effdd7fa640 1 -- 172.21.15.12:0/4148231392 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd80b8b80 con 0x7effe00c9ad0 2023-11-28T21:44:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 -- 172.21.15.12:0/4148231392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe0143920 con 0x7effe00c9ad0 2023-11-28T21:44:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 -- 172.21.15.12:0/4148231392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe0143640 con 0x7effe00c9ad0 2023-11-28T21:44:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.886+0000 7effe6cea640 1 -- 172.21.15.12:0/4148231392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe0074f60 con 0x7effe00c9ad0 2023-11-28T21:44:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.890+0000 7effdd7fa640 1 -- 172.21.15.12:0/4148231392 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7effd80b8ce0 con 0x7effe00c9ad0 2023-11-28T21:44:29.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.890+0000 7effdd7fa640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7effc40664a0 0x7effc4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:29.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.890+0000 7effdd7fa640 1 -- 172.21.15.12:0/4148231392 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7effd811f9a0 con 0x7effe00c9ad0 2023-11-28T21:44:29.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.890+0000 7effe4a86640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7effc40664a0 0x7effc4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:29.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.890+0000 7effe4a86640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7effc40664a0 0x7effc4068960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7effd4009970 tx=0x7effd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:29.894+0000 7effdd7fa640 1 -- 172.21.15.12:0/4148231392 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7effd80eca40 con 0x7effe00c9ad0 2023-11-28T21:44:30.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.082+0000 7effe6cea640 1 -- 172.21.15.12:0/4148231392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effe01f3860 con 0x7effe00c9ad0 2023-11-28T21:44:30.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.082+0000 7effdd7fa640 1 -- 172.21.15.12:0/4148231392 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5721 (secure 0 0 0) 0x7effd8009510 con 0x7effe00c9ad0 2023-11-28T21:44:30.090 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:30.090 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:26.854401+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:clientreplay","state_seq":22,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:30.090 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 42 2023-11-28T21:44:30.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 -- 172.21.15.12:0/4148231392 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7effc40664a0 msgr2=0x7effc4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:30.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7effc40664a0 0x7effc4068960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7effd4009970 tx=0x7effd4008040 comp rx=0 tx=0).stop 2023-11-28T21:44:30.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 -- 172.21.15.12:0/4148231392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe00c9ad0 msgr2=0x7effe0141960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:30.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe00c9ad0 0x7effe0141960 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7effd801a640 tx=0x7effd80197b0 comp rx=0 tx=0).stop 2023-11-28T21:44:30.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 -- 172.21.15.12:0/4148231392 shutdown_connections 2023-11-28T21:44:30.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7effc40664a0 0x7effc4068960 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:30.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe00c9ad0 0x7effe0141960 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:30.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe0151af0 0x7effe013fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:30.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 --2- 172.21.15.12:0/4148231392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7effe006e300 0x7effe0142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:30.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 -- 172.21.15.12:0/4148231392 >> 172.21.15.12:0/4148231392 conn(0x7effe0066360 msgr2=0x7effe014f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:30.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 -- 172.21.15.12:0/4148231392 shutdown_connections 2023-11-28T21:44:30.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:30.086+0000 7effe6cea640 1 -- 172.21.15.12:0/4148231392 wait complete. 2023-11-28T21:44:30.111 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:44:32.112 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:40.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc1c572640 1 Processor -- start 2023-11-28T21:44:40.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc1c572640 1 -- start start 2023-11-28T21:44:40.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc1c572640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1406b130 0x7fdc140c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc1c572640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdc140cb020 0x7fdc14159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc1c572640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc140cb4b0 con 0x7fdc140cac40 2023-11-28T21:44:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc1c572640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1406b550 con 0x7fdc140cb020 2023-11-28T21:44:40.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc1c572640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1415a5e0 con 0x7fdc1406b130 2023-11-28T21:44:40.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc1ab0f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdc140cb020 0x7fdc14159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:40.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc19b0d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1406b130 0x7fdc140c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:40.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc1ab0f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdc140cb020 0x7fdc14159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:44162/0 (socket says 172.21.15.12:44162) 2023-11-28T21:44:40.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc1ab0f640 1 -- 172.21.15.12:0/1509307495 learned_addr learned my addr 172.21.15.12:0/1509307495 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:40.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc19b0d640 1 -- 172.21.15.12:0/1509307495 >> v2:172.21.15.144:3300/0 conn(0x7fdc140cb020 msgr2=0x7fdc14159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:40.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc19b0d640 1 --2- 172.21.15.12:0/1509307495 >> v2:172.21.15.144:3300/0 conn(0x7fdc140cb020 0x7fdc14159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:40.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc19b0d640 1 -- 172.21.15.12:0/1509307495 >> v1:172.21.15.144:6789/0 conn(0x7fdc140cac40 legacy=0x7fdc140ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:40.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.198+0000 7fdc19b0d640 1 -- 172.21.15.12:0/1509307495 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1415a8c0 con 0x7fdc1406b130 2023-11-28T21:44:40.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc19b0d640 1 --2- 172.21.15.12:0/1509307495 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1406b130 0x7fdc140c4980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc040097c0 tx=0x7fdc0401ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2f5ea4b723a190 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:40.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1930c640 1 -- 172.21.15.12:0/1509307495 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc040a8970 con 0x7fdc1406b130 2023-11-28T21:44:40.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1930c640 1 -- 172.21.15.12:0/1509307495 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc040a8ad0 con 0x7fdc1406b130 2023-11-28T21:44:40.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1930c640 1 -- 172.21.15.12:0/1509307495 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc040a8e60 con 0x7fdc1406b130 2023-11-28T21:44:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1c572640 1 -- 172.21.15.12:0/1509307495 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1406b130 msgr2=0x7fdc140c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1c572640 1 --2- 172.21.15.12:0/1509307495 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1406b130 0x7fdc140c4980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc040097c0 tx=0x7fdc0401ec70 comp rx=0 tx=0).stop 2023-11-28T21:44:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1c572640 1 -- 172.21.15.12:0/1509307495 shutdown_connections 2023-11-28T21:44:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1c572640 1 --2- 172.21.15.12:0/1509307495 >> v2:172.21.15.144:3300/0 conn(0x7fdc140cb020 0x7fdc14159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1c572640 1 --2- 172.21.15.12:0/1509307495 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1406b130 0x7fdc140c4980 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:40.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1c572640 1 -- 172.21.15.12:0/1509307495 >> 172.21.15.12:0/1509307495 conn(0x7fdc140720e0 msgr2=0x7fdc140724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:40.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1c572640 1 -- 172.21.15.12:0/1509307495 shutdown_connections 2023-11-28T21:44:40.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.202+0000 7fdc1c572640 1 -- 172.21.15.12:0/1509307495 wait complete. 2023-11-28T21:44:40.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1c572640 1 Processor -- start 2023-11-28T21:44:40.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1c572640 1 -- start start 2023-11-28T21:44:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1c572640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1406b130 0x7fdc1414a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1c572640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc140cac40 0x7fdc1414a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1c572640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc140cb020 0x7fdc14159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1c572640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc141f0c20 con 0x7fdc1406b130 2023-11-28T21:44:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1c572640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc141e3ac0 con 0x7fdc140cb020 2023-11-28T21:44:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1c572640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc141e3da0 con 0x7fdc140cac40 2023-11-28T21:44:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1a30e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1406b130 0x7fdc1414a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1a30e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1406b130 0x7fdc1414a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49498/0 (socket says 172.21.15.12:49498) 2023-11-28T21:44:40.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1a30e640 1 -- 172.21.15.12:0/1110724873 learned_addr learned my addr 172.21.15.12:0/1110724873 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc19b0d640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc140cac40 0x7fdc1414a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:40.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1ab0f640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc140cb020 0x7fdc14159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1a30e640 1 -- 172.21.15.12:0/1110724873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc140cac40 msgr2=0x7fdc1414a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1a30e640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc140cac40 0x7fdc1414a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1a30e640 1 -- 172.21.15.12:0/1110724873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc140cb020 msgr2=0x7fdc14159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1a30e640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc140cb020 0x7fdc14159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1a30e640 1 -- 172.21.15.12:0/1110724873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc04009420 con 0x7fdc1406b130 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1a30e640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1406b130 0x7fdc1414a070 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c370 tx=0x7fdc1000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc0affd640 1 -- 172.21.15.12:0/1110724873 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc100052d0 con 0x7fdc1406b130 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc0affd640 1 -- 172.21.15.12:0/1110724873 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc10005430 con 0x7fdc1406b130 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc0affd640 1 -- 172.21.15.12:0/1110724873 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10007260 con 0x7fdc1406b130 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1c572640 1 -- 172.21.15.12:0/1110724873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc141e4080 con 0x7fdc1406b130 2023-11-28T21:44:40.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.206+0000 7fdc1c572640 1 -- 172.21.15.12:0/1110724873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc141e46b0 con 0x7fdc1406b130 2023-11-28T21:44:40.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.210+0000 7fdc1c572640 1 -- 172.21.15.12:0/1110724873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc14067c70 con 0x7fdc1406b130 2023-11-28T21:44:40.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.210+0000 7fdc0affd640 1 -- 172.21.15.12:0/1110724873 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdc100073c0 con 0x7fdc1406b130 2023-11-28T21:44:40.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.210+0000 7fdc0affd640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdbf8066470 0x7fdbf8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:40.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.210+0000 7fdc0affd640 1 -- 172.21.15.12:0/1110724873 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7fdc100837c0 con 0x7fdc1406b130 2023-11-28T21:44:40.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.210+0000 7fdc19b0d640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdbf8066470 0x7fdbf8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:40.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.210+0000 7fdc19b0d640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdbf8066470 0x7fdbf8068930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdc040b2000 tx=0x7fdc040aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:40.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.214+0000 7fdc0affd640 1 -- 172.21.15.12:0/1110724873 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdc100506b0 con 0x7fdc1406b130 2023-11-28T21:44:40.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.406+0000 7fdc1c572640 1 -- 172.21.15.12:0/1110724873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc141e48a0 con 0x7fdc1406b130 2023-11-28T21:44:40.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc0affd640 1 -- 172.21.15.12:0/1110724873 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fdc10053cd0 con 0x7fdc1406b130 2023-11-28T21:44:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:40.414 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:40.415 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-11-28T21:44:40.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 -- 172.21.15.12:0/1110724873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdbf8066470 msgr2=0x7fdbf8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:40.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdbf8066470 0x7fdbf8068930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdc040b2000 tx=0x7fdc040aa070 comp rx=0 tx=0).stop 2023-11-28T21:44:40.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 -- 172.21.15.12:0/1110724873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1406b130 msgr2=0x7fdc1414a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:40.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1406b130 0x7fdc1414a070 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000c370 tx=0x7fdc1000c830 comp rx=0 tx=0).stop 2023-11-28T21:44:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 -- 172.21.15.12:0/1110724873 shutdown_connections 2023-11-28T21:44:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdbf8066470 0x7fdbf8068930 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc140cb020 0x7fdc14159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc140cac40 0x7fdc1414a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 --2- 172.21.15.12:0/1110724873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc1406b130 0x7fdc1414a070 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 -- 172.21.15.12:0/1110724873 >> 172.21.15.12:0/1110724873 conn(0x7fdc140720e0 msgr2=0x7fdc140b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 -- 172.21.15.12:0/1110724873 shutdown_connections 2023-11-28T21:44:40.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:40.410+0000 7fdc1c572640 1 -- 172.21.15.12:0/1110724873 wait complete. 2023-11-28T21:44:40.435 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=0, replacing gid=5568 2023-11-28T21:44:40.435 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.a 2023-11-28T21:44:40.467 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-11-28T21:44:40.462+0000 7f629fc54640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T21:44:44.030 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:44:44.034 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:44:48.436 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2023-11-28T21:44:48.437 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T21:44:48.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T21:44:48.441 INFO:tasks.ceph.mds.a:Started 2023-11-28T21:44:48.441 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:48.537 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-11-28T21:44:48.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.590+0000 7fdaac29e640 1 Processor -- start 2023-11-28T21:44:48.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.590+0000 7fdaac29e640 1 -- start start 2023-11-28T21:44:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.590+0000 7fdaac29e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdaa40c27c0 0x7fdaa41500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.590+0000 7fdaac29e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdaa406cf40 0x7fdaa406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.590+0000 7fdaac29e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa40c6ce0 con 0x7fdaa4156af0 2023-11-28T21:44:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaac29e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa40c7170 con 0x7fdaa406cf40 2023-11-28T21:44:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaac29e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa40c2ba0 con 0x7fdaa40c27c0 2023-11-28T21:44:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaaa83b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdaa406cf40 0x7fdaa406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaa9839640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdaa4156af0 0x7fdaa40c2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:38540/0 (socket says 172.21.15.12:38540) 2023-11-28T21:44:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaa9839640 1 -- 172.21.15.12:0/4203307407 learned_addr learned my addr 172.21.15.12:0/4203307407 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:44:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaaa03a640 1 --2- 172.21.15.12:0/4203307407 >> v2:172.21.15.144:3300/0 conn(0x7fdaa40c27c0 0x7fdaa41500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:48.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaaa83b640 1 -- 172.21.15.12:0/4203307407 >> v2:172.21.15.144:3300/0 conn(0x7fdaa40c27c0 msgr2=0x7fdaa41500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:48.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaaa83b640 1 --2- 172.21.15.12:0/4203307407 >> v2:172.21.15.144:3300/0 conn(0x7fdaa40c27c0 0x7fdaa41500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:48.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaaa83b640 1 -- 172.21.15.12:0/4203307407 >> v1:172.21.15.12:6789/0 conn(0x7fdaa4156af0 legacy=0x7fdaa40c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:48.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaaa83b640 1 -- 172.21.15.12:0/4203307407 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa4156f60 con 0x7fdaa406cf40 2023-11-28T21:44:48.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaa9038640 1 -- 172.21.15.12:0/4203307407 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 333582538 0 0) 0x7fdaa40c6ce0 con 0x7fdaa4156af0 2023-11-28T21:44:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaaa83b640 1 --2- 172.21.15.12:0/4203307407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa406cf40 0x7fdaa406d350 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4150930 tx=0x7fda9c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec9b83cbf96ea47 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaa9038640 1 -- 172.21.15.12:0/4203307407 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0a59d0 con 0x7fdaa406cf40 2023-11-28T21:44:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaa9038640 1 -- 172.21.15.12:0/4203307407 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9c0a5b30 con 0x7fdaa406cf40 2023-11-28T21:44:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.594+0000 7fdaa9038640 1 -- 172.21.15.12:0/4203307407 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0ad440 con 0x7fdaa406cf40 2023-11-28T21:44:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 -- 172.21.15.12:0/4203307407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa406cf40 msgr2=0x7fdaa406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 --2- 172.21.15.12:0/4203307407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa406cf40 0x7fdaa406d350 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4150930 tx=0x7fda9c01c910 comp rx=0 tx=0).stop 2023-11-28T21:44:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 -- 172.21.15.12:0/4203307407 shutdown_connections 2023-11-28T21:44:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 --2- 172.21.15.12:0/4203307407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa406cf40 0x7fdaa406d350 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 --2- 172.21.15.12:0/4203307407 >> v2:172.21.15.144:3300/0 conn(0x7fdaa40c27c0 0x7fdaa41500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 -- 172.21.15.12:0/4203307407 >> 172.21.15.12:0/4203307407 conn(0x7fdaa4066360 msgr2=0x7fdaa4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 -- 172.21.15.12:0/4203307407 shutdown_connections 2023-11-28T21:44:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 -- 172.21.15.12:0/4203307407 wait complete. 2023-11-28T21:44:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 Processor -- start 2023-11-28T21:44:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 -- start start 2023-11-28T21:44:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa406cf40 0x7fdaa4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa40c27c0 0x7fdaa41ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaa4156af0 0x7fdaa41ec260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa414d290 con 0x7fdaa40c27c0 2023-11-28T21:44:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa41f2670 con 0x7fdaa4156af0 2023-11-28T21:44:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaac29e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa41f2950 con 0x7fdaa406cf40 2023-11-28T21:44:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaa9839640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa40c27c0 0x7fdaa41ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaa9839640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa40c27c0 0x7fdaa41ebd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40774/0 (socket says 172.21.15.12:40774) 2023-11-28T21:44:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaa9839640 1 -- 172.21.15.12:0/3204319916 learned_addr learned my addr 172.21.15.12:0/3204319916 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaaa03a640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa406cf40 0x7fdaa4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.598+0000 7fdaaa83b640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaa4156af0 0x7fdaa41ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaa9839640 1 -- 172.21.15.12:0/3204319916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa406cf40 msgr2=0x7fdaa4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaa9839640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa406cf40 0x7fdaa4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaa9839640 1 -- 172.21.15.12:0/3204319916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaa4156af0 msgr2=0x7fdaa41ec260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaa9839640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaa4156af0 0x7fdaa41ec260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaa9839640 1 -- 172.21.15.12:0/3204319916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c009a70 con 0x7fdaa40c27c0 2023-11-28T21:44:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaaa83b640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaa4156af0 0x7fdaa41ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:44:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaaa03a640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa406cf40 0x7fdaa4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:44:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaa9839640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa40c27c0 0x7fdaa41ebd20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fda9400c480 tx=0x7fda9400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fda9affd640 1 -- 172.21.15.12:0/3204319916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda940074e0 con 0x7fdaa40c27c0 2023-11-28T21:44:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fda9affd640 1 -- 172.21.15.12:0/3204319916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda94007660 con 0x7fdaa40c27c0 2023-11-28T21:44:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaac29e640 1 -- 172.21.15.12:0/3204319916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa41f2c30 con 0x7fdaa40c27c0 2023-11-28T21:44:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fda9affd640 1 -- 172.21.15.12:0/3204319916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda94004a30 con 0x7fdaa40c27c0 2023-11-28T21:44:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaac29e640 1 -- 172.21.15.12:0/3204319916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa41f31f0 con 0x7fdaa40c27c0 2023-11-28T21:44:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaac29e640 1 -- 172.21.15.12:0/3204319916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa41eba40 con 0x7fdaa40c27c0 2023-11-28T21:44:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fda9affd640 1 -- 172.21.15.12:0/3204319916 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fda94004b90 con 0x7fdaa40c27c0 2023-11-28T21:44:48.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fda9affd640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda88066470 0x7fda88068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:48.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fda9affd640 1 -- 172.21.15.12:0/3204319916 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7fda940835e0 con 0x7fdaa40c27c0 2023-11-28T21:44:48.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.602+0000 7fdaaa03a640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda88066470 0x7fda88068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:48.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.606+0000 7fdaaa03a640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda88066470 0x7fda88068930 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009bb0 tx=0x7fdaa0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:48.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.606+0000 7fda9affd640 1 -- 172.21.15.12:0/3204319916 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdaa41eba40 con 0x7fdaa40c27c0 2023-11-28T21:44:48.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 -- 172.21.15.12:0/3204319916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaa41eba40 con 0x7fdaa40c27c0 2023-11-28T21:44:48.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fda9affd640 1 -- 172.21.15.12:0/3204319916 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fdaa41eba40 con 0x7fdaa40c27c0 2023-11-28T21:44:48.804 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:48.804 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:48.804 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 43 2023-11-28T21:44:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 -- 172.21.15.12:0/3204319916 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda88066470 msgr2=0x7fda88068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda88066470 0x7fda88068930 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009bb0 tx=0x7fdaa0008040 comp rx=0 tx=0).stop 2023-11-28T21:44:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 -- 172.21.15.12:0/3204319916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa40c27c0 msgr2=0x7fdaa41ebd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa40c27c0 0x7fdaa41ebd20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fda9400c480 tx=0x7fda9400c940 comp rx=0 tx=0).stop 2023-11-28T21:44:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 -- 172.21.15.12:0/3204319916 shutdown_connections 2023-11-28T21:44:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fda88066470 0x7fda88068930 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaa4156af0 0x7fdaa41ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdaa40c27c0 0x7fdaa41ebd20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 --2- 172.21.15.12:0/3204319916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa406cf40 0x7fdaa4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:48.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 -- 172.21.15.12:0/3204319916 >> 172.21.15.12:0/3204319916 conn(0x7fdaa4066360 msgr2=0x7fdaa40c2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.798+0000 7fdaac29e640 1 -- 172.21.15.12:0/3204319916 shutdown_connections 2023-11-28T21:44:48.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:48.802+0000 7fdaac29e640 1 -- 172.21.15.12:0/3204319916 wait complete. 2023-11-28T21:44:48.825 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2023-11-28T21:44:50.826 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:44:50.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.974+0000 7f96914fe640 1 Processor -- start 2023-11-28T21:44:50.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.974+0000 7f96914fe640 1 -- start start 2023-11-28T21:44:50.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f96914fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f968c150c20 0x7f968c156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f96914fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06cf40 0x7f968c158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f96914fe640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c067410 con 0x7f968c06e300 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f96914fe640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c06e090 con 0x7f968c150c20 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f96914fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c157f30 con 0x7f968c06cf40 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f968affd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f968c150c20 0x7f968c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f968affd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f968c150c20 0x7f968c156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40786/0 (socket says 172.21.15.12:40786) 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f968affd640 1 -- 172.21.15.12:0/996019475 learned_addr learned my addr 172.21.15.12:0/996019475 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f968b7fe640 1 --2- 172.21.15.12:0/996019475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06cf40 0x7f968c158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f968affd640 1 -- 172.21.15.12:0/996019475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06cf40 msgr2=0x7f968c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f968affd640 1 --2- 172.21.15.12:0/996019475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06cf40 0x7f968c158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f968affd640 1 -- 172.21.15.12:0/996019475 >> v1:172.21.15.144:6789/0 conn(0x7f968c06e300 legacy=0x7f968c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:50.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f968affd640 1 -- 172.21.15.12:0/996019475 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c151000 con 0x7f968c150c20 2023-11-28T21:44:50.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.978+0000 7f968affd640 1 --2- 172.21.15.12:0/996019475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c150c20 0x7f968c156910 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f967c00bb10 tx=0x7f967c01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cbb1fcfef7aa2e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f968a7fc640 1 -- 172.21.15.12:0/996019475 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c0a7af0 con 0x7f968c150c20 2023-11-28T21:44:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f968a7fc640 1 -- 172.21.15.12:0/996019475 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c0a7c50 con 0x7f968c150c20 2023-11-28T21:44:50.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f968a7fc640 1 -- 172.21.15.12:0/996019475 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c0af440 con 0x7f968c150c20 2023-11-28T21:44:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 -- 172.21.15.12:0/996019475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c150c20 msgr2=0x7f968c156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 --2- 172.21.15.12:0/996019475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c150c20 0x7f968c156910 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f967c00bb10 tx=0x7f967c01ec10 comp rx=0 tx=0).stop 2023-11-28T21:44:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 -- 172.21.15.12:0/996019475 shutdown_connections 2023-11-28T21:44:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 --2- 172.21.15.12:0/996019475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06cf40 0x7f968c158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 --2- 172.21.15.12:0/996019475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c150c20 0x7f968c156910 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 -- 172.21.15.12:0/996019475 >> 172.21.15.12:0/996019475 conn(0x7f968c066360 msgr2=0x7f968c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 -- 172.21.15.12:0/996019475 shutdown_connections 2023-11-28T21:44:50.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 -- 172.21.15.12:0/996019475 wait complete. 2023-11-28T21:44:50.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 Processor -- start 2023-11-28T21:44:50.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 -- start start 2023-11-28T21:44:50.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06cf40 0x7f968c0c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:50.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c06e300 0x7f968c14c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:50.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f968c150c20 0x7f968c156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:50.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c155bc0 con 0x7f968c06e300 2023-11-28T21:44:50.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0c4850 con 0x7f968c150c20 2023-11-28T21:44:50.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f96914fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0c3da0 con 0x7f968c06cf40 2023-11-28T21:44:50.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f9683fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c06e300 0x7f968c14c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:50.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f9683fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c06e300 0x7f968c14c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40800/0 (socket says 172.21.15.12:40800) 2023-11-28T21:44:50.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f9683fff640 1 -- 172.21.15.12:0/1211819702 learned_addr learned my addr 172.21.15.12:0/1211819702 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:44:50.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f968affd640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06cf40 0x7f968c0c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:50.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.982+0000 7f968b7fe640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f968c150c20 0x7f968c156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:50.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f9683fff640 1 -- 172.21.15.12:0/1211819702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06cf40 msgr2=0x7f968c0c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:50.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f9683fff640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06cf40 0x7f968c0c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:50.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f9683fff640 1 -- 172.21.15.12:0/1211819702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f968c150c20 msgr2=0x7f968c156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:50.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f9683fff640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f968c150c20 0x7f968c156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:50.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f9683fff640 1 -- 172.21.15.12:0/1211819702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c00b770 con 0x7f968c06e300 2023-11-28T21:44:50.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f9683fff640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c06e300 0x7f968c14c540 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f967400c4d0 tx=0x7f967400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:50.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f96837fe640 1 -- 172.21.15.12:0/1211819702 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96740052d0 con 0x7f968c06e300 2023-11-28T21:44:50.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f96837fe640 1 -- 172.21.15.12:0/1211819702 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9674005430 con 0x7f968c06e300 2023-11-28T21:44:50.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f96837fe640 1 -- 172.21.15.12:0/1211819702 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674012830 con 0x7f968c06e300 2023-11-28T21:44:50.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f96914fe640 1 -- 172.21.15.12:0/1211819702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c0c4af0 con 0x7f968c06e300 2023-11-28T21:44:50.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f96914fe640 1 -- 172.21.15.12:0/1211819702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c0c50b0 con 0x7f968c06e300 2023-11-28T21:44:50.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f96914fe640 1 -- 172.21.15.12:0/1211819702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c155d70 con 0x7f968c06e300 2023-11-28T21:44:50.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f96837fe640 1 -- 172.21.15.12:0/1211819702 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9674012990 con 0x7f968c06e300 2023-11-28T21:44:50.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f96837fe640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f966c0664a0 0x7f966c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:44:50.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f96837fe640 1 -- 172.21.15.12:0/1211819702 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f9674083f70 con 0x7f968c06e300 2023-11-28T21:44:50.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.986+0000 7f968affd640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f966c0664a0 0x7f966c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:44:50.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.990+0000 7f968affd640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f966c0664a0 0x7f966c068960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f967c0b1000 tx=0x7f967c0a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:44:50.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:50.990+0000 7f96837fe640 1 -- 172.21.15.12:0/1211819702 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f968c155d70 con 0x7f968c06e300 2023-11-28T21:44:51.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.182+0000 7f96914fe640 1 -- 172.21.15.12:0/1211819702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f968c074f60 con 0x7f968c06e300 2023-11-28T21:44:51.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.182+0000 7f96837fe640 1 -- 172.21.15.12:0/1211819702 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f9674050db0 con 0x7f968c06e300 2023-11-28T21:44:51.189 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:44:51.189 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:44:51.189 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-11-28T21:44:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 -- 172.21.15.12:0/1211819702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f966c0664a0 msgr2=0x7f966c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f966c0664a0 0x7f966c068960 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f967c0b1000 tx=0x7f967c0a9070 comp rx=0 tx=0).stop 2023-11-28T21:44:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 -- 172.21.15.12:0/1211819702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c06e300 msgr2=0x7f968c14c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:44:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c06e300 0x7f968c14c540 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f967400c4d0 tx=0x7f967400c990 comp rx=0 tx=0).stop 2023-11-28T21:44:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 -- 172.21.15.12:0/1211819702 shutdown_connections 2023-11-28T21:44:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f966c0664a0 0x7f966c068960 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f968c150c20 0x7f968c156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f968c06e300 0x7f968c14c540 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:51.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 --2- 172.21.15.12:0/1211819702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06cf40 0x7f968c0c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:44:51.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 -- 172.21.15.12:0/1211819702 >> 172.21.15.12:0/1211819702 conn(0x7f968c066360 msgr2=0x7f968c14fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:44:51.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 -- 172.21.15.12:0/1211819702 shutdown_connections 2023-11-28T21:44:51.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:44:51.186+0000 7f96914fe640 1 -- 172.21.15.12:0/1211819702 wait complete. 2023-11-28T21:44:51.210 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2023-11-28T21:44:51.210 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 15 secs before thrashing 2023-11-28T21:45:06.211 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5abe78d640 1 Processor -- start 2023-11-28T21:45:06.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5abe78d640 1 -- start start 2023-11-28T21:45:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5abe78d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 0x7f5ab8158040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5abe78d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5ab8158580 0x7f5ab80712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5abe78d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab815af70 con 0x7f5ab806f050 2023-11-28T21:45:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5abe78d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab8156b50 con 0x7f5ab8158580 2023-11-28T21:45:06.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5abe78d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab8156e10 con 0x7f5ab8150e00 2023-11-28T21:45:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5ab7fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 0x7f5ab8158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5ab77fe640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f5ab806f050 0x7f5ab8070ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:60582/0 (socket says 172.21.15.12:60582) 2023-11-28T21:45:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5ab77fe640 1 -- 172.21.15.12:0/2102603019 learned_addr learned my addr 172.21.15.12:0/2102603019 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:45:06.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.554+0000 7f5abcd2a640 1 --2- 172.21.15.12:0/2102603019 >> v2:172.21.15.144:3300/0 conn(0x7f5ab8158580 0x7f5ab80712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.558+0000 7f5ab7fff640 1 -- 172.21.15.12:0/2102603019 >> v2:172.21.15.144:3300/0 conn(0x7f5ab8158580 msgr2=0x7f5ab80712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.558+0000 7f5ab7fff640 1 --2- 172.21.15.12:0/2102603019 >> v2:172.21.15.144:3300/0 conn(0x7f5ab8158580 0x7f5ab80712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.558+0000 7f5ab7fff640 1 -- 172.21.15.12:0/2102603019 >> v1:172.21.15.144:6789/0 conn(0x7f5ab806f050 legacy=0x7f5ab8070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:06.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.558+0000 7f5ab7fff640 1 -- 172.21.15.12:0/2102603019 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab81570f0 con 0x7f5ab8150e00 2023-11-28T21:45:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.562+0000 7f5ab7fff640 1 --2- 172.21.15.12:0/2102603019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 0x7f5ab8158040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009d90 tx=0x7f5ab001c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=292d7afd2d15b1ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.570+0000 7f5ab6ffd640 1 -- 172.21.15.12:0/2102603019 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00a5890 con 0x7f5ab8150e00 2023-11-28T21:45:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.570+0000 7f5ab6ffd640 1 -- 172.21.15.12:0/2102603019 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab00a59f0 con 0x7f5ab8150e00 2023-11-28T21:45:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.570+0000 7f5ab6ffd640 1 -- 172.21.15.12:0/2102603019 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00a5d80 con 0x7f5ab8150e00 2023-11-28T21:45:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.570+0000 7f5abe78d640 1 -- 172.21.15.12:0/2102603019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 msgr2=0x7f5ab8158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.570+0000 7f5abe78d640 1 --2- 172.21.15.12:0/2102603019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 0x7f5ab8158040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009d90 tx=0x7f5ab001c870 comp rx=0 tx=0).stop 2023-11-28T21:45:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.574+0000 7f5abe78d640 1 -- 172.21.15.12:0/2102603019 shutdown_connections 2023-11-28T21:45:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.574+0000 7f5abe78d640 1 --2- 172.21.15.12:0/2102603019 >> v2:172.21.15.144:3300/0 conn(0x7f5ab8158580 0x7f5ab80712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.574+0000 7f5abe78d640 1 --2- 172.21.15.12:0/2102603019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 0x7f5ab8158040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:06.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.574+0000 7f5abe78d640 1 -- 172.21.15.12:0/2102603019 >> 172.21.15.12:0/2102603019 conn(0x7f5ab806b6c0 msgr2=0x7f5ab806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.574+0000 7f5abe78d640 1 -- 172.21.15.12:0/2102603019 shutdown_connections 2023-11-28T21:45:06.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.574+0000 7f5abe78d640 1 -- 172.21.15.12:0/2102603019 wait complete. 2023-11-28T21:45:06.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.578+0000 7f5abe78d640 1 Processor -- start 2023-11-28T21:45:06.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.578+0000 7f5abe78d640 1 -- start start 2023-11-28T21:45:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.578+0000 7f5abe78d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab806f050 0x7f5ab80cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.578+0000 7f5abe78d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 0x7f5ab80cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.578+0000 7f5abe78d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab8158580 0x7f5ab81f4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abe78d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab814ec50 con 0x7f5ab8158580 2023-11-28T21:45:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abe78d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab814ef10 con 0x7f5ab806f050 2023-11-28T21:45:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abe78d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80c8050 con 0x7f5ab8150e00 2023-11-28T21:45:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5ab7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab806f050 0x7f5ab80cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:06.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5ab7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab806f050 0x7f5ab80cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:47928/0 (socket says 172.21.15.12:47928) 2023-11-28T21:45:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5ab7fff640 1 -- 172.21.15.12:0/2905392067 learned_addr learned my addr 172.21.15.12:0/2905392067 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abcd2a640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab8158580 0x7f5ab81f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5ab77fe640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 0x7f5ab80cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abcd2a640 1 -- 172.21.15.12:0/2905392067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 msgr2=0x7f5ab80cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:06.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abcd2a640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 0x7f5ab80cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abcd2a640 1 -- 172.21.15.12:0/2905392067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab806f050 msgr2=0x7f5ab80cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abcd2a640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab806f050 0x7f5ab80cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abcd2a640 1 -- 172.21.15.12:0/2905392067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab0009a70 con 0x7f5ab8158580 2023-11-28T21:45:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abcd2a640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab8158580 0x7f5ab81f4450 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800e950 tx=0x7f5aa800ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5ab4ff9640 1 -- 172.21.15.12:0/2905392067 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa800dd40 con 0x7f5ab8158580 2023-11-28T21:45:06.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5ab4ff9640 1 -- 172.21.15.12:0/2905392067 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa8010440 con 0x7f5ab8158580 2023-11-28T21:45:06.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5ab4ff9640 1 -- 172.21.15.12:0/2905392067 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa800dea0 con 0x7f5ab8158580 2023-11-28T21:45:06.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abe78d640 1 -- 172.21.15.12:0/2905392067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab80c8330 con 0x7f5ab8158580 2023-11-28T21:45:06.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.582+0000 7f5abe78d640 1 -- 172.21.15.12:0/2905392067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab80c88f0 con 0x7f5ab8158580 2023-11-28T21:45:06.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.590+0000 7f5abe78d640 1 -- 172.21.15.12:0/2905392067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab8074f60 con 0x7f5ab8158580 2023-11-28T21:45:06.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.590+0000 7f5ab4ff9640 1 -- 172.21.15.12:0/2905392067 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5aa80107f0 con 0x7f5ab8158580 2023-11-28T21:45:06.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.594+0000 7f5ab4ff9640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa4066540 0x7f5aa4068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:06.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.594+0000 7f5ab4ff9640 1 -- 172.21.15.12:0/2905392067 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f5aa8014070 con 0x7f5ab8158580 2023-11-28T21:45:06.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.598+0000 7f5ab7fff640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa4066540 0x7f5aa4068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:06.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.598+0000 7f5ab7fff640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa4066540 0x7f5aa4068a00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0000f90 tx=0x7f5ab00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:06.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:06.606+0000 7f5ab4ff9640 1 -- 172.21.15.12:0/2905392067 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5aa8051840 con 0x7f5ab8158580 2023-11-28T21:45:07.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.006+0000 7f5abe78d640 1 -- 172.21.15.12:0/2905392067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ab8154c60 con 0x7f5ab8158580 2023-11-28T21:45:07.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5ab4ff9640 1 -- 172.21.15.12:0/2905392067 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f5aa8051840 con 0x7f5ab8158580 2023-11-28T21:45:07.016 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:07.016 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:07.017 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-11-28T21:45:07.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 -- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa4066540 msgr2=0x7f5aa4068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:07.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa4066540 0x7f5aa4068a00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0000f90 tx=0x7f5ab00a7070 comp rx=0 tx=0).stop 2023-11-28T21:45:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 -- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab8158580 msgr2=0x7f5ab81f4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab8158580 0x7f5ab81f4450 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800e950 tx=0x7f5aa800ee10 comp rx=0 tx=0).stop 2023-11-28T21:45:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 -- 172.21.15.12:0/2905392067 shutdown_connections 2023-11-28T21:45:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa4066540 0x7f5aa4068a00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab8158580 0x7f5ab81f4450 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab8150e00 0x7f5ab80cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 --2- 172.21.15.12:0/2905392067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab806f050 0x7f5ab80cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 -- 172.21.15.12:0/2905392067 >> 172.21.15.12:0/2905392067 conn(0x7f5ab806b6c0 msgr2=0x7f5ab8071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 -- 172.21.15.12:0/2905392067 shutdown_connections 2023-11-28T21:45:07.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.010+0000 7f5abe78d640 1 -- 172.21.15.12:0/2905392067 wait complete. 2023-11-28T21:45:07.042 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2023-11-28T21:45:07.042 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-11-28T21:45:07.042 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:45:07.066 INFO:tasks.ceph.mds.e:Stopped 2023-11-28T21:45:07.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:07.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.250+0000 7f47c3980640 1 Processor -- start 2023-11-28T21:45:07.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.250+0000 7f47c3980640 1 -- start start 2023-11-28T21:45:07.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c3980640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f47bc158420 0x7f47bc06d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:07.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c3980640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f47bc161600 0x7f47bc161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:07.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c3980640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47bc0c3460 con 0x7f47bc0c38f0 2023-11-28T21:45:07.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c3980640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47bc158800 con 0x7f47bc158420 2023-11-28T21:45:07.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c3980640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47bc0c52c0 con 0x7f47bc161600 2023-11-28T21:45:07.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c171c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f47bc158420 0x7f47bc06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:07.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c171c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f47bc158420 0x7f47bc06d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55074/0 (socket says 172.21.15.12:55074) 2023-11-28T21:45:07.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c171c640 1 -- 172.21.15.12:0/1897166431 learned_addr learned my addr 172.21.15.12:0/1897166431 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:07.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c1f1d640 1 --2- 172.21.15.12:0/1897166431 >> v2:172.21.15.144:3300/0 conn(0x7f47bc161600 0x7f47bc161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:07.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c171c640 1 -- 172.21.15.12:0/1897166431 >> v2:172.21.15.144:3300/0 conn(0x7f47bc161600 msgr2=0x7f47bc161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:07.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c171c640 1 --2- 172.21.15.12:0/1897166431 >> v2:172.21.15.144:3300/0 conn(0x7f47bc161600 0x7f47bc161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c171c640 1 -- 172.21.15.12:0/1897166431 >> v1:172.21.15.12:6789/0 conn(0x7f47bc0c38f0 legacy=0x7f47bc0c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:07.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.254+0000 7f47c171c640 1 -- 172.21.15.12:0/1897166431 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc0c2b70 con 0x7f47bc158420 2023-11-28T21:45:07.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c171c640 1 --2- 172.21.15.12:0/1897166431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 0x7f47bc06d950 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0c5510 tx=0x7f47b801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897d34474ff3ca18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:07.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47b3fff640 1 -- 172.21.15.12:0/1897166431 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80ab600 con 0x7f47bc158420 2023-11-28T21:45:07.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47b3fff640 1 -- 172.21.15.12:0/1897166431 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b80a7050 con 0x7f47bc158420 2023-11-28T21:45:07.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47b3fff640 1 -- 172.21.15.12:0/1897166431 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80ab980 con 0x7f47bc158420 2023-11-28T21:45:07.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c3980640 1 -- 172.21.15.12:0/1897166431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 msgr2=0x7f47bc06d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:07.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c3980640 1 --2- 172.21.15.12:0/1897166431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 0x7f47bc06d950 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0c5510 tx=0x7f47b801c500 comp rx=0 tx=0).stop 2023-11-28T21:45:07.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c3980640 1 -- 172.21.15.12:0/1897166431 shutdown_connections 2023-11-28T21:45:07.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c3980640 1 --2- 172.21.15.12:0/1897166431 >> v2:172.21.15.144:3300/0 conn(0x7f47bc161600 0x7f47bc161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.263 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c3980640 1 --2- 172.21.15.12:0/1897166431 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 0x7f47bc06d950 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c3980640 1 -- 172.21.15.12:0/1897166431 >> 172.21.15.12:0/1897166431 conn(0x7f47bc066360 msgr2=0x7f47bc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:07.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c3980640 1 -- 172.21.15.12:0/1897166431 shutdown_connections 2023-11-28T21:45:07.264 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c3980640 1 -- 172.21.15.12:0/1897166431 wait complete. 2023-11-28T21:45:07.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c3980640 1 Processor -- start 2023-11-28T21:45:07.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.258+0000 7f47c3980640 1 -- start start 2023-11-28T21:45:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c3980640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47bc0c38f0 0x7f47bc15f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c3980640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 0x7f47bc14a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c3980640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47bc161600 0x7f47bc14e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c3980640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc1613e0 con 0x7f47bc158420 2023-11-28T21:45:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c3980640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc158ab0 con 0x7f47bc161600 2023-11-28T21:45:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c3980640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc161100 con 0x7f47bc0c38f0 2023-11-28T21:45:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c0f1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 0x7f47bc14a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:07.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c0f1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 0x7f47bc14a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55084/0 (socket says 172.21.15.12:55084) 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c0f1b640 1 -- 172.21.15.12:0/1760344621 learned_addr learned my addr 172.21.15.12:0/1760344621 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c171c640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47bc0c38f0 0x7f47bc15f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c1f1d640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47bc161600 0x7f47bc14e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c0f1b640 1 -- 172.21.15.12:0/1760344621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47bc0c38f0 msgr2=0x7f47bc15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c0f1b640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47bc0c38f0 0x7f47bc15f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c0f1b640 1 -- 172.21.15.12:0/1760344621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47bc161600 msgr2=0x7f47bc14e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c0f1b640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47bc161600 0x7f47bc14e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c0f1b640 1 -- 172.21.15.12:0/1760344621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b80094d0 con 0x7f47bc158420 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c0f1b640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 0x7f47bc14a930 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c500 tx=0x7f47ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47b1ffb640 1 -- 172.21.15.12:0/1760344621 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac005220 con 0x7f47bc158420 2023-11-28T21:45:07.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47b1ffb640 1 -- 172.21.15.12:0/1760344621 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ac005380 con 0x7f47bc158420 2023-11-28T21:45:07.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47b1ffb640 1 -- 172.21.15.12:0/1760344621 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ac007270 con 0x7f47bc158420 2023-11-28T21:45:07.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c3980640 1 -- 172.21.15.12:0/1760344621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47bc155530 con 0x7f47bc158420 2023-11-28T21:45:07.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c3980640 1 -- 172.21.15.12:0/1760344621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc155af0 con 0x7f47bc158420 2023-11-28T21:45:07.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47c3980640 1 -- 172.21.15.12:0/1760344621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47bc06c1d0 con 0x7f47bc158420 2023-11-28T21:45:07.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.262+0000 7f47b1ffb640 1 -- 172.21.15.12:0/1760344621 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f47ac0073d0 con 0x7f47bc158420 2023-11-28T21:45:07.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.266+0000 7f47b1ffb640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f47a00664a0 0x7f47a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:07.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.266+0000 7f47b1ffb640 1 -- 172.21.15.12:0/1760344621 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f47ac083d10 con 0x7f47bc158420 2023-11-28T21:45:07.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.266+0000 7f47c171c640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f47a00664a0 0x7f47a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:07.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.266+0000 7f47c171c640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f47a00664a0 0x7f47a0068960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f47b80ad000 tx=0x7f47b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:07.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.270+0000 7f47b1ffb640 1 -- 172.21.15.12:0/1760344621 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f47ac050bf0 con 0x7f47bc158420 2023-11-28T21:45:07.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.462+0000 7f47c3980640 1 -- 172.21.15.12:0/1760344621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47bc074f60 con 0x7f47bc158420 2023-11-28T21:45:07.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.462+0000 7f47b1ffb640 1 -- 172.21.15.12:0/1760344621 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f47ac054210 con 0x7f47bc158420 2023-11-28T21:45:07.468 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:07.468 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:07.468 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-11-28T21:45:07.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 -- 172.21.15.12:0/1760344621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f47a00664a0 msgr2=0x7f47a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:07.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f47a00664a0 0x7f47a0068960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f47b80ad000 tx=0x7f47b8008040 comp rx=0 tx=0).stop 2023-11-28T21:45:07.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 -- 172.21.15.12:0/1760344621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 msgr2=0x7f47bc14a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:07.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 0x7f47bc14a930 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c500 tx=0x7f47ac00c9c0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 -- 172.21.15.12:0/1760344621 shutdown_connections 2023-11-28T21:45:07.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f47a00664a0 0x7f47a0068960 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47bc161600 0x7f47bc14e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47bc158420 0x7f47bc14a930 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 --2- 172.21.15.12:0/1760344621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47bc0c38f0 0x7f47bc15f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:07.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 -- 172.21.15.12:0/1760344621 >> 172.21.15.12:0/1760344621 conn(0x7f47bc066360 msgr2=0x7f47bc0c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:07.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 -- 172.21.15.12:0/1760344621 shutdown_connections 2023-11-28T21:45:07.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:07.466+0000 7f47c3980640 1 -- 172.21.15.12:0/1760344621 wait complete. 2023-11-28T21:45:07.489 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:45:09.490 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:09.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcafb846640 1 Processor -- start 2023-11-28T21:45:09.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcafb846640 1 -- start start 2023-11-28T21:45:09.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcafb846640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcaf40cd410 0x7fcaf40cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:09.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcafb846640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaf415d7f0 0x7fcaf415dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:09.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcafb846640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf40c7710 con 0x7fcaf4153e00 2023-11-28T21:45:09.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcafb846640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf41541e0 con 0x7fcaf40cd410 2023-11-28T21:45:09.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcafb846640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf415e180 con 0x7fcaf415d7f0 2023-11-28T21:45:09.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcaf8de1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcaf40cd410 0x7fcaf40cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:09.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcaf8de1640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcaf40cd410 0x7fcaf40cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55088/0 (socket says 172.21.15.12:55088) 2023-11-28T21:45:09.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcaf8de1640 1 -- 172.21.15.12:0/511147665 learned_addr learned my addr 172.21.15.12:0/511147665 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:09.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcaf9de3640 1 --2- 172.21.15.12:0/511147665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaf415d7f0 0x7fcaf415dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:09.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcaf8de1640 1 -- 172.21.15.12:0/511147665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaf415d7f0 msgr2=0x7fcaf415dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:09.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcaf8de1640 1 --2- 172.21.15.12:0/511147665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaf415d7f0 0x7fcaf415dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:09.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcaf8de1640 1 -- 172.21.15.12:0/511147665 >> v1:172.21.15.12:6789/0 conn(0x7fcaf4153e00 legacy=0x7fcaf406b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:09.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.642+0000 7fcaf8de1640 1 -- 172.21.15.12:0/511147665 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf415a010 con 0x7fcaf40cd410 2023-11-28T21:45:09.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcaf8de1640 1 --2- 172.21.15.12:0/511147665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf40cd410 0x7fcaf40cd8a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009870 tx=0x7fcae401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4b04de3b32b42 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:09.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcaebfff640 1 -- 172.21.15.12:0/511147665 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae40a8a30 con 0x7fcaf40cd410 2023-11-28T21:45:09.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcaebfff640 1 -- 172.21.15.12:0/511147665 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae40a8b90 con 0x7fcaf40cd410 2023-11-28T21:45:09.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcaebfff640 1 -- 172.21.15.12:0/511147665 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae40b0440 con 0x7fcaf40cd410 2023-11-28T21:45:09.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcafb846640 1 -- 172.21.15.12:0/511147665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf40cd410 msgr2=0x7fcaf40cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:09.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcafb846640 1 --2- 172.21.15.12:0/511147665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf40cd410 0x7fcaf40cd8a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009870 tx=0x7fcae401ed30 comp rx=0 tx=0).stop 2023-11-28T21:45:09.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcafb846640 1 -- 172.21.15.12:0/511147665 shutdown_connections 2023-11-28T21:45:09.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcafb846640 1 --2- 172.21.15.12:0/511147665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaf415d7f0 0x7fcaf415dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:09.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcafb846640 1 --2- 172.21.15.12:0/511147665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf40cd410 0x7fcaf40cd8a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:09.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcafb846640 1 -- 172.21.15.12:0/511147665 >> 172.21.15.12:0/511147665 conn(0x7fcaf4066360 msgr2=0x7fcaf4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:09.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcafb846640 1 -- 172.21.15.12:0/511147665 shutdown_connections 2023-11-28T21:45:09.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.646+0000 7fcafb846640 1 -- 172.21.15.12:0/511147665 wait complete. 2023-11-28T21:45:09.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcafb846640 1 Processor -- start 2023-11-28T21:45:09.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcafb846640 1 -- start start 2023-11-28T21:45:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcafb846640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40cd410 0x7fcaf40c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcafb846640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaf4153e00 0x7fcaf414a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcafb846640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf415d7f0 0x7fcaf4145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcafb846640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf41f2ba0 con 0x7fcaf415d7f0 2023-11-28T21:45:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcafb846640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40c9740 con 0x7fcaf40cd410 2023-11-28T21:45:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcafb846640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40c8e50 con 0x7fcaf4153e00 2023-11-28T21:45:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf9de3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf415d7f0 0x7fcaf4145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf9de3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf415d7f0 0x7fcaf4145be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55100/0 (socket says 172.21.15.12:55100) 2023-11-28T21:45:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf9de3640 1 -- 172.21.15.12:0/3367108280 learned_addr learned my addr 172.21.15.12:0/3367108280 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf95e2640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40cd410 0x7fcaf40c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf8de1640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaf4153e00 0x7fcaf414a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf9de3640 1 -- 172.21.15.12:0/3367108280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaf4153e00 msgr2=0x7fcaf414a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf9de3640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaf4153e00 0x7fcaf414a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf9de3640 1 -- 172.21.15.12:0/3367108280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40cd410 msgr2=0x7fcaf40c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf9de3640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40cd410 0x7fcaf40c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf9de3640 1 -- 172.21.15.12:0/3367108280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae40094d0 con 0x7fcaf415d7f0 2023-11-28T21:45:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcaf9de3640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf415d7f0 0x7fcaf4145be0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00efc0 tx=0x7fcaec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcae9ffb640 1 -- 172.21.15.12:0/3367108280 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec009ba0 con 0x7fcaf415d7f0 2023-11-28T21:45:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcae9ffb640 1 -- 172.21.15.12:0/3367108280 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec00f040 con 0x7fcaf415d7f0 2023-11-28T21:45:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcae9ffb640 1 -- 172.21.15.12:0/3367108280 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0146c0 con 0x7fcaf415d7f0 2023-11-28T21:45:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.650+0000 7fcafb846640 1 -- 172.21.15.12:0/3367108280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf40c9440 con 0x7fcaf415d7f0 2023-11-28T21:45:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.654+0000 7fcafb846640 1 -- 172.21.15.12:0/3367108280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf4151a00 con 0x7fcaf415d7f0 2023-11-28T21:45:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.654+0000 7fcafb846640 1 -- 172.21.15.12:0/3367108280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf406c620 con 0x7fcaf415d7f0 2023-11-28T21:45:09.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.654+0000 7fcae9ffb640 1 -- 172.21.15.12:0/3367108280 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fcaec014820 con 0x7fcaf415d7f0 2023-11-28T21:45:09.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.654+0000 7fcae9ffb640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcad80664a0 0x7fcad8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:09.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.654+0000 7fcae9ffb640 1 -- 172.21.15.12:0/3367108280 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7fcaec0851f0 con 0x7fcaf415d7f0 2023-11-28T21:45:09.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.654+0000 7fcaf95e2640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcad80664a0 0x7fcad8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:09.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.654+0000 7fcaf95e2640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcad80664a0 0x7fcad8068960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0000f70 tx=0x7fcaf0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:09.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.662+0000 7fcae9ffb640 1 -- 172.21.15.12:0/3367108280 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcaec052150 con 0x7fcaf415d7f0 2023-11-28T21:45:09.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 -- 172.21.15.12:0/3367108280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaf40c98b0 con 0x7fcaf415d7f0 2023-11-28T21:45:09.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcae9ffb640 1 -- 172.21.15.12:0/3367108280 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fcaec055770 con 0x7fcaf415d7f0 2023-11-28T21:45:09.855 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:09.855 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:09.856 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-11-28T21:45:09.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 -- 172.21.15.12:0/3367108280 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcad80664a0 msgr2=0x7fcad8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:09.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcad80664a0 0x7fcad8068960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0000f70 tx=0x7fcaf0009040 comp rx=0 tx=0).stop 2023-11-28T21:45:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 -- 172.21.15.12:0/3367108280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf415d7f0 msgr2=0x7fcaf4145be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf415d7f0 0x7fcaf4145be0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00efc0 tx=0x7fcaec00c490 comp rx=0 tx=0).stop 2023-11-28T21:45:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 -- 172.21.15.12:0/3367108280 shutdown_connections 2023-11-28T21:45:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcad80664a0 0x7fcad8068960 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcaf415d7f0 0x7fcaf4145be0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcaf4153e00 0x7fcaf414a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 --2- 172.21.15.12:0/3367108280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcaf40cd410 0x7fcaf40c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 -- 172.21.15.12:0/3367108280 >> 172.21.15.12:0/3367108280 conn(0x7fcaf4066360 msgr2=0x7fcaf40cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 -- 172.21.15.12:0/3367108280 shutdown_connections 2023-11-28T21:45:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:09.850+0000 7fcafb846640 1 -- 172.21.15.12:0/3367108280 wait complete. 2023-11-28T21:45:09.877 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:45:11.878 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:12.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4489df5640 1 Processor -- start 2023-11-28T21:45:12.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4489df5640 1 -- start start 2023-11-28T21:45:12.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4489df5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415d320 0x7f448406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:12.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4489df5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448415e270 con 0x7f448406aaf0 2023-11-28T21:45:12.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4489df5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44840c4e60 con 0x7f4484154100 2023-11-28T21:45:12.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4489df5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448415d740 con 0x7f448415d320 2023-11-28T21:45:12.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4483fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f448406aaf0 0x7f44840c52f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45420/0 (socket says 172.21.15.12:45420) 2023-11-28T21:45:12.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4483fff640 1 -- 172.21.15.12:0/369906156 learned_addr learned my addr 172.21.15.12:0/369906156 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:45:12.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4482ffd640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415d320 0x7f448406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:12.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417495699 0 0) 0x7f448415e270 con 0x7f448406aaf0 2023-11-28T21:45:12.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f446c003530 con 0x7f448406aaf0 2023-11-28T21:45:12.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270197209 0 0) 0x7f44840c4e60 con 0x7f4484154100 2023-11-28T21:45:12.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f448415e270 con 0x7f4484154100 2023-11-28T21:45:12.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3265080434 0 0) 0x7f446c003530 con 0x7f448406aaf0 2023-11-28T21:45:12.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415d320 msgr2=0x7f448406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:12.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415d320 0x7f448406a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:12.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 >> v1:172.21.15.144:6789/0 conn(0x7f4484154100 legacy=0x7f44840c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:12.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44841544e0 con 0x7f448406aaf0 2023-11-28T21:45:12.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f447c001a00 con 0x7f448406aaf0 2023-11-28T21:45:12.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 >> v1:172.21.15.12:6789/0 conn(0x7f448406aaf0 legacy=0x7f44840c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:12.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415d320 0x7f446c003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004110 0x7f446c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f446c004d10 0x7f446c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448415d740 con 0x7f446c004d10 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c00bad0 con 0x7f446c004110 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c00bc90 con 0x7f448415d320 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4483fff640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f446c004d10 0x7f446c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f44837fe640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415d320 0x7f446c003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4482ffd640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004110 0x7f446c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4483fff640 1 -- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415d320 msgr2=0x7f446c003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4483fff640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415d320 0x7f446c003bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:12.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4483fff640 1 -- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004110 msgr2=0x7f446c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:12.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4483fff640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004110 0x7f446c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:12.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.022+0000 7f4483fff640 1 -- 172.21.15.12:0/369906156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c003530 con 0x7f446c004d10 2023-11-28T21:45:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f4483fff640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f446c004d10 0x7f446c005190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f447c00a7a0 tx=0x7f447c01fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1d062a588be95a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0aa630 con 0x7f446c004d10 2023-11-28T21:45:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c0aa7b0 con 0x7f446c004d10 2023-11-28T21:45:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f44827fc640 1 -- 172.21.15.12:0/369906156 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c0aaba0 con 0x7f446c004d10 2023-11-28T21:45:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f4489df5640 1 -- 172.21.15.12:0/369906156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f446c004d10 msgr2=0x7f446c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f4489df5640 1 --2- 172.21.15.12:0/369906156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f446c004d10 0x7f446c005190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f447c00a7a0 tx=0x7f447c01fa00 comp rx=0 tx=0).stop 2023-11-28T21:45:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f4483fff640 1 -- 172.21.15.12:0/369906156 reap_dead start 2023-11-28T21:45:12.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f4489df5640 1 -- 172.21.15.12:0/369906156 shutdown_connections 2023-11-28T21:45:12.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f4489df5640 1 -- 172.21.15.12:0/369906156 >> 172.21.15.12:0/369906156 conn(0x7f448406c690 msgr2=0x7f4484066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:12.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f4489df5640 1 -- 172.21.15.12:0/369906156 shutdown_connections 2023-11-28T21:45:12.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.026+0000 7f4489df5640 1 -- 172.21.15.12:0/369906156 wait complete. 2023-11-28T21:45:12.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4489df5640 1 Processor -- start 2023-11-28T21:45:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4489df5640 1 -- start start 2023-11-28T21:45:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4489df5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004d10 0x7f44841f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4489df5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44840c6f00 0x7f44841493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4489df5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415a970 0x7f44840b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4489df5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44840cc7e0 con 0x7f44840c6f00 2023-11-28T21:45:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4489df5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44840c8a70 con 0x7f446c004d10 2023-11-28T21:45:12.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4489df5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44840c8d30 con 0x7f448415a970 2023-11-28T21:45:12.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4482ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44840c6f00 0x7f44841493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:12.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4482ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44840c6f00 0x7f44841493e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55112/0 (socket says 172.21.15.12:55112) 2023-11-28T21:45:12.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4482ffd640 1 -- 172.21.15.12:0/1353458029 learned_addr learned my addr 172.21.15.12:0/1353458029 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:12.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4483fff640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415a970 0x7f44840b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:12.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f44837fe640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004d10 0x7f44841f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:12.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4482ffd640 1 -- 172.21.15.12:0/1353458029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415a970 msgr2=0x7f44840b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4482ffd640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415a970 0x7f44840b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4482ffd640 1 -- 172.21.15.12:0/1353458029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004d10 msgr2=0x7f44841f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4482ffd640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004d10 0x7f44841f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4482ffd640 1 -- 172.21.15.12:0/1353458029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44841f2ae0 con 0x7f44840c6f00 2023-11-28T21:45:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f44837fe640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004d10 0x7f44841f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:45:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4482ffd640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44840c6f00 0x7f44841493e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f44780119d0 tx=0x7f4478011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:12.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f446bfff640 1 -- 172.21.15.12:0/1353458029 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800fb50 con 0x7f44840c6f00 2023-11-28T21:45:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f446bfff640 1 -- 172.21.15.12:0/1353458029 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447800fcb0 con 0x7f44840c6f00 2023-11-28T21:45:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4489df5640 1 -- 172.21.15.12:0/1353458029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44841f2dc0 con 0x7f44840c6f00 2023-11-28T21:45:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f446bfff640 1 -- 172.21.15.12:0/1353458029 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478014440 con 0x7f44840c6f00 2023-11-28T21:45:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.030+0000 7f4489df5640 1 -- 172.21.15.12:0/1353458029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44840c8710 con 0x7f44840c6f00 2023-11-28T21:45:12.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.034+0000 7f4489df5640 1 -- 172.21.15.12:0/1353458029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4484074f60 con 0x7f44840c6f00 2023-11-28T21:45:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.034+0000 7f446bfff640 1 -- 172.21.15.12:0/1353458029 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f44780145a0 con 0x7f44840c6f00 2023-11-28T21:45:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.034+0000 7f446bfff640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44700664a0 0x7f4470068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.034+0000 7f446bfff640 1 -- 172.21.15.12:0/1353458029 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f4478084d00 con 0x7f44840c6f00 2023-11-28T21:45:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.034+0000 7f44837fe640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44700664a0 0x7f4470068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:12.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.034+0000 7f44837fe640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44700664a0 0x7f4470068960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4474007fd0 tx=0x7f4474009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:12.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.038+0000 7f446bfff640 1 -- 172.21.15.12:0/1353458029 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4478051c90 con 0x7f44840c6f00 2023-11-28T21:45:12.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.226+0000 7f4489df5640 1 -- 172.21.15.12:0/1353458029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44840b2bf0 con 0x7f44840c6f00 2023-11-28T21:45:12.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.226+0000 7f446bfff640 1 -- 172.21.15.12:0/1353458029 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f4478022020 con 0x7f44840c6f00 2023-11-28T21:45:12.233 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:12.234 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:12.235 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-11-28T21:45:12.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 -- 172.21.15.12:0/1353458029 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44700664a0 msgr2=0x7f4470068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:12.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44700664a0 0x7f4470068960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4474007fd0 tx=0x7f4474009040 comp rx=0 tx=0).stop 2023-11-28T21:45:12.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 -- 172.21.15.12:0/1353458029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44840c6f00 msgr2=0x7f44841493e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:12.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44840c6f00 0x7f44841493e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f44780119d0 tx=0x7f4478011e90 comp rx=0 tx=0).stop 2023-11-28T21:45:12.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 -- 172.21.15.12:0/1353458029 shutdown_connections 2023-11-28T21:45:12.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f44700664a0 0x7f4470068960 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:12.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f448415a970 0x7f44840b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:12.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f44840c6f00 0x7f44841493e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:12.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 --2- 172.21.15.12:0/1353458029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f446c004d10 0x7f44841f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:12.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 -- 172.21.15.12:0/1353458029 >> 172.21.15.12:0/1353458029 conn(0x7f448406c690 msgr2=0x7f44840cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:12.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 -- 172.21.15.12:0/1353458029 shutdown_connections 2023-11-28T21:45:12.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:12.230+0000 7f4489df5640 1 -- 172.21.15.12:0/1353458029 wait complete. 2023-11-28T21:45:12.255 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:45:14.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:45:14.095 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:45:14.257 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:14.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365c12a640 1 Processor -- start 2023-11-28T21:45:14.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365c12a640 1 -- start start 2023-11-28T21:45:14.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365c12a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f365415d320 0x7f365406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:14.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365c12a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f365406aaf0 0x7f36540c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:14.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365c12a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365415e270 con 0x7f3654154100 2023-11-28T21:45:14.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365c12a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36540c4e60 con 0x7f365406aaf0 2023-11-28T21:45:14.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365c12a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365415d740 con 0x7f365415d320 2023-11-28T21:45:14.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365a6c7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f365406aaf0 0x7f36540c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:14.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365a6c7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f365406aaf0 0x7f36540c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55120/0 (socket says 172.21.15.12:55120) 2023-11-28T21:45:14.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365a6c7640 1 -- 172.21.15.12:0/1722060841 learned_addr learned my addr 172.21.15.12:0/1722060841 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:14.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f36596c5640 1 --2- 172.21.15.12:0/1722060841 >> v2:172.21.15.144:3300/0 conn(0x7f365415d320 0x7f365406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:14.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365a6c7640 1 -- 172.21.15.12:0/1722060841 >> v2:172.21.15.144:3300/0 conn(0x7f365415d320 msgr2=0x7f365406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:14.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365a6c7640 1 --2- 172.21.15.12:0/1722060841 >> v2:172.21.15.144:3300/0 conn(0x7f365415d320 0x7f365406a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:14.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365a6c7640 1 -- 172.21.15.12:0/1722060841 >> v1:172.21.15.144:6789/0 conn(0x7f3654154100 legacy=0x7f36540c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:14.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.406+0000 7f365a6c7640 1 -- 172.21.15.12:0/1722060841 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36541544e0 con 0x7f365406aaf0 2023-11-28T21:45:14.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f365a6c7640 1 --2- 172.21.15.12:0/1722060841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365406aaf0 0x7f36540c52f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f365406ce00 tx=0x7f364c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a589490f1a2882f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:14.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f3658ec4640 1 -- 172.21.15.12:0/1722060841 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0a5850 con 0x7f365406aaf0 2023-11-28T21:45:14.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f3658ec4640 1 -- 172.21.15.12:0/1722060841 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c0a59b0 con 0x7f365406aaf0 2023-11-28T21:45:14.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f3658ec4640 1 -- 172.21.15.12:0/1722060841 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0a5d40 con 0x7f365406aaf0 2023-11-28T21:45:14.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f365c12a640 1 -- 172.21.15.12:0/1722060841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365406aaf0 msgr2=0x7f36540c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:14.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f365c12a640 1 --2- 172.21.15.12:0/1722060841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365406aaf0 0x7f36540c52f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f365406ce00 tx=0x7f364c01c990 comp rx=0 tx=0).stop 2023-11-28T21:45:14.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f365c12a640 1 -- 172.21.15.12:0/1722060841 shutdown_connections 2023-11-28T21:45:14.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f365c12a640 1 --2- 172.21.15.12:0/1722060841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365406aaf0 0x7f36540c52f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:14.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f365c12a640 1 --2- 172.21.15.12:0/1722060841 >> v2:172.21.15.144:3300/0 conn(0x7f365415d320 0x7f365406a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:14.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f365c12a640 1 -- 172.21.15.12:0/1722060841 >> 172.21.15.12:0/1722060841 conn(0x7f365406c690 msgr2=0x7f3654066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:14.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f365c12a640 1 -- 172.21.15.12:0/1722060841 shutdown_connections 2023-11-28T21:45:14.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.410+0000 7f365c12a640 1 -- 172.21.15.12:0/1722060841 wait complete. 2023-11-28T21:45:14.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365c12a640 1 Processor -- start 2023-11-28T21:45:14.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365c12a640 1 -- start start 2023-11-28T21:45:14.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365c12a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365406aaf0 0x7f36541505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:14.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365c12a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3654154100 0x7f3654152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:14.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365c12a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365415d320 0x7f3654144f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:14.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365c12a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36541f3740 con 0x7f365415d320 2023-11-28T21:45:14.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365c12a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365414dd60 con 0x7f365406aaf0 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365c12a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36541474d0 con 0x7f3654154100 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365a6c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365415d320 0x7f3654144f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365a6c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365415d320 0x7f3654144f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55124/0 (socket says 172.21.15.12:55124) 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365a6c7640 1 -- 172.21.15.12:0/674347739 learned_addr learned my addr 172.21.15.12:0/674347739 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f3659ec6640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365406aaf0 0x7f36541505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f36596c5640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3654154100 0x7f3654152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365a6c7640 1 -- 172.21.15.12:0/674347739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3654154100 msgr2=0x7f3654152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365a6c7640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3654154100 0x7f3654152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365a6c7640 1 -- 172.21.15.12:0/674347739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365406aaf0 msgr2=0x7f36541505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365a6c7640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365406aaf0 0x7f36541505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365a6c7640 1 -- 172.21.15.12:0/674347739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c009b30 con 0x7f365415d320 2023-11-28T21:45:14.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365a6c7640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365415d320 0x7f3654144f70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f364c01a5c0 tx=0x7f364c019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:14.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f364a7fc640 1 -- 172.21.15.12:0/674347739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0b8670 con 0x7f365415d320 2023-11-28T21:45:14.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f364a7fc640 1 -- 172.21.15.12:0/674347739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c0b8810 con 0x7f365415d320 2023-11-28T21:45:14.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365c12a640 1 -- 172.21.15.12:0/674347739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36541f3990 con 0x7f365415d320 2023-11-28T21:45:14.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f364a7fc640 1 -- 172.21.15.12:0/674347739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0b8bc0 con 0x7f365415d320 2023-11-28T21:45:14.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.414+0000 7f365c12a640 1 -- 172.21.15.12:0/674347739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36541514f0 con 0x7f365415d320 2023-11-28T21:45:14.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.418+0000 7f365c12a640 1 -- 172.21.15.12:0/674347739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3654144c90 con 0x7f365415d320 2023-11-28T21:45:14.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.418+0000 7f364a7fc640 1 -- 172.21.15.12:0/674347739 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f364c0b8d20 con 0x7f365415d320 2023-11-28T21:45:14.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.418+0000 7f364a7fc640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36380664a0 0x7f3638068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:14.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.418+0000 7f364a7fc640 1 -- 172.21.15.12:0/674347739 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f364c11f410 con 0x7f365415d320 2023-11-28T21:45:14.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.418+0000 7f3659ec6640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36380664a0 0x7f3638068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:14.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.418+0000 7f3659ec6640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36380664a0 0x7f3638068960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f365000b810 tx=0x7f3650008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:14.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.422+0000 7f364a7fc640 1 -- 172.21.15.12:0/674347739 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3654144c90 con 0x7f365415d320 2023-11-28T21:45:14.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.614+0000 7f365c12a640 1 -- 172.21.15.12:0/674347739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3654144c90 con 0x7f365415d320 2023-11-28T21:45:14.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f364a7fc640 1 -- 172.21.15.12:0/674347739 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f3654144c90 con 0x7f365415d320 2023-11-28T21:45:14.623 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:14.623 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:14.623 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-11-28T21:45:14.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 -- 172.21.15.12:0/674347739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36380664a0 msgr2=0x7f3638068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36380664a0 0x7f3638068960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f365000b810 tx=0x7f3650008040 comp rx=0 tx=0).stop 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 -- 172.21.15.12:0/674347739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365415d320 msgr2=0x7f3654144f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365415d320 0x7f3654144f70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f364c01a5c0 tx=0x7f364c019730 comp rx=0 tx=0).stop 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 -- 172.21.15.12:0/674347739 shutdown_connections 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36380664a0 0x7f3638068960 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f365415d320 0x7f3654144f70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3654154100 0x7f3654152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 --2- 172.21.15.12:0/674347739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365406aaf0 0x7f36541505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 -- 172.21.15.12:0/674347739 >> 172.21.15.12:0/674347739 conn(0x7f365406c690 msgr2=0x7f36540cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 -- 172.21.15.12:0/674347739 shutdown_connections 2023-11-28T21:45:14.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:14.618+0000 7f365c12a640 1 -- 172.21.15.12:0/674347739 wait complete. 2023-11-28T21:45:14.644 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:45:16.644 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ed2832640 1 Processor -- start 2023-11-28T21:45:16.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ed2832640 1 -- start start 2023-11-28T21:45:16.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ed2832640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3ecc06e1b0 0x7f3ecc066e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:16.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ed2832640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3ecc159a10 0x7f3ecc159e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:16.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ed2832640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ecc156140 con 0x7f3ecc14cae0 2023-11-28T21:45:16.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ed2832640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc0673c0 con 0x7f3ecc06e1b0 2023-11-28T21:45:16.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ed2832640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc06fb80 con 0x7f3ecc159a10 2023-11-28T21:45:16.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ecb7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3ecc06e1b0 0x7f3ecc066e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:16.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ecb7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3ecc06e1b0 0x7f3ecc066e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54920/0 (socket says 172.21.15.12:54920) 2023-11-28T21:45:16.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ecb7fe640 1 -- 172.21.15.12:0/4176806943 learned_addr learned my addr 172.21.15.12:0/4176806943 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:16.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.794+0000 7f3ecbfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3ecc14cae0 0x7f3ecc06d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53228/0 (socket says 172.21.15.12:53228) 2023-11-28T21:45:16.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ed0dcf640 1 --2- 172.21.15.12:0/4176806943 >> v2:172.21.15.144:3300/0 conn(0x7f3ecc159a10 0x7f3ecc159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:16.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ecb7fe640 1 -- 172.21.15.12:0/4176806943 >> v2:172.21.15.144:3300/0 conn(0x7f3ecc159a10 msgr2=0x7f3ecc159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:16.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ecb7fe640 1 --2- 172.21.15.12:0/4176806943 >> v2:172.21.15.144:3300/0 conn(0x7f3ecc159a10 0x7f3ecc159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:16.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ecb7fe640 1 -- 172.21.15.12:0/4176806943 >> v1:172.21.15.12:6789/0 conn(0x7f3ecc14cae0 legacy=0x7f3ecc06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:16.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ecb7fe640 1 -- 172.21.15.12:0/4176806943 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc15a380 con 0x7f3ecc06e1b0 2023-11-28T21:45:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ecb7fe640 1 --2- 172.21.15.12:0/4176806943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc06e1b0 0x7f3ecc066e80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0c9100 tx=0x7f3ec001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92818ba123e8215c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ecaffd640 1 -- 172.21.15.12:0/4176806943 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00a89f0 con 0x7f3ecc06e1b0 2023-11-28T21:45:16.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ecaffd640 1 -- 172.21.15.12:0/4176806943 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec00a8b50 con 0x7f3ecc06e1b0 2023-11-28T21:45:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ecaffd640 1 -- 172.21.15.12:0/4176806943 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00b0440 con 0x7f3ecc06e1b0 2023-11-28T21:45:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ed2832640 1 -- 172.21.15.12:0/4176806943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc06e1b0 msgr2=0x7f3ecc066e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.798+0000 7f3ed2832640 1 --2- 172.21.15.12:0/4176806943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc06e1b0 0x7f3ecc066e80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0c9100 tx=0x7f3ec001ecf0 comp rx=0 tx=0).stop 2023-11-28T21:45:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 -- 172.21.15.12:0/4176806943 shutdown_connections 2023-11-28T21:45:16.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 --2- 172.21.15.12:0/4176806943 >> v2:172.21.15.144:3300/0 conn(0x7f3ecc159a10 0x7f3ecc159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 --2- 172.21.15.12:0/4176806943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc06e1b0 0x7f3ecc066e80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 -- 172.21.15.12:0/4176806943 >> 172.21.15.12:0/4176806943 conn(0x7f3ecc066360 msgr2=0x7f3ecc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 -- 172.21.15.12:0/4176806943 shutdown_connections 2023-11-28T21:45:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 -- 172.21.15.12:0/4176806943 wait complete. 2023-11-28T21:45:16.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 Processor -- start 2023-11-28T21:45:16.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 -- start start 2023-11-28T21:45:16.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ecc06e1b0 0x7f3ecc0c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc14cae0 0x7f3ecc1ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ecc159a10 0x7f3ecc1ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc1efd20 con 0x7f3ecc14cae0 2023-11-28T21:45:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc158f80 con 0x7f3ecc06e1b0 2023-11-28T21:45:16.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed2832640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0ca610 con 0x7f3ecc159a10 2023-11-28T21:45:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ecb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc14cae0 0x7f3ecc1ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ecb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc14cae0 0x7f3ecc1ef2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54930/0 (socket says 172.21.15.12:54930) 2023-11-28T21:45:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ecb7fe640 1 -- 172.21.15.12:0/3484785137 learned_addr learned my addr 172.21.15.12:0/3484785137 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ecbfff640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ecc06e1b0 0x7f3ecc0c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:16.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ed0dcf640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ecc159a10 0x7f3ecc1ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ecb7fe640 1 -- 172.21.15.12:0/3484785137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ecc159a10 msgr2=0x7f3ecc1ef7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ecb7fe640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ecc159a10 0x7f3ecc1ef7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ecb7fe640 1 -- 172.21.15.12:0/3484785137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ecc06e1b0 msgr2=0x7f3ecc0c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ecb7fe640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ecc06e1b0 0x7f3ecc0c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.802+0000 7f3ecb7fe640 1 -- 172.21.15.12:0/3484785137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec00094d0 con 0x7f3ecc14cae0 2023-11-28T21:45:16.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ecb7fe640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc14cae0 0x7f3ecc1ef2a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3ec001b900 tx=0x7f3ec001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ec8ff9640 1 -- 172.21.15.12:0/3484785137 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00ba600 con 0x7f3ecc14cae0 2023-11-28T21:45:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ec8ff9640 1 -- 172.21.15.12:0/3484785137 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec00ba760 con 0x7f3ecc14cae0 2023-11-28T21:45:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ec8ff9640 1 -- 172.21.15.12:0/3484785137 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00bab10 con 0x7f3ecc14cae0 2023-11-28T21:45:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ed2832640 1 -- 172.21.15.12:0/3484785137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc1591f0 con 0x7f3ecc14cae0 2023-11-28T21:45:16.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ed2832640 1 -- 172.21.15.12:0/3484785137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc157300 con 0x7f3ecc14cae0 2023-11-28T21:45:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ed2832640 1 -- 172.21.15.12:0/3484785137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc074f60 con 0x7f3ecc14cae0 2023-11-28T21:45:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ec8ff9640 1 -- 172.21.15.12:0/3484785137 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3ec00bac70 con 0x7f3ecc14cae0 2023-11-28T21:45:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ec8ff9640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3eb80664a0 0x7f3eb8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ec8ff9640 1 -- 172.21.15.12:0/3484785137 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f3ec0122a10 con 0x7f3ecc14cae0 2023-11-28T21:45:16.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.806+0000 7f3ecbfff640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3eb80664a0 0x7f3eb8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.810+0000 7f3ecbfff640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3eb80664a0 0x7f3eb8068960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc000f70 tx=0x7f3ebc007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:16.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.810+0000 7f3ec8ff9640 1 -- 172.21.15.12:0/3484785137 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3ec00ef970 con 0x7f3ecc14cae0 2023-11-28T21:45:17.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:16.998+0000 7f3ed2832640 1 -- 172.21.15.12:0/3484785137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ecc153d90 con 0x7f3ecc14cae0 2023-11-28T21:45:17.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ec8ff9640 1 -- 172.21.15.12:0/3484785137 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f3ec00b5090 con 0x7f3ecc14cae0 2023-11-28T21:45:17.007 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:17.007 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:17.007 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-11-28T21:45:17.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 -- 172.21.15.12:0/3484785137 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3eb80664a0 msgr2=0x7f3eb8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:17.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3eb80664a0 0x7f3eb8068960 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc000f70 tx=0x7f3ebc007c90 comp rx=0 tx=0).stop 2023-11-28T21:45:17.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 -- 172.21.15.12:0/3484785137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc14cae0 msgr2=0x7f3ecc1ef2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:17.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc14cae0 0x7f3ecc1ef2a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3ec001b900 tx=0x7f3ec001ba90 comp rx=0 tx=0).stop 2023-11-28T21:45:17.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 -- 172.21.15.12:0/3484785137 shutdown_connections 2023-11-28T21:45:17.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3eb80664a0 0x7f3eb8068960 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:17.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ecc159a10 0x7f3ecc1ef7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:17.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ecc14cae0 0x7f3ecc1ef2a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:17.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 --2- 172.21.15.12:0/3484785137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ecc06e1b0 0x7f3ecc0c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:17.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 -- 172.21.15.12:0/3484785137 >> 172.21.15.12:0/3484785137 conn(0x7f3ecc066360 msgr2=0x7f3ecc154d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:17.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 -- 172.21.15.12:0/3484785137 shutdown_connections 2023-11-28T21:45:17.009 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:17.002+0000 7f3ed2832640 1 -- 172.21.15.12:0/3484785137 wait complete. 2023-11-28T21:45:17.028 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:45:19.029 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:19.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fe6a17640 1 Processor -- start 2023-11-28T21:45:19.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fe6a17640 1 -- start start 2023-11-28T21:45:19.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fe6a17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fe0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:19.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fe6a17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe0067410 con 0x7f4fe006cf40 2023-11-28T21:45:19.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fe6a17640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe006e090 con 0x7f4fe0150c20 2023-11-28T21:45:19.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fe6a17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0157f30 con 0x7f4fe006e300 2023-11-28T21:45:19.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fe4fb4640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4fe006cf40 0x7f4fe0158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53242/0 (socket says 172.21.15.12:53242) 2023-11-28T21:45:19.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fe4fb4640 1 -- 172.21.15.12:0/551379875 learned_addr learned my addr 172.21.15.12:0/551379875 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:45:19.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fdf7fe640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fe0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:19.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682308054 0 0) 0x7f4fe0067410 con 0x7f4fe006cf40 2023-11-28T21:45:19.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.178+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc8003530 con 0x7f4fe006cf40 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3045027455 0 0) 0x7f4fc8003530 con 0x7f4fe006cf40 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 msgr2=0x7f4fe0156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fe0156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 >> v1:172.21.15.144:6789/0 conn(0x7f4fe0150c20 legacy=0x7f4fe00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe0151000 con 0x7f4fe006cf40 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f4fd8001a00 con 0x7f4fe006cf40 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 >> v1:172.21.15.12:6789/0 conn(0x7f4fe006cf40 legacy=0x7f4fe0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fc8003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc8003fa0 0x7f4fc80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc8004d10 0x7f4fc8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0157f30 con 0x7f4fc8003fa0 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc800dd00 con 0x7f4fc8004d10 2023-11-28T21:45:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc800dec0 con 0x7f4fe006e300 2023-11-28T21:45:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdf7fe640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc8003fa0 0x7f4fc80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdffff640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fc8003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fe4fb4640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc8004d10 0x7f4fc8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdf7fe640 1 -- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 msgr2=0x7f4fc8003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdf7fe640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fc8003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdf7fe640 1 -- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc8004d10 msgr2=0x7f4fc8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdf7fe640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc8004d10 0x7f4fc8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fdf7fe640 1 -- 172.21.15.12:0/551379875 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc8003530 con 0x7f4fc8003fa0 2023-11-28T21:45:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.182+0000 7f4fe4fb4640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fc8004d10 0x7f4fc8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:45:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fdf7fe640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc8003fa0 0x7f4fc80047d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009ae0 tx=0x7f4fd401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32896928ec8917fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:19.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40a5580 con 0x7f4fc8003fa0 2023-11-28T21:45:19.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd40a56e0 con 0x7f4fc8003fa0 2023-11-28T21:45:19.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fdeffd640 1 -- 172.21.15.12:0/551379875 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40a5a90 con 0x7f4fc8003fa0 2023-11-28T21:45:19.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe6a17640 1 -- 172.21.15.12:0/551379875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc8003fa0 msgr2=0x7f4fc80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:19.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe6a17640 1 --2- 172.21.15.12:0/551379875 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fc8003fa0 0x7f4fc80047d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009ae0 tx=0x7f4fd401c990 comp rx=0 tx=0).stop 2023-11-28T21:45:19.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe4fb4640 1 -- 172.21.15.12:0/551379875 reap_dead start 2023-11-28T21:45:19.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe6a17640 1 -- 172.21.15.12:0/551379875 shutdown_connections 2023-11-28T21:45:19.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe6a17640 1 -- 172.21.15.12:0/551379875 >> 172.21.15.12:0/551379875 conn(0x7f4fe0066360 msgr2=0x7f4fe0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:19.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe6a17640 1 -- 172.21.15.12:0/551379875 shutdown_connections 2023-11-28T21:45:19.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe6a17640 1 -- 172.21.15.12:0/551379875 wait complete. 2023-11-28T21:45:19.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe6a17640 1 Processor -- start 2023-11-28T21:45:19.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe6a17640 1 -- start start 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe6a17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fc8003fa0 0x7f4fe00c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.186+0000 7f4fe6a17640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe0072aa0 0x7f4fe01f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe6a17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0151240 0x7f4fe01f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe6a17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0156c00 con 0x7f4fe0151240 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe6a17640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe014bcd0 con 0x7f4fe0072aa0 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe6a17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01f3e80 con 0x7f4fc8003fa0 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe4fb4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0151240 0x7f4fe01f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe4fb4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0151240 0x7f4fe01f05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54960/0 (socket says 172.21.15.12:54960) 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe4fb4640 1 -- 172.21.15.12:0/3414545186 learned_addr learned my addr 172.21.15.12:0/3414545186 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fdffff640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fc8003fa0 0x7f4fe00c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fdf7fe640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe0072aa0 0x7f4fe01f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:19.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe4fb4640 1 -- 172.21.15.12:0/3414545186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fc8003fa0 msgr2=0x7f4fe00c1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:19.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe4fb4640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fc8003fa0 0x7f4fe00c1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:19.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe4fb4640 1 -- 172.21.15.12:0/3414545186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe0072aa0 msgr2=0x7f4fe01f00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:19.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe4fb4640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe0072aa0 0x7f4fe01f00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:19.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe4fb4640 1 -- 172.21.15.12:0/3414545186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe01f3300 con 0x7f4fe0151240 2023-11-28T21:45:19.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fdf7fe640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe0072aa0 0x7f4fe01f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:45:19.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe4fb4640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0151240 0x7f4fe01f05e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0157a00 tx=0x7f4fd8011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:19.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fdcff9640 1 -- 172.21.15.12:0/3414545186 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800f8a0 con 0x7f4fe0151240 2023-11-28T21:45:19.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fdcff9640 1 -- 172.21.15.12:0/3414545186 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd800fa00 con 0x7f4fe0151240 2023-11-28T21:45:19.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fdcff9640 1 -- 172.21.15.12:0/3414545186 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800fdb0 con 0x7f4fe0151240 2023-11-28T21:45:19.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe6a17640 1 -- 172.21.15.12:0/3414545186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe01f35e0 con 0x7f4fe0151240 2023-11-28T21:45:19.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe6a17640 1 -- 172.21.15.12:0/3414545186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe01f3ba0 con 0x7f4fe0151240 2023-11-28T21:45:19.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fe6a17640 1 -- 172.21.15.12:0/3414545186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe0154500 con 0x7f4fe0151240 2023-11-28T21:45:19.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.190+0000 7f4fdcff9640 1 -- 172.21.15.12:0/3414545186 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4fd801f460 con 0x7f4fe0151240 2023-11-28T21:45:19.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.194+0000 7f4fdcff9640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fcc0664c0 0x7f4fcc068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:19.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.194+0000 7f4fdcff9640 1 -- 172.21.15.12:0/3414545186 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f4fd8087370 con 0x7f4fe0151240 2023-11-28T21:45:19.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.194+0000 7f4fdffff640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fcc0664c0 0x7f4fcc068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:19.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.194+0000 7f4fdffff640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fcc0664c0 0x7f4fcc068980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0007fd0 tx=0x7f4fd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:19.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.198+0000 7f4fdcff9640 1 -- 172.21.15.12:0/3414545186 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4fd8054230 con 0x7f4fe0151240 2023-11-28T21:45:19.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.386+0000 7f4fe6a17640 1 -- 172.21.15.12:0/3414545186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fe0074f60 con 0x7f4fe0151240 2023-11-28T21:45:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.386+0000 7f4fdcff9640 1 -- 172.21.15.12:0/3414545186 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f4fd8001210 con 0x7f4fe0151240 2023-11-28T21:45:19.394 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:19.394 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-11-28T21:45:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 -- 172.21.15.12:0/3414545186 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fcc0664c0 msgr2=0x7f4fcc068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fcc0664c0 0x7f4fcc068980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0007fd0 tx=0x7f4fd0009040 comp rx=0 tx=0).stop 2023-11-28T21:45:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 -- 172.21.15.12:0/3414545186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0151240 msgr2=0x7f4fe01f05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0151240 0x7f4fe01f05e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0157a00 tx=0x7f4fd8011c20 comp rx=0 tx=0).stop 2023-11-28T21:45:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 -- 172.21.15.12:0/3414545186 shutdown_connections 2023-11-28T21:45:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fcc0664c0 0x7f4fcc068980 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0151240 0x7f4fe01f05e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe0072aa0 0x7f4fe01f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 --2- 172.21.15.12:0/3414545186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fc8003fa0 0x7f4fe00c1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 -- 172.21.15.12:0/3414545186 >> 172.21.15.12:0/3414545186 conn(0x7f4fe0066360 msgr2=0x7f4fe0154b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 -- 172.21.15.12:0/3414545186 shutdown_connections 2023-11-28T21:45:19.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:19.390+0000 7f4fe6a17640 1 -- 172.21.15.12:0/3414545186 wait complete. 2023-11-28T21:45:19.416 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:45:21.417 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.566+0000 7f9aa233e640 1 Processor -- start 2023-11-28T21:45:21.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa233e640 1 -- start start 2023-11-28T21:45:21.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa233e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a9c1500d0 0x7f9a9c0c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa233e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9a9c06d7d0 0x7f9a9c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa233e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9c0c6ce0 con 0x7f9a9c156af0 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa233e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c06e120 con 0x7f9a9c06d7d0 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa233e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c0c7170 con 0x7f9a9c1500d0 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa08db640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9a9c06d7d0 0x7f9a9c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa08db640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9a9c06d7d0 0x7f9a9c06dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54972/0 (socket says 172.21.15.12:54972) 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa08db640 1 -- 172.21.15.12:0/647943784 learned_addr learned my addr 172.21.15.12:0/647943784 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9a9bfff640 1 --2- 172.21.15.12:0/647943784 >> v2:172.21.15.144:3300/0 conn(0x7f9a9c1500d0 0x7f9a9c0c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa08db640 1 -- 172.21.15.12:0/647943784 >> v2:172.21.15.144:3300/0 conn(0x7f9a9c1500d0 msgr2=0x7f9a9c0c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa08db640 1 --2- 172.21.15.12:0/647943784 >> v2:172.21.15.144:3300/0 conn(0x7f9a9c1500d0 0x7f9a9c0c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:21.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa08db640 1 -- 172.21.15.12:0/647943784 >> v1:172.21.15.144:6789/0 conn(0x7f9a9c156af0 legacy=0x7f9a9c0c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:21.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.570+0000 7f9aa08db640 1 -- 172.21.15.12:0/647943784 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c156f60 con 0x7f9a9c06d7d0 2023-11-28T21:45:21.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa08db640 1 --2- 172.21.15.12:0/647943784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c06d7d0 0x7f9a9c06dbe0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c150930 tx=0x7f9a9401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d14463ae805dbf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:21.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9a9affd640 1 -- 172.21.15.12:0/647943784 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a940a5820 con 0x7f9a9c06d7d0 2023-11-28T21:45:21.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9a9affd640 1 -- 172.21.15.12:0/647943784 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a940a5980 con 0x7f9a9c06d7d0 2023-11-28T21:45:21.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9a9affd640 1 -- 172.21.15.12:0/647943784 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a940a5d10 con 0x7f9a9c06d7d0 2023-11-28T21:45:21.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa233e640 1 -- 172.21.15.12:0/647943784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c06d7d0 msgr2=0x7f9a9c06dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa233e640 1 --2- 172.21.15.12:0/647943784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c06d7d0 0x7f9a9c06dbe0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c150930 tx=0x7f9a9401c960 comp rx=0 tx=0).stop 2023-11-28T21:45:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa233e640 1 -- 172.21.15.12:0/647943784 shutdown_connections 2023-11-28T21:45:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa233e640 1 --2- 172.21.15.12:0/647943784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c06d7d0 0x7f9a9c06dbe0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa233e640 1 --2- 172.21.15.12:0/647943784 >> v2:172.21.15.144:3300/0 conn(0x7f9a9c1500d0 0x7f9a9c0c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa233e640 1 -- 172.21.15.12:0/647943784 >> 172.21.15.12:0/647943784 conn(0x7f9a9c066360 msgr2=0x7f9a9c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa233e640 1 -- 172.21.15.12:0/647943784 shutdown_connections 2023-11-28T21:45:21.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa233e640 1 -- 172.21.15.12:0/647943784 wait complete. 2023-11-28T21:45:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa233e640 1 Processor -- start 2023-11-28T21:45:21.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.574+0000 7f9aa233e640 1 -- start start 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9aa233e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a9c06d7d0 0x7f9a9c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9aa233e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c1500d0 0x7f9a9c14a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9aa233e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a9c156af0 0x7f9a9c14a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9aa233e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c1f6670 con 0x7f9a9c1500d0 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9aa233e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c149920 con 0x7f9a9c156af0 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9aa233e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c149ee0 con 0x7f9a9c06d7d0 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a9b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c1500d0 0x7f9a9c14a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a9b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c1500d0 0x7f9a9c14a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54974/0 (socket says 172.21.15.12:54974) 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a9b7fe640 1 -- 172.21.15.12:0/4070689279 learned_addr learned my addr 172.21.15.12:0/4070689279 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a9bfff640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a9c06d7d0 0x7f9a9c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:21.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9aa08db640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a9c156af0 0x7f9a9c14a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a9b7fe640 1 -- 172.21.15.12:0/4070689279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a9c06d7d0 msgr2=0x7f9a9c148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a9b7fe640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a9c06d7d0 0x7f9a9c148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a9b7fe640 1 -- 172.21.15.12:0/4070689279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a9c156af0 msgr2=0x7f9a9c14a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a9b7fe640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a9c156af0 0x7f9a9c14a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a9b7fe640 1 -- 172.21.15.12:0/4070689279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a94009b30 con 0x7f9a9c1500d0 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a9b7fe640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c1500d0 0x7f9a9c14a370 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000c4b0 tx=0x7f9a9000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a98ff9640 1 -- 172.21.15.12:0/4070689279 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90012830 con 0x7f9a9c1500d0 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a98ff9640 1 -- 172.21.15.12:0/4070689279 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a900129b0 con 0x7f9a9c1500d0 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9a98ff9640 1 -- 172.21.15.12:0/4070689279 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90012d60 con 0x7f9a9c1500d0 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9aa233e640 1 -- 172.21.15.12:0/4070689279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c1f5af0 con 0x7f9a9c1500d0 2023-11-28T21:45:21.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9aa233e640 1 -- 172.21.15.12:0/4070689279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c1f60b0 con 0x7f9a9c1500d0 2023-11-28T21:45:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.578+0000 7f9aa233e640 1 -- 172.21.15.12:0/4070689279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9c149cb0 con 0x7f9a9c1500d0 2023-11-28T21:45:21.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.582+0000 7f9a98ff9640 1 -- 172.21.15.12:0/4070689279 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9a9001c460 con 0x7f9a9c1500d0 2023-11-28T21:45:21.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.582+0000 7f9a98ff9640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a880664a0 0x7f9a88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.582+0000 7f9a98ff9640 1 -- 172.21.15.12:0/4070689279 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7007+0+0 (secure 0 0 0) 0x7f9a90084260 con 0x7f9a9c1500d0 2023-11-28T21:45:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.582+0000 7f9a9bfff640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a880664a0 0x7f9a88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:21.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.582+0000 7f9a9bfff640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a880664a0 0x7f9a88068960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00b9d0 tx=0x7f9a8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:21.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.586+0000 7f9a98ff9640 1 -- 172.21.15.12:0/4070689279 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a9c149cb0 con 0x7f9a9c1500d0 2023-11-28T21:45:21.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.778+0000 7f9aa233e640 1 -- 172.21.15.12:0/4070689279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a9c14a090 con 0x7f9a9c1500d0 2023-11-28T21:45:21.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.778+0000 7f9a98ff9640 1 -- 172.21.15.12:0/4070689279 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f9a9c14a090 con 0x7f9a9c1500d0 2023-11-28T21:45:21.785 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:21.785 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:44:33.723470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5727":{"gid":5727,"name":"e","rank":0,"incarnation":39,"state":"up:active","state_seq":24,"addr":"172.21.15.12:6839/3299030262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3299030262},{"type":"v1","addr":"172.21.15.12:6839","nonce":3299030262}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:21.785 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 44 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 -- 172.21.15.12:0/4070689279 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a880664a0 msgr2=0x7f9a88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a880664a0 0x7f9a88068960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00b9d0 tx=0x7f9a8c008040 comp rx=0 tx=0).stop 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 -- 172.21.15.12:0/4070689279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c1500d0 msgr2=0x7f9a9c14a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c1500d0 0x7f9a9c14a370 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000c4b0 tx=0x7f9a9000c970 comp rx=0 tx=0).stop 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 -- 172.21.15.12:0/4070689279 shutdown_connections 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a880664a0 0x7f9a88068960 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a9c156af0 0x7f9a9c14a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a9c1500d0 0x7f9a9c14a370 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 --2- 172.21.15.12:0/4070689279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a9c06d7d0 0x7f9a9c148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 -- 172.21.15.12:0/4070689279 >> 172.21.15.12:0/4070689279 conn(0x7f9a9c066360 msgr2=0x7f9a9c1560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 -- 172.21.15.12:0/4070689279 shutdown_connections 2023-11-28T21:45:21.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:21.782+0000 7f9aa233e640 1 -- 172.21.15.12:0/4070689279 wait complete. 2023-11-28T21:45:21.806 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:45:23.807 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:23.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.954+0000 7f53e0a09640 1 Processor -- start 2023-11-28T21:45:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.954+0000 7f53e0a09640 1 -- start start 2023-11-28T21:45:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53e0a09640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc158b20 0x7f53dc156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53e0a09640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53dc067410 con 0x7f53dc150c20 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53e0a09640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53dc06e090 con 0x7f53dc06e300 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53e0a09640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc157f30 con 0x7f53dc158b20 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53da59c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f53dc150c20 0x7f53dc0c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53252/0 (socket says 172.21.15.12:53252) 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53da59c640 1 -- 172.21.15.12:0/551016822 learned_addr learned my addr 172.21.15.12:0/551016822 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53dad9d640 1 --2- 172.21.15.12:0/551016822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc158b20 0x7f53dc156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53d9d9b640 1 -- 172.21.15.12:0/551016822 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003926942 0 0) 0x7f53dc067410 con 0x7f53dc150c20 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53d9d9b640 1 -- 172.21.15.12:0/551016822 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c8005740 con 0x7f53dc150c20 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53d9d9b640 1 -- 172.21.15.12:0/551016822 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162226936 0 0) 0x7f53dc06e090 con 0x7f53dc06e300 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53d9d9b640 1 -- 172.21.15.12:0/551016822 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53dc067410 con 0x7f53dc06e300 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53dad9d640 1 -- 172.21.15.12:0/551016822 >> v1:172.21.15.144:6789/0 conn(0x7f53dc06e300 legacy=0x7f53dc158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53d9d9b640 1 -- 172.21.15.12:0/551016822 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2283822253 0 0) 0x7f53c8005740 con 0x7f53dc150c20 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53dad9d640 1 -- 172.21.15.12:0/551016822 >> v1:172.21.15.12:6789/0 conn(0x7f53dc150c20 legacy=0x7f53dc0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.958+0000 7f53dad9d640 1 -- 172.21.15.12:0/551016822 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc151000 con 0x7f53dc158b20 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53dad9d640 1 --2- 172.21.15.12:0/551016822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc158b20 0x7f53dc156910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009dc0 tx=0x7f53d401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3019731a6d4289cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53d9d9b640 1 -- 172.21.15.12:0/551016822 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc001680 con 0x7f53dc158b20 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53d9d9b640 1 -- 172.21.15.12:0/551016822 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d40a64f0 con 0x7f53dc158b20 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53d9d9b640 1 -- 172.21.15.12:0/551016822 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d40a6880 con 0x7f53dc158b20 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53e0a09640 1 -- 172.21.15.12:0/551016822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc158b20 msgr2=0x7f53dc156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53e0a09640 1 --2- 172.21.15.12:0/551016822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc158b20 0x7f53dc156910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009dc0 tx=0x7f53d401c890 comp rx=0 tx=0).stop 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53e0a09640 1 -- 172.21.15.12:0/551016822 shutdown_connections 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53e0a09640 1 --2- 172.21.15.12:0/551016822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc158b20 0x7f53dc156910 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53e0a09640 1 -- 172.21.15.12:0/551016822 >> 172.21.15.12:0/551016822 conn(0x7f53dc066360 msgr2=0x7f53dc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53e0a09640 1 -- 172.21.15.12:0/551016822 shutdown_connections 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53e0a09640 1 -- 172.21.15.12:0/551016822 wait complete. 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53e0a09640 1 Processor -- start 2023-11-28T21:45:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.962+0000 7f53e0a09640 1 -- start start 2023-11-28T21:45:24.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53e0a09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc06e300 0x7f53dc15c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:24.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53e0a09640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc150c20 0x7f53dc14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53e0a09640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53dc158b20 0x7f53dc15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53e0a09640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc1f2e80 con 0x7f53dc06e300 2023-11-28T21:45:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53e0a09640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc1f30f0 con 0x7f53dc158b20 2023-11-28T21:45:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53e0a09640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53dc1f33d0 con 0x7f53dc150c20 2023-11-28T21:45:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc06e300 0x7f53dc15c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc06e300 0x7f53dc15c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54982/0 (socket says 172.21.15.12:54982) 2023-11-28T21:45:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53da59c640 1 -- 172.21.15.12:0/2218256171 learned_addr learned my addr 172.21.15.12:0/2218256171 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53d3fff640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc150c20 0x7f53dc14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:24.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53dad9d640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53dc158b20 0x7f53dc15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53da59c640 1 -- 172.21.15.12:0/2218256171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc150c20 msgr2=0x7f53dc14ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53da59c640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc150c20 0x7f53dc14ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53da59c640 1 -- 172.21.15.12:0/2218256171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53dc158b20 msgr2=0x7f53dc15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53da59c640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53dc158b20 0x7f53dc15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53da59c640 1 -- 172.21.15.12:0/2218256171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4009a70 con 0x7f53dc06e300 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53da59c640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc06e300 0x7f53dc15c6c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00c750 tx=0x7f53cc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53d2ffd640 1 -- 172.21.15.12:0/2218256171 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc005af0 con 0x7f53dc06e300 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53d2ffd640 1 -- 172.21.15.12:0/2218256171 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc005c50 con 0x7f53dc06e300 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53d2ffd640 1 -- 172.21.15.12:0/2218256171 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc007070 con 0x7f53dc06e300 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53e0a09640 1 -- 172.21.15.12:0/2218256171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53dc1f36b0 con 0x7f53dc06e300 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53e0a09640 1 -- 172.21.15.12:0/2218256171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53dc15cc00 con 0x7f53dc06e300 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.966+0000 7f53e0a09640 1 -- 172.21.15.12:0/2218256171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53dc074f60 con 0x7f53dc06e300 2023-11-28T21:45:24.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.970+0000 7f53d2ffd640 1 -- 172.21.15.12:0/2218256171 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f53cc0071d0 con 0x7f53dc06e300 2023-11-28T21:45:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.970+0000 7f53d2ffd640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53bc066470 0x7f53bc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.970+0000 7f53d2ffd640 1 -- 172.21.15.12:0/2218256171 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f53cc084090 con 0x7f53dc06e300 2023-11-28T21:45:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.970+0000 7f53d3fff640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53bc066470 0x7f53bc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.970+0000 7f53d3fff640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53bc066470 0x7f53bc068930 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0cb070 tx=0x7f53c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:23.974+0000 7f53d2ffd640 1 -- 172.21.15.12:0/2218256171 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f53cc050fc0 con 0x7f53dc06e300 2023-11-28T21:45:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.166+0000 7f53e0a09640 1 -- 172.21.15.12:0/2218256171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53dc162060 con 0x7f53dc06e300 2023-11-28T21:45:24.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.166+0000 7f53d2ffd640 1 -- 172.21.15.12:0/2218256171 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f53cc0545e0 con 0x7f53dc06e300 2023-11-28T21:45:24.959 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:24.960 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:23.480048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 45 2023-11-28T21:45:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 -- 172.21.15.12:0/2218256171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53bc066470 msgr2=0x7f53bc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53bc066470 0x7f53bc068930 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0cb070 tx=0x7f53c4009040 comp rx=0 tx=0).stop 2023-11-28T21:45:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 -- 172.21.15.12:0/2218256171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc06e300 msgr2=0x7f53dc15c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc06e300 0x7f53dc15c6c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00c750 tx=0x7f53cc00cc10 comp rx=0 tx=0).stop 2023-11-28T21:45:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 -- 172.21.15.12:0/2218256171 shutdown_connections 2023-11-28T21:45:24.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53bc066470 0x7f53bc068930 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53dc158b20 0x7f53dc15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53dc150c20 0x7f53dc14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 --2- 172.21.15.12:0/2218256171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53dc06e300 0x7f53dc15c6c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 -- 172.21.15.12:0/2218256171 >> 172.21.15.12:0/2218256171 conn(0x7f53dc066360 msgr2=0x7f53dc0c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 -- 172.21.15.12:0/2218256171 shutdown_connections 2023-11-28T21:45:24.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:24.170+0000 7f53e0a09640 1 -- 172.21.15.12:0/2218256171 wait complete. 2023-11-28T21:45:24.965 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-11-28T21:45:24.966 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:45:24.966 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa42d6640 1 Processor -- start 2023-11-28T21:45:25.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa42d6640 1 -- start start 2023-11-28T21:45:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa42d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9f9c1540f0 0x7f9f9c0c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa42d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c06d2a0 con 0x7f9f9c15d210 2023-11-28T21:45:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa42d6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c06d580 con 0x7f9f9c15b180 2023-11-28T21:45:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa42d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c06e200 con 0x7f9f9c1540f0 2023-11-28T21:45:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa2072640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9f9c1540f0 0x7f9f9c0c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa2072640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9f9c1540f0 0x7f9f9c0c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39376/0 (socket says 172.21.15.12:39376) 2023-11-28T21:45:25.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa2072640 1 -- 172.21.15.12:0/1380229943 learned_addr learned my addr 172.21.15.12:0/1380229943 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa2873640 1 --1- 172.21.15.12:0/1380229943 >> v1:172.21.15.12:6789/0 conn(0x7f9f9c15d210 0x7f9f9c15d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35098/0 (socket says 172.21.15.12:35098) 2023-11-28T21:45:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa2072640 1 -- 172.21.15.12:0/1380229943 >> v1:172.21.15.144:6789/0 conn(0x7f9f9c15b180 legacy=0x7f9f9c15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa2072640 1 -- 172.21.15.12:0/1380229943 >> v1:172.21.15.12:6789/0 conn(0x7f9f9c15d210 legacy=0x7f9f9c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.118+0000 7f9fa2072640 1 -- 172.21.15.12:0/1380229943 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c06d860 con 0x7f9f9c1540f0 2023-11-28T21:45:25.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa2072640 1 --2- 172.21.15.12:0/1380229943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c1540f0 0x7f9f9c0c3be0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009870 tx=0x7f9f9801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90d4893a9fe3653 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa1070640 1 -- 172.21.15.12:0/1380229943 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980aa430 con 0x7f9f9c1540f0 2023-11-28T21:45:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa1070640 1 -- 172.21.15.12:0/1380229943 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f980aa590 con 0x7f9f9c1540f0 2023-11-28T21:45:25.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa1070640 1 -- 172.21.15.12:0/1380229943 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980aa920 con 0x7f9f9c1540f0 2023-11-28T21:45:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa42d6640 1 -- 172.21.15.12:0/1380229943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c1540f0 msgr2=0x7f9f9c0c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa42d6640 1 --2- 172.21.15.12:0/1380229943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c1540f0 0x7f9f9c0c3be0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009870 tx=0x7f9f9801c4d0 comp rx=0 tx=0).stop 2023-11-28T21:45:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa42d6640 1 -- 172.21.15.12:0/1380229943 shutdown_connections 2023-11-28T21:45:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa42d6640 1 --2- 172.21.15.12:0/1380229943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c1540f0 0x7f9f9c0c3be0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa42d6640 1 -- 172.21.15.12:0/1380229943 >> 172.21.15.12:0/1380229943 conn(0x7f9f9c066360 msgr2=0x7f9f9c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa42d6640 1 -- 172.21.15.12:0/1380229943 shutdown_connections 2023-11-28T21:45:25.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.122+0000 7f9fa42d6640 1 -- 172.21.15.12:0/1380229943 wait complete. 2023-11-28T21:45:25.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa42d6640 1 Processor -- start 2023-11-28T21:45:25.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa42d6640 1 -- start start 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa42d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f9c1540f0 0x7f9f9c146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa42d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c15b180 0x7f9f9c14f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa42d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f9c15d210 0x7f9f9c14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa42d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c148690 con 0x7f9f9c15b180 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa42d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c148b80 con 0x7f9f9c1540f0 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa42d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c1f2e70 con 0x7f9f9c15d210 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa1871640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c15b180 0x7f9f9c14f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa1871640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c15b180 0x7f9f9c14f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39384/0 (socket says 172.21.15.12:39384) 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa1871640 1 -- 172.21.15.12:0/3161286637 learned_addr learned my addr 172.21.15.12:0/3161286637 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa2072640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f9c1540f0 0x7f9f9c146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:25.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa2873640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f9c15d210 0x7f9f9c14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa1871640 1 -- 172.21.15.12:0/3161286637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f9c15d210 msgr2=0x7f9f9c14f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa1871640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f9c15d210 0x7f9f9c14f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa1871640 1 -- 172.21.15.12:0/3161286637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f9c1540f0 msgr2=0x7f9f9c146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa1871640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f9c1540f0 0x7f9f9c146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa1871640 1 -- 172.21.15.12:0/3161286637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f980094d0 con 0x7f9f9c15b180 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa2072640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f9c1540f0 0x7f9f9c146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa1871640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c15b180 0x7f9f9c14f1d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00c480 tx=0x7f9f8c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9f92ffd640 1 -- 172.21.15.12:0/3161286637 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c012830 con 0x7f9f9c15b180 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9f92ffd640 1 -- 172.21.15.12:0/3161286637 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0129b0 con 0x7f9f9c15b180 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9f92ffd640 1 -- 172.21.15.12:0/3161286637 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c012d60 con 0x7f9f9c15b180 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa42d6640 1 -- 172.21.15.12:0/3161286637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c147300 con 0x7f9f9c15b180 2023-11-28T21:45:25.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.126+0000 7f9fa42d6640 1 -- 172.21.15.12:0/3161286637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c1f2820 con 0x7f9f9c15b180 2023-11-28T21:45:25.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.130+0000 7f9fa42d6640 1 -- 172.21.15.12:0/3161286637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c1f2680 con 0x7f9f9c15b180 2023-11-28T21:45:25.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.130+0000 7f9f92ffd640 1 -- 172.21.15.12:0/3161286637 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9f8c01c460 con 0x7f9f9c15b180 2023-11-28T21:45:25.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.130+0000 7f9f92ffd640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9f80066470 0x7f9f80068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:25.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.130+0000 7f9f92ffd640 1 -- 172.21.15.12:0/3161286637 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f9f8c084260 con 0x7f9f9c15b180 2023-11-28T21:45:25.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.130+0000 7f9fa2072640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9f80066470 0x7f9f80068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:25.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.130+0000 7f9fa2072640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9f80066470 0x7f9f80068930 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f980ae000 tx=0x7f9f980a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:25.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.134+0000 7f9f92ffd640 1 -- 172.21.15.12:0/3161286637 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9f9c1f2680 con 0x7f9f9c15b180 2023-11-28T21:45:25.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.326+0000 7f9fa42d6640 1 -- 172.21.15.12:0/3161286637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f9c1f2680 con 0x7f9f9c15b180 2023-11-28T21:45:25.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.326+0000 7f9f92ffd640 1 -- 172.21.15.12:0/3161286637 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f9f9c1f2680 con 0x7f9f9c15b180 2023-11-28T21:45:25.333 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:25.334 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:23.480048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:25.334 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 45 2023-11-28T21:45:25.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 -- 172.21.15.12:0/3161286637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9f80066470 msgr2=0x7f9f80068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:25.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9f80066470 0x7f9f80068930 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f980ae000 tx=0x7f9f980a7000 comp rx=0 tx=0).stop 2023-11-28T21:45:25.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 -- 172.21.15.12:0/3161286637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c15b180 msgr2=0x7f9f9c14f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:25.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c15b180 0x7f9f9c14f1d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00c480 tx=0x7f9f8c00c940 comp rx=0 tx=0).stop 2023-11-28T21:45:25.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 -- 172.21.15.12:0/3161286637 shutdown_connections 2023-11-28T21:45:25.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9f80066470 0x7f9f80068930 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:25.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f9c15d210 0x7f9f9c14f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:25.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f9c15b180 0x7f9f9c14f1d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:25.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 --2- 172.21.15.12:0/3161286637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f9c1540f0 0x7f9f9c146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:25.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 -- 172.21.15.12:0/3161286637 >> 172.21.15.12:0/3161286637 conn(0x7f9f9c066360 msgr2=0x7f9f9c15d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:25.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 -- 172.21.15.12:0/3161286637 shutdown_connections 2023-11-28T21:45:25.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:25.330+0000 7f9fa42d6640 1 -- 172.21.15.12:0/3161286637 wait complete. 2023-11-28T21:45:25.356 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 45, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 45, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5871": { "addr": "172.21.15.12:6838/3474424840", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3474424840, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 3474424840, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5871, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 47, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:45:23.480048+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5871 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:45:27.358 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:27.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.506+0000 7fe8dc9d8640 1 Processor -- start 2023-11-28T21:45:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.506+0000 7fe8dc9d8640 1 -- start start 2023-11-28T21:45:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.506+0000 7fe8dc9d8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe8d8150c20 0x7fe8d8156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.506+0000 7fe8dc9d8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe8d806e300 0x7fe8d80c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8dc9d8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d8067410 con 0x7fe8d806cf40 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8dc9d8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d806e090 con 0x7fe8d806e300 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8dc9d8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d8157f30 con 0x7fe8d8150c20 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8cffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe8d806e300 0x7fe8d80c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8cffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe8d806e300 0x7fe8d80c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39398/0 (socket says 172.21.15.12:39398) 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8cffff640 1 -- 172.21.15.12:0/1978245239 learned_addr learned my addr 172.21.15.12:0/1978245239 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8d659c640 1 --2- 172.21.15.12:0/1978245239 >> v2:172.21.15.144:3300/0 conn(0x7fe8d8150c20 0x7fe8d8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8cffff640 1 -- 172.21.15.12:0/1978245239 >> v2:172.21.15.144:3300/0 conn(0x7fe8d8150c20 msgr2=0x7fe8d8156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8cffff640 1 --2- 172.21.15.12:0/1978245239 >> v2:172.21.15.144:3300/0 conn(0x7fe8d8150c20 0x7fe8d8156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8cffff640 1 -- 172.21.15.12:0/1978245239 >> v1:172.21.15.12:6789/0 conn(0x7fe8d806cf40 legacy=0x7fe8d8158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8cffff640 1 -- 172.21.15.12:0/1978245239 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d8151000 con 0x7fe8d806e300 2023-11-28T21:45:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8cffff640 1 --2- 172.21.15.12:0/1978245239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806e300 0x7fe8d80c6dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe8d806c6a0 tx=0x7fe8c001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b5b82905c6eaf1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8d5d9b640 1 -- 172.21.15.12:0/1978245239 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c00ab610 con 0x7fe8d806e300 2023-11-28T21:45:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8d5d9b640 1 -- 172.21.15.12:0/1978245239 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c00ab770 con 0x7fe8d806e300 2023-11-28T21:45:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.510+0000 7fe8d5d9b640 1 -- 172.21.15.12:0/1978245239 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c00abb00 con 0x7fe8d806e300 2023-11-28T21:45:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1978245239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806e300 msgr2=0x7fe8d80c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 --2- 172.21.15.12:0/1978245239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806e300 0x7fe8d80c6dd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe8d806c6a0 tx=0x7fe8c001c500 comp rx=0 tx=0).stop 2023-11-28T21:45:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1978245239 shutdown_connections 2023-11-28T21:45:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 --2- 172.21.15.12:0/1978245239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806e300 0x7fe8d80c6dd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 --2- 172.21.15.12:0/1978245239 >> v2:172.21.15.144:3300/0 conn(0x7fe8d8150c20 0x7fe8d8156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1978245239 >> 172.21.15.12:0/1978245239 conn(0x7fe8d8066360 msgr2=0x7fe8d8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1978245239 shutdown_connections 2023-11-28T21:45:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1978245239 wait complete. 2023-11-28T21:45:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 Processor -- start 2023-11-28T21:45:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 -- start start 2023-11-28T21:45:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806cf40 0x7fe8d814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d806e300 0x7fe8d814a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8d8150c20 0x7fe8d8147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80c4e70 con 0x7fe8d806cf40 2023-11-28T21:45:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80c9750 con 0x7fe8d8150c20 2023-11-28T21:45:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8dc9d8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d8163610 con 0x7fe8d806e300 2023-11-28T21:45:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8d659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806cf40 0x7fe8d814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8d659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806cf40 0x7fe8d814e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39402/0 (socket says 172.21.15.12:39402) 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.514+0000 7fe8d659c640 1 -- 172.21.15.12:0/1372955647 learned_addr learned my addr 172.21.15.12:0/1372955647 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8cffff640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d806e300 0x7fe8d814a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8d6d9d640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8d8150c20 0x7fe8d8147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8d659c640 1 -- 172.21.15.12:0/1372955647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d806e300 msgr2=0x7fe8d814a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8d659c640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d806e300 0x7fe8d814a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8d659c640 1 -- 172.21.15.12:0/1372955647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8d8150c20 msgr2=0x7fe8d8147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8d659c640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8d8150c20 0x7fe8d8147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8d659c640 1 -- 172.21.15.12:0/1372955647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00094d0 con 0x7fe8d806cf40 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8d659c640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806cf40 0x7fe8d814e200 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800d5f0 tx=0x7fe8c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8ceffd640 1 -- 172.21.15.12:0/1372955647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80125f0 con 0x7fe8d806cf40 2023-11-28T21:45:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8ceffd640 1 -- 172.21.15.12:0/1372955647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c8012750 con 0x7fe8d806cf40 2023-11-28T21:45:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1372955647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d8163180 con 0x7fe8d806cf40 2023-11-28T21:45:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8ceffd640 1 -- 172.21.15.12:0/1372955647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8012ae0 con 0x7fe8d806cf40 2023-11-28T21:45:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1372955647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d8163c90 con 0x7fe8d806cf40 2023-11-28T21:45:27.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1372955647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d8074f60 con 0x7fe8d806cf40 2023-11-28T21:45:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8ceffd640 1 -- 172.21.15.12:0/1372955647 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe8c8012c40 con 0x7fe8d806cf40 2023-11-28T21:45:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8ceffd640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8b80664a0 0x7fe8b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8ceffd640 1 -- 172.21.15.12:0/1372955647 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fe8c80849b0 con 0x7fe8d806cf40 2023-11-28T21:45:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.518+0000 7fe8cffff640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8b80664a0 0x7fe8b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:27.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.522+0000 7fe8cffff640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8b80664a0 0x7fe8b8068960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00af000 tx=0x7fe8c00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:27.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.522+0000 7fe8ceffd640 1 -- 172.21.15.12:0/1372955647 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe8c8051820 con 0x7fe8d806cf40 2023-11-28T21:45:27.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.714+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1372955647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8d8161030 con 0x7fe8d806cf40 2023-11-28T21:45:27.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8ceffd640 1 -- 172.21.15.12:0/1372955647 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5714 (secure 0 0 0) 0x7fe8c8054e40 con 0x7fe8d806cf40 2023-11-28T21:45:27.722 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:27.722 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:23.480048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:27.723 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 45 2023-11-28T21:45:27.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1372955647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8b80664a0 msgr2=0x7fe8b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:27.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8b80664a0 0x7fe8b8068960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00af000 tx=0x7fe8c00a7000 comp rx=0 tx=0).stop 2023-11-28T21:45:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1372955647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806cf40 msgr2=0x7fe8d814e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806cf40 0x7fe8d814e200 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800d5f0 tx=0x7fe8c800dab0 comp rx=0 tx=0).stop 2023-11-28T21:45:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1372955647 shutdown_connections 2023-11-28T21:45:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe8b80664a0 0x7fe8b8068960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8d8150c20 0x7fe8d8147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8d806e300 0x7fe8d814a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 --2- 172.21.15.12:0/1372955647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe8d806cf40 0x7fe8d814e200 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1372955647 >> 172.21.15.12:0/1372955647 conn(0x7fe8d8066360 msgr2=0x7fe8d80c2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1372955647 shutdown_connections 2023-11-28T21:45:27.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:27.718+0000 7fe8dc9d8640 1 -- 172.21.15.12:0/1372955647 wait complete. 2023-11-28T21:45:27.744 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:45:29.745 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.886+0000 7f453cf55640 1 Processor -- start 2023-11-28T21:45:29.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.886+0000 7f453cf55640 1 -- start start 2023-11-28T21:45:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f453cf55640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4538151ed0 0x7f45380c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f453cf55640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f453806e300 0x7f4538072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f453cf55640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4538157b70 con 0x7f4538151af0 2023-11-28T21:45:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f453cf55640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45380707f0 con 0x7f4538151ed0 2023-11-28T21:45:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f453cf55640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45380c7b90 con 0x7f453806e300 2023-11-28T21:45:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f452ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4538151ed0 0x7f45380c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f452ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4538151ed0 0x7f45380c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39410/0 (socket says 172.21.15.12:39410) 2023-11-28T21:45:29.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f452ffff640 1 -- 172.21.15.12:0/1164223532 learned_addr learned my addr 172.21.15.12:0/1164223532 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:29.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f4536d9d640 1 --2- 172.21.15.12:0/1164223532 >> v2:172.21.15.144:3300/0 conn(0x7f453806e300 0x7f4538072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:29.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f452ffff640 1 -- 172.21.15.12:0/1164223532 >> v2:172.21.15.144:3300/0 conn(0x7f453806e300 msgr2=0x7f4538072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:29.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f452ffff640 1 --2- 172.21.15.12:0/1164223532 >> v2:172.21.15.144:3300/0 conn(0x7f453806e300 0x7f4538072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:29.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f452ffff640 1 -- 172.21.15.12:0/1164223532 >> v1:172.21.15.144:6789/0 conn(0x7f4538151af0 legacy=0x7f45380c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:29.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.890+0000 7f452ffff640 1 -- 172.21.15.12:0/1164223532 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4538157e50 con 0x7f4538151ed0 2023-11-28T21:45:29.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f452ffff640 1 --2- 172.21.15.12:0/1164223532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4538151ed0 0x7f45380c7650 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f45380716f0 tx=0x7f452001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f974b92ed684669a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f4535d9b640 1 -- 172.21.15.12:0/1164223532 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200ab600 con 0x7f4538151ed0 2023-11-28T21:45:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f4535d9b640 1 -- 172.21.15.12:0/1164223532 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45200a7050 con 0x7f4538151ed0 2023-11-28T21:45:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f4535d9b640 1 -- 172.21.15.12:0/1164223532 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200ab980 con 0x7f4538151ed0 2023-11-28T21:45:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f453cf55640 1 -- 172.21.15.12:0/1164223532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4538151ed0 msgr2=0x7f45380c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f453cf55640 1 --2- 172.21.15.12:0/1164223532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4538151ed0 0x7f45380c7650 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f45380716f0 tx=0x7f452001c4c0 comp rx=0 tx=0).stop 2023-11-28T21:45:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f453cf55640 1 -- 172.21.15.12:0/1164223532 shutdown_connections 2023-11-28T21:45:29.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f453cf55640 1 --2- 172.21.15.12:0/1164223532 >> v2:172.21.15.144:3300/0 conn(0x7f453806e300 0x7f4538072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f453cf55640 1 --2- 172.21.15.12:0/1164223532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4538151ed0 0x7f45380c7650 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f453cf55640 1 -- 172.21.15.12:0/1164223532 >> 172.21.15.12:0/1164223532 conn(0x7f4538066360 msgr2=0x7f4538066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f453cf55640 1 -- 172.21.15.12:0/1164223532 shutdown_connections 2023-11-28T21:45:29.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.894+0000 7f453cf55640 1 -- 172.21.15.12:0/1164223532 wait complete. 2023-11-28T21:45:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453cf55640 1 Processor -- start 2023-11-28T21:45:29.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453cf55640 1 -- start start 2023-11-28T21:45:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453cf55640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f453806e300 0x7f45381f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:29.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453cf55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4538151af0 0x7f45381f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:29.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453cf55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45380c2330 0x7f45381432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:29.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453cf55640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45381f2c20 con 0x7f45380c2330 2023-11-28T21:45:29.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453cf55640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45380c3710 con 0x7f4538151af0 2023-11-28T21:45:29.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453cf55640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4538142380 con 0x7f453806e300 2023-11-28T21:45:29.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f4536d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45380c2330 0x7f45381432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:29.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f4536d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45380c2330 0x7f45381432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39426/0 (socket says 172.21.15.12:39426) 2023-11-28T21:45:29.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f4536d9d640 1 -- 172.21.15.12:0/3157375727 learned_addr learned my addr 172.21.15.12:0/3157375727 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:29.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f452ffff640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4538151af0 0x7f45381f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:29.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453659c640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f453806e300 0x7f45381f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:29.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f4536d9d640 1 -- 172.21.15.12:0/3157375727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f453806e300 msgr2=0x7f45381f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f4536d9d640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f453806e300 0x7f45381f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f4536d9d640 1 -- 172.21.15.12:0/3157375727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4538151af0 msgr2=0x7f45381f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f4536d9d640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4538151af0 0x7f45381f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f4536d9d640 1 -- 172.21.15.12:0/3157375727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45200094d0 con 0x7f45380c2330 2023-11-28T21:45:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f4536d9d640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45380c2330 0x7f45381432a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4530007e30 tx=0x7f453000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f452effd640 1 -- 172.21.15.12:0/3157375727 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530014620 con 0x7f45380c2330 2023-11-28T21:45:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f452effd640 1 -- 172.21.15.12:0/3157375727 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453000ee90 con 0x7f45380c2330 2023-11-28T21:45:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f452effd640 1 -- 172.21.15.12:0/3157375727 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45300149c0 con 0x7f45380c2330 2023-11-28T21:45:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453cf55640 1 -- 172.21.15.12:0/3157375727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4538142b50 con 0x7f45380c2330 2023-11-28T21:45:29.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.898+0000 7f453cf55640 1 -- 172.21.15.12:0/3157375727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45381f6350 con 0x7f45380c2330 2023-11-28T21:45:29.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.902+0000 7f453cf55640 1 -- 172.21.15.12:0/3157375727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45381f64e0 con 0x7f45380c2330 2023-11-28T21:45:29.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.902+0000 7f452effd640 1 -- 172.21.15.12:0/3157375727 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4530014b20 con 0x7f45380c2330 2023-11-28T21:45:29.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.902+0000 7f452effd640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4518066470 0x7f4518068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:29.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.902+0000 7f452effd640 1 -- 172.21.15.12:0/3157375727 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f4530084d60 con 0x7f45380c2330 2023-11-28T21:45:29.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.902+0000 7f453659c640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4518066470 0x7f4518068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:29.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.902+0000 7f453659c640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4518066470 0x7f4518068930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4528004e30 tx=0x7f4528009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:29.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:29.910+0000 7f452effd640 1 -- 172.21.15.12:0/3157375727 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4530051c70 con 0x7f45380c2330 2023-11-28T21:45:30.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 -- 172.21.15.12:0/3157375727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4538074f60 con 0x7f45380c2330 2023-11-28T21:45:30.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f452effd640 1 -- 172.21.15.12:0/3157375727 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f4530055290 con 0x7f45380c2330 2023-11-28T21:45:30.119 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:30.119 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:23.480048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:30.120 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 45 2023-11-28T21:45:30.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 -- 172.21.15.12:0/3157375727 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4518066470 msgr2=0x7f4518068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:30.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4518066470 0x7f4518068930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4528004e30 tx=0x7f4528009040 comp rx=0 tx=0).stop 2023-11-28T21:45:30.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 -- 172.21.15.12:0/3157375727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45380c2330 msgr2=0x7f45381432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:30.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45380c2330 0x7f45381432a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4530007e30 tx=0x7f453000e4b0 comp rx=0 tx=0).stop 2023-11-28T21:45:30.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 -- 172.21.15.12:0/3157375727 shutdown_connections 2023-11-28T21:45:30.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4518066470 0x7f4518068930 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:30.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45380c2330 0x7f45381432a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:30.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4538151af0 0x7f45381f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:30.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 --2- 172.21.15.12:0/3157375727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f453806e300 0x7f45381f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:30.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 -- 172.21.15.12:0/3157375727 >> 172.21.15.12:0/3157375727 conn(0x7f4538066360 msgr2=0x7f45380c6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:30.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 -- 172.21.15.12:0/3157375727 shutdown_connections 2023-11-28T21:45:30.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:30.114+0000 7f453cf55640 1 -- 172.21.15.12:0/3157375727 wait complete. 2023-11-28T21:45:30.143 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:45:32.144 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:32.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c5cf5640 1 Processor -- start 2023-11-28T21:45:32.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c5cf5640 1 -- start start 2023-11-28T21:45:32.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c5cf5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb4c0153fb0 0x7fb4c00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:32.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c5cf5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4c015a980 0x7fb4c00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:32.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c5cf5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c015d390 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c5cf5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c00b2b60 con 0x7fb4c0153fb0 2023-11-28T21:45:32.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c5cf5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c015e1b0 con 0x7fb4c015a980 2023-11-28T21:45:32.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c4cf3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb4c0153fb0 0x7fb4c00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:32.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c4cf3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb4c0153fb0 0x7fb4c00ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39434/0 (socket says 172.21.15.12:39434) 2023-11-28T21:45:32.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c4cf3640 1 -- 172.21.15.12:0/3355703520 learned_addr learned my addr 172.21.15.12:0/3355703520 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:32.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4bffff640 1 --2- 172.21.15.12:0/3355703520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4c015a980 0x7fb4c00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:32.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c4cf3640 1 -- 172.21.15.12:0/3355703520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4c015a980 msgr2=0x7fb4c00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:32.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c4cf3640 1 --2- 172.21.15.12:0/3355703520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4c015a980 0x7fb4c00caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:32.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c4cf3640 1 -- 172.21.15.12:0/3355703520 >> v1:172.21.15.144:6789/0 conn(0x7fb4c00b3ad0 legacy=0x7fb4c00b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:32.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.286+0000 7fb4c4cf3640 1 -- 172.21.15.12:0/3355703520 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c015d630 con 0x7fb4c0153fb0 2023-11-28T21:45:32.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c4cf3640 1 --2- 172.21.15.12:0/3355703520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c0153fb0 0x7fb4c00ca4f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009870 tx=0x7fb4b401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def948bd844bb621 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:32.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4bf7fe640 1 -- 172.21.15.12:0/3355703520 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40aa670 con 0x7fb4c0153fb0 2023-11-28T21:45:32.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4bf7fe640 1 -- 172.21.15.12:0/3355703520 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b40aa7d0 con 0x7fb4c0153fb0 2023-11-28T21:45:32.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4bf7fe640 1 -- 172.21.15.12:0/3355703520 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40aab60 con 0x7fb4c0153fb0 2023-11-28T21:45:32.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/3355703520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c0153fb0 msgr2=0x7fb4c00ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:32.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c5cf5640 1 --2- 172.21.15.12:0/3355703520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c0153fb0 0x7fb4c00ca4f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009870 tx=0x7fb4b401c510 comp rx=0 tx=0).stop 2023-11-28T21:45:32.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/3355703520 shutdown_connections 2023-11-28T21:45:32.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c5cf5640 1 --2- 172.21.15.12:0/3355703520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4c015a980 0x7fb4c00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:32.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c5cf5640 1 --2- 172.21.15.12:0/3355703520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c0153fb0 0x7fb4c00ca4f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:32.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/3355703520 >> 172.21.15.12:0/3355703520 conn(0x7fb4c0071ce0 msgr2=0x7fb4c00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:32.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/3355703520 shutdown_connections 2023-11-28T21:45:32.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/3355703520 wait complete. 2023-11-28T21:45:32.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c5cf5640 1 Processor -- start 2023-11-28T21:45:32.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.290+0000 7fb4c5cf5640 1 -- start start 2023-11-28T21:45:32.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c5cf5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c00b3ad0 0x7fb4c01560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:32.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c5cf5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4c0153fb0 0x7fb4c0156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:32.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c5cf5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4c015a980 0x7fb4c01f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:32.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c5cf5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c0155060 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c5cf5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c01523f0 con 0x7fb4c015a980 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c5cf5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c01f71a0 con 0x7fb4c0153fb0 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c4cf3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c00b3ad0 0x7fb4c01560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c4cf3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c00b3ad0 0x7fb4c01560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39442/0 (socket says 172.21.15.12:39442) 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c4cf3640 1 -- 172.21.15.12:0/2381448949 learned_addr learned my addr 172.21.15.12:0/2381448949 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c54f4640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4c015a980 0x7fb4c01f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4bffff640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4c0153fb0 0x7fb4c0156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c4cf3640 1 -- 172.21.15.12:0/2381448949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4c0153fb0 msgr2=0x7fb4c0156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c4cf3640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4c0153fb0 0x7fb4c0156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c4cf3640 1 -- 172.21.15.12:0/2381448949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4c015a980 msgr2=0x7fb4c01f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c4cf3640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4c015a980 0x7fb4c01f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:32.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c4cf3640 1 -- 172.21.15.12:0/2381448949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b40094d0 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c4cf3640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c00b3ad0 0x7fb4c01560c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40ac040 tx=0x7fb4b401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4bd7fa640 1 -- 172.21.15.12:0/2381448949 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40bd3f0 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4bd7fa640 1 -- 172.21.15.12:0/2381448949 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b40bd570 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4bd7fa640 1 -- 172.21.15.12:0/2381448949 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40bd920 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/2381448949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c0152690 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/2381448949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c0157500 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/2381448949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c01576f0 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.294+0000 7fb4bd7fa640 1 -- 172.21.15.12:0/2381448949 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb4b40b33f0 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.298+0000 7fb4bd7fa640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb4a40664a0 0x7fb4a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:32.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.298+0000 7fb4bd7fa640 1 -- 172.21.15.12:0/2381448949 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fb4b411e910 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.298+0000 7fb4bffff640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb4a40664a0 0x7fb4a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:32.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.298+0000 7fb4bffff640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb4a40664a0 0x7fb4a4068960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0067390 tx=0x7fb4ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:32.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.302+0000 7fb4bd7fa640 1 -- 172.21.15.12:0/2381448949 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4c01576f0 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.490+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/2381448949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4c0155720 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.490+0000 7fb4bd7fa640 1 -- 172.21.15.12:0/2381448949 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5714 (secure 0 0 0) 0x7fb4c0155720 con 0x7fb4c00b3ad0 2023-11-28T21:45:32.498 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:32.498 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:23.480048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:32.500 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 45 2023-11-28T21:45:32.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/2381448949 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb4a40664a0 msgr2=0x7fb4a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb4a40664a0 0x7fb4a4068960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0067390 tx=0x7fb4ac008040 comp rx=0 tx=0).stop 2023-11-28T21:45:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/2381448949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c00b3ad0 msgr2=0x7fb4c01560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c00b3ad0 0x7fb4c01560c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40ac040 tx=0x7fb4b401ad50 comp rx=0 tx=0).stop 2023-11-28T21:45:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/2381448949 shutdown_connections 2023-11-28T21:45:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb4a40664a0 0x7fb4a4068960 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4c015a980 0x7fb4c01f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4c0153fb0 0x7fb4c0156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 --2- 172.21.15.12:0/2381448949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4c00b3ad0 0x7fb4c01560c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/2381448949 >> 172.21.15.12:0/2381448949 conn(0x7fb4c0071ce0 msgr2=0x7fb4c0159d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/2381448949 shutdown_connections 2023-11-28T21:45:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:32.494+0000 7fb4c5cf5640 1 -- 172.21.15.12:0/2381448949 wait complete. 2023-11-28T21:45:32.519 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:45:34.520 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:34.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff60e05a640 1 Processor -- start 2023-11-28T21:45:34.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff60e05a640 1 -- start start 2023-11-28T21:45:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff60e05a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff608150c20 0x7ff608156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff60e05a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff60806e300 0x7ff6080c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff60e05a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff608067410 con 0x7ff60806cf40 2023-11-28T21:45:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff60e05a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60806e090 con 0x7ff60806e300 2023-11-28T21:45:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff60e05a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff608157f30 con 0x7ff608150c20 2023-11-28T21:45:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff5fffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff60806e300 0x7ff6080c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff5fffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff60806e300 0x7ff6080c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54922/0 (socket says 172.21.15.12:54922) 2023-11-28T21:45:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff5fffff640 1 -- 172.21.15.12:0/3409129216 learned_addr learned my addr 172.21.15.12:0/3409129216 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:34.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff6077fe640 1 --2- 172.21.15.12:0/3409129216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff608150c20 0x7ff608156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:34.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff5fffff640 1 -- 172.21.15.12:0/3409129216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff608150c20 msgr2=0x7ff608156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:34.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff5fffff640 1 --2- 172.21.15.12:0/3409129216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff608150c20 0x7ff608156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:34.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff5fffff640 1 -- 172.21.15.12:0/3409129216 >> v1:172.21.15.144:6789/0 conn(0x7ff60806cf40 legacy=0x7ff608158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:34.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.670+0000 7ff5fffff640 1 -- 172.21.15.12:0/3409129216 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608151000 con 0x7ff60806e300 2023-11-28T21:45:34.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff5fffff640 1 --2- 172.21.15.12:0/3409129216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806e300 0x7ff6080c6dd0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009870 tx=0x7ff5f001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288e076066d5304b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:34.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff606ffd640 1 -- 172.21.15.12:0/3409129216 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00ab650 con 0x7ff60806e300 2023-11-28T21:45:34.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff606ffd640 1 -- 172.21.15.12:0/3409129216 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f00ab7b0 con 0x7ff60806e300 2023-11-28T21:45:34.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff606ffd640 1 -- 172.21.15.12:0/3409129216 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00abb40 con 0x7ff60806e300 2023-11-28T21:45:34.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff60e05a640 1 -- 172.21.15.12:0/3409129216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806e300 msgr2=0x7ff6080c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:34.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff60e05a640 1 --2- 172.21.15.12:0/3409129216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806e300 0x7ff6080c6dd0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009870 tx=0x7ff5f001c540 comp rx=0 tx=0).stop 2023-11-28T21:45:34.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff60e05a640 1 -- 172.21.15.12:0/3409129216 shutdown_connections 2023-11-28T21:45:34.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff60e05a640 1 --2- 172.21.15.12:0/3409129216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806e300 0x7ff6080c6dd0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:34.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff60e05a640 1 --2- 172.21.15.12:0/3409129216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff608150c20 0x7ff608156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:34.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff60e05a640 1 -- 172.21.15.12:0/3409129216 >> 172.21.15.12:0/3409129216 conn(0x7ff608066360 msgr2=0x7ff608066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:34.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff60e05a640 1 -- 172.21.15.12:0/3409129216 shutdown_connections 2023-11-28T21:45:34.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.674+0000 7ff60e05a640 1 -- 172.21.15.12:0/3409129216 wait complete. 2023-11-28T21:45:34.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff60e05a640 1 Processor -- start 2023-11-28T21:45:34.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff60e05a640 1 -- start start 2023-11-28T21:45:34.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff60e05a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806cf40 0x7ff60814f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:34.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff60e05a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60806e300 0x7ff60814dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff60e05a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608150c20 0x7ff6080c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff60e05a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6081f4120 con 0x7ff60806cf40 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff60e05a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6081f3810 con 0x7ff608150c20 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff60e05a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60814f8d0 con 0x7ff60806e300 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff6077fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806cf40 0x7ff60814f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff6077fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806cf40 0x7ff60814f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54926/0 (socket says 172.21.15.12:54926) 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff6077fe640 1 -- 172.21.15.12:0/1321429256 learned_addr learned my addr 172.21.15.12:0/1321429256 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff5fffff640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60806e300 0x7ff60814dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff607fff640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608150c20 0x7ff6080c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff6077fe640 1 -- 172.21.15.12:0/1321429256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60806e300 msgr2=0x7ff60814dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff6077fe640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60806e300 0x7ff60814dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:34.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff6077fe640 1 -- 172.21.15.12:0/1321429256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608150c20 msgr2=0x7ff6080c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:34.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff6077fe640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608150c20 0x7ff6080c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:34.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff6077fe640 1 -- 172.21.15.12:0/1321429256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f00094d0 con 0x7ff60806cf40 2023-11-28T21:45:34.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff6077fe640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806cf40 0x7ff60814f390 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800d5f0 tx=0x7ff5f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:34.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff604ff9640 1 -- 172.21.15.12:0/1321429256 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80125f0 con 0x7ff60806cf40 2023-11-28T21:45:34.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff604ff9640 1 -- 172.21.15.12:0/1321429256 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f8012750 con 0x7ff60806cf40 2023-11-28T21:45:34.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff60e05a640 1 -- 172.21.15.12:0/1321429256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60814fba0 con 0x7ff60806cf40 2023-11-28T21:45:34.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff604ff9640 1 -- 172.21.15.12:0/1321429256 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8012b00 con 0x7ff60806cf40 2023-11-28T21:45:34.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.678+0000 7ff60e05a640 1 -- 172.21.15.12:0/1321429256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6081f3ad0 con 0x7ff60806cf40 2023-11-28T21:45:34.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.682+0000 7ff60e05a640 1 -- 172.21.15.12:0/1321429256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff608074f60 con 0x7ff60806cf40 2023-11-28T21:45:34.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.682+0000 7ff604ff9640 1 -- 172.21.15.12:0/1321429256 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff5f8012c60 con 0x7ff60806cf40 2023-11-28T21:45:34.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.682+0000 7ff604ff9640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5e8066470 0x7ff5e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:34.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.682+0000 7ff604ff9640 1 -- 172.21.15.12:0/1321429256 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7ff5f8084a00 con 0x7ff60806cf40 2023-11-28T21:45:34.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.682+0000 7ff5fffff640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5e8066470 0x7ff5e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:34.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.682+0000 7ff5fffff640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5e8066470 0x7ff5e8068930 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00af000 tx=0x7ff5f00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:34.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.686+0000 7ff604ff9640 1 -- 172.21.15.12:0/1321429256 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff5f8051870 con 0x7ff60806cf40 2023-11-28T21:45:34.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.878+0000 7ff60e05a640 1 -- 172.21.15.12:0/1321429256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6080c2410 con 0x7ff60806cf40 2023-11-28T21:45:34.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.878+0000 7ff604ff9640 1 -- 172.21.15.12:0/1321429256 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5718 (secure 0 0 0) 0x7ff5f8054e90 con 0x7ff60806cf40 2023-11-28T21:45:34.884 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:34.885 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:32.125948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:34.885 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 46 2023-11-28T21:45:34.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 -- 172.21.15.12:0/1321429256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5e8066470 msgr2=0x7ff5e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:34.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5e8066470 0x7ff5e8068930 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00af000 tx=0x7ff5f00a7000 comp rx=0 tx=0).stop 2023-11-28T21:45:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 -- 172.21.15.12:0/1321429256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806cf40 msgr2=0x7ff60814f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806cf40 0x7ff60814f390 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800d5f0 tx=0x7ff5f800dab0 comp rx=0 tx=0).stop 2023-11-28T21:45:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 -- 172.21.15.12:0/1321429256 shutdown_connections 2023-11-28T21:45:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff5e8066470 0x7ff5e8068930 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff608150c20 0x7ff6080c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff60806e300 0x7ff60814dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 --2- 172.21.15.12:0/1321429256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff60806cf40 0x7ff60814f390 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 -- 172.21.15.12:0/1321429256 >> 172.21.15.12:0/1321429256 conn(0x7ff608066360 msgr2=0x7ff6080c5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 -- 172.21.15.12:0/1321429256 shutdown_connections 2023-11-28T21:45:34.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:34.882+0000 7ff60e05a640 1 -- 172.21.15.12:0/1321429256 wait complete. 2023-11-28T21:45:34.906 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:45:36.907 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:37.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.054+0000 7fa65111d640 1 Processor -- start 2023-11-28T21:45:37.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.054+0000 7fa65111d640 1 -- start start 2023-11-28T21:45:37.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa65111d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64c151ed0 0x7fa64c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:37.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa65111d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa64c06e300 0x7fa64c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:37.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa65111d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c157b70 con 0x7fa64c151af0 2023-11-28T21:45:37.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa65111d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0707f0 con 0x7fa64c06e300 2023-11-28T21:45:37.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa65111d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c7b90 con 0x7fa64c151ed0 2023-11-28T21:45:37.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64ad9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa64c151af0 0x7fa64c0c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33628/0 (socket says 172.21.15.12:33628) 2023-11-28T21:45:37.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64ad9d640 1 -- 172.21.15.12:0/1004825306 learned_addr learned my addr 172.21.15.12:0/1004825306 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:45:37.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa643fff640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64c151ed0 0x7fa64c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64b59e640 1 --2- 172.21.15.12:0/1004825306 >> v2:172.21.15.144:3300/0 conn(0x7fa64c06e300 0x7fa64c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211927784 0 0) 0x7fa64c157b70 con 0x7fa64c151af0 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa638003530 con 0x7fa64c151af0 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3131249081 0 0) 0x7fa638003530 con 0x7fa64c151af0 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64c151ed0 msgr2=0x7fa64c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64c151ed0 0x7fa64c0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 >> v2:172.21.15.144:3300/0 conn(0x7fa64c06e300 msgr2=0x7fa64c072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 --2- 172.21.15.12:0/1004825306 >> v2:172.21.15.144:3300/0 conn(0x7fa64c06e300 0x7fa64c072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c157e50 con 0x7fa64c151af0 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fa63c001680 con 0x7fa64c151af0 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 >> v1:172.21.15.12:6789/0 conn(0x7fa64c151af0 legacy=0x7fa64c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:37.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6380039f0 0x7fa638003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa638004350 0x7fa638004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 0x7fa63800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0707f0 con 0x7fa638004ea0 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c7b90 con 0x7fa638004350 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6380056a0 con 0x7fa6380039f0 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64b59e640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 0x7fa63800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64ad9d640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6380039f0 0x7fa638003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa643fff640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa638004350 0x7fa638004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64b59e640 1 -- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6380039f0 msgr2=0x7fa638003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64b59e640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6380039f0 0x7fa638003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64b59e640 1 -- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa638004350 msgr2=0x7fa638004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:37.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64b59e640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa638004350 0x7fa638004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:37.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.058+0000 7fa64b59e640 1 -- 172.21.15.12:0/1004825306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa638003530 con 0x7fa638004ea0 2023-11-28T21:45:37.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa64b59e640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 0x7fa63800b990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa644009f90 tx=0x7fa64401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31f1ca6f2ab2652 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:37.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440a5c70 con 0x7fa638004ea0 2023-11-28T21:45:37.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64400f040 con 0x7fa638004ea0 2023-11-28T21:45:37.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa64a59c640 1 -- 172.21.15.12:0/1004825306 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440ad440 con 0x7fa638004ea0 2023-11-28T21:45:37.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa65111d640 1 -- 172.21.15.12:0/1004825306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 msgr2=0x7fa63800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:37.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa65111d640 1 --2- 172.21.15.12:0/1004825306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 0x7fa63800b990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa644009f90 tx=0x7fa64401cc60 comp rx=0 tx=0).stop 2023-11-28T21:45:37.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa64b59e640 1 -- 172.21.15.12:0/1004825306 reap_dead start 2023-11-28T21:45:37.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa65111d640 1 -- 172.21.15.12:0/1004825306 shutdown_connections 2023-11-28T21:45:37.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa65111d640 1 -- 172.21.15.12:0/1004825306 >> 172.21.15.12:0/1004825306 conn(0x7fa64c066360 msgr2=0x7fa64c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:37.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa65111d640 1 -- 172.21.15.12:0/1004825306 shutdown_connections 2023-11-28T21:45:37.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.062+0000 7fa65111d640 1 -- 172.21.15.12:0/1004825306 wait complete. 2023-11-28T21:45:37.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa65111d640 1 Processor -- start 2023-11-28T21:45:37.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa65111d640 1 -- start start 2023-11-28T21:45:37.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa65111d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 0x7fa64c0c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:37.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa65111d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa64c1462a0 0x7fa64c0ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:37.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa65111d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64c1f2570 0x7fa64c1f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:37.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa65111d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0c9520 con 0x7fa638004ea0 2023-11-28T21:45:37.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa65111d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0cafb0 con 0x7fa64c1462a0 2023-11-28T21:45:37.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa65111d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c1ef000 con 0x7fa64c1f2570 2023-11-28T21:45:37.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 0x7fa64c0c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 0x7fa64c0c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54954/0 (socket says 172.21.15.12:54954) 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64ad9d640 1 -- 172.21.15.12:0/1719308911 learned_addr learned my addr 172.21.15.12:0/1719308911 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa643fff640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa64c1462a0 0x7fa64c0ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64b59e640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64c1f2570 0x7fa64c1f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64ad9d640 1 -- 172.21.15.12:0/1719308911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64c1f2570 msgr2=0x7fa64c1f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64ad9d640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64c1f2570 0x7fa64c1f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64ad9d640 1 -- 172.21.15.12:0/1719308911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa64c1462a0 msgr2=0x7fa64c0ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64ad9d640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa64c1462a0 0x7fa64c0ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64ad9d640 1 -- 172.21.15.12:0/1719308911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c1effd0 con 0x7fa638004ea0 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64b59e640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64c1f2570 0x7fa64c1f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa64ad9d640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 0x7fa64c0c6cd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00e850 tx=0x7fa63c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa6437fe640 1 -- 172.21.15.12:0/1719308911 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00c870 con 0x7fa638004ea0 2023-11-28T21:45:37.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa6437fe640 1 -- 172.21.15.12:0/1719308911 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c00c9f0 con 0x7fa638004ea0 2023-11-28T21:45:37.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa6437fe640 1 -- 172.21.15.12:0/1719308911 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00cda0 con 0x7fa638004ea0 2023-11-28T21:45:37.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa65111d640 1 -- 172.21.15.12:0/1719308911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c1efb40 con 0x7fa638004ea0 2023-11-28T21:45:37.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.066+0000 7fa65111d640 1 -- 172.21.15.12:0/1719308911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c1ef210 con 0x7fa638004ea0 2023-11-28T21:45:37.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.070+0000 7fa65111d640 1 -- 172.21.15.12:0/1719308911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c074f60 con 0x7fa638004ea0 2023-11-28T21:45:37.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.070+0000 7fa6437fe640 1 -- 172.21.15.12:0/1719308911 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa63c013420 con 0x7fa638004ea0 2023-11-28T21:45:37.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.070+0000 7fa6437fe640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa62c0664a0 0x7fa62c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:37.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.070+0000 7fa6437fe640 1 -- 172.21.15.12:0/1719308911 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fa63c084180 con 0x7fa638004ea0 2023-11-28T21:45:37.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.070+0000 7fa643fff640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa62c0664a0 0x7fa62c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:37.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.070+0000 7fa643fff640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa62c0664a0 0x7fa62c068960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa64c1ecc30 tx=0x7fa634008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:37.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.074+0000 7fa6437fe640 1 -- 172.21.15.12:0/1719308911 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa63c050f60 con 0x7fa638004ea0 2023-11-28T21:45:37.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.262+0000 7fa65111d640 1 -- 172.21.15.12:0/1719308911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa64c072c10 con 0x7fa638004ea0 2023-11-28T21:45:37.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.262+0000 7fa6437fe640 1 -- 172.21.15.12:0/1719308911 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5718 (secure 0 0 0) 0x7fa63c054580 con 0x7fa638004ea0 2023-11-28T21:45:37.269 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:37.269 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:32.125948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:37.270 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 46 2023-11-28T21:45:37.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 -- 172.21.15.12:0/1719308911 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa62c0664a0 msgr2=0x7fa62c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa62c0664a0 0x7fa62c068960 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa64c1ecc30 tx=0x7fa634008040 comp rx=0 tx=0).stop 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 -- 172.21.15.12:0/1719308911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 msgr2=0x7fa64c0c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 0x7fa64c0c6cd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00e850 tx=0x7fa63c00ed10 comp rx=0 tx=0).stop 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 -- 172.21.15.12:0/1719308911 shutdown_connections 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa62c0664a0 0x7fa62c068960 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa64c1f2570 0x7fa64c1f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa64c1462a0 0x7fa64c0ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 --2- 172.21.15.12:0/1719308911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa638004ea0 0x7fa64c0c6cd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 -- 172.21.15.12:0/1719308911 >> 172.21.15.12:0/1719308911 conn(0x7fa64c066360 msgr2=0x7fa64c155580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 -- 172.21.15.12:0/1719308911 shutdown_connections 2023-11-28T21:45:37.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:37.266+0000 7fa65111d640 1 -- 172.21.15.12:0/1719308911 wait complete. 2023-11-28T21:45:37.291 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 46, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 46, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5871": { "addr": "172.21.15.12:6838/3474424840", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3474424840, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 3474424840, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5871, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 12 } }, "last_failure": 0, "last_failure_osd_epoch": 47, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:45:32.125948+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5871 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:45:39.292 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:39.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d6c3c640 1 Processor -- start 2023-11-28T21:45:39.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d6c3c640 1 -- start start 2023-11-28T21:45:39.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d6c3c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53d0153fb0 0x7f53d00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:39.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d6c3c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d015d390 con 0x7f53d00b41e0 2023-11-28T21:45:39.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d6c3c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d015d630 con 0x7f53d015a980 2023-11-28T21:45:39.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d6c3c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d00b2b60 con 0x7f53d0153fb0 2023-11-28T21:45:39.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d643b640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f53d00b41e0 0x7f53d00cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33636/0 (socket says 172.21.15.12:33636) 2023-11-28T21:45:39.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d643b640 1 -- 172.21.15.12:0/373061695 learned_addr learned my addr 172.21.15.12:0/373061695 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:45:39.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d5c3a640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53d0153fb0 0x7f53d00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:39.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510298266 0 0) 0x7f53d015d390 con 0x7f53d00b41e0 2023-11-28T21:45:39.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53bc003530 con 0x7f53d00b41e0 2023-11-28T21:45:39.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4223558348 0 0) 0x7f53bc003530 con 0x7f53d00b41e0 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53d0153fb0 msgr2=0x7f53d00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d4c38640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53d0153fb0 0x7f53d00ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 >> v1:172.21.15.144:6789/0 conn(0x7f53d015a980 legacy=0x7f53d00b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d015d910 con 0x7f53d00b41e0 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f53c8001a00 con 0x7f53d00b41e0 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 >> v1:172.21.15.12:6789/0 conn(0x7f53d00b41e0 legacy=0x7f53d00cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.442+0000 7f53d4c38640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53bc0039b0 0x7f53bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d4c38640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53d0153fb0 0x7f53bc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d4c38640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53bc004d10 0x7f53bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d00b2b60 con 0x7f53bc004d10 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc00b990 con 0x7f53bc0039b0 2023-11-28T21:45:39.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc00bb50 con 0x7f53d0153fb0 2023-11-28T21:45:39.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d643b640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53bc004d10 0x7f53bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:39.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d5439640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53d0153fb0 0x7f53bc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:39.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d5c3a640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53bc0039b0 0x7f53bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:39.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d643b640 1 -- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53d0153fb0 msgr2=0x7f53bc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:39.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d643b640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53d0153fb0 0x7f53bc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:39.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d643b640 1 -- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53bc0039b0 msgr2=0x7f53bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:39.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d643b640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53bc0039b0 0x7f53bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:39.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d643b640 1 -- 172.21.15.12:0/373061695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc003530 con 0x7f53bc004d10 2023-11-28T21:45:39.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d643b640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53bc004d10 0x7f53bc005190 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f53c800a840 tx=0x7f53c801fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7819b756604205f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80aa6b0 con 0x7f53bc004d10 2023-11-28T21:45:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c80aa810 con 0x7f53bc004d10 2023-11-28T21:45:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.446+0000 7f53d4c38640 1 -- 172.21.15.12:0/373061695 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80aabc0 con 0x7f53bc004d10 2023-11-28T21:45:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 -- 172.21.15.12:0/373061695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53bc004d10 msgr2=0x7f53bc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 --2- 172.21.15.12:0/373061695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53bc004d10 0x7f53bc005190 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f53c800a840 tx=0x7f53c801fa80 comp rx=0 tx=0).stop 2023-11-28T21:45:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d643b640 1 -- 172.21.15.12:0/373061695 reap_dead start 2023-11-28T21:45:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 -- 172.21.15.12:0/373061695 shutdown_connections 2023-11-28T21:45:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 -- 172.21.15.12:0/373061695 >> 172.21.15.12:0/373061695 conn(0x7f53d0071ce0 msgr2=0x7f53d00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 -- 172.21.15.12:0/373061695 shutdown_connections 2023-11-28T21:45:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 -- 172.21.15.12:0/373061695 wait complete. 2023-11-28T21:45:39.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 Processor -- start 2023-11-28T21:45:39.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 -- start start 2023-11-28T21:45:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc004d10 0x7f53d014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53d01e5600 0x7f53d01f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d00cfd70 0x7f53d01e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d015e1b0 con 0x7f53d01e5600 2023-11-28T21:45:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d00c5f90 con 0x7f53d00cfd70 2023-11-28T21:45:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d6c3c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d01ef5a0 con 0x7f53bc004d10 2023-11-28T21:45:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d5439640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53d01e5600 0x7f53d01f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d5439640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53d01e5600 0x7f53d01f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54970/0 (socket says 172.21.15.12:54970) 2023-11-28T21:45:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d5439640 1 -- 172.21.15.12:0/3683940096 learned_addr learned my addr 172.21.15.12:0/3683940096 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d5c3a640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc004d10 0x7f53d014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.450+0000 7f53d643b640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d00cfd70 0x7f53d01e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d5439640 1 -- 172.21.15.12:0/3683940096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc004d10 msgr2=0x7f53d014a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d5439640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc004d10 0x7f53d014a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d5439640 1 -- 172.21.15.12:0/3683940096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d00cfd70 msgr2=0x7f53d01e66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d5439640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d00cfd70 0x7f53d01e66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d5439640 1 -- 172.21.15.12:0/3683940096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d00cf900 con 0x7f53d01e5600 2023-11-28T21:45:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d5c3a640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc004d10 0x7f53d014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:45:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d5439640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53d01e5600 0x7f53d01f2a00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f53c000f7c0 tx=0x7f53c000fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:39.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53c67fc640 1 -- 172.21.15.12:0/3683940096 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c000fe70 con 0x7f53d01e5600 2023-11-28T21:45:39.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53c67fc640 1 -- 172.21.15.12:0/3683940096 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c00052e0 con 0x7f53d01e5600 2023-11-28T21:45:39.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53c67fc640 1 -- 172.21.15.12:0/3683940096 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c000d970 con 0x7f53d01e5600 2023-11-28T21:45:39.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d6c3c640 1 -- 172.21.15.12:0/3683940096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d01ef840 con 0x7f53d01e5600 2023-11-28T21:45:39.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d6c3c640 1 -- 172.21.15.12:0/3683940096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d01efe00 con 0x7f53d01e5600 2023-11-28T21:45:39.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d6c3c640 1 -- 172.21.15.12:0/3683940096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d0074f60 con 0x7f53d01e5600 2023-11-28T21:45:39.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53c67fc640 1 -- 172.21.15.12:0/3683940096 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f53c000dad0 con 0x7f53d01e5600 2023-11-28T21:45:39.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53c67fc640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53b4066470 0x7f53b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:39.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53c67fc640 1 -- 172.21.15.12:0/3683940096 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f53c00843a0 con 0x7f53d01e5600 2023-11-28T21:45:39.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.454+0000 7f53d5c3a640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53b4066470 0x7f53b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:39.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.458+0000 7f53d5c3a640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53b4066470 0x7f53b4068930 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009be0 tx=0x7f53cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:39.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.458+0000 7f53c67fc640 1 -- 172.21.15.12:0/3683940096 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f53c00512a0 con 0x7f53d01e5600 2023-11-28T21:45:39.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.646+0000 7f53d6c3c640 1 -- 172.21.15.12:0/3683940096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53d015d080 con 0x7f53d01e5600 2023-11-28T21:45:39.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53c67fc640 1 -- 172.21.15.12:0/3683940096 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f53c00548c0 con 0x7f53d01e5600 2023-11-28T21:45:39.655 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:39.655 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:37.764569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:rejoin","state_seq":14,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:39.655 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 47 2023-11-28T21:45:39.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 -- 172.21.15.12:0/3683940096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53b4066470 msgr2=0x7f53b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:39.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53b4066470 0x7f53b4068930 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009be0 tx=0x7f53cc008040 comp rx=0 tx=0).stop 2023-11-28T21:45:39.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 -- 172.21.15.12:0/3683940096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53d01e5600 msgr2=0x7f53d01f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:39.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53d01e5600 0x7f53d01f2a00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f53c000f7c0 tx=0x7f53c000fc80 comp rx=0 tx=0).stop 2023-11-28T21:45:39.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 -- 172.21.15.12:0/3683940096 shutdown_connections 2023-11-28T21:45:39.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f53b4066470 0x7f53b4068930 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:39.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d00cfd70 0x7f53d01e66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:39.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f53d01e5600 0x7f53d01f2a00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:39.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 --2- 172.21.15.12:0/3683940096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc004d10 0x7f53d014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:39.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 -- 172.21.15.12:0/3683940096 >> 172.21.15.12:0/3683940096 conn(0x7f53d0071ce0 msgr2=0x7f53d0152a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:39.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 -- 172.21.15.12:0/3683940096 shutdown_connections 2023-11-28T21:45:39.658 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:39.650+0000 7f53d6c3c640 1 -- 172.21.15.12:0/3683940096 wait complete. 2023-11-28T21:45:39.675 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:45:41.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:41.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f073a8bb640 1 Processor -- start 2023-11-28T21:45:41.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f073a8bb640 1 -- start start 2023-11-28T21:45:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f073a8bb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0734151af0 0x7f07340c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f073a8bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f073406e300 0x7f0734072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f073a8bb640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0734157b70 con 0x7f0734151ed0 2023-11-28T21:45:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f073a8bb640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07340707f0 con 0x7f0734151af0 2023-11-28T21:45:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f073a8bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07340c7b90 con 0x7f073406e300 2023-11-28T21:45:41.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f0733fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0734151af0 0x7f07340c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:41.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f0733fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0734151af0 0x7f07340c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54984/0 (socket says 172.21.15.12:54984) 2023-11-28T21:45:41.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f0733fff640 1 -- 172.21.15.12:0/272946739 learned_addr learned my addr 172.21.15.12:0/272946739 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:41.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f0738e58640 1 --2- 172.21.15.12:0/272946739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f073406e300 0x7f0734072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:41.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f0733fff640 1 -- 172.21.15.12:0/272946739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f073406e300 msgr2=0x7f0734072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:41.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f0733fff640 1 --2- 172.21.15.12:0/272946739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f073406e300 0x7f0734072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:41.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f0733fff640 1 -- 172.21.15.12:0/272946739 >> v1:172.21.15.144:6789/0 conn(0x7f0734151ed0 legacy=0x7f07340c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:41.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.826+0000 7f0733fff640 1 -- 172.21.15.12:0/272946739 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734157e50 con 0x7f0734151af0 2023-11-28T21:45:41.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f0733fff640 1 --2- 172.21.15.12:0/272946739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0734151af0 0x7f07340c7650 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0724009870 tx=0x7f072401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c81d9212fb9061 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:41.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f07337fe640 1 -- 172.21.15.12:0/272946739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240ab600 con 0x7f0734151af0 2023-11-28T21:45:41.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f07337fe640 1 -- 172.21.15.12:0/272946739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07240a7050 con 0x7f0734151af0 2023-11-28T21:45:41.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f07337fe640 1 -- 172.21.15.12:0/272946739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240ab980 con 0x7f0734151af0 2023-11-28T21:45:41.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f073a8bb640 1 -- 172.21.15.12:0/272946739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0734151af0 msgr2=0x7f07340c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:41.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f073a8bb640 1 --2- 172.21.15.12:0/272946739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0734151af0 0x7f07340c7650 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0724009870 tx=0x7f072401c540 comp rx=0 tx=0).stop 2023-11-28T21:45:41.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f073a8bb640 1 -- 172.21.15.12:0/272946739 shutdown_connections 2023-11-28T21:45:41.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f073a8bb640 1 --2- 172.21.15.12:0/272946739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f073406e300 0x7f0734072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:41.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f073a8bb640 1 --2- 172.21.15.12:0/272946739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0734151af0 0x7f07340c7650 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:41.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f073a8bb640 1 -- 172.21.15.12:0/272946739 >> 172.21.15.12:0/272946739 conn(0x7f0734066360 msgr2=0x7f0734066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:41.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f073a8bb640 1 -- 172.21.15.12:0/272946739 shutdown_connections 2023-11-28T21:45:41.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.830+0000 7f073a8bb640 1 -- 172.21.15.12:0/272946739 wait complete. 2023-11-28T21:45:41.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f073a8bb640 1 Processor -- start 2023-11-28T21:45:41.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f073a8bb640 1 -- start start 2023-11-28T21:45:41.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f073a8bb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f073406e300 0x7f07340c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:41.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f073a8bb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0734151af0 0x7f07341f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:41.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f073a8bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f073413f700 0x7f073413e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:41.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f073a8bb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340c7030 con 0x7f073406e300 2023-11-28T21:45:41.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f073a8bb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340c9160 con 0x7f0734151af0 2023-11-28T21:45:41.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f073a8bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07340c9440 con 0x7f073413f700 2023-11-28T21:45:41.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f0733fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f073406e300 0x7f07340c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:41.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f0733fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f073406e300 0x7f07340c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54990/0 (socket says 172.21.15.12:54990) 2023-11-28T21:45:41.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f0733fff640 1 -- 172.21.15.12:0/2490756695 learned_addr learned my addr 172.21.15.12:0/2490756695 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:41.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f072bfff640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0734151af0 0x7f07341f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:41.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f0738e58640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f073413f700 0x7f073413e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:41.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f0733fff640 1 -- 172.21.15.12:0/2490756695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f073413f700 msgr2=0x7f073413e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:41.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f0733fff640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f073413f700 0x7f073413e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:41.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f0733fff640 1 -- 172.21.15.12:0/2490756695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0734151af0 msgr2=0x7f07341f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:41.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f0733fff640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0734151af0 0x7f07341f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:41.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f0733fff640 1 -- 172.21.15.12:0/2490756695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07240094d0 con 0x7f073406e300 2023-11-28T21:45:41.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f0733fff640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f073406e300 0x7f07340c7450 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f072401abf0 tx=0x7f07240192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:41.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f07317fa640 1 -- 172.21.15.12:0/2490756695 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240b4460 con 0x7f073406e300 2023-11-28T21:45:41.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f07317fa640 1 -- 172.21.15.12:0/2490756695 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07240b45c0 con 0x7f073406e300 2023-11-28T21:45:41.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f073a8bb640 1 -- 172.21.15.12:0/2490756695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07340c9720 con 0x7f073406e300 2023-11-28T21:45:41.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f07317fa640 1 -- 172.21.15.12:0/2490756695 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240b4970 con 0x7f073406e300 2023-11-28T21:45:41.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.834+0000 7f073a8bb640 1 -- 172.21.15.12:0/2490756695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07340c6c50 con 0x7f073406e300 2023-11-28T21:45:41.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.838+0000 7f073a8bb640 1 -- 172.21.15.12:0/2490756695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07340c72a0 con 0x7f073406e300 2023-11-28T21:45:41.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.838+0000 7f07317fa640 1 -- 172.21.15.12:0/2490756695 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f07240b2070 con 0x7f073406e300 2023-11-28T21:45:41.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.838+0000 7f07317fa640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0714066470 0x7f0714068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:41.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.838+0000 7f07317fa640 1 -- 172.21.15.12:0/2490756695 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f072411e260 con 0x7f073406e300 2023-11-28T21:45:41.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.838+0000 7f072bfff640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0714066470 0x7f0714068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:41.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.838+0000 7f072bfff640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0714066470 0x7f0714068930 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f07341622b0 tx=0x7f071c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:41.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:41.842+0000 7f07317fa640 1 -- 172.21.15.12:0/2490756695 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f07240eb160 con 0x7f073406e300 2023-11-28T21:45:42.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.030+0000 7f073a8bb640 1 -- 172.21.15.12:0/2490756695 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0734074f60 con 0x7f073406e300 2023-11-28T21:45:42.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f07317fa640 1 -- 172.21.15.12:0/2490756695 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f07240ee780 con 0x7f073406e300 2023-11-28T21:45:42.038 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:42.038 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:37.764569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:rejoin","state_seq":14,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:42.039 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 47 2023-11-28T21:45:42.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 -- 172.21.15.12:0/2490756695 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0714066470 msgr2=0x7f0714068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:42.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0714066470 0x7f0714068930 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f07341622b0 tx=0x7f071c009040 comp rx=0 tx=0).stop 2023-11-28T21:45:42.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 -- 172.21.15.12:0/2490756695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f073406e300 msgr2=0x7f07340c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:42.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f073406e300 0x7f07340c7450 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f072401abf0 tx=0x7f07240192e0 comp rx=0 tx=0).stop 2023-11-28T21:45:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 -- 172.21.15.12:0/2490756695 shutdown_connections 2023-11-28T21:45:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0714066470 0x7f0714068930 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f073413f700 0x7f073413e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0734151af0 0x7f07341f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 --2- 172.21.15.12:0/2490756695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f073406e300 0x7f07340c7450 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 -- 172.21.15.12:0/2490756695 >> 172.21.15.12:0/2490756695 conn(0x7f0734066360 msgr2=0x7f073406faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 -- 172.21.15.12:0/2490756695 shutdown_connections 2023-11-28T21:45:42.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:42.034+0000 7f073a8bb640 1 -- 172.21.15.12:0/2490756695 wait complete. 2023-11-28T21:45:42.059 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:45:44.061 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:44.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.210+0000 7f4bd6e9c640 1 Processor -- start 2023-11-28T21:45:44.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.210+0000 7f4bd6e9c640 1 -- start start 2023-11-28T21:45:44.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.210+0000 7f4bd6e9c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4bd0158b20 0x7f4bd0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:44.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.210+0000 7f4bd6e9c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd0067410 con 0x7f4bd0150c20 2023-11-28T21:45:44.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.210+0000 7f4bd6e9c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd006e090 con 0x7f4bd006e300 2023-11-28T21:45:44.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.210+0000 7f4bd6e9c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd0157f30 con 0x7f4bd0158b20 2023-11-28T21:45:44.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.214+0000 7f4bd4c38640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4bd0150c20 0x7f4bd00c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33644/0 (socket says 172.21.15.12:33644) 2023-11-28T21:45:44.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.214+0000 7f4bd4c38640 1 -- 172.21.15.12:0/179757475 learned_addr learned my addr 172.21.15.12:0/179757475 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:45:44.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.214+0000 7f4bd5439640 1 --2- 172.21.15.12:0/179757475 >> v2:172.21.15.144:3300/0 conn(0x7f4bd0158b20 0x7f4bd0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:44.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.214+0000 7f4bcf7fe640 1 -- 172.21.15.12:0/179757475 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576460991 0 0) 0x7f4bd0067410 con 0x7f4bd0150c20 2023-11-28T21:45:44.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.214+0000 7f4bcf7fe640 1 -- 172.21.15.12:0/179757475 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc0003530 con 0x7f4bd0150c20 2023-11-28T21:45:44.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.214+0000 7f4bcf7fe640 1 -- 172.21.15.12:0/179757475 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2362919984 0 0) 0x7f4bc0003530 con 0x7f4bd0150c20 2023-11-28T21:45:44.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.214+0000 7f4bd5439640 1 -- 172.21.15.12:0/179757475 >> v1:172.21.15.144:6789/0 conn(0x7f4bd006e300 legacy=0x7f4bd0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:44.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.214+0000 7f4bd5439640 1 -- 172.21.15.12:0/179757475 >> v1:172.21.15.12:6789/0 conn(0x7f4bd0150c20 legacy=0x7f4bd00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:44.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.214+0000 7f4bd5439640 1 -- 172.21.15.12:0/179757475 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0151000 con 0x7f4bd0158b20 2023-11-28T21:45:44.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.214+0000 7f4bd5439640 1 --2- 172.21.15.12:0/179757475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bd0158b20 0x7f4bd0156910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009d90 tx=0x7f4bc801c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb4567a0a633466 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:44.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bcf7fe640 1 -- 172.21.15.12:0/179757475 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4001680 con 0x7f4bd0158b20 2023-11-28T21:45:44.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bcf7fe640 1 -- 172.21.15.12:0/179757475 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc80a55c0 con 0x7f4bd0158b20 2023-11-28T21:45:44.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bcf7fe640 1 -- 172.21.15.12:0/179757475 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80a5930 con 0x7f4bd0158b20 2023-11-28T21:45:44.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/179757475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bd0158b20 msgr2=0x7f4bd0156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:44.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 --2- 172.21.15.12:0/179757475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bd0158b20 0x7f4bd0156910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009d90 tx=0x7f4bc801c870 comp rx=0 tx=0).stop 2023-11-28T21:45:44.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/179757475 shutdown_connections 2023-11-28T21:45:44.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 --2- 172.21.15.12:0/179757475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bd0158b20 0x7f4bd0156910 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:44.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/179757475 >> 172.21.15.12:0/179757475 conn(0x7f4bd0066360 msgr2=0x7f4bd0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:44.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/179757475 shutdown_connections 2023-11-28T21:45:44.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/179757475 wait complete. 2023-11-28T21:45:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 Processor -- start 2023-11-28T21:45:44.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 -- start start 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bd006e300 0x7f4bd00c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bd0150c20 0x7f4bd006ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.218+0000 7f4bd6e9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd0158b20 0x7f4bd014f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd6e9c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0153880 con 0x7f4bd0158b20 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd6e9c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd00c2990 con 0x7f4bd006e300 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd6e9c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd00b9050 con 0x7f4bd0150c20 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd5439640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd0158b20 0x7f4bd014f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd5439640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd0158b20 0x7f4bd014f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55002/0 (socket says 172.21.15.12:55002) 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd5439640 1 -- 172.21.15.12:0/602817348 learned_addr learned my addr 172.21.15.12:0/602817348 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd4c38640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bd006e300 0x7f4bd00c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:44.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bcffff640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bd0150c20 0x7f4bd006ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:44.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd5439640 1 -- 172.21.15.12:0/602817348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bd0150c20 msgr2=0x7f4bd006ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:44.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd5439640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bd0150c20 0x7f4bd006ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:44.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd5439640 1 -- 172.21.15.12:0/602817348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bd006e300 msgr2=0x7f4bd00c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:44.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd5439640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bd006e300 0x7f4bd00c3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:44.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd5439640 1 -- 172.21.15.12:0/602817348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8009a70 con 0x7f4bd0158b20 2023-11-28T21:45:44.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd4c38640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bd006e300 0x7f4bd00c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:45:44.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd5439640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd0158b20 0x7f4bd014f890 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8019480 tx=0x7f4bc8019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:44.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bcd7fa640 1 -- 172.21.15.12:0/602817348 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80bd070 con 0x7f4bd0158b20 2023-11-28T21:45:44.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bcd7fa640 1 -- 172.21.15.12:0/602817348 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc80a5940 con 0x7f4bd0158b20 2023-11-28T21:45:44.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/602817348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd00c4420 con 0x7f4bd0158b20 2023-11-28T21:45:44.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bcd7fa640 1 -- 172.21.15.12:0/602817348 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80a5cd0 con 0x7f4bd0158b20 2023-11-28T21:45:44.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/602817348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd00c49e0 con 0x7f4bd0158b20 2023-11-28T21:45:44.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/602817348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd00c4bd0 con 0x7f4bd0158b20 2023-11-28T21:45:44.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.222+0000 7f4bcd7fa640 1 -- 172.21.15.12:0/602817348 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4bc8004040 con 0x7f4bd0158b20 2023-11-28T21:45:44.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.226+0000 7f4bcd7fa640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4bb4066470 0x7f4bb4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:44.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.226+0000 7f4bcd7fa640 1 -- 172.21.15.12:0/602817348 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f4bc811eaf0 con 0x7f4bd0158b20 2023-11-28T21:45:44.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.226+0000 7f4bd4c38640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4bb4066470 0x7f4bb4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:44.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.226+0000 7f4bd4c38640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4bb4066470 0x7f4bb4068930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4005890 tx=0x7f4bc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:44.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.230+0000 7f4bcd7fa640 1 -- 172.21.15.12:0/602817348 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4bc80eb960 con 0x7f4bd0158b20 2023-11-28T21:45:44.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.430+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/602817348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bd0074f60 con 0x7f4bd0158b20 2023-11-28T21:45:44.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.430+0000 7f4bcd7fa640 1 -- 172.21.15.12:0/602817348 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5721 (secure 0 0 0) 0x7f4bc80eef80 con 0x7f4bd0158b20 2023-11-28T21:45:44.437 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:44.437 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:42.925654+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:clientreplay","state_seq":16,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:44.437 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 48 2023-11-28T21:45:44.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/602817348 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4bb4066470 msgr2=0x7f4bb4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:44.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4bb4066470 0x7f4bb4068930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4005890 tx=0x7f4bc4009040 comp rx=0 tx=0).stop 2023-11-28T21:45:44.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/602817348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd0158b20 msgr2=0x7f4bd014f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:44.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd0158b20 0x7f4bd014f890 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8019480 tx=0x7f4bc8019610 comp rx=0 tx=0).stop 2023-11-28T21:45:44.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/602817348 shutdown_connections 2023-11-28T21:45:44.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4bb4066470 0x7f4bb4068930 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:44.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4bd0158b20 0x7f4bd014f890 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:44.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bd0150c20 0x7f4bd006ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:44.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 --2- 172.21.15.12:0/602817348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bd006e300 0x7f4bd00c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:44.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/602817348 >> 172.21.15.12:0/602817348 conn(0x7f4bd0066360 msgr2=0x7f4bd0155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:44.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/602817348 shutdown_connections 2023-11-28T21:45:44.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:44.434+0000 7f4bd6e9c640 1 -- 172.21.15.12:0/602817348 wait complete. 2023-11-28T21:45:44.458 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:45:45.305 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:45:45.310 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:45:46.459 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:46.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.610+0000 7f24e5ec9640 1 Processor -- start 2023-11-28T21:45:46.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.610+0000 7f24e5ec9640 1 -- start start 2023-11-28T21:45:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.610+0000 7f24e5ec9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f24e0150c20 0x7f24e0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.610+0000 7f24e5ec9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e006e300 0x7f24e00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:46.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.610+0000 7f24e5ec9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f24e006cf40 0x7f24e00c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.610+0000 7f24e5ec9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e0067410 con 0x7f24e0150c20 2023-11-28T21:45:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.610+0000 7f24e5ec9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e006e090 con 0x7f24e006cf40 2023-11-28T21:45:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.610+0000 7f24e5ec9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0157f30 con 0x7f24e006e300 2023-11-28T21:45:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24df7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f24e0150c20 0x7f24e0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24df7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f24e0150c20 0x7f24e0156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53718/0 (socket says 172.21.15.12:53718) 2023-11-28T21:45:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24df7fe640 1 -- 172.21.15.12:0/2049777238 learned_addr learned my addr 172.21.15.12:0/2049777238 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24deffd640 1 --2- 172.21.15.12:0/2049777238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e006e300 0x7f24e00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:46.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24dffff640 1 --2- 172.21.15.12:0/2049777238 >> v2:172.21.15.144:3300/0 conn(0x7f24e006cf40 0x7f24e00c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24df7fe640 1 -- 172.21.15.12:0/2049777238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e006e300 msgr2=0x7f24e00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24df7fe640 1 --2- 172.21.15.12:0/2049777238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e006e300 0x7f24e00c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24df7fe640 1 -- 172.21.15.12:0/2049777238 >> v2:172.21.15.144:3300/0 conn(0x7f24e006cf40 msgr2=0x7f24e00c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24df7fe640 1 --2- 172.21.15.12:0/2049777238 >> v2:172.21.15.144:3300/0 conn(0x7f24e006cf40 0x7f24e00c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24df7fe640 1 -- 172.21.15.12:0/2049777238 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e0151000 con 0x7f24e0150c20 2023-11-28T21:45:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24df7fe640 1 --2- 172.21.15.12:0/2049777238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e0150c20 0x7f24e0156910 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f24e00c7d70 tx=0x7f24d001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ae36d8a06cfaf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:46.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24de7fc640 1 -- 172.21.15.12:0/2049777238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d00ac630 con 0x7f24e0150c20 2023-11-28T21:45:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24de7fc640 1 -- 172.21.15.12:0/2049777238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d00ac790 con 0x7f24e0150c20 2023-11-28T21:45:46.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.614+0000 7f24de7fc640 1 -- 172.21.15.12:0/2049777238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d00acb20 con 0x7f24e0150c20 2023-11-28T21:45:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 -- 172.21.15.12:0/2049777238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e0150c20 msgr2=0x7f24e0156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 --2- 172.21.15.12:0/2049777238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e0150c20 0x7f24e0156910 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f24e00c7d70 tx=0x7f24d001c500 comp rx=0 tx=0).stop 2023-11-28T21:45:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 -- 172.21.15.12:0/2049777238 shutdown_connections 2023-11-28T21:45:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 --2- 172.21.15.12:0/2049777238 >> v2:172.21.15.144:3300/0 conn(0x7f24e006cf40 0x7f24e00c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 --2- 172.21.15.12:0/2049777238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e006e300 0x7f24e00c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 --2- 172.21.15.12:0/2049777238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e0150c20 0x7f24e0156910 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 -- 172.21.15.12:0/2049777238 >> 172.21.15.12:0/2049777238 conn(0x7f24e0066360 msgr2=0x7f24e0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 -- 172.21.15.12:0/2049777238 shutdown_connections 2023-11-28T21:45:46.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 -- 172.21.15.12:0/2049777238 wait complete. 2023-11-28T21:45:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 Processor -- start 2023-11-28T21:45:46.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 -- start start 2023-11-28T21:45:46.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e006cf40 0x7f24e014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e006e300 0x7f24e014b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e0150c20 0x7f24e0071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e01ee410 con 0x7f24e006e300 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e01ee6f0 con 0x7f24e006cf40 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24e5ec9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e01ee9d0 con 0x7f24e0150c20 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24deffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e006e300 0x7f24e014b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24deffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e006e300 0x7f24e014b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53724/0 (socket says 172.21.15.12:53724) 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24deffd640 1 -- 172.21.15.12:0/876383637 learned_addr learned my addr 172.21.15.12:0/876383637 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24df7fe640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e006cf40 0x7f24e014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.618+0000 7f24dffff640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e0150c20 0x7f24e0071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:46.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24deffd640 1 -- 172.21.15.12:0/876383637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e0150c20 msgr2=0x7f24e0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:46.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24deffd640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e0150c20 0x7f24e0071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24deffd640 1 -- 172.21.15.12:0/876383637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e006cf40 msgr2=0x7f24e014c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:46.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24deffd640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e006cf40 0x7f24e014c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24deffd640 1 -- 172.21.15.12:0/876383637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00094d0 con 0x7f24e006e300 2023-11-28T21:45:46.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24deffd640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e006e300 0x7f24e014b660 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f24d400d800 tx=0x7f24d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:46.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24c7fff640 1 -- 172.21.15.12:0/876383637 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d400b6c0 con 0x7f24e006e300 2023-11-28T21:45:46.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24e5ec9640 1 -- 172.21.15.12:0/876383637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e01eecb0 con 0x7f24e006e300 2023-11-28T21:45:46.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24c7fff640 1 -- 172.21.15.12:0/876383637 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d400b820 con 0x7f24e006e300 2023-11-28T21:45:46.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24c7fff640 1 -- 172.21.15.12:0/876383637 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d400bb90 con 0x7f24e006e300 2023-11-28T21:45:46.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24e5ec9640 1 -- 172.21.15.12:0/876383637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e00c97a0 con 0x7f24e006e300 2023-11-28T21:45:46.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24e5ec9640 1 -- 172.21.15.12:0/876383637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e0074f60 con 0x7f24e006e300 2023-11-28T21:45:46.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24c7fff640 1 -- 172.21.15.12:0/876383637 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f24d400bcf0 con 0x7f24e006e300 2023-11-28T21:45:46.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24c7fff640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f24cc066470 0x7f24cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:46.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24c7fff640 1 -- 172.21.15.12:0/876383637 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f24d40848f0 con 0x7f24e006e300 2023-11-28T21:45:46.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.622+0000 7f24df7fe640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f24cc066470 0x7f24cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:46.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.626+0000 7f24df7fe640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f24cc066470 0x7f24cc068930 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f24d00ae000 tx=0x7f24d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:46.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.626+0000 7f24c7fff640 1 -- 172.21.15.12:0/876383637 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f24d4051790 con 0x7f24e006e300 2023-11-28T21:45:46.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.818+0000 7f24e5ec9640 1 -- 172.21.15.12:0/876383637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24e00c2c30 con 0x7f24e006e300 2023-11-28T21:45:46.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.818+0000 7f24c7fff640 1 -- 172.21.15.12:0/876383637 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5721 (secure 0 0 0) 0x7f24d4054db0 con 0x7f24e006e300 2023-11-28T21:45:46.826 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:46.826 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:42.925654+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:clientreplay","state_seq":16,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:46.828 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 48 2023-11-28T21:45:46.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 -- 172.21.15.12:0/876383637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f24cc066470 msgr2=0x7f24cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:46.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f24cc066470 0x7f24cc068930 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f24d00ae000 tx=0x7f24d0008040 comp rx=0 tx=0).stop 2023-11-28T21:45:46.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 -- 172.21.15.12:0/876383637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e006e300 msgr2=0x7f24e014b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:46.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e006e300 0x7f24e014b660 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f24d400d800 tx=0x7f24d400dcc0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 -- 172.21.15.12:0/876383637 shutdown_connections 2023-11-28T21:45:46.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f24cc066470 0x7f24cc068930 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e0150c20 0x7f24e0071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24e006e300 0x7f24e014b660 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 --2- 172.21.15.12:0/876383637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e006cf40 0x7f24e014c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:46.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 -- 172.21.15.12:0/876383637 >> 172.21.15.12:0/876383637 conn(0x7f24e0066360 msgr2=0x7f24e00c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:46.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 -- 172.21.15.12:0/876383637 shutdown_connections 2023-11-28T21:45:46.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:46.822+0000 7f24e5ec9640 1 -- 172.21.15.12:0/876383637 wait complete. 2023-11-28T21:45:46.848 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:45:48.849 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:49.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.042+0000 7fcf25c17640 1 Processor -- start 2023-11-28T21:45:49.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.042+0000 7fcf25c17640 1 -- start start 2023-11-28T21:45:49.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf25c17640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcf200c7300 0x7fcf20156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:49.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf25c17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2006b7c0 con 0x7fcf200c6f20 2023-11-28T21:45:49.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf25c17640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2006a8f0 con 0x7fcf200c26b0 2023-11-28T21:45:49.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf25c17640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2006abb0 con 0x7fcf200c7300 2023-11-28T21:45:49.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf17fff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcf200c26b0 0x7fcf20150840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:47348/0 (socket says 172.21.15.12:47348) 2023-11-28T21:45:49.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf17fff640 1 -- 172.21.15.12:0/904715876 learned_addr learned my addr 172.21.15.12:0/904715876 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:45:49.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf25416640 1 --2- 172.21.15.12:0/904715876 >> v2:172.21.15.12:3300/0 conn(0x7fcf200c7300 0x7fcf20156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:49.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf177fe640 1 -- 172.21.15.12:0/904715876 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953452925 0 0) 0x7fcf2006a8f0 con 0x7fcf200c26b0 2023-11-28T21:45:49.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf177fe640 1 -- 172.21.15.12:0/904715876 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf08005740 con 0x7fcf200c26b0 2023-11-28T21:45:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf25416640 1 -- 172.21.15.12:0/904715876 >> v1:172.21.15.144:6789/0 conn(0x7fcf200c26b0 legacy=0x7fcf20150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf25416640 1 -- 172.21.15.12:0/904715876 >> v1:172.21.15.12:6789/0 conn(0x7fcf200c6f20 legacy=0x7fcf20158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:49.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.046+0000 7fcf25416640 1 -- 172.21.15.12:0/904715876 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2006ae90 con 0x7fcf200c7300 2023-11-28T21:45:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf25416640 1 --2- 172.21.15.12:0/904715876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf200c7300 0x7fcf20156ac0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009d90 tx=0x7fcf1801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36fe5f70108933c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf177fe640 1 -- 172.21.15.12:0/904715876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf180a5910 con 0x7fcf200c7300 2023-11-28T21:45:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf177fe640 1 -- 172.21.15.12:0/904715876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf180a5a70 con 0x7fcf200c7300 2023-11-28T21:45:49.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf177fe640 1 -- 172.21.15.12:0/904715876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf180a5e00 con 0x7fcf200c7300 2023-11-28T21:45:49.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf25c17640 1 -- 172.21.15.12:0/904715876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf200c7300 msgr2=0x7fcf20156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:49.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf25c17640 1 --2- 172.21.15.12:0/904715876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf200c7300 0x7fcf20156ac0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009d90 tx=0x7fcf1801c950 comp rx=0 tx=0).stop 2023-11-28T21:45:49.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf25c17640 1 -- 172.21.15.12:0/904715876 shutdown_connections 2023-11-28T21:45:49.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf25c17640 1 --2- 172.21.15.12:0/904715876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf200c7300 0x7fcf20156ac0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:49.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf25c17640 1 -- 172.21.15.12:0/904715876 >> 172.21.15.12:0/904715876 conn(0x7fcf20066360 msgr2=0x7fcf20066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:49.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf25c17640 1 -- 172.21.15.12:0/904715876 shutdown_connections 2023-11-28T21:45:49.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.050+0000 7fcf25c17640 1 -- 172.21.15.12:0/904715876 wait complete. 2023-11-28T21:45:49.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25c17640 1 Processor -- start 2023-11-28T21:45:49.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25c17640 1 -- start start 2023-11-28T21:45:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25c17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf200c26b0 0x7fcf201fad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25c17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf200c6f20 0x7fcf200c1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25c17640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf200c7300 0x7fcf20142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25c17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf20156aa0 con 0x7fcf200c6f20 2023-11-28T21:45:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25c17640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf201fb2a0 con 0x7fcf200c7300 2023-11-28T21:45:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25c17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf201f6f80 con 0x7fcf200c26b0 2023-11-28T21:45:49.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf200c6f20 0x7fcf200c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25416640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf200c7300 0x7fcf20142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf24c15640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf200c26b0 0x7fcf201fad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25416640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf200c7300 0x7fcf20142b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:53806/0 (socket says 172.21.15.12:53806) 2023-11-28T21:45:49.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25416640 1 -- 172.21.15.12:0/3154075993 learned_addr learned my addr 172.21.15.12:0/3154075993 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf24c15640 1 -- 172.21.15.12:0/3154075993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf200c7300 msgr2=0x7fcf20142b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf24c15640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf200c7300 0x7fcf20142b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf24c15640 1 -- 172.21.15.12:0/3154075993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf200c6f20 msgr2=0x7fcf200c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf24c15640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf200c6f20 0x7fcf200c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf24c15640 1 -- 172.21.15.12:0/3154075993 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf18009a70 con 0x7fcf200c26b0 2023-11-28T21:45:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.054+0000 7fcf25416640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf200c7300 0x7fcf20142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:45:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.058+0000 7fcf24c15640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf200c26b0 0x7fcf201fad60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c3c0 tx=0x7fcf1c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.058+0000 7fcf157fa640 1 -- 172.21.15.12:0/3154075993 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0052d0 con 0x7fcf200c26b0 2023-11-28T21:45:49.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.058+0000 7fcf157fa640 1 -- 172.21.15.12:0/3154075993 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c005430 con 0x7fcf200c26b0 2023-11-28T21:45:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.058+0000 7fcf25c17640 1 -- 172.21.15.12:0/3154075993 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf201f7220 con 0x7fcf200c26b0 2023-11-28T21:45:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.058+0000 7fcf157fa640 1 -- 172.21.15.12:0/3154075993 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c007260 con 0x7fcf200c26b0 2023-11-28T21:45:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.058+0000 7fcf25c17640 1 -- 172.21.15.12:0/3154075993 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf201f77e0 con 0x7fcf200c26b0 2023-11-28T21:45:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.058+0000 7fcf25c17640 1 -- 172.21.15.12:0/3154075993 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf20067c70 con 0x7fcf200c26b0 2023-11-28T21:45:49.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.058+0000 7fcf157fa640 1 -- 172.21.15.12:0/3154075993 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fcf1c0073c0 con 0x7fcf200c26b0 2023-11-28T21:45:49.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.058+0000 7fcf157fa640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf0c066470 0x7fcf0c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:49.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.058+0000 7fcf157fa640 1 -- 172.21.15.12:0/3154075993 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fcf1c083780 con 0x7fcf200c26b0 2023-11-28T21:45:49.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.062+0000 7fcf17fff640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf0c066470 0x7fcf0c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:49.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.062+0000 7fcf17fff640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf0c066470 0x7fcf0c068930 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcf20150020 tx=0x7fcf10007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:49.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.066+0000 7fcf157fa640 1 -- 172.21.15.12:0/3154075993 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcf1c050620 con 0x7fcf200c26b0 2023-11-28T21:45:49.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.326+0000 7fcf25c17640 1 -- 172.21.15.12:0/3154075993 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf2006fb40 con 0x7fcf200c26b0 2023-11-28T21:45:49.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.326+0000 7fcf157fa640 1 -- 172.21.15.12:0/3154075993 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5721 (secure 0 0 0) 0x7fcf1c053c40 con 0x7fcf200c26b0 2023-11-28T21:45:49.333 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:49.333 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:42.925654+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:clientreplay","state_seq":16,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:49.333 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 48 2023-11-28T21:45:49.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 -- 172.21.15.12:0/3154075993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf0c066470 msgr2=0x7fcf0c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:49.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf0c066470 0x7fcf0c068930 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcf20150020 tx=0x7fcf10007d40 comp rx=0 tx=0).stop 2023-11-28T21:45:49.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 -- 172.21.15.12:0/3154075993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf200c26b0 msgr2=0x7fcf201fad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:49.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf200c26b0 0x7fcf201fad60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c3c0 tx=0x7fcf1c00c880 comp rx=0 tx=0).stop 2023-11-28T21:45:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 -- 172.21.15.12:0/3154075993 shutdown_connections 2023-11-28T21:45:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf0c066470 0x7fcf0c068930 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf200c7300 0x7fcf20142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf200c6f20 0x7fcf200c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 --2- 172.21.15.12:0/3154075993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf200c26b0 0x7fcf201fad60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:49.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 -- 172.21.15.12:0/3154075993 >> 172.21.15.12:0/3154075993 conn(0x7fcf20066360 msgr2=0x7fcf2014ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:49.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 -- 172.21.15.12:0/3154075993 shutdown_connections 2023-11-28T21:45:49.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:49.330+0000 7fcf25c17640 1 -- 172.21.15.12:0/3154075993 wait complete. 2023-11-28T21:45:49.355 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 48, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 48, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5871": { "addr": "172.21.15.12:6838/3474424840", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 3474424840, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 3474424840, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5871, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:clientreplay", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 47, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:45:42.925654+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 5871 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:45:51.356 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.506+0000 7ff7dd32a640 1 Processor -- start 2023-11-28T21:45:51.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.506+0000 7ff7dd32a640 1 -- start start 2023-11-28T21:45:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.506+0000 7ff7dd32a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8150c20 0x7ff7d8156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.506+0000 7ff7dd32a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff7d806cf40 0x7ff7d8158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.506+0000 7ff7dd32a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d8067410 con 0x7ff7d806e300 2023-11-28T21:45:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.506+0000 7ff7dd32a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d806e090 con 0x7ff7d806cf40 2023-11-28T21:45:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.506+0000 7ff7dd32a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8157f30 con 0x7ff7d8150c20 2023-11-28T21:45:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.506+0000 7ff7cffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff7d806e300 0x7ff7d80c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:41744/0 (socket says 172.21.15.12:41744) 2023-11-28T21:45:51.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.506+0000 7ff7cffff640 1 -- 172.21.15.12:0/225481272 learned_addr learned my addr 172.21.15.12:0/225481272 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:45:51.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.510+0000 7ff7d6ffd640 1 --2- 172.21.15.12:0/225481272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8150c20 0x7ff7d8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:51.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.510+0000 7ff7d77fe640 1 --2- 172.21.15.12:0/225481272 >> v2:172.21.15.144:3300/0 conn(0x7ff7d806cf40 0x7ff7d8158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:51.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.510+0000 7ff7d67fc640 1 -- 172.21.15.12:0/225481272 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719770808 0 0) 0x7ff7d8067410 con 0x7ff7d806e300 2023-11-28T21:45:51.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.510+0000 7ff7d67fc640 1 -- 172.21.15.12:0/225481272 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c4003530 con 0x7ff7d806e300 2023-11-28T21:45:51.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.510+0000 7ff7d6ffd640 1 -- 172.21.15.12:0/225481272 >> v2:172.21.15.144:3300/0 conn(0x7ff7d806cf40 msgr2=0x7ff7d8158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:51.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.510+0000 7ff7d6ffd640 1 --2- 172.21.15.12:0/225481272 >> v2:172.21.15.144:3300/0 conn(0x7ff7d806cf40 0x7ff7d8158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:51.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.510+0000 7ff7d6ffd640 1 -- 172.21.15.12:0/225481272 >> v1:172.21.15.12:6789/0 conn(0x7ff7d806e300 legacy=0x7ff7d80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:51.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.510+0000 7ff7d6ffd640 1 -- 172.21.15.12:0/225481272 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d8151000 con 0x7ff7d8150c20 2023-11-28T21:45:51.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.510+0000 7ff7d6ffd640 1 --2- 172.21.15.12:0/225481272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8150c20 0x7ff7d8156910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80097c0 tx=0x7ff7c801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db32ce5ddc4d3281 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7d67fc640 1 -- 172.21.15.12:0/225481272 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80ad640 con 0x7ff7d8150c20 2023-11-28T21:45:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7d67fc640 1 -- 172.21.15.12:0/225481272 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c80ad7a0 con 0x7ff7d8150c20 2023-11-28T21:45:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7d67fc640 1 -- 172.21.15.12:0/225481272 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80adb30 con 0x7ff7d8150c20 2023-11-28T21:45:51.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7dd32a640 1 -- 172.21.15.12:0/225481272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8150c20 msgr2=0x7ff7d8156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7dd32a640 1 --2- 172.21.15.12:0/225481272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8150c20 0x7ff7d8156910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80097c0 tx=0x7ff7c801c480 comp rx=0 tx=0).stop 2023-11-28T21:45:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7dd32a640 1 -- 172.21.15.12:0/225481272 shutdown_connections 2023-11-28T21:45:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7dd32a640 1 --2- 172.21.15.12:0/225481272 >> v2:172.21.15.144:3300/0 conn(0x7ff7d806cf40 0x7ff7d8158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7dd32a640 1 --2- 172.21.15.12:0/225481272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d8150c20 0x7ff7d8156910 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7dd32a640 1 -- 172.21.15.12:0/225481272 >> 172.21.15.12:0/225481272 conn(0x7ff7d8066360 msgr2=0x7ff7d8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7dd32a640 1 -- 172.21.15.12:0/225481272 shutdown_connections 2023-11-28T21:45:51.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.518+0000 7ff7dd32a640 1 -- 172.21.15.12:0/225481272 wait complete. 2023-11-28T21:45:51.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7dd32a640 1 Processor -- start 2023-11-28T21:45:51.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7dd32a640 1 -- start start 2023-11-28T21:45:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7dd32a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d806cf40 0x7ff7d80c3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7dd32a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d806e300 0x7ff7d80c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7dd32a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d8150c20 0x7ff7d8147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7dd32a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8072e70 con 0x7ff7d8150c20 2023-11-28T21:45:51.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7dd32a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d8072bd0 con 0x7ff7d806e300 2023-11-28T21:45:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7dd32a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d81f3120 con 0x7ff7d806cf40 2023-11-28T21:45:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7d77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d8150c20 0x7ff7d8147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7d77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d8150c20 0x7ff7d8147bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53752/0 (socket says 172.21.15.12:53752) 2023-11-28T21:45:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7d77fe640 1 -- 172.21.15.12:0/1433005711 learned_addr learned my addr 172.21.15.12:0/1433005711 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7d6ffd640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d806cf40 0x7ff7d80c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7cffff640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d806e300 0x7ff7d80c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:51.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7d77fe640 1 -- 172.21.15.12:0/1433005711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d806cf40 msgr2=0x7ff7d80c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7d77fe640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d806cf40 0x7ff7d80c3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7d77fe640 1 -- 172.21.15.12:0/1433005711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d806e300 msgr2=0x7ff7d80c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7d77fe640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d806e300 0x7ff7d80c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7d77fe640 1 -- 172.21.15.12:0/1433005711 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c8009420 con 0x7ff7d8150c20 2023-11-28T21:45:51.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7d77fe640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d8150c20 0x7ff7d8147bc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000efc0 tx=0x7ff7d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7cf7fe640 1 -- 172.21.15.12:0/1433005711 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0014480 con 0x7ff7d8150c20 2023-11-28T21:45:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7cf7fe640 1 -- 172.21.15.12:0/1433005711 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d000f040 con 0x7ff7d8150c20 2023-11-28T21:45:51.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7cf7fe640 1 -- 172.21.15.12:0/1433005711 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0014820 con 0x7ff7d8150c20 2023-11-28T21:45:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7dd32a640 1 -- 172.21.15.12:0/1433005711 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d81f33c0 con 0x7ff7d8150c20 2023-11-28T21:45:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.522+0000 7ff7dd32a640 1 -- 172.21.15.12:0/1433005711 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d8072560 con 0x7ff7d8150c20 2023-11-28T21:45:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.526+0000 7ff7dd32a640 1 -- 172.21.15.12:0/1433005711 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d8072780 con 0x7ff7d8150c20 2023-11-28T21:45:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.526+0000 7ff7cf7fe640 1 -- 172.21.15.12:0/1433005711 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff7d0014980 con 0x7ff7d8150c20 2023-11-28T21:45:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.526+0000 7ff7cf7fe640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7b80664a0 0x7ff7b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:51.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.526+0000 7ff7cf7fe640 1 -- 172.21.15.12:0/1433005711 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7ff7d0085230 con 0x7ff7d8150c20 2023-11-28T21:45:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.526+0000 7ff7d6ffd640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7b80664a0 0x7ff7b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:51.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.526+0000 7ff7d6ffd640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7b80664a0 0x7ff7b8068960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80af000 tx=0x7ff7c80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:51.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.530+0000 7ff7cf7fe640 1 -- 172.21.15.12:0/1433005711 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff7d00520b0 con 0x7ff7d8150c20 2023-11-28T21:45:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 -- 172.21.15.12:0/1433005711 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7d8074f60 con 0x7ff7d8150c20 2023-11-28T21:45:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7cf7fe640 1 -- 172.21.15.12:0/1433005711 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5715 (secure 0 0 0) 0x7ff7d0018090 con 0x7ff7d8150c20 2023-11-28T21:45:51.727 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:51.727 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:51.727 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 49 2023-11-28T21:45:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 -- 172.21.15.12:0/1433005711 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7b80664a0 msgr2=0x7ff7b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7b80664a0 0x7ff7b8068960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80af000 tx=0x7ff7c80a7070 comp rx=0 tx=0).stop 2023-11-28T21:45:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 -- 172.21.15.12:0/1433005711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d8150c20 msgr2=0x7ff7d8147bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d8150c20 0x7ff7d8147bc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000efc0 tx=0x7ff7d000c490 comp rx=0 tx=0).stop 2023-11-28T21:45:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 -- 172.21.15.12:0/1433005711 shutdown_connections 2023-11-28T21:45:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7b80664a0 0x7ff7b8068960 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7d8150c20 0x7ff7d8147bc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7d806e300 0x7ff7d80c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 --2- 172.21.15.12:0/1433005711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7d806cf40 0x7ff7d80c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:51.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 -- 172.21.15.12:0/1433005711 >> 172.21.15.12:0/1433005711 conn(0x7ff7d8066360 msgr2=0x7ff7d80c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 -- 172.21.15.12:0/1433005711 shutdown_connections 2023-11-28T21:45:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:51.722+0000 7ff7dd32a640 1 -- 172.21.15.12:0/1433005711 wait complete. 2023-11-28T21:45:51.748 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=5727 2023-11-28T21:45:51.748 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.e 2023-11-28T21:45:54.554 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-11-28T21:45:54.550+0000 7f33e5109640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T21:45:54.717 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 0% (97/12117) Updating files: 1% (122/12117) Updating files: 2% (243/12117) Updating files: 2% (353/12117) Updating files: 3% (364/12117) Updating files: 4% (485/12117) Updating files: 4% (496/12117) Updating files: 5% (606/12117) Updating files: 5% (685/12117) Updating files: 6% (728/12117) Updating files: 6% (834/12117) Updating files: 7% (849/12117) Updating files: 7% (935/12117) Updating files: 8% (970/12117) Updating files: 8% (992/12117) Updating files: 8% (995/12117) Updating files: 9% (1091/12117) Updating files: 9% (1149/12117) Updating files: 10% (1212/12117) Updating files: 10% (1267/12117) Updating files: 10% (1316/12117) Updating files: 11% (1333/12117) Updating files: 11% (1413/12117) Updating files: 12% (1455/12117) Updating files: 12% (1499/12117) Updating files: 13% (1576/12117) Updating files: 14% (1697/12117) Updating files: 14% (1774/12117) Updating files: 15% (1818/12117) Updating files: 16% (1939/12117) Updating files: 17% (2060/12117) Updating files: 17% (2097/12117) Updating files: 17% (2178/12117) Updating files: 18% (2182/12117) Updating files: 18% (2223/12117) Updating files: 19% (2303/12117) Updating files: 19% (2346/12117) Updating files: 20% (2424/12117) Updating files: 21% (2545/12117) Updating files: 21% (2597/12117) Updating files: 22% (2666/12117) Updating files: 22% (2704/12117) Updating files: 23% (2787/12117) Updating files: 23% (2869/12117) Updating files: 24% (2909/12117) Updating files: 25% (3030/12117) Updating files: 26% (3151/12117) Updating files: 26% (3173/12117) Updating files: 27% (3272/12117) Updating files: 28% (3393/12117) Updating files: 28% (3504/12117) Updating files: 29% (3514/12117) Updating files: 30% (3636/12117) Updating files: 30% (3679/12117) Updating files: 30% (3691/12117) Updating files: 30% (3748/12117) Updating files: 31% (3757/12117) Updating files: 32% (3878/12117) Updating files: 32% (3939/12117) Updating files: 33% (3999/12117) Updating files: 34% (4120/12117) Updating files: 34% (4229/12117) Updating files: 35% (4241/12117) Updating files: 35% (4269/12117) Updating files: 35% (4324/12117) Updating files: 35% (4334/12117) Updating files: 36% (4363/12117) Updating files: 37% (4484/12117) Updating files: 37% (4556/12117) Updating files: 38% (4605/12117) Updating files: 38% (4651/12117) Updating files: 39% (4726/12117) Updating files: 39% (4735/12117) Updating files: 39% (4832/12117) Updating files: 40% (4847/12117) Updating files: 40% (4931/12117) Updating files: 41% (4968/12117) Updating files: 41% (4975/12117) Updating files: 42% (5090/12117) Updating files: 42% (5153/12117) Updating files: 43% (5211/12117) Updating files: 43% (5263/12117) Updating files: 44% (5332/12117) Updating files: 44% (5354/12117) Updating files: 44% (5410/12117) Updating files: 44% (5442/12117) Updating files: 45% (5453/12117) Updating files: 45% (5477/12117) Updating files: 46% (5574/12117) Updating files: 47% (5695/12117) Updating files: 47% (5706/12117) Updating files: 47% (5771/12117) Updating files: 48% (5817/12117) Updating files: 49% (5938/12117) Updating files: 49% (5957/12117) Updating files: 50% (6059/12117) Updating files: 51% (6180/12117) Updating files: 51% (6219/12117) Updating files: 51% (6287/12117) Updating files: 52% (6301/12117) Updating files: 52% (6401/12117) Updating files: 53% (6423/12117) Updating files: 53% (6445/12117) Updating files: 53% (6500/12117) Updating files: 54% (6544/12117) Updating files: 54% (6574/12117) Updating files: 55% (6665/12117) Updating files: 55% (6709/12117) Updating files: 56% (6786/12117) Updating files: 56% (6792/12117) Updating files: 56% (6804/12117) Updating files: 56% (6806/12117) Updating files: 57% (6907/12117) Updating files: 57% (6924/12117) Updating files: 57% (6963/12117) Updating files: 58% (7028/12117) Updating files: 59% (7150/12117) Updating files: 60% (7271/12117) Updating files: 61% (7392/12117) Updating files: 61% (7433/12117) Updating files: 62% (7513/12117) Updating files: 63% (7634/12117) Updating files: 63% (7743/12117) Updating files: 64% (7755/12117) Updating files: 64% (7861/12117) Updating files: 65% (7877/12117) Updating files: 65% (7910/12117) Updating files: 66% (7998/12117) Updating files: 66% (8036/12117) Updating files: 66% (8052/12117) Updating files: 67% (8119/12117) Updating files: 68% (8240/12117) Updating files: 69% (8361/12117) Updating files: 69% (8381/12117) Updating files: 70% (8482/12117) Updating files: 70% (8521/12117) Updating files: 71% (8604/12117) Updating files: 71% (8679/12117) Updating files: 72% (8725/12117) Updating files: 72% (8781/12117) Updating files: 72% (8785/12117) Updating files: 72% (8832/12117) Updating files: 73% (8846/12117) Updating files: 73% (8963/12117) Updating files: 74% (8967/12117) Updating files: 74% (8973/12117) Updating files: 74% (9030/12117) Updating files: 74% (9046/12117) Updating files: 75% (9088/12117) Updating files: 75% (9178/12117) Updating files: 76% (9209/12117) Updating files: 77% (9331/12117) Updating files: 78% (9452/12117) Updating files: 78% (9482/12117) Updating files: 79% (9573/12117) Updating files: 79% (9657/12117) Updating files: 80% (9694/12117) Updating files: 80% (9779/12117) Updating files: 80% (9805/12117) Updating files: 81% (9815/12117) Updating files: 81% (9829/12117) error: unable to create file src/python-common/ceph/deployment/__init__.py: File exists 2023-11-28T21:45:55.748 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-11-28T21:45:55.749 INFO:tasks.ceph.mds.e:Restarting daemon 2023-11-28T21:45:55.749 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-11-28T21:45:55.753 INFO:tasks.ceph.mds.e:Started 2023-11-28T21:45:55.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:55.843 INFO:tasks.ceph.mds.e.smithi012.stdout:starting mds.e at 2023-11-28T21:45:55.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5ce45a640 1 Processor -- start 2023-11-28T21:45:55.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5ce45a640 1 -- start start 2023-11-28T21:45:55.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5ce45a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c8151af0 0x7fb5c80c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:55.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5ce45a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb5c8151ed0 0x7fb5c80c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:55.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5ce45a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb5c806e300 0x7fb5c80c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:55.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5ce45a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80707f0 con 0x7fb5c806e300 2023-11-28T21:45:55.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5ce45a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80c7b90 con 0x7fb5c8151ed0 2023-11-28T21:45:55.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5ce45a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8071c20 con 0x7fb5c8151af0 2023-11-28T21:45:55.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5cc9f7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb5c806e300 0x7fb5c80c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:55.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5cc9f7640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb5c806e300 0x7fb5c80c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41064/0 (socket says 172.21.15.12:41064) 2023-11-28T21:45:55.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5cc9f7640 1 -- 172.21.15.12:0/154434271 learned_addr learned my addr 172.21.15.12:0/154434271 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:55.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5c7fff640 1 --2- 172.21.15.12:0/154434271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c8151af0 0x7fb5c80c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:55.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5c77fe640 1 --2- 172.21.15.12:0/154434271 >> v2:172.21.15.144:3300/0 conn(0x7fb5c8151ed0 0x7fb5c80c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:55.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5cc9f7640 1 -- 172.21.15.12:0/154434271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c8151af0 msgr2=0x7fb5c80c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:55.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5cc9f7640 1 --2- 172.21.15.12:0/154434271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c8151af0 0x7fb5c80c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:55.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5cc9f7640 1 -- 172.21.15.12:0/154434271 >> v2:172.21.15.144:3300/0 conn(0x7fb5c8151ed0 msgr2=0x7fb5c80c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:55.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5cc9f7640 1 --2- 172.21.15.12:0/154434271 >> v2:172.21.15.144:3300/0 conn(0x7fb5c8151ed0 0x7fb5c80c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:55.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5cc9f7640 1 -- 172.21.15.12:0/154434271 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c8157b70 con 0x7fb5c806e300 2023-11-28T21:45:55.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.894+0000 7fb5c77fe640 1 --2- 172.21.15.12:0/154434271 >> v2:172.21.15.144:3300/0 conn(0x7fb5c8151ed0 0x7fb5c80c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:45:55.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5cc9f7640 1 --2- 172.21.15.12:0/154434271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c806e300 0x7fb5c80c4f10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb5c806c7c0 tx=0x7fb5c001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a431ef4983e47d5c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5c6ffd640 1 -- 172.21.15.12:0/154434271 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a5740 con 0x7fb5c806e300 2023-11-28T21:45:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5c6ffd640 1 -- 172.21.15.12:0/154434271 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c00a58a0 con 0x7fb5c806e300 2023-11-28T21:45:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5c6ffd640 1 -- 172.21.15.12:0/154434271 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a5c30 con 0x7fb5c806e300 2023-11-28T21:45:55.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5ce45a640 1 -- 172.21.15.12:0/154434271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c806e300 msgr2=0x7fb5c80c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:55.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5ce45a640 1 --2- 172.21.15.12:0/154434271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c806e300 0x7fb5c80c4f10 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb5c806c7c0 tx=0x7fb5c001c9d0 comp rx=0 tx=0).stop 2023-11-28T21:45:55.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5ce45a640 1 -- 172.21.15.12:0/154434271 shutdown_connections 2023-11-28T21:45:55.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5ce45a640 1 --2- 172.21.15.12:0/154434271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c806e300 0x7fb5c80c4f10 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:55.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5ce45a640 1 --2- 172.21.15.12:0/154434271 >> v2:172.21.15.144:3300/0 conn(0x7fb5c8151ed0 0x7fb5c80c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:55.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5ce45a640 1 --2- 172.21.15.12:0/154434271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c8151af0 0x7fb5c80c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:55.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5ce45a640 1 -- 172.21.15.12:0/154434271 >> 172.21.15.12:0/154434271 conn(0x7fb5c8066360 msgr2=0x7fb5c8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:55.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5ce45a640 1 -- 172.21.15.12:0/154434271 shutdown_connections 2023-11-28T21:45:55.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.898+0000 7fb5ce45a640 1 -- 172.21.15.12:0/154434271 wait complete. 2023-11-28T21:45:55.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 Processor -- start 2023-11-28T21:45:55.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 -- start start 2023-11-28T21:45:55.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c806e300 0x7fb5c806d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c8151af0 0x7fb5c80ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c814c7b0 0x7fb5c80c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80c9150 con 0x7fb5c8151af0 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c81f09a0 con 0x7fb5c806e300 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c81f0c60 con 0x7fb5c814c7b0 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c8151af0 0x7fb5c80ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c8151af0 0x7fb5c80ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41066/0 (socket says 172.21.15.12:41066) 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c77fe640 1 -- 172.21.15.12:0/1161481256 learned_addr learned my addr 172.21.15.12:0/1161481256 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5cc9f7640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c814c7b0 0x7fb5c80c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c7fff640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c806e300 0x7fb5c806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:55.909 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c77fe640 1 -- 172.21.15.12:0/1161481256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c814c7b0 msgr2=0x7fb5c80c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c77fe640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c814c7b0 0x7fb5c80c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c77fe640 1 -- 172.21.15.12:0/1161481256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c806e300 msgr2=0x7fb5c806d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c77fe640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c806e300 0x7fb5c806d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c77fe640 1 -- 172.21.15.12:0/1161481256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0009b30 con 0x7fb5c8151af0 2023-11-28T21:45:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c77fe640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c8151af0 0x7fb5c80ca9b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00d7d0 tx=0x7fb5bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c4ff9640 1 -- 172.21.15.12:0/1161481256 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0096a0 con 0x7fb5c8151af0 2023-11-28T21:45:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c4ff9640 1 -- 172.21.15.12:0/1161481256 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc009800 con 0x7fb5c8151af0 2023-11-28T21:45:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5c4ff9640 1 -- 172.21.15.12:0/1161481256 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc009bb0 con 0x7fb5c8151af0 2023-11-28T21:45:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 -- 172.21.15.12:0/1161481256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c81f0f40 con 0x7fb5c8151af0 2023-11-28T21:45:55.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 -- 172.21.15.12:0/1161481256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c81f1500 con 0x7fb5c8151af0 2023-11-28T21:45:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.902+0000 7fb5ce45a640 1 -- 172.21.15.12:0/1161481256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c8074f60 con 0x7fb5c8151af0 2023-11-28T21:45:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.906+0000 7fb5c4ff9640 1 -- 172.21.15.12:0/1161481256 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb5bc009d10 con 0x7fb5c8151af0 2023-11-28T21:45:55.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.906+0000 7fb5c4ff9640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5b40664a0 0x7fb5b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:55.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.906+0000 7fb5c4ff9640 1 -- 172.21.15.12:0/1161481256 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fb5bc084b40 con 0x7fb5c8151af0 2023-11-28T21:45:55.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.906+0000 7fb5c7fff640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5b40664a0 0x7fb5b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:55.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.906+0000 7fb5c7fff640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5b40664a0 0x7fb5b4068960 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009a70 tx=0x7fb5b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:55.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:55.910+0000 7fb5c4ff9640 1 -- 172.21.15.12:0/1161481256 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb5bc051920 con 0x7fb5c8151af0 2023-11-28T21:45:56.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.102+0000 7fb5ce45a640 1 -- 172.21.15.12:0/1161481256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5c8072c10 con 0x7fb5c8151af0 2023-11-28T21:45:56.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.102+0000 7fb5c4ff9640 1 -- 172.21.15.12:0/1161481256 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fb5bc054f40 con 0x7fb5c8151af0 2023-11-28T21:45:56.108 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:56.108 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:56.108 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 49 2023-11-28T21:45:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.102+0000 7fb5ce45a640 1 -- 172.21.15.12:0/1161481256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5b40664a0 msgr2=0x7fb5b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.102+0000 7fb5ce45a640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5b40664a0 0x7fb5b4068960 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009a70 tx=0x7fb5b8008040 comp rx=0 tx=0).stop 2023-11-28T21:45:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.102+0000 7fb5ce45a640 1 -- 172.21.15.12:0/1161481256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c8151af0 msgr2=0x7fb5c80ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.102+0000 7fb5ce45a640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c8151af0 0x7fb5c80ca9b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00d7d0 tx=0x7fb5bc00dc90 comp rx=0 tx=0).stop 2023-11-28T21:45:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.106+0000 7fb5ce45a640 1 -- 172.21.15.12:0/1161481256 shutdown_connections 2023-11-28T21:45:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.106+0000 7fb5ce45a640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb5b40664a0 0x7fb5b4068960 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:56.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.106+0000 7fb5ce45a640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5c814c7b0 0x7fb5c80c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:56.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.106+0000 7fb5ce45a640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c8151af0 0x7fb5c80ca9b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:56.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.106+0000 7fb5ce45a640 1 --2- 172.21.15.12:0/1161481256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5c806e300 0x7fb5c806d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:56.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.106+0000 7fb5ce45a640 1 -- 172.21.15.12:0/1161481256 >> 172.21.15.12:0/1161481256 conn(0x7fb5c8066360 msgr2=0x7fb5c8150900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:56.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.106+0000 7fb5ce45a640 1 -- 172.21.15.12:0/1161481256 shutdown_connections 2023-11-28T21:45:56.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:56.106+0000 7fb5ce45a640 1 -- 172.21.15.12:0/1161481256 wait complete. 2023-11-28T21:45:56.129 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-11-28T21:45:58.130 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:45:58.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe2d69c640 1 Processor -- start 2023-11-28T21:45:58.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe2d69c640 1 -- start start 2023-11-28T21:45:58.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe2d69c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe280cba30 0x7efe2806ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:58.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe2d69c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efe2806b1b0 0x7efe280caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:58.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe2d69c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe2815ab40 con 0x7efe28153e00 2023-11-28T21:45:58.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe2d69c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe2806a850 con 0x7efe2806b1b0 2023-11-28T21:45:58.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe2d69c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe280b2670 con 0x7efe280cba30 2023-11-28T21:45:58.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe26ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efe28153e00 0x7efe280ca3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50712/0 (socket says 172.21.15.12:50712) 2023-11-28T21:45:58.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe26ffd640 1 -- 172.21.15.12:0/3129288666 learned_addr learned my addr 172.21.15.12:0/3129288666 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:45:58.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe277fe640 1 --2- 172.21.15.12:0/3129288666 >> v2:172.21.15.12:3300/0 conn(0x7efe2806b1b0 0x7efe280caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:58.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe267fc640 1 --2- 172.21.15.12:0/3129288666 >> v2:172.21.15.144:3300/0 conn(0x7efe280cba30 0x7efe2806ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:58.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe277fe640 1 -- 172.21.15.12:0/3129288666 >> v2:172.21.15.144:3300/0 conn(0x7efe280cba30 msgr2=0x7efe2806ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:45:58.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe277fe640 1 --2- 172.21.15.12:0/3129288666 >> v2:172.21.15.144:3300/0 conn(0x7efe280cba30 0x7efe2806ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:58.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe277fe640 1 -- 172.21.15.12:0/3129288666 >> v1:172.21.15.12:6789/0 conn(0x7efe28153e00 legacy=0x7efe280ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:58.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe277fe640 1 -- 172.21.15.12:0/3129288666 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe28159fe0 con 0x7efe2806b1b0 2023-11-28T21:45:58.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.282+0000 7efe25ffb640 1 -- 172.21.15.12:0/3129288666 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976839980 0 0) 0x7efe2815ab40 con 0x7efe28153e00 2023-11-28T21:45:58.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe277fe640 1 --2- 172.21.15.12:0/3129288666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 0x7efe280caec0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efe2806ca30 tx=0x7efe2001f100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad47d645596c3f86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:58.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe25ffb640 1 -- 172.21.15.12:0/3129288666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe200af600 con 0x7efe2806b1b0 2023-11-28T21:45:58.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe25ffb640 1 -- 172.21.15.12:0/3129288666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe200ab050 con 0x7efe2806b1b0 2023-11-28T21:45:58.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe25ffb640 1 -- 172.21.15.12:0/3129288666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe200af980 con 0x7efe2806b1b0 2023-11-28T21:45:58.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe2d69c640 1 -- 172.21.15.12:0/3129288666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 msgr2=0x7efe280caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:58.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe2d69c640 1 --2- 172.21.15.12:0/3129288666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 0x7efe280caec0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efe2806ca30 tx=0x7efe2001f100 comp rx=0 tx=0).stop 2023-11-28T21:45:58.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe2d69c640 1 -- 172.21.15.12:0/3129288666 shutdown_connections 2023-11-28T21:45:58.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe2d69c640 1 --2- 172.21.15.12:0/3129288666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 0x7efe280caec0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:58.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe2d69c640 1 --2- 172.21.15.12:0/3129288666 >> v2:172.21.15.144:3300/0 conn(0x7efe280cba30 0x7efe2806ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:58.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe2d69c640 1 -- 172.21.15.12:0/3129288666 >> 172.21.15.12:0/3129288666 conn(0x7efe280cf4e0 msgr2=0x7efe280cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:58.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe2d69c640 1 -- 172.21.15.12:0/3129288666 shutdown_connections 2023-11-28T21:45:58.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.286+0000 7efe2d69c640 1 -- 172.21.15.12:0/3129288666 wait complete. 2023-11-28T21:45:58.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 Processor -- start 2023-11-28T21:45:58.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 -- start start 2023-11-28T21:45:58.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 0x7efe280ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:58.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe280cba30 0x7efe2815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:58.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe28153e00 0x7efe28143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:58.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe281f3bb0 con 0x7efe2806b1b0 2023-11-28T21:45:58.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe281f2fa0 con 0x7efe280cba30 2023-11-28T21:45:58.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe281f3280 con 0x7efe28153e00 2023-11-28T21:45:58.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe26ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 0x7efe280ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:58.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe26ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 0x7efe280ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41086/0 (socket says 172.21.15.12:41086) 2023-11-28T21:45:58.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe26ffd640 1 -- 172.21.15.12:0/551528496 learned_addr learned my addr 172.21.15.12:0/551528496 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:45:58.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe267fc640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe280cba30 0x7efe2815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:58.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe277fe640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe28153e00 0x7efe28143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:58.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe26ffd640 1 -- 172.21.15.12:0/551528496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe28153e00 msgr2=0x7efe28143f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:58.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe26ffd640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe28153e00 0x7efe28143f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:58.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe26ffd640 1 -- 172.21.15.12:0/551528496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe280cba30 msgr2=0x7efe2815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:58.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe26ffd640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe280cba30 0x7efe2815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:58.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe26ffd640 1 -- 172.21.15.12:0/551528496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe20009a70 con 0x7efe2806b1b0 2023-11-28T21:45:58.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe26ffd640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 0x7efe280ce6a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efe1800c480 tx=0x7efe1800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:58.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe0f7fe640 1 -- 172.21.15.12:0/551528496 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe180074e0 con 0x7efe2806b1b0 2023-11-28T21:45:58.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe0f7fe640 1 -- 172.21.15.12:0/551528496 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe18007660 con 0x7efe2806b1b0 2023-11-28T21:45:58.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 -- 172.21.15.12:0/551528496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe281f3560 con 0x7efe2806b1b0 2023-11-28T21:45:58.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe0f7fe640 1 -- 172.21.15.12:0/551528496 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18004a30 con 0x7efe2806b1b0 2023-11-28T21:45:58.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 -- 172.21.15.12:0/551528496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2815af40 con 0x7efe2806b1b0 2023-11-28T21:45:58.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.290+0000 7efe2d69c640 1 -- 172.21.15.12:0/551528496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe28074f60 con 0x7efe2806b1b0 2023-11-28T21:45:58.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.294+0000 7efe0f7fe640 1 -- 172.21.15.12:0/551528496 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7efe18004b90 con 0x7efe2806b1b0 2023-11-28T21:45:58.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.294+0000 7efe0f7fe640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efe14066470 0x7efe14068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:45:58.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.294+0000 7efe0f7fe640 1 -- 172.21.15.12:0/551528496 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7efe18083930 con 0x7efe2806b1b0 2023-11-28T21:45:58.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.294+0000 7efe267fc640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efe14066470 0x7efe14068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:45:58.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.294+0000 7efe267fc640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efe14066470 0x7efe14068930 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efe280b55b0 tx=0x7efe1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:45:58.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.298+0000 7efe0f7fe640 1 -- 172.21.15.12:0/551528496 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efe180542c0 con 0x7efe2806b1b0 2023-11-28T21:45:58.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.486+0000 7efe2d69c640 1 -- 172.21.15.12:0/551528496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe2806c510 con 0x7efe2806b1b0 2023-11-28T21:45:58.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.486+0000 7efe0f7fe640 1 -- 172.21.15.12:0/551528496 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6506 (secure 0 0 0) 0x7efe18018090 con 0x7efe2806b1b0 2023-11-28T21:45:58.493 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:45:58.493 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6024,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:45:58.493 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-11-28T21:45:58.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 -- 172.21.15.12:0/551528496 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efe14066470 msgr2=0x7efe14068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:58.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efe14066470 0x7efe14068930 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efe280b55b0 tx=0x7efe1c008040 comp rx=0 tx=0).stop 2023-11-28T21:45:58.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 -- 172.21.15.12:0/551528496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 msgr2=0x7efe280ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:45:58.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 0x7efe280ce6a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efe1800c480 tx=0x7efe1800c940 comp rx=0 tx=0).stop 2023-11-28T21:45:58.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 -- 172.21.15.12:0/551528496 shutdown_connections 2023-11-28T21:45:58.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efe14066470 0x7efe14068930 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:58.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe28153e00 0x7efe28143f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:58.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe280cba30 0x7efe2815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:58.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 --2- 172.21.15.12:0/551528496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe2806b1b0 0x7efe280ce6a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:45:58.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 -- 172.21.15.12:0/551528496 >> 172.21.15.12:0/551528496 conn(0x7efe280cf4e0 msgr2=0x7efe2806ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:45:58.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 -- 172.21.15.12:0/551528496 shutdown_connections 2023-11-28T21:45:58.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:45:58.490+0000 7efe2d69c640 1 -- 172.21.15.12:0/551528496 wait complete. 2023-11-28T21:45:58.514 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-11-28T21:45:58.514 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 31 secs before thrashing 2023-11-28T21:46:14.765 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 81% (9872/12117) Updating files: 82% (9936/12117) Updating files: 82% (9952/12117) Updating files: 83% (10058/12117) Updating files: 83% (10088/12117) Updating files: 84% (10179/12117) Updating files: 85% (10300/12117) Updating files: 85% (10385/12117) Updating files: 86% (10421/12117) Updating files: 86% (10487/12117) Updating files: 87% (10542/12117) Updating files: 88% (10663/12117) Updating files: 88% (10769/12117) Updating files: 89% (10785/12117) Updating files: 90% (10906/12117) Updating files: 90% (10961/12117) Updating files: 90% (11023/12117) Updating files: 91% (11027/12117) Updating files: 91% (11104/12117) Updating files: 92% (11148/12117) Updating files: 93% (11269/12117) Updating files: 93% (11370/12117) Updating files: 94% (11390/12117) Updating files: 94% (11465/12117) Updating files: 95% (11512/12117) Updating files: 96% (11633/12117) Updating files: 96% (11649/12117) Updating files: 96% (11675/12117) Updating files: 96% (11746/12117) Updating files: 97% (11754/12117) Updating files: 97% (11812/12117) Updating files: 98% (11875/12117) Updating files: 98% (11878/12117) Updating files: 98% (11983/12117) Updating files: 99% (11996/12117) Updating files: 100% (12117/12117) Updating files: 100% (12117/12117), done. 2023-11-28T21:46:14.766 INFO:tasks.workunit.client.0.smithi012.stderr:fatal: unable to checkout working tree 2023-11-28T21:46:14.766 INFO:tasks.workunit.client.0.smithi012.stderr:warning: Clone succeeded, but checkout failed. 2023-11-28T21:46:14.767 INFO:tasks.workunit.client.0.smithi012.stderr:You can inspect what was checked out with 'git status' 2023-11-28T21:46:14.767 INFO:tasks.workunit.client.0.smithi012.stderr:and retry with 'git restore --source=HEAD :/' 2023-11-28T21:46:14.767 INFO:tasks.workunit.client.0.smithi012.stderr: 2023-11-28T21:46:14.782 INFO:tasks.workunit.client.0.smithi012.stderr:+ retry 2023-11-28T21:46:14.782 INFO:tasks.workunit.client.0.smithi012.stderr:+ rm -rf ceph 2023-11-28T21:46:15.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:46:15.372 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:46:29.516 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:29.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.697+0000 7fb6ae2bb640 1 Processor -- start 2023-11-28T21:46:29.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.697+0000 7fb6ae2bb640 1 -- start start 2023-11-28T21:46:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6ae2bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 0x7fb6a80c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6ae2bb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb6a8159d20 0x7fb6a815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6ae2bb640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a80c1660 con 0x7fb6a80c6580 2023-11-28T21:46:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6ae2bb640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a80c93b0 con 0x7fb6a8159d20 2023-11-28T21:46:29.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6ae2bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a815a6f0 con 0x7fb6a80c8a90 2023-11-28T21:46:29.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6ad2b9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 0x7fb6a80c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:29.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6adaba640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb6a8159d20 0x7fb6a815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:29.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6acab8640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb6a80c6580 0x7fb6a80c7a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:47790/0 (socket says 172.21.15.12:47790) 2023-11-28T21:46:29.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6acab8640 1 -- 172.21.15.12:0/434641532 learned_addr learned my addr 172.21.15.12:0/434641532 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:46:29.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb69ffff640 1 -- 172.21.15.12:0/434641532 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912944429 0 0) 0x7fb6a80c1660 con 0x7fb6a80c6580 2023-11-28T21:46:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb69ffff640 1 -- 172.21.15.12:0/434641532 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb694003530 con 0x7fb6a80c6580 2023-11-28T21:46:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6ad2b9640 1 -- 172.21.15.12:0/434641532 >> v2:172.21.15.144:3300/0 conn(0x7fb6a8159d20 msgr2=0x7fb6a815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6ad2b9640 1 --2- 172.21.15.12:0/434641532 >> v2:172.21.15.144:3300/0 conn(0x7fb6a8159d20 0x7fb6a815a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6ad2b9640 1 -- 172.21.15.12:0/434641532 >> v1:172.21.15.144:6789/0 conn(0x7fb6a80c6580 legacy=0x7fb6a80c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:29.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.701+0000 7fb6ad2b9640 1 -- 172.21.15.12:0/434641532 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80c82b0 con 0x7fb6a80c8a90 2023-11-28T21:46:29.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.705+0000 7fb6ad2b9640 1 --2- 172.21.15.12:0/434641532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 0x7fb6a80c8e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40097c0 tx=0x7fb6a401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=641586ba722c1051 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:29.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.705+0000 7fb69ffff640 1 -- 172.21.15.12:0/434641532 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a40ad640 con 0x7fb6a80c8a90 2023-11-28T21:46:29.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.705+0000 7fb69ffff640 1 -- 172.21.15.12:0/434641532 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a40ad7a0 con 0x7fb6a80c8a90 2023-11-28T21:46:29.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.705+0000 7fb69ffff640 1 -- 172.21.15.12:0/434641532 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a40adb30 con 0x7fb6a80c8a90 2023-11-28T21:46:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/434641532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 msgr2=0x7fb6a80c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 --2- 172.21.15.12:0/434641532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 0x7fb6a80c8e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6a40097c0 tx=0x7fb6a401c480 comp rx=0 tx=0).stop 2023-11-28T21:46:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/434641532 shutdown_connections 2023-11-28T21:46:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 --2- 172.21.15.12:0/434641532 >> v2:172.21.15.144:3300/0 conn(0x7fb6a8159d20 0x7fb6a815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:29.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 --2- 172.21.15.12:0/434641532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 0x7fb6a80c8e70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:29.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/434641532 >> 172.21.15.12:0/434641532 conn(0x7fb6a806e220 msgr2=0x7fb6a806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:29.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/434641532 shutdown_connections 2023-11-28T21:46:29.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/434641532 wait complete. 2023-11-28T21:46:29.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 Processor -- start 2023-11-28T21:46:29.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 -- start start 2023-11-28T21:46:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a80c6580 0x7fb6a814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 0x7fb6a80c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6a8159d20 0x7fb6a814b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a80c53d0 con 0x7fb6a80c6580 2023-11-28T21:46:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a81ec660 con 0x7fb6a8159d20 2023-11-28T21:46:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ae2bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a81ecaf0 con 0x7fb6a80c8a90 2023-11-28T21:46:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ad2b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a80c6580 0x7fb6a814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ad2b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a80c6580 0x7fb6a814ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32796/0 (socket says 172.21.15.12:32796) 2023-11-28T21:46:29.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6ad2b9640 1 -- 172.21.15.12:0/103457680 learned_addr learned my addr 172.21.15.12:0/103457680 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.709+0000 7fb6acab8640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 0x7fb6a80c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6adaba640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6a8159d20 0x7fb6a814b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6ad2b9640 1 -- 172.21.15.12:0/103457680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 msgr2=0x7fb6a80c2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6ad2b9640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 0x7fb6a80c2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6ad2b9640 1 -- 172.21.15.12:0/103457680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6a8159d20 msgr2=0x7fb6a814b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6ad2b9640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6a8159d20 0x7fb6a814b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6ad2b9640 1 -- 172.21.15.12:0/103457680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a4009420 con 0x7fb6a80c6580 2023-11-28T21:46:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6acab8640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 0x7fb6a80c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:46:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6ad2b9640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a80c6580 0x7fb6a814ea30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb6a401a0b0 tx=0x7fb6a4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb69dffb640 1 -- 172.21.15.12:0/103457680 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a40b6610 con 0x7fb6a80c6580 2023-11-28T21:46:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb69dffb640 1 -- 172.21.15.12:0/103457680 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a40a6040 con 0x7fb6a80c6580 2023-11-28T21:46:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb69dffb640 1 -- 172.21.15.12:0/103457680 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a40b69b0 con 0x7fb6a80c6580 2023-11-28T21:46:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/103457680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a81eba50 con 0x7fb6a80c6580 2023-11-28T21:46:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/103457680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a81ec010 con 0x7fb6a80c6580 2023-11-28T21:46:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.713+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/103457680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a8074f60 con 0x7fb6a80c6580 2023-11-28T21:46:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.717+0000 7fb69dffb640 1 -- 172.21.15.12:0/103457680 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb6a40b4040 con 0x7fb6a80c6580 2023-11-28T21:46:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.717+0000 7fb69dffb640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb68c066470 0x7fb68c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.717+0000 7fb69dffb640 1 -- 172.21.15.12:0/103457680 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fb6a411e3c0 con 0x7fb6a80c6580 2023-11-28T21:46:29.723 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.717+0000 7fb6acab8640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb68c066470 0x7fb68c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:29.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.717+0000 7fb6acab8640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb68c066470 0x7fb68c068930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8150720 tx=0x7fb698009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:29.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:29.725+0000 7fb69dffb640 1 -- 172.21.15.12:0/103457680 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb6a40eb230 con 0x7fb6a80c6580 2023-11-28T21:46:30.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.057+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/103457680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6a80cc2e0 con 0x7fb6a80c6580 2023-11-28T21:46:30.066 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:30.066 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6024,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:30.066 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-11-28T21:46:30.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.057+0000 7fb69dffb640 1 -- 172.21.15.12:0/103457680 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fb6a40ee850 con 0x7fb6a80c6580 2023-11-28T21:46:30.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/103457680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb68c066470 msgr2=0x7fb68c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:30.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb68c066470 0x7fb68c068930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8150720 tx=0x7fb698009040 comp rx=0 tx=0).stop 2023-11-28T21:46:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/103457680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a80c6580 msgr2=0x7fb6a814ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:30.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a80c6580 0x7fb6a814ea30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb6a401a0b0 tx=0x7fb6a4019220 comp rx=0 tx=0).stop 2023-11-28T21:46:30.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/103457680 shutdown_connections 2023-11-28T21:46:30.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb68c066470 0x7fb68c068930 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6a8159d20 0x7fb6a814b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6a80c8a90 0x7fb6a80c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 --2- 172.21.15.12:0/103457680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6a80c6580 0x7fb6a814ea30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.073 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/103457680 >> 172.21.15.12:0/103457680 conn(0x7fb6a806e220 msgr2=0x7fb6a806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/103457680 shutdown_connections 2023-11-28T21:46:30.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.065+0000 7fb6ae2bb640 1 -- 172.21.15.12:0/103457680 wait complete. 2023-11-28T21:46:30.118 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2023-11-28T21:46:30.118 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T21:46:30.119 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:46:30.148 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T21:46:30.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:30.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.325+0000 7f8d01d57640 1 Processor -- start 2023-11-28T21:46:30.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.325+0000 7f8d01d57640 1 -- start start 2023-11-28T21:46:30.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8d01d57640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8cfc06e300 0x7f8cfc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:30.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8d01d57640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc157b70 con 0x7f8cfc151ed0 2023-11-28T21:46:30.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8d01d57640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc157e50 con 0x7f8cfc151af0 2023-11-28T21:46:30.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8d01d57640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc0707f0 con 0x7f8cfc06e300 2023-11-28T21:46:30.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfbfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8cfc06e300 0x7f8cfc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:30.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfaffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8cfc151ed0 0x7f8cfc072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39878/0 (socket says 172.21.15.12:39878) 2023-11-28T21:46:30.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfaffd640 1 -- 172.21.15.12:0/2309091259 learned_addr learned my addr 172.21.15.12:0/2309091259 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:46:30.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfbfff640 1 -- 172.21.15.12:0/2309091259 >> v1:172.21.15.144:6789/0 conn(0x7f8cfc151af0 legacy=0x7f8cfc0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:30.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfbfff640 1 -- 172.21.15.12:0/2309091259 >> v1:172.21.15.12:6789/0 conn(0x7f8cfc151ed0 legacy=0x7f8cfc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:30.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfa7fc640 1 -- 172.21.15.12:0/2309091259 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077349433 0 0) 0x7f8cfc157b70 con 0x7f8cfc151ed0 2023-11-28T21:46:30.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfbfff640 1 -- 172.21.15.12:0/2309091259 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc158130 con 0x7f8cfc06e300 2023-11-28T21:46:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfbfff640 1 --2- 172.21.15.12:0/2309091259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc06e300 0x7f8cfc0c7650 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009d90 tx=0x7f8cf401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2031cc2749dd9c98 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:30.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfa7fc640 1 -- 172.21.15.12:0/2309091259 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40a5810 con 0x7f8cfc06e300 2023-11-28T21:46:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfa7fc640 1 -- 172.21.15.12:0/2309091259 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf40a5970 con 0x7f8cfc06e300 2023-11-28T21:46:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.329+0000 7f8cfa7fc640 1 -- 172.21.15.12:0/2309091259 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40a5d00 con 0x7f8cfc06e300 2023-11-28T21:46:30.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 -- 172.21.15.12:0/2309091259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc06e300 msgr2=0x7f8cfc0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 --2- 172.21.15.12:0/2309091259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc06e300 0x7f8cfc0c7650 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009d90 tx=0x7f8cf401c950 comp rx=0 tx=0).stop 2023-11-28T21:46:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 -- 172.21.15.12:0/2309091259 shutdown_connections 2023-11-28T21:46:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 --2- 172.21.15.12:0/2309091259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc06e300 0x7f8cfc0c7650 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 -- 172.21.15.12:0/2309091259 >> 172.21.15.12:0/2309091259 conn(0x7f8cfc066360 msgr2=0x7f8cfc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 -- 172.21.15.12:0/2309091259 shutdown_connections 2023-11-28T21:46:30.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 -- 172.21.15.12:0/2309091259 wait complete. 2023-11-28T21:46:30.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 Processor -- start 2023-11-28T21:46:30.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 -- start start 2023-11-28T21:46:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc06e300 0x7f8cfc155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc151af0 0x7f8cfc0c1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc162f40 0x7f8cfc156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c3520 con 0x7f8cfc151af0 2023-11-28T21:46:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c3c30 con 0x7f8cfc06e300 2023-11-28T21:46:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8d01d57640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c3ed0 con 0x7f8cfc162f40 2023-11-28T21:46:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8cfaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc151af0 0x7f8cfc0c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8cfaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc151af0 0x7f8cfc0c1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32808/0 (socket says 172.21.15.12:32808) 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8cfaffd640 1 -- 172.21.15.12:0/3007228915 learned_addr learned my addr 172.21.15.12:0/3007228915 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8cfb7fe640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc06e300 0x7f8cfc155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.333+0000 7f8cfbfff640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc162f40 0x7f8cfc156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8cfaffd640 1 -- 172.21.15.12:0/3007228915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc162f40 msgr2=0x7f8cfc156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8cfaffd640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc162f40 0x7f8cfc156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8cfaffd640 1 -- 172.21.15.12:0/3007228915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc06e300 msgr2=0x7f8cfc155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8cfaffd640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc06e300 0x7f8cfc155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8cfaffd640 1 -- 172.21.15.12:0/3007228915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf4009a70 con 0x7f8cfc151af0 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8cfbfff640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc162f40 0x7f8cfc156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8cfb7fe640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc06e300 0x7f8cfc155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:46:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8cfaffd640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc151af0 0x7f8cfc0c1990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000c3e0 tx=0x7f8cf000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8ce3fff640 1 -- 172.21.15.12:0/3007228915 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf00074e0 con 0x7f8cfc151af0 2023-11-28T21:46:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8ce3fff640 1 -- 172.21.15.12:0/3007228915 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf0007640 con 0x7f8cfc151af0 2023-11-28T21:46:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8ce3fff640 1 -- 172.21.15.12:0/3007228915 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0004a30 con 0x7f8cfc151af0 2023-11-28T21:46:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8d01d57640 1 -- 172.21.15.12:0/3007228915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc0c40f0 con 0x7f8cfc151af0 2023-11-28T21:46:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8d01d57640 1 -- 172.21.15.12:0/3007228915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc0c3730 con 0x7f8cfc151af0 2023-11-28T21:46:30.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8d01d57640 1 -- 172.21.15.12:0/3007228915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc0c3050 con 0x7f8cfc151af0 2023-11-28T21:46:30.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8ce3fff640 1 -- 172.21.15.12:0/3007228915 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8cf0004b90 con 0x7f8cfc151af0 2023-11-28T21:46:30.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8ce3fff640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8ce8066470 0x7f8ce8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:30.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8ce3fff640 1 -- 172.21.15.12:0/3007228915 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f8cf0084250 con 0x7f8cfc151af0 2023-11-28T21:46:30.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.337+0000 7f8cfb7fe640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8ce8066470 0x7f8ce8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:30.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.341+0000 7f8cfb7fe640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8ce8066470 0x7f8ce8068930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000f70 tx=0x7f8cec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:30.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.341+0000 7f8ce3fff640 1 -- 172.21.15.12:0/3007228915 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8cf00510c0 con 0x7f8cfc151af0 2023-11-28T21:46:30.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 -- 172.21.15.12:0/3007228915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cfc074f60 con 0x7f8cfc151af0 2023-11-28T21:46:30.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8ce3fff640 1 -- 172.21.15.12:0/3007228915 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f8cf00546e0 con 0x7f8cfc151af0 2023-11-28T21:46:30.540 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:30.540 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6024,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-11-28T21:46:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 -- 172.21.15.12:0/3007228915 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8ce8066470 msgr2=0x7f8ce8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8ce8066470 0x7f8ce8068930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000f70 tx=0x7f8cec009040 comp rx=0 tx=0).stop 2023-11-28T21:46:30.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 -- 172.21.15.12:0/3007228915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc151af0 msgr2=0x7f8cfc0c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc151af0 0x7f8cfc0c1990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000c3e0 tx=0x7f8cf000c8a0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 -- 172.21.15.12:0/3007228915 shutdown_connections 2023-11-28T21:46:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8ce8066470 0x7f8ce8068930 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.543 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8cfc162f40 0x7f8cfc156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8cfc151af0 0x7f8cfc0c1990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 --2- 172.21.15.12:0/3007228915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8cfc06e300 0x7f8cfc155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 -- 172.21.15.12:0/3007228915 >> 172.21.15.12:0/3007228915 conn(0x7f8cfc066360 msgr2=0x7f8cfc14fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 -- 172.21.15.12:0/3007228915 shutdown_connections 2023-11-28T21:46:30.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:30.533+0000 7f8d01d57640 1 -- 172.21.15.12:0/3007228915 wait complete. 2023-11-28T21:46:30.561 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:46:32.562 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65e650f640 1 Processor -- start 2023-11-28T21:46:32.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65e650f640 1 -- start start 2023-11-28T21:46:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65e650f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f65e0151af0 0x7f65e00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65e650f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f65e0151ed0 0x7f65e00c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65e650f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e0157b70 con 0x7f65e006e300 2023-11-28T21:46:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65e650f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e00707f0 con 0x7f65e0151af0 2023-11-28T21:46:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65e650f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e00c7b90 con 0x7f65e0151ed0 2023-11-28T21:46:32.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65dffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f65e0151af0 0x7f65e00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65dffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f65e0151af0 0x7f65e00c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32820/0 (socket says 172.21.15.12:32820) 2023-11-28T21:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65dffff640 1 -- 172.21.15.12:0/4195365218 learned_addr learned my addr 172.21.15.12:0/4195365218 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65df7fe640 1 --2- 172.21.15.12:0/4195365218 >> v2:172.21.15.144:3300/0 conn(0x7f65e0151ed0 0x7f65e00c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65dffff640 1 -- 172.21.15.12:0/4195365218 >> v2:172.21.15.144:3300/0 conn(0x7f65e0151ed0 msgr2=0x7f65e00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65dffff640 1 --2- 172.21.15.12:0/4195365218 >> v2:172.21.15.144:3300/0 conn(0x7f65e0151ed0 0x7f65e00c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65dffff640 1 -- 172.21.15.12:0/4195365218 >> v1:172.21.15.144:6789/0 conn(0x7f65e006e300 legacy=0x7f65e0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:32.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.713+0000 7f65dffff640 1 -- 172.21.15.12:0/4195365218 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0157e50 con 0x7f65e0151af0 2023-11-28T21:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65dffff640 1 --2- 172.21.15.12:0/4195365218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 0x7f65e00c7650 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f65e00c8e60 tx=0x7f65d001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f93779376faa63 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65deffd640 1 -- 172.21.15.12:0/4195365218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00ab600 con 0x7f65e0151af0 2023-11-28T21:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65deffd640 1 -- 172.21.15.12:0/4195365218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d00a7050 con 0x7f65e0151af0 2023-11-28T21:46:32.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65deffd640 1 -- 172.21.15.12:0/4195365218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d00ab980 con 0x7f65e0151af0 2023-11-28T21:46:32.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65e650f640 1 -- 172.21.15.12:0/4195365218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 msgr2=0x7f65e00c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:32.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65e650f640 1 --2- 172.21.15.12:0/4195365218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 0x7f65e00c7650 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f65e00c8e60 tx=0x7f65d001c500 comp rx=0 tx=0).stop 2023-11-28T21:46:32.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65e650f640 1 -- 172.21.15.12:0/4195365218 shutdown_connections 2023-11-28T21:46:32.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65e650f640 1 --2- 172.21.15.12:0/4195365218 >> v2:172.21.15.144:3300/0 conn(0x7f65e0151ed0 0x7f65e00c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:32.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65e650f640 1 --2- 172.21.15.12:0/4195365218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 0x7f65e00c7650 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:32.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65e650f640 1 -- 172.21.15.12:0/4195365218 >> 172.21.15.12:0/4195365218 conn(0x7f65e0066360 msgr2=0x7f65e0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:32.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65e650f640 1 -- 172.21.15.12:0/4195365218 shutdown_connections 2023-11-28T21:46:32.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.717+0000 7f65e650f640 1 -- 172.21.15.12:0/4195365218 wait complete. 2023-11-28T21:46:32.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e650f640 1 Processor -- start 2023-11-28T21:46:32.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e650f640 1 -- start start 2023-11-28T21:46:32.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e650f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e006e300 0x7f65e0156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:32.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e650f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 0x7f65e0157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:32.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e650f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e0155e20 0x7f65e0154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:32.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e650f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e01566a0 con 0x7f65e0151af0 2023-11-28T21:46:32.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e650f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0155bf0 con 0x7f65e0155e20 2023-11-28T21:46:32.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e650f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e00c9bf0 con 0x7f65e006e300 2023-11-28T21:46:32.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65df7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 0x7f65e0157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:32.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65df7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 0x7f65e0157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32832/0 (socket says 172.21.15.12:32832) 2023-11-28T21:46:32.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65df7fe640 1 -- 172.21.15.12:0/2112025603 learned_addr learned my addr 172.21.15.12:0/2112025603 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:32.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65dffff640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e006e300 0x7f65e0156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e4aac640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e0155e20 0x7f65e0154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65df7fe640 1 -- 172.21.15.12:0/2112025603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e006e300 msgr2=0x7f65e0156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65df7fe640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e006e300 0x7f65e0156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65df7fe640 1 -- 172.21.15.12:0/2112025603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e0155e20 msgr2=0x7f65e0154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65df7fe640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e0155e20 0x7f65e0154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65df7fe640 1 -- 172.21.15.12:0/2112025603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00094d0 con 0x7f65e0151af0 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e4aac640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e0155e20 0x7f65e0154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65df7fe640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 0x7f65e0157070 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f65d400d5d0 tx=0x7f65d400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65dcff9640 1 -- 172.21.15.12:0/2112025603 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40125f0 con 0x7f65e0151af0 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65dcff9640 1 -- 172.21.15.12:0/2112025603 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d4012750 con 0x7f65e0151af0 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65dcff9640 1 -- 172.21.15.12:0/2112025603 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4012b00 con 0x7f65e0151af0 2023-11-28T21:46:32.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e650f640 1 -- 172.21.15.12:0/2112025603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e00c9ed0 con 0x7f65e0151af0 2023-11-28T21:46:32.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.721+0000 7f65e650f640 1 -- 172.21.15.12:0/2112025603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e01f1d30 con 0x7f65e0151af0 2023-11-28T21:46:32.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.725+0000 7f65e650f640 1 -- 172.21.15.12:0/2112025603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e01f1ef0 con 0x7f65e0151af0 2023-11-28T21:46:32.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.725+0000 7f65dcff9640 1 -- 172.21.15.12:0/2112025603 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f65d4012c60 con 0x7f65e0151af0 2023-11-28T21:46:32.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.725+0000 7f65dcff9640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65cc0664c0 0x7f65cc068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.725+0000 7f65dcff9640 1 -- 172.21.15.12:0/2112025603 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f65d4084600 con 0x7f65e0151af0 2023-11-28T21:46:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.725+0000 7f65dffff640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65cc0664c0 0x7f65cc068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:32.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.725+0000 7f65dffff640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65cc0664c0 0x7f65cc068980 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f65d00ad000 tx=0x7f65d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:32.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.729+0000 7f65dcff9640 1 -- 172.21.15.12:0/2112025603 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f65e01f1ef0 con 0x7f65e0151af0 2023-11-28T21:46:32.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.921+0000 7f65e650f640 1 -- 172.21.15.12:0/2112025603 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e01f1ec0 con 0x7f65e0151af0 2023-11-28T21:46:32.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.921+0000 7f65dcff9640 1 -- 172.21.15.12:0/2112025603 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f65d40514f0 con 0x7f65e0151af0 2023-11-28T21:46:32.929 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:32.929 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6024,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:32.929 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-11-28T21:46:32.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 -- 172.21.15.12:0/2112025603 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65cc0664c0 msgr2=0x7f65cc068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:32.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65cc0664c0 0x7f65cc068980 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f65d00ad000 tx=0x7f65d0008040 comp rx=0 tx=0).stop 2023-11-28T21:46:32.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 -- 172.21.15.12:0/2112025603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 msgr2=0x7f65e0157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:32.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 0x7f65e0157070 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f65d400d5d0 tx=0x7f65d400da90 comp rx=0 tx=0).stop 2023-11-28T21:46:32.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 -- 172.21.15.12:0/2112025603 shutdown_connections 2023-11-28T21:46:32.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65cc0664c0 0x7f65cc068980 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:32.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e0155e20 0x7f65e0154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:32.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65e0151af0 0x7f65e0157070 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 --2- 172.21.15.12:0/2112025603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e006e300 0x7f65e0156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 -- 172.21.15.12:0/2112025603 >> 172.21.15.12:0/2112025603 conn(0x7f65e0066360 msgr2=0x7f65e006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 -- 172.21.15.12:0/2112025603 shutdown_connections 2023-11-28T21:46:32.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:32.925+0000 7f65e650f640 1 -- 172.21.15.12:0/2112025603 wait complete. 2023-11-28T21:46:32.950 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:46:34.951 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.097+0000 7f46fbae3640 1 Processor -- start 2023-11-28T21:46:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.097+0000 7f46fbae3640 1 -- start start 2023-11-28T21:46:35.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46fbae3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f46f4153e00 0x7f46f406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:35.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46fbae3640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f46f415d7f0 0x7f46f415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:35.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46fbae3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f40c7710 con 0x7f46f406bc00 2023-11-28T21:46:35.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46fbae3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f41541e0 con 0x7f46f4153e00 2023-11-28T21:46:35.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46fbae3640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f415e160 con 0x7f46f415d7f0 2023-11-28T21:46:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46f987f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f46f4153e00 0x7f46f406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46f987f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f46f4153e00 0x7f46f406b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42330/0 (socket says 172.21.15.12:42330) 2023-11-28T21:46:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46f987f640 1 -- 172.21.15.12:0/3330811869 learned_addr learned my addr 172.21.15.12:0/3330811869 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46fa080640 1 --2- 172.21.15.12:0/3330811869 >> v2:172.21.15.144:3300/0 conn(0x7f46f415d7f0 0x7f46f415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46f987f640 1 -- 172.21.15.12:0/3330811869 >> v2:172.21.15.144:3300/0 conn(0x7f46f415d7f0 msgr2=0x7f46f415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46f987f640 1 --2- 172.21.15.12:0/3330811869 >> v2:172.21.15.144:3300/0 conn(0x7f46f415d7f0 0x7f46f415dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:35.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46f987f640 1 -- 172.21.15.12:0/3330811869 >> v1:172.21.15.12:6789/0 conn(0x7f46f406bc00 legacy=0x7f46f40cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:35.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.101+0000 7f46f987f640 1 -- 172.21.15.12:0/3330811869 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f415a010 con 0x7f46f4153e00 2023-11-28T21:46:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46f987f640 1 --2- 172.21.15.12:0/3330811869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f4153e00 0x7f46f406b6c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f46f406d510 tx=0x7f46f001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e070eb78ac7ffc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:35.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46f887d640 1 -- 172.21.15.12:0/3330811869 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00a89f0 con 0x7f46f4153e00 2023-11-28T21:46:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46f887d640 1 -- 172.21.15.12:0/3330811869 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f00a8b50 con 0x7f46f4153e00 2023-11-28T21:46:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46f887d640 1 -- 172.21.15.12:0/3330811869 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00b0440 con 0x7f46f4153e00 2023-11-28T21:46:35.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46fbae3640 1 -- 172.21.15.12:0/3330811869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f4153e00 msgr2=0x7f46f406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46fbae3640 1 --2- 172.21.15.12:0/3330811869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f4153e00 0x7f46f406b6c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f46f406d510 tx=0x7f46f001ecf0 comp rx=0 tx=0).stop 2023-11-28T21:46:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46fbae3640 1 -- 172.21.15.12:0/3330811869 shutdown_connections 2023-11-28T21:46:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46fbae3640 1 --2- 172.21.15.12:0/3330811869 >> v2:172.21.15.144:3300/0 conn(0x7f46f415d7f0 0x7f46f415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46fbae3640 1 --2- 172.21.15.12:0/3330811869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f4153e00 0x7f46f406b6c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46fbae3640 1 -- 172.21.15.12:0/3330811869 >> 172.21.15.12:0/3330811869 conn(0x7f46f4066360 msgr2=0x7f46f4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46fbae3640 1 -- 172.21.15.12:0/3330811869 shutdown_connections 2023-11-28T21:46:35.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46fbae3640 1 -- 172.21.15.12:0/3330811869 wait complete. 2023-11-28T21:46:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46fbae3640 1 Processor -- start 2023-11-28T21:46:35.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.105+0000 7f46fbae3640 1 -- start start 2023-11-28T21:46:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fbae3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f406bc00 0x7f46f414fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fbae3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f4153e00 0x7f46f4150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fbae3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f415d7f0 0x7f46f40cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fbae3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40cbf30 con 0x7f46f406bc00 2023-11-28T21:46:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fbae3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40cb360 con 0x7f46f415d7f0 2023-11-28T21:46:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fbae3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40cb600 con 0x7f46f4153e00 2023-11-28T21:46:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46f987f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f406bc00 0x7f46f414fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:35.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46f987f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f406bc00 0x7f46f414fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42340/0 (socket says 172.21.15.12:42340) 2023-11-28T21:46:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46f987f640 1 -- 172.21.15.12:0/123315871 learned_addr learned my addr 172.21.15.12:0/123315871 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fa080640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f415d7f0 0x7f46f40cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46f907e640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f4153e00 0x7f46f4150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46f987f640 1 -- 172.21.15.12:0/123315871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f4153e00 msgr2=0x7f46f4150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46f987f640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f4153e00 0x7f46f4150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46f987f640 1 -- 172.21.15.12:0/123315871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f415d7f0 msgr2=0x7f46f40cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46f987f640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f415d7f0 0x7f46f40cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46f987f640 1 -- 172.21.15.12:0/123315871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f00094d0 con 0x7f46f406bc00 2023-11-28T21:46:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fa080640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f415d7f0 0x7f46f40cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:46:35.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46f987f640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f406bc00 0x7f46f414fbb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f46f001b900 tx=0x7f46f001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:35.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46ea7fc640 1 -- 172.21.15.12:0/123315871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00ba600 con 0x7f46f406bc00 2023-11-28T21:46:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46ea7fc640 1 -- 172.21.15.12:0/123315871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f00ba760 con 0x7f46f406bc00 2023-11-28T21:46:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fbae3640 1 -- 172.21.15.12:0/123315871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f40cb8e0 con 0x7f46f406bc00 2023-11-28T21:46:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46ea7fc640 1 -- 172.21.15.12:0/123315871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00bab10 con 0x7f46f406bc00 2023-11-28T21:46:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fbae3640 1 -- 172.21.15.12:0/123315871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f4066af0 con 0x7f46f406bc00 2023-11-28T21:46:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46fbae3640 1 -- 172.21.15.12:0/123315871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f406c620 con 0x7f46f406bc00 2023-11-28T21:46:35.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.109+0000 7f46ea7fc640 1 -- 172.21.15.12:0/123315871 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f46f00bac70 con 0x7f46f406bc00 2023-11-28T21:46:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.113+0000 7f46ea7fc640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46d8066470 0x7f46d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.113+0000 7f46ea7fc640 1 -- 172.21.15.12:0/123315871 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7f46f01229a0 con 0x7f46f406bc00 2023-11-28T21:46:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.113+0000 7f46f907e640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46d8066470 0x7f46d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:35.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.113+0000 7f46f907e640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46d8066470 0x7f46d8068930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f46f4157890 tx=0x7f46e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:35.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.117+0000 7f46ea7fc640 1 -- 172.21.15.12:0/123315871 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46f00ef8a0 con 0x7f46f406bc00 2023-11-28T21:46:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 -- 172.21.15.12:0/123315871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46f40cbd90 con 0x7f46f406bc00 2023-11-28T21:46:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46ea7fc640 1 -- 172.21.15.12:0/123315871 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f46f40cbd90 con 0x7f46f406bc00 2023-11-28T21:46:35.312 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:35.312 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6024,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-11-28T21:46:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 -- 172.21.15.12:0/123315871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46d8066470 msgr2=0x7f46d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46d8066470 0x7f46d8068930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f46f4157890 tx=0x7f46e4009040 comp rx=0 tx=0).stop 2023-11-28T21:46:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 -- 172.21.15.12:0/123315871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f406bc00 msgr2=0x7f46f414fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f406bc00 0x7f46f414fbb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f46f001b900 tx=0x7f46f001ba90 comp rx=0 tx=0).stop 2023-11-28T21:46:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 -- 172.21.15.12:0/123315871 shutdown_connections 2023-11-28T21:46:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f46d8066470 0x7f46d8068930 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46f415d7f0 0x7f46f40cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:35.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46f4153e00 0x7f46f4150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:35.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 --2- 172.21.15.12:0/123315871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f46f406bc00 0x7f46f414fbb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:35.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 -- 172.21.15.12:0/123315871 >> 172.21.15.12:0/123315871 conn(0x7f46f4066360 msgr2=0x7f46f415c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:35.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 -- 172.21.15.12:0/123315871 shutdown_connections 2023-11-28T21:46:35.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:35.305+0000 7f46fbae3640 1 -- 172.21.15.12:0/123315871 wait complete. 2023-11-28T21:46:35.333 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:46:37.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:37.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.485+0000 7fdf1046b640 1 Processor -- start 2023-11-28T21:46:37.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.485+0000 7fdf1046b640 1 -- start start 2023-11-28T21:46:37.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.485+0000 7fdf1046b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdf08150c20 0x7fdf08156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:37.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.485+0000 7fdf1046b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf0806e300 0x7fdf080c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:37.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.485+0000 7fdf1046b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf08067410 con 0x7fdf0806cf40 2023-11-28T21:46:37.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.485+0000 7fdf1046b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0806e090 con 0x7fdf08150c20 2023-11-28T21:46:37.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.485+0000 7fdf1046b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08157f30 con 0x7fdf0806e300 2023-11-28T21:46:37.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0e207640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdf08150c20 0x7fdf08156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:37.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0e207640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdf08150c20 0x7fdf08156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42352/0 (socket says 172.21.15.12:42352) 2023-11-28T21:46:37.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0e207640 1 -- 172.21.15.12:0/1021576581 learned_addr learned my addr 172.21.15.12:0/1021576581 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:37.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0da06640 1 --2- 172.21.15.12:0/1021576581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf0806e300 0x7fdf080c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:37.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0e207640 1 -- 172.21.15.12:0/1021576581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf0806e300 msgr2=0x7fdf080c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:37.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0e207640 1 --2- 172.21.15.12:0/1021576581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf0806e300 0x7fdf080c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:37.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0e207640 1 -- 172.21.15.12:0/1021576581 >> v1:172.21.15.144:6789/0 conn(0x7fdf0806cf40 legacy=0x7fdf08158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:37.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0e207640 1 -- 172.21.15.12:0/1021576581 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf08151000 con 0x7fdf08150c20 2023-11-28T21:46:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0e207640 1 --2- 172.21.15.12:0/1021576581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08150c20 0x7fdf08156910 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009870 tx=0x7fdf0401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d638429ab70331 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0d205640 1 -- 172.21.15.12:0/1021576581 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040ab650 con 0x7fdf08150c20 2023-11-28T21:46:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0d205640 1 -- 172.21.15.12:0/1021576581 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf040ab7b0 con 0x7fdf08150c20 2023-11-28T21:46:37.498 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.489+0000 7fdf0d205640 1 -- 172.21.15.12:0/1021576581 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040abb40 con 0x7fdf08150c20 2023-11-28T21:46:37.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 -- 172.21.15.12:0/1021576581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08150c20 msgr2=0x7fdf08156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:37.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 --2- 172.21.15.12:0/1021576581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08150c20 0x7fdf08156910 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009870 tx=0x7fdf0401c540 comp rx=0 tx=0).stop 2023-11-28T21:46:37.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 -- 172.21.15.12:0/1021576581 shutdown_connections 2023-11-28T21:46:37.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 --2- 172.21.15.12:0/1021576581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf0806e300 0x7fdf080c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:37.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 --2- 172.21.15.12:0/1021576581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08150c20 0x7fdf08156910 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:37.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 -- 172.21.15.12:0/1021576581 >> 172.21.15.12:0/1021576581 conn(0x7fdf08066360 msgr2=0x7fdf08066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:37.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 -- 172.21.15.12:0/1021576581 shutdown_connections 2023-11-28T21:46:37.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 -- 172.21.15.12:0/1021576581 wait complete. 2023-11-28T21:46:37.506 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 Processor -- start 2023-11-28T21:46:37.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 -- start start 2023-11-28T21:46:37.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf0806cf40 0x7fdf08145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:37.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf0806e300 0x7fdf081f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:37.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08150c20 0x7fdf08141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:37.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080c3ad0 con 0x7fdf0806cf40 2023-11-28T21:46:37.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf080c3720 con 0x7fdf08150c20 2023-11-28T21:46:37.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf1046b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf081f6ff0 con 0x7fdf0806e300 2023-11-28T21:46:37.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf0e207640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf0806cf40 0x7fdf08145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:37.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf0e207640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf0806cf40 0x7fdf08145f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42358/0 (socket says 172.21.15.12:42358) 2023-11-28T21:46:37.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.493+0000 7fdf0e207640 1 -- 172.21.15.12:0/214262025 learned_addr learned my addr 172.21.15.12:0/214262025 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:37.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf0da06640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf0806e300 0x7fdf081f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:37.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf0ea08640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08150c20 0x7fdf08141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf0e207640 1 -- 172.21.15.12:0/214262025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf0806e300 msgr2=0x7fdf081f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf0e207640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf0806e300 0x7fdf081f66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf0e207640 1 -- 172.21.15.12:0/214262025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08150c20 msgr2=0x7fdf08141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf0e207640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08150c20 0x7fdf08141d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf0e207640 1 -- 172.21.15.12:0/214262025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040094d0 con 0x7fdf0806cf40 2023-11-28T21:46:37.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf0e207640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf0806cf40 0x7fdf08145f60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdf040ad040 tx=0x7fdf0401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdefeffd640 1 -- 172.21.15.12:0/214262025 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040b4800 con 0x7fdf0806cf40 2023-11-28T21:46:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdefeffd640 1 -- 172.21.15.12:0/214262025 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf040b4980 con 0x7fdf0806cf40 2023-11-28T21:46:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdefeffd640 1 -- 172.21.15.12:0/214262025 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040b4d30 con 0x7fdf0806cf40 2023-11-28T21:46:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf1046b640 1 -- 172.21.15.12:0/214262025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf08146fb0 con 0x7fdf0806cf40 2023-11-28T21:46:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf1046b640 1 -- 172.21.15.12:0/214262025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf08147570 con 0x7fdf0806cf40 2023-11-28T21:46:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf1046b640 1 -- 172.21.15.12:0/214262025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf08074f60 con 0x7fdf0806cf40 2023-11-28T21:46:37.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdefeffd640 1 -- 172.21.15.12:0/214262025 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdf040c7440 con 0x7fdf0806cf40 2023-11-28T21:46:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdefeffd640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdeec0664c0 0x7fdeec068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdefeffd640 1 -- 172.21.15.12:0/214262025 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fdf0411efd0 con 0x7fdf0806cf40 2023-11-28T21:46:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.497+0000 7fdf0da06640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdeec0664c0 0x7fdeec068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:37.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.501+0000 7fdf0da06640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdeec0664c0 0x7fdeec068980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdf08149370 tx=0x7fdef8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:37.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.501+0000 7fdefeffd640 1 -- 172.21.15.12:0/214262025 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdf040ec070 con 0x7fdf0806cf40 2023-11-28T21:46:37.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.693+0000 7fdf1046b640 1 -- 172.21.15.12:0/214262025 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf080c2c30 con 0x7fdf0806cf40 2023-11-28T21:46:37.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.693+0000 7fdefeffd640 1 -- 172.21.15.12:0/214262025 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fdf040ef690 con 0x7fdf0806cf40 2023-11-28T21:46:37.701 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:37.701 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6024,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:37.701 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-11-28T21:46:37.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 -- 172.21.15.12:0/214262025 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdeec0664c0 msgr2=0x7fdeec068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:37.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdeec0664c0 0x7fdeec068980 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdf08149370 tx=0x7fdef8008040 comp rx=0 tx=0).stop 2023-11-28T21:46:37.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 -- 172.21.15.12:0/214262025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf0806cf40 msgr2=0x7fdf08145f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:37.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf0806cf40 0x7fdf08145f60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdf040ad040 tx=0x7fdf0401ad80 comp rx=0 tx=0).stop 2023-11-28T21:46:37.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 -- 172.21.15.12:0/214262025 shutdown_connections 2023-11-28T21:46:37.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdeec0664c0 0x7fdeec068980 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:37.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf08150c20 0x7fdf08141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf0806e300 0x7fdf081f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 --2- 172.21.15.12:0/214262025 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf0806cf40 0x7fdf08145f60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 -- 172.21.15.12:0/214262025 >> 172.21.15.12:0/214262025 conn(0x7fdf08066360 msgr2=0x7fdf08160230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 -- 172.21.15.12:0/214262025 shutdown_connections 2023-11-28T21:46:37.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:37.697+0000 7fdf1046b640 1 -- 172.21.15.12:0/214262025 wait complete. 2023-11-28T21:46:37.722 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:46:39.726 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.873+0000 7fb46a101640 1 Processor -- start 2023-11-28T21:46:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.873+0000 7fb46a101640 1 -- start start 2023-11-28T21:46:39.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.873+0000 7fb46a101640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb464151ed0 0x7fb4640c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:39.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.873+0000 7fb46a101640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb46406e300 0x7fb464072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:39.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.873+0000 7fb46a101640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb464157b70 con 0x7fb464151af0 2023-11-28T21:46:39.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.873+0000 7fb46a101640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640707f0 con 0x7fb464151ed0 2023-11-28T21:46:39.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.873+0000 7fb46a101640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640c7b90 con 0x7fb46406e300 2023-11-28T21:46:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.873+0000 7fb462ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb464151ed0 0x7fb4640c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.873+0000 7fb4637fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb464151af0 0x7fb4640c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50512/0 (socket says 172.21.15.12:50512) 2023-11-28T21:46:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.873+0000 7fb4637fe640 1 -- 172.21.15.12:0/1391674049 learned_addr learned my addr 172.21.15.12:0/1391674049 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:46:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb463fff640 1 --2- 172.21.15.12:0/1391674049 >> v2:172.21.15.144:3300/0 conn(0x7fb46406e300 0x7fb464072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:39.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb462ffd640 1 -- 172.21.15.12:0/1391674049 >> v2:172.21.15.144:3300/0 conn(0x7fb46406e300 msgr2=0x7fb464072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb462ffd640 1 --2- 172.21.15.12:0/1391674049 >> v2:172.21.15.144:3300/0 conn(0x7fb46406e300 0x7fb464072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb462ffd640 1 -- 172.21.15.12:0/1391674049 >> v1:172.21.15.12:6789/0 conn(0x7fb464151af0 legacy=0x7fb4640c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb462ffd640 1 -- 172.21.15.12:0/1391674049 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb464157e50 con 0x7fb464151ed0 2023-11-28T21:46:39.884 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb462ffd640 1 --2- 172.21.15.12:0/1391674049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151ed0 0x7fb4640c7650 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb4640716f0 tx=0x7fb45801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b844de0c81a1aec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:39.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb4627fc640 1 -- 172.21.15.12:0/1391674049 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580ab600 con 0x7fb464151ed0 2023-11-28T21:46:39.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb4627fc640 1 -- 172.21.15.12:0/1391674049 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4580a7050 con 0x7fb464151ed0 2023-11-28T21:46:39.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb4627fc640 1 -- 172.21.15.12:0/1391674049 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580ab980 con 0x7fb464151ed0 2023-11-28T21:46:39.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb46a101640 1 -- 172.21.15.12:0/1391674049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151ed0 msgr2=0x7fb4640c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:39.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb46a101640 1 --2- 172.21.15.12:0/1391674049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151ed0 0x7fb4640c7650 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb4640716f0 tx=0x7fb45801c460 comp rx=0 tx=0).stop 2023-11-28T21:46:39.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb46a101640 1 -- 172.21.15.12:0/1391674049 shutdown_connections 2023-11-28T21:46:39.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb46a101640 1 --2- 172.21.15.12:0/1391674049 >> v2:172.21.15.144:3300/0 conn(0x7fb46406e300 0x7fb464072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:39.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb46a101640 1 --2- 172.21.15.12:0/1391674049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151ed0 0x7fb4640c7650 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:39.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.877+0000 7fb46a101640 1 -- 172.21.15.12:0/1391674049 >> 172.21.15.12:0/1391674049 conn(0x7fb464066360 msgr2=0x7fb464066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb46a101640 1 -- 172.21.15.12:0/1391674049 shutdown_connections 2023-11-28T21:46:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb46a101640 1 -- 172.21.15.12:0/1391674049 wait complete. 2023-11-28T21:46:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb46a101640 1 Processor -- start 2023-11-28T21:46:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb46a101640 1 -- start start 2023-11-28T21:46:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb46a101640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46406e300 0x7fb464140d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb46a101640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151af0 0x7fb46406a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb46a101640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4640c93f0 0x7fb46413f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb46a101640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464147320 con 0x7fb464151af0 2023-11-28T21:46:39.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb46a101640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4641424a0 con 0x7fb46406e300 2023-11-28T21:46:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb46a101640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb464141250 con 0x7fb4640c93f0 2023-11-28T21:46:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb462ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151af0 0x7fb46406a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb462ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151af0 0x7fb46406a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42380/0 (socket says 172.21.15.12:42380) 2023-11-28T21:46:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb462ffd640 1 -- 172.21.15.12:0/1458596683 learned_addr learned my addr 172.21.15.12:0/1458596683 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb4637fe640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46406e300 0x7fb464140d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb463fff640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4640c93f0 0x7fb46413f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:39.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb462ffd640 1 -- 172.21.15.12:0/1458596683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4640c93f0 msgr2=0x7fb46413f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb462ffd640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4640c93f0 0x7fb46413f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb462ffd640 1 -- 172.21.15.12:0/1458596683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46406e300 msgr2=0x7fb464140d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb462ffd640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46406e300 0x7fb464140d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb462ffd640 1 -- 172.21.15.12:0/1458596683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458009420 con 0x7fb464151af0 2023-11-28T21:46:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb4637fe640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46406e300 0x7fb464140d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:46:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.881+0000 7fb462ffd640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151af0 0x7fb46406a810 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb4580ae040 tx=0x7fb458019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.885+0000 7fb44bfff640 1 -- 172.21.15.12:0/1458596683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580b4840 con 0x7fb464151af0 2023-11-28T21:46:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.885+0000 7fb44bfff640 1 -- 172.21.15.12:0/1458596683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4580b49c0 con 0x7fb464151af0 2023-11-28T21:46:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.885+0000 7fb44bfff640 1 -- 172.21.15.12:0/1458596683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580b4d70 con 0x7fb464151af0 2023-11-28T21:46:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.885+0000 7fb46a101640 1 -- 172.21.15.12:0/1458596683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4641f3840 con 0x7fb464151af0 2023-11-28T21:46:39.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.885+0000 7fb46a101640 1 -- 172.21.15.12:0/1458596683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4641f3b20 con 0x7fb464151af0 2023-11-28T21:46:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.885+0000 7fb46a101640 1 -- 172.21.15.12:0/1458596683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb464142910 con 0x7fb464151af0 2023-11-28T21:46:39.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.885+0000 7fb44bfff640 1 -- 172.21.15.12:0/1458596683 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb4580b2040 con 0x7fb464151af0 2023-11-28T21:46:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.885+0000 7fb44bfff640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb450066470 0x7fb450068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.885+0000 7fb44bfff640 1 -- 172.21.15.12:0/1458596683 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fb45811e830 con 0x7fb464151af0 2023-11-28T21:46:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.885+0000 7fb4637fe640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb450066470 0x7fb450068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:39.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.889+0000 7fb4637fe640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb450066470 0x7fb450068930 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb454005890 tx=0x7fb454009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:39.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:39.889+0000 7fb44bfff640 1 -- 172.21.15.12:0/1458596683 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4580eb610 con 0x7fb464151af0 2023-11-28T21:46:40.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.085+0000 7fb46a101640 1 -- 172.21.15.12:0/1458596683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb464074f60 con 0x7fb464151af0 2023-11-28T21:46:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.085+0000 7fb44bfff640 1 -- 172.21.15.12:0/1458596683 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fb4580eec30 con 0x7fb464151af0 2023-11-28T21:46:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:40.092 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6024,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:40.092 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-11-28T21:46:40.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.085+0000 7fb46a101640 1 -- 172.21.15.12:0/1458596683 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb450066470 msgr2=0x7fb450068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:40.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.085+0000 7fb46a101640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb450066470 0x7fb450068930 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb454005890 tx=0x7fb454009040 comp rx=0 tx=0).stop 2023-11-28T21:46:40.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.089+0000 7fb46a101640 1 -- 172.21.15.12:0/1458596683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151af0 msgr2=0x7fb46406a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:40.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.089+0000 7fb46a101640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151af0 0x7fb46406a810 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb4580ae040 tx=0x7fb458019200 comp rx=0 tx=0).stop 2023-11-28T21:46:40.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.089+0000 7fb46a101640 1 -- 172.21.15.12:0/1458596683 shutdown_connections 2023-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.089+0000 7fb46a101640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb450066470 0x7fb450068930 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.089+0000 7fb46a101640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4640c93f0 0x7fb46413f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.089+0000 7fb46a101640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb464151af0 0x7fb46406a810 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.089+0000 7fb46a101640 1 --2- 172.21.15.12:0/1458596683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb46406e300 0x7fb464140d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.089+0000 7fb46a101640 1 -- 172.21.15.12:0/1458596683 >> 172.21.15.12:0/1458596683 conn(0x7fb464066360 msgr2=0x7fb46414f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.089+0000 7fb46a101640 1 -- 172.21.15.12:0/1458596683 shutdown_connections 2023-11-28T21:46:40.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:40.089+0000 7fb46a101640 1 -- 172.21.15.12:0/1458596683 wait complete. 2023-11-28T21:46:40.114 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:46:42.115 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:42.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.261+0000 7fe36925f640 1 Processor -- start 2023-11-28T21:46:42.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.261+0000 7fe36925f640 1 -- start start 2023-11-28T21:46:42.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36925f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe364151af0 0x7fe3640c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:42.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36925f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe364157b70 con 0x7fe364151ed0 2023-11-28T21:46:42.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36925f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe364157e50 con 0x7fe36406e300 2023-11-28T21:46:42.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36925f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3640707f0 con 0x7fe364151af0 2023-11-28T21:46:42.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe35bfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe364151ed0 0x7fe3640c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50524/0 (socket says 172.21.15.12:50524) 2023-11-28T21:46:42.271 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe35bfff640 1 -- 172.21.15.12:0/1773841862 learned_addr learned my addr 172.21.15.12:0/1773841862 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:46:42.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe362d9d640 1 --2- 172.21.15.12:0/1773841862 >> v2:172.21.15.144:3300/0 conn(0x7fe364151af0 0x7fe3640c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:42.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137888196 0 0) 0x7fe364157b70 con 0x7fe364151ed0 2023-11-28T21:46:42.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe350003530 con 0x7fe364151ed0 2023-11-28T21:46:42.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2521323183 0 0) 0x7fe350003530 con 0x7fe364151ed0 2023-11-28T21:46:42.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 >> v2:172.21.15.144:3300/0 conn(0x7fe364151af0 msgr2=0x7fe3640c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 --2- 172.21.15.12:0/1773841862 >> v2:172.21.15.144:3300/0 conn(0x7fe364151af0 0x7fe3640c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 >> v1:172.21.15.144:6789/0 conn(0x7fe36406e300 legacy=0x7fe364072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe364158130 con 0x7fe364151ed0 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fe34c001680 con 0x7fe364151ed0 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 >> v1:172.21.15.12:6789/0 conn(0x7fe364151ed0 legacy=0x7fe3640c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3500039f0 0x7fe350003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3500047d0 0x7fe350004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe35000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640707f0 con 0x7fe350005190 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35000c0c0 con 0x7fe3500047d0 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35000c280 con 0x7fe3500039f0 2023-11-28T21:46:42.274 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36359e640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe35000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:42.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe35bfff640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3500047d0 0x7fe350004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:42.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe362d9d640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3500039f0 0x7fe350003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:42.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36359e640 1 -- 172.21.15.12:0/1773841862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3500039f0 msgr2=0x7fe350003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:42.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36359e640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3500039f0 0x7fe350003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:42.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36359e640 1 -- 172.21.15.12:0/1773841862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3500047d0 msgr2=0x7fe350004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:42.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36359e640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3500047d0 0x7fe350004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:42.275 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.265+0000 7fe36359e640 1 -- 172.21.15.12:0/1773841862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350003530 con 0x7fe350005190 2023-11-28T21:46:42.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36359e640 1 -- 172.21.15.12:0/1773841862 reap_dead start 2023-11-28T21:46:42.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36359e640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe35000b990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe35c001e40 tx=0x7fe35c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33019ab990a45dbd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:42.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c003b80 con 0x7fe350005190 2023-11-28T21:46:42.276 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c003ce0 con 0x7fe350005190 2023-11-28T21:46:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36259c640 1 -- 172.21.15.12:0/1773841862 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c005260 con 0x7fe350005190 2023-11-28T21:46:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36925f640 1 -- 172.21.15.12:0/1773841862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 msgr2=0x7fe35000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:42.277 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36925f640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe35000b990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe35c001e40 tx=0x7fe35c01c740 comp rx=0 tx=0).stop 2023-11-28T21:46:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36925f640 1 -- 172.21.15.12:0/1773841862 shutdown_connections 2023-11-28T21:46:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36925f640 1 --2- 172.21.15.12:0/1773841862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe35000b990 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36925f640 1 -- 172.21.15.12:0/1773841862 >> 172.21.15.12:0/1773841862 conn(0x7fe364066360 msgr2=0x7fe364066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36925f640 1 -- 172.21.15.12:0/1773841862 shutdown_connections 2023-11-28T21:46:42.278 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.269+0000 7fe36925f640 1 -- 172.21.15.12:0/1773841862 wait complete. 2023-11-28T21:46:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36925f640 1 Processor -- start 2023-11-28T21:46:42.279 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36925f640 1 -- start start 2023-11-28T21:46:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36925f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe364146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36925f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe364146dd0 0x7fe36414fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:42.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36925f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3641503a0 0x7fe364148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36925f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641ed280 con 0x7fe350005190 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36925f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe364150820 con 0x7fe3641503a0 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36925f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe364147230 con 0x7fe364146dd0 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe362d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe364146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe362d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe364146890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42384/0 (socket says 172.21.15.12:42384) 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe362d9d640 1 -- 172.21.15.12:0/2369736291 learned_addr learned my addr 172.21.15.12:0/2369736291 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe35bfff640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe364146dd0 0x7fe36414fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36359e640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3641503a0 0x7fe364148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe362d9d640 1 -- 172.21.15.12:0/2369736291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe364146dd0 msgr2=0x7fe36414fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe362d9d640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe364146dd0 0x7fe36414fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe362d9d640 1 -- 172.21.15.12:0/2369736291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3641503a0 msgr2=0x7fe364148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:42.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe362d9d640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3641503a0 0x7fe364148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:42.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe362d9d640 1 -- 172.21.15.12:0/2369736291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36414ee50 con 0x7fe350005190 2023-11-28T21:46:42.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36359e640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3641503a0 0x7fe364148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:46:42.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe362d9d640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe364146890 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe354011990 tx=0x7fe354011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:42.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe35b7fe640 1 -- 172.21.15.12:0/2369736291 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35400fa30 con 0x7fe350005190 2023-11-28T21:46:42.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe35b7fe640 1 -- 172.21.15.12:0/2369736291 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35400fb90 con 0x7fe350005190 2023-11-28T21:46:42.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe35b7fe640 1 -- 172.21.15.12:0/2369736291 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354014440 con 0x7fe350005190 2023-11-28T21:46:42.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36925f640 1 -- 172.21.15.12:0/2369736291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36414f9d0 con 0x7fe350005190 2023-11-28T21:46:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.273+0000 7fe36925f640 1 -- 172.21.15.12:0/2369736291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36414f0a0 con 0x7fe350005190 2023-11-28T21:46:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.277+0000 7fe36925f640 1 -- 172.21.15.12:0/2369736291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe364074f60 con 0x7fe350005190 2023-11-28T21:46:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.277+0000 7fe35b7fe640 1 -- 172.21.15.12:0/2369736291 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe3540145a0 con 0x7fe350005190 2023-11-28T21:46:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.277+0000 7fe35b7fe640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3440664a0 0x7fe344068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.277+0000 7fe35b7fe640 1 -- 172.21.15.12:0/2369736291 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fe354084bd0 con 0x7fe350005190 2023-11-28T21:46:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.277+0000 7fe35bfff640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3440664a0 0x7fe344068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.277+0000 7fe35bfff640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3440664a0 0x7fe344068960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe34c001060 tx=0x7fe34c001520 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.281+0000 7fe35b7fe640 1 -- 172.21.15.12:0/2369736291 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe354051af0 con 0x7fe350005190 2023-11-28T21:46:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.469+0000 7fe36925f640 1 -- 172.21.15.12:0/2369736291 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe364072c10 con 0x7fe350005190 2023-11-28T21:46:42.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.469+0000 7fe35b7fe640 1 -- 172.21.15.12:0/2369736291 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fe354022090 con 0x7fe350005190 2023-11-28T21:46:42.478 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:42.478 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6024,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:42.478 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-11-28T21:46:42.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 -- 172.21.15.12:0/2369736291 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3440664a0 msgr2=0x7fe344068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:42.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3440664a0 0x7fe344068960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe34c001060 tx=0x7fe34c001520 comp rx=0 tx=0).stop 2023-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 -- 172.21.15.12:0/2369736291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 msgr2=0x7fe364146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe364146890 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe354011990 tx=0x7fe354011d90 comp rx=0 tx=0).stop 2023-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 -- 172.21.15.12:0/2369736291 shutdown_connections 2023-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe3440664a0 0x7fe344068960 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3641503a0 0x7fe364148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe364146dd0 0x7fe36414fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 --2- 172.21.15.12:0/2369736291 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe350005190 0x7fe364146890 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 -- 172.21.15.12:0/2369736291 >> 172.21.15.12:0/2369736291 conn(0x7fe364066360 msgr2=0x7fe36406ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 -- 172.21.15.12:0/2369736291 shutdown_connections 2023-11-28T21:46:42.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:42.473+0000 7fe36925f640 1 -- 172.21.15.12:0/2369736291 wait complete. 2023-11-28T21:46:42.499 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:46:44.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.645+0000 7fd27ba9c640 1 Processor -- start 2023-11-28T21:46:44.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd27ba9c640 1 -- start start 2023-11-28T21:46:44.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd27ba9c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27406e300 0x7fd2740c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:44.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd27ba9c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd274157b70 con 0x7fd274151af0 2023-11-28T21:46:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd27ba9c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd274157e50 con 0x7fd274151ed0 2023-11-28T21:46:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd27ba9c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2740707f0 con 0x7fd27406e300 2023-11-28T21:46:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd279838640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd274151af0 0x7fd2740c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36232/0 (socket says 172.21.15.12:36232) 2023-11-28T21:46:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd279838640 1 -- 172.21.15.12:0/2227893216 learned_addr learned my addr 172.21.15.12:0/2227893216 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:46:44.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd27a039640 1 --2- 172.21.15.12:0/2227893216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27406e300 0x7fd2740c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:44.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd278836640 1 -- 172.21.15.12:0/2227893216 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696559450 0 0) 0x7fd274157b70 con 0x7fd274151af0 2023-11-28T21:46:44.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd278836640 1 -- 172.21.15.12:0/2227893216 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd260003530 con 0x7fd274151af0 2023-11-28T21:46:44.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd278836640 1 -- 172.21.15.12:0/2227893216 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1614263784 0 0) 0x7fd260003530 con 0x7fd274151af0 2023-11-28T21:46:44.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd27a039640 1 -- 172.21.15.12:0/2227893216 >> v1:172.21.15.144:6789/0 conn(0x7fd274151ed0 legacy=0x7fd274072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:44.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd27a039640 1 -- 172.21.15.12:0/2227893216 >> v1:172.21.15.12:6789/0 conn(0x7fd274151af0 legacy=0x7fd2740c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:44.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.649+0000 7fd27a039640 1 -- 172.21.15.12:0/2227893216 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274158130 con 0x7fd27406e300 2023-11-28T21:46:44.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd27a039640 1 --2- 172.21.15.12:0/2227893216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27406e300 0x7fd2740c7650 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009dc0 tx=0x7fd26c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af83ab4c9cba6ad6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:44.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd278836640 1 -- 172.21.15.12:0/2227893216 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270001680 con 0x7fd27406e300 2023-11-28T21:46:44.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd278836640 1 -- 172.21.15.12:0/2227893216 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c0a64e0 con 0x7fd27406e300 2023-11-28T21:46:44.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd278836640 1 -- 172.21.15.12:0/2227893216 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c0a6850 con 0x7fd27406e300 2023-11-28T21:46:44.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd27ba9c640 1 -- 172.21.15.12:0/2227893216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27406e300 msgr2=0x7fd2740c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:44.660 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd27ba9c640 1 --2- 172.21.15.12:0/2227893216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27406e300 0x7fd2740c7650 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009dc0 tx=0x7fd26c01c890 comp rx=0 tx=0).stop 2023-11-28T21:46:44.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd27ba9c640 1 -- 172.21.15.12:0/2227893216 shutdown_connections 2023-11-28T21:46:44.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd27ba9c640 1 --2- 172.21.15.12:0/2227893216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27406e300 0x7fd2740c7650 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:44.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd27ba9c640 1 -- 172.21.15.12:0/2227893216 >> 172.21.15.12:0/2227893216 conn(0x7fd274066360 msgr2=0x7fd274066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:44.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd27ba9c640 1 -- 172.21.15.12:0/2227893216 shutdown_connections 2023-11-28T21:46:44.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.653+0000 7fd27ba9c640 1 -- 172.21.15.12:0/2227893216 wait complete. 2023-11-28T21:46:44.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 Processor -- start 2023-11-28T21:46:44.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 -- start start 2023-11-28T21:46:44.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd27406e300 0x7fd2740c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:44.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd274151af0 0x7fd2741569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:44.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2740c4520 0x7fd2740c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:44.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2741f39e0 con 0x7fd274151af0 2023-11-28T21:46:44.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2741f3e70 con 0x7fd27406e300 2023-11-28T21:46:44.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2740c40e0 con 0x7fd2740c4520 2023-11-28T21:46:44.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279037640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd274151af0 0x7fd2741569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:44.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279037640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd274151af0 0x7fd2741569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50536/0 (socket says 172.21.15.12:50536) 2023-11-28T21:46:44.665 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279037640 1 -- 172.21.15.12:0/53671608 learned_addr learned my addr 172.21.15.12:0/53671608 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279838640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd27406e300 0x7fd2740c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27a039640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2740c4520 0x7fd2740c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279037640 1 -- 172.21.15.12:0/53671608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2740c4520 msgr2=0x7fd2740c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279037640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2740c4520 0x7fd2740c2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279037640 1 -- 172.21.15.12:0/53671608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd27406e300 msgr2=0x7fd2740c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279037640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd27406e300 0x7fd2740c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279037640 1 -- 172.21.15.12:0/53671608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c009a70 con 0x7fd274151af0 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279838640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd27406e300 0x7fd2740c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd279037640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd274151af0 0x7fd2741569f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd26400c3c0 tx=0x7fd26400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd26a7fc640 1 -- 172.21.15.12:0/53671608 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2640052d0 con 0x7fd274151af0 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd26a7fc640 1 -- 172.21.15.12:0/53671608 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd264005430 con 0x7fd274151af0 2023-11-28T21:46:44.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd26a7fc640 1 -- 172.21.15.12:0/53671608 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264003840 con 0x7fd274151af0 2023-11-28T21:46:44.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 -- 172.21.15.12:0/53671608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2740c3e00 con 0x7fd274151af0 2023-11-28T21:46:44.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 -- 172.21.15.12:0/53671608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2741f3390 con 0x7fd274151af0 2023-11-28T21:46:44.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.657+0000 7fd27ba9c640 1 -- 172.21.15.12:0/53671608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd274074f60 con 0x7fd274151af0 2023-11-28T21:46:44.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.661+0000 7fd26a7fc640 1 -- 172.21.15.12:0/53671608 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd2640039a0 con 0x7fd274151af0 2023-11-28T21:46:44.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.661+0000 7fd26a7fc640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd258066470 0x7fd258068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:44.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.661+0000 7fd26a7fc640 1 -- 172.21.15.12:0/53671608 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fd264083900 con 0x7fd274151af0 2023-11-28T21:46:44.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.661+0000 7fd279838640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd258066470 0x7fd258068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:44.668 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.661+0000 7fd279838640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd258066470 0x7fd258068930 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd270005890 tx=0x7fd270009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:44.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.665+0000 7fd26a7fc640 1 -- 172.21.15.12:0/53671608 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd264050820 con 0x7fd274151af0 2023-11-28T21:46:44.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.853+0000 7fd27ba9c640 1 -- 172.21.15.12:0/53671608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2741f3b20 con 0x7fd274151af0 2023-11-28T21:46:44.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.853+0000 7fd26a7fc640 1 -- 172.21.15.12:0/53671608 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fd264053e40 con 0x7fd274151af0 2023-11-28T21:46:44.861 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:44.861 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6024,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:44.862 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 -- 172.21.15.12:0/53671608 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd258066470 msgr2=0x7fd258068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd258066470 0x7fd258068930 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd270005890 tx=0x7fd270009040 comp rx=0 tx=0).stop 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 -- 172.21.15.12:0/53671608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd274151af0 msgr2=0x7fd2741569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd274151af0 0x7fd2741569f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd26400c3c0 tx=0x7fd26400c880 comp rx=0 tx=0).stop 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 -- 172.21.15.12:0/53671608 shutdown_connections 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd258066470 0x7fd258068930 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2740c4520 0x7fd2740c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd274151af0 0x7fd2741569f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 --2- 172.21.15.12:0/53671608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd27406e300 0x7fd2740c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 -- 172.21.15.12:0/53671608 >> 172.21.15.12:0/53671608 conn(0x7fd274066360 msgr2=0x7fd274072070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:44.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 -- 172.21.15.12:0/53671608 shutdown_connections 2023-11-28T21:46:44.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:44.857+0000 7fd27ba9c640 1 -- 172.21.15.12:0/53671608 wait complete. 2023-11-28T21:46:44.883 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:46:45.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:46:45.441 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:46:46.884 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:47.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd6aad640 1 Processor -- start 2023-11-28T21:46:47.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd6aad640 1 -- start start 2023-11-28T21:46:47.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd6aad640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd0153fb0 0x7fddd00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:47.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd6aad640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fddd015a980 0x7fddd00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:47.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd6aad640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd015d390 con 0x7fddd00b3ad0 2023-11-28T21:46:47.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd6aad640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd00b2b60 con 0x7fddd015a980 2023-11-28T21:46:47.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd6aad640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd015e1b0 con 0x7fddd0153fb0 2023-11-28T21:46:47.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd62ac640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fddd00b3ad0 0x7fddd00b3ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36246/0 (socket says 172.21.15.12:36246) 2023-11-28T21:46:47.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd62ac640 1 -- 172.21.15.12:0/3076831271 learned_addr learned my addr 172.21.15.12:0/3076831271 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:46:47.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd5aab640 1 --2- 172.21.15.12:0/3076831271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd0153fb0 0x7fddd00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:47.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd52aa640 1 --2- 172.21.15.12:0/3076831271 >> v2:172.21.15.144:3300/0 conn(0x7fddd015a980 0x7fddd00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:47.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd4aa9640 1 -- 172.21.15.12:0/3076831271 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926253999 0 0) 0x7fddd015d390 con 0x7fddd00b3ad0 2023-11-28T21:46:47.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd4aa9640 1 -- 172.21.15.12:0/3076831271 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddbc003530 con 0x7fddd00b3ad0 2023-11-28T21:46:47.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd4aa9640 1 -- 172.21.15.12:0/3076831271 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3619793175 0 0) 0x7fddbc003530 con 0x7fddd00b3ad0 2023-11-28T21:46:47.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd5aab640 1 -- 172.21.15.12:0/3076831271 >> v2:172.21.15.144:3300/0 conn(0x7fddd015a980 msgr2=0x7fddd00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:47.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd5aab640 1 --2- 172.21.15.12:0/3076831271 >> v2:172.21.15.144:3300/0 conn(0x7fddd015a980 0x7fddd00caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:47.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd5aab640 1 -- 172.21.15.12:0/3076831271 >> v1:172.21.15.12:6789/0 conn(0x7fddd00b3ad0 legacy=0x7fddd00b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:47.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd5aab640 1 -- 172.21.15.12:0/3076831271 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd015d630 con 0x7fddd0153fb0 2023-11-28T21:46:47.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.033+0000 7fddd52aa640 1 --2- 172.21.15.12:0/3076831271 >> v2:172.21.15.144:3300/0 conn(0x7fddd015a980 0x7fddd00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:46:47.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd5aab640 1 --2- 172.21.15.12:0/3076831271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd0153fb0 0x7fddd00ca4f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0097c0 tx=0x7fddcc01c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49c4c15501d0373 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:47.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd4aa9640 1 -- 172.21.15.12:0/3076831271 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8001a00 con 0x7fddd0153fb0 2023-11-28T21:46:47.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd4aa9640 1 -- 172.21.15.12:0/3076831271 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc0a7050 con 0x7fddd0153fb0 2023-11-28T21:46:47.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd4aa9640 1 -- 172.21.15.12:0/3076831271 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0ab600 con 0x7fddd0153fb0 2023-11-28T21:46:47.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd6aad640 1 -- 172.21.15.12:0/3076831271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd0153fb0 msgr2=0x7fddd00ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:47.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd6aad640 1 --2- 172.21.15.12:0/3076831271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd0153fb0 0x7fddd00ca4f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0097c0 tx=0x7fddcc01c370 comp rx=0 tx=0).stop 2023-11-28T21:46:47.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd6aad640 1 -- 172.21.15.12:0/3076831271 shutdown_connections 2023-11-28T21:46:47.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd6aad640 1 --2- 172.21.15.12:0/3076831271 >> v2:172.21.15.144:3300/0 conn(0x7fddd015a980 0x7fddd00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:47.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd6aad640 1 --2- 172.21.15.12:0/3076831271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd0153fb0 0x7fddd00ca4f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:47.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd6aad640 1 -- 172.21.15.12:0/3076831271 >> 172.21.15.12:0/3076831271 conn(0x7fddd0071ce0 msgr2=0x7fddd00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:47.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd6aad640 1 -- 172.21.15.12:0/3076831271 shutdown_connections 2023-11-28T21:46:47.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.037+0000 7fddd6aad640 1 -- 172.21.15.12:0/3076831271 wait complete. 2023-11-28T21:46:47.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd6aad640 1 Processor -- start 2023-11-28T21:46:47.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd6aad640 1 -- start start 2023-11-28T21:46:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd6aad640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd00b3ad0 0x7fddd0158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd6aad640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd0153fb0 0x7fddd0158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd6aad640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddd015a980 0x7fddd01f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd6aad640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0146ec0 con 0x7fddd015a980 2023-11-28T21:46:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd6aad640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0157bc0 con 0x7fddd0153fb0 2023-11-28T21:46:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd6aad640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd015be30 con 0x7fddd00b3ad0 2023-11-28T21:46:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd62ac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddd015a980 0x7fddd01f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd62ac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddd015a980 0x7fddd01f6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50540/0 (socket says 172.21.15.12:50540) 2023-11-28T21:46:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd62ac640 1 -- 172.21.15.12:0/708834241 learned_addr learned my addr 172.21.15.12:0/708834241 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:47.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd5aab640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd00b3ad0 0x7fddd0158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:47.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd52aa640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd0153fb0 0x7fddd0158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:47.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd62ac640 1 -- 172.21.15.12:0/708834241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd00b3ad0 msgr2=0x7fddd0158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:47.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd62ac640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd00b3ad0 0x7fddd0158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:47.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd62ac640 1 -- 172.21.15.12:0/708834241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd0153fb0 msgr2=0x7fddd0158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:47.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd62ac640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd0153fb0 0x7fddd0158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:47.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd62ac640 1 -- 172.21.15.12:0/708834241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0009420 con 0x7fddd015a980 2023-11-28T21:46:47.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd62ac640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddd015a980 0x7fddd01f6180 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fddc8001840 tx=0x7fddc800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:47.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddc67fc640 1 -- 172.21.15.12:0/708834241 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8012400 con 0x7fddd015a980 2023-11-28T21:46:47.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddc67fc640 1 -- 172.21.15.12:0/708834241 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc8012560 con 0x7fddd015a980 2023-11-28T21:46:47.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddc67fc640 1 -- 172.21.15.12:0/708834241 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8012910 con 0x7fddd015a980 2023-11-28T21:46:47.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd6aad640 1 -- 172.21.15.12:0/708834241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddcc009420 con 0x7fddd015a980 2023-11-28T21:46:47.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.041+0000 7fddd6aad640 1 -- 172.21.15.12:0/708834241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd0157e60 con 0x7fddd015a980 2023-11-28T21:46:47.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.045+0000 7fddd6aad640 1 -- 172.21.15.12:0/708834241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd0074f60 con 0x7fddd015a980 2023-11-28T21:46:47.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.045+0000 7fddc67fc640 1 -- 172.21.15.12:0/708834241 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fddc8012a70 con 0x7fddd015a980 2023-11-28T21:46:47.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.045+0000 7fddc67fc640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fddb40664a0 0x7fddb4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:47.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.045+0000 7fddc67fc640 1 -- 172.21.15.12:0/708834241 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7093+0+0 (secure 0 0 0) 0x7fddc8084b10 con 0x7fddd015a980 2023-11-28T21:46:47.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.045+0000 7fddd5aab640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fddb40664a0 0x7fddb4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:47.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.045+0000 7fddd5aab640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fddb40664a0 0x7fddb4068960 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0ad000 tx=0x7fddcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:47.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.049+0000 7fddc67fc640 1 -- 172.21.15.12:0/708834241 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fddc80519a0 con 0x7fddd015a980 2023-11-28T21:46:47.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.237+0000 7fddd6aad640 1 -- 172.21.15.12:0/708834241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddd00cb810 con 0x7fddd015a980 2023-11-28T21:46:47.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.237+0000 7fddc67fc640 1 -- 172.21.15.12:0/708834241 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fddc8054fc0 con 0x7fddd015a980 2023-11-28T21:46:47.245 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:47.245 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6024,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:45:48.986753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_5871":{"gid":5871,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.12:6838/3474424840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":3474424840},{"type":"v1","addr":"172.21.15.12:6838","nonce":3474424840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:47.245 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 50 2023-11-28T21:46:47.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 -- 172.21.15.12:0/708834241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fddb40664a0 msgr2=0x7fddb4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:47.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fddb40664a0 0x7fddb4068960 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0ad000 tx=0x7fddcc008040 comp rx=0 tx=0).stop 2023-11-28T21:46:47.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 -- 172.21.15.12:0/708834241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddd015a980 msgr2=0x7fddd01f6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:47.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddd015a980 0x7fddd01f6180 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fddc8001840 tx=0x7fddc800a430 comp rx=0 tx=0).stop 2023-11-28T21:46:47.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 -- 172.21.15.12:0/708834241 shutdown_connections 2023-11-28T21:46:47.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fddb40664a0 0x7fddb4068960 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:47.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fddd015a980 0x7fddd01f6180 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:47.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd0153fb0 0x7fddd0158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:47.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 --2- 172.21.15.12:0/708834241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddd00b3ad0 0x7fddd0158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:47.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 -- 172.21.15.12:0/708834241 >> 172.21.15.12:0/708834241 conn(0x7fddd0071ce0 msgr2=0x7fddd00720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:47.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 -- 172.21.15.12:0/708834241 shutdown_connections 2023-11-28T21:46:47.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:47.241+0000 7fddd6aad640 1 -- 172.21.15.12:0/708834241 wait complete. 2023-11-28T21:46:47.266 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:46:49.267 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:49.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39cf9d640 1 Processor -- start 2023-11-28T21:46:49.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39cf9d640 1 -- start start 2023-11-28T21:46:49.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39cf9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc398150c20 0x7fc398156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39cf9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc39806e300 0x7fc3980c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39cf9d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc398067410 con 0x7fc39806cf40 2023-11-28T21:46:49.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39cf9d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc39806e090 con 0x7fc398150c20 2023-11-28T21:46:49.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39cf9d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc398157f30 con 0x7fc39806e300 2023-11-28T21:46:49.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc398150c20 0x7fc398156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc396d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc39806cf40 0x7fc398158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36248/0 (socket says 172.21.15.12:36248) 2023-11-28T21:46:49.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc396d9d640 1 -- 172.21.15.12:0/2358414523 learned_addr learned my addr 172.21.15.12:0/2358414523 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:46:49.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc395d9b640 1 --2- 172.21.15.12:0/2358414523 >> v2:172.21.15.144:3300/0 conn(0x7fc39806e300 0x7fc3980c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39659c640 1 -- 172.21.15.12:0/2358414523 >> v2:172.21.15.144:3300/0 conn(0x7fc39806e300 msgr2=0x7fc3980c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:49.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39659c640 1 --2- 172.21.15.12:0/2358414523 >> v2:172.21.15.144:3300/0 conn(0x7fc39806e300 0x7fc3980c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39659c640 1 -- 172.21.15.12:0/2358414523 >> v1:172.21.15.12:6789/0 conn(0x7fc39806cf40 legacy=0x7fc398158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39659c640 1 -- 172.21.15.12:0/2358414523 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc398151000 con 0x7fc398150c20 2023-11-28T21:46:49.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.413+0000 7fc39559a640 1 -- 172.21.15.12:0/2358414523 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303656942 0 0) 0x7fc398067410 con 0x7fc39806cf40 2023-11-28T21:46:49.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39659c640 1 --2- 172.21.15.12:0/2358414523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc398150c20 0x7fc398156910 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc3980c8e60 tx=0x7fc38801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bf9d8b7f055cab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:49.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39559a640 1 -- 172.21.15.12:0/2358414523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880ab600 con 0x7fc398150c20 2023-11-28T21:46:49.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39559a640 1 -- 172.21.15.12:0/2358414523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3880a7050 con 0x7fc398150c20 2023-11-28T21:46:49.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39559a640 1 -- 172.21.15.12:0/2358414523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880ab980 con 0x7fc398150c20 2023-11-28T21:46:49.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2358414523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc398150c20 msgr2=0x7fc398156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39cf9d640 1 --2- 172.21.15.12:0/2358414523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc398150c20 0x7fc398156910 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc3980c8e60 tx=0x7fc38801c460 comp rx=0 tx=0).stop 2023-11-28T21:46:49.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2358414523 shutdown_connections 2023-11-28T21:46:49.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39cf9d640 1 --2- 172.21.15.12:0/2358414523 >> v2:172.21.15.144:3300/0 conn(0x7fc39806e300 0x7fc3980c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39cf9d640 1 --2- 172.21.15.12:0/2358414523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc398150c20 0x7fc398156910 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2358414523 >> 172.21.15.12:0/2358414523 conn(0x7fc398066360 msgr2=0x7fc398066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:49.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2358414523 shutdown_connections 2023-11-28T21:46:49.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.417+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2358414523 wait complete. 2023-11-28T21:46:49.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39cf9d640 1 Processor -- start 2023-11-28T21:46:49.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39cf9d640 1 -- start start 2023-11-28T21:46:49.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39cf9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc39806cf40 0x7fc398070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39cf9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc39806e300 0x7fc3981560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39cf9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc398150c20 0x7fc398160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39cf9d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc398157220 con 0x7fc39806cf40 2023-11-28T21:46:49.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39cf9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc398155640 con 0x7fc39806e300 2023-11-28T21:46:49.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39cf9d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc398156c80 con 0x7fc398150c20 2023-11-28T21:46:49.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc39806cf40 0x7fc398070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc39806cf40 0x7fc398070c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50568/0 (socket says 172.21.15.12:50568) 2023-11-28T21:46:49.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39659c640 1 -- 172.21.15.12:0/2071908076 learned_addr learned my addr 172.21.15.12:0/2071908076 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:49.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc395d9b640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc39806e300 0x7fc3981560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc396d9d640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc398150c20 0x7fc398160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39659c640 1 -- 172.21.15.12:0/2071908076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc398150c20 msgr2=0x7fc398160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39659c640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc398150c20 0x7fc398160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39659c640 1 -- 172.21.15.12:0/2071908076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc39806e300 msgr2=0x7fc3981560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39659c640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc39806e300 0x7fc3981560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39659c640 1 -- 172.21.15.12:0/2071908076 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388009420 con 0x7fc39806cf40 2023-11-28T21:46:49.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc395d9b640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc39806e300 0x7fc3981560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:46:49.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39659c640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc39806cf40 0x7fc398070c20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ae040 tx=0x7fc38801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:49.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc386ffd640 1 -- 172.21.15.12:0/2071908076 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880b7650 con 0x7fc39806cf40 2023-11-28T21:46:49.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc386ffd640 1 -- 172.21.15.12:0/2071908076 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3880b77d0 con 0x7fc39806cf40 2023-11-28T21:46:49.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2071908076 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc398156f40 con 0x7fc39806cf40 2023-11-28T21:46:49.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc386ffd640 1 -- 172.21.15.12:0/2071908076 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880b7b80 con 0x7fc39806cf40 2023-11-28T21:46:49.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.421+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2071908076 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc398155bc0 con 0x7fc39806cf40 2023-11-28T21:46:49.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.425+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2071908076 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc398074f60 con 0x7fc39806cf40 2023-11-28T21:46:49.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.425+0000 7fc386ffd640 1 -- 172.21.15.12:0/2071908076 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc3880b7ce0 con 0x7fc39806cf40 2023-11-28T21:46:49.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.425+0000 7fc386ffd640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc3780664a0 0x7fc378068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.425+0000 7fc386ffd640 1 -- 172.21.15.12:0/2071908076 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7fc38811ecb0 con 0x7fc39806cf40 2023-11-28T21:46:49.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.425+0000 7fc395d9b640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc3780664a0 0x7fc378068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.425+0000 7fc395d9b640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc3780664a0 0x7fc378068960 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc39814deb0 tx=0x7fc38c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:49.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.429+0000 7fc386ffd640 1 -- 172.21.15.12:0/2071908076 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc3880eba40 con 0x7fc39806cf40 2023-11-28T21:46:49.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.621+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2071908076 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3980c90c0 con 0x7fc39806cf40 2023-11-28T21:46:49.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.621+0000 7fc386ffd640 1 -- 172.21.15.12:0/2071908076 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+5714 (secure 0 0 0) 0x7fc3880b0090 con 0x7fc39806cf40 2023-11-28T21:46:49.629 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:49.630 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:46:48.503189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 51 2023-11-28T21:46:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2071908076 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc3780664a0 msgr2=0x7fc378068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc3780664a0 0x7fc378068960 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc39814deb0 tx=0x7fc38c008040 comp rx=0 tx=0).stop 2023-11-28T21:46:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2071908076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc39806cf40 msgr2=0x7fc398070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc39806cf40 0x7fc398070c20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ae040 tx=0x7fc38801aca0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2071908076 shutdown_connections 2023-11-28T21:46:49.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc3780664a0 0x7fc378068960 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc398150c20 0x7fc398160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc39806e300 0x7fc3981560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 --2- 172.21.15.12:0/2071908076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc39806cf40 0x7fc398070c20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2071908076 >> 172.21.15.12:0/2071908076 conn(0x7fc398066360 msgr2=0x7fc3980ca140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:49.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2071908076 shutdown_connections 2023-11-28T21:46:49.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.625+0000 7fc39cf9d640 1 -- 172.21.15.12:0/2071908076 wait complete. 2023-11-28T21:46:49.650 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2023-11-28T21:46:49.650 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:46:49.650 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a233f5640 1 Processor -- start 2023-11-28T21:46:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a233f5640 1 -- start start 2023-11-28T21:46:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a233f5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a1c15b180 0x7f9a1c15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a233f5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c06d2a0 con 0x7f9a1c15d210 2023-11-28T21:46:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a233f5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c06d580 con 0x7f9a1c1540f0 2023-11-28T21:46:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a233f5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c06e200 con 0x7f9a1c15b180 2023-11-28T21:46:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a21992640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9a1c15d210 0x7f9a1c15d640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36260/0 (socket says 172.21.15.12:36260) 2023-11-28T21:46:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a21992640 1 -- 172.21.15.12:0/517676961 learned_addr learned my addr 172.21.15.12:0/517676961 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:46:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a20990640 1 --2- 172.21.15.12:0/517676961 >> v2:172.21.15.144:3300/0 conn(0x7f9a1c15b180 0x7f9a1c15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a13fff640 1 -- 172.21.15.12:0/517676961 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553927410 0 0) 0x7f9a1c06d2a0 con 0x7f9a1c15d210 2023-11-28T21:46:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a13fff640 1 -- 172.21.15.12:0/517676961 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a08003530 con 0x7f9a1c15d210 2023-11-28T21:46:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a13fff640 1 -- 172.21.15.12:0/517676961 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492208045 0 0) 0x7f9a1c06d580 con 0x7f9a1c1540f0 2023-11-28T21:46:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a13fff640 1 -- 172.21.15.12:0/517676961 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a1c06d2a0 con 0x7f9a1c1540f0 2023-11-28T21:46:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a20990640 1 -- 172.21.15.12:0/517676961 >> v1:172.21.15.144:6789/0 conn(0x7f9a1c1540f0 legacy=0x7f9a1c0c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a20990640 1 -- 172.21.15.12:0/517676961 >> v1:172.21.15.12:6789/0 conn(0x7f9a1c15d210 legacy=0x7f9a1c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.789+0000 7f9a20990640 1 -- 172.21.15.12:0/517676961 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c06d860 con 0x7f9a1c15b180 2023-11-28T21:46:49.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.793+0000 7f9a20990640 1 --2- 172.21.15.12:0/517676961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15b180 0x7f9a1c15b5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0097c0 tx=0x7f9a0c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29d76e02559180cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.793+0000 7f9a13fff640 1 -- 172.21.15.12:0/517676961 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0ab680 con 0x7f9a1c15b180 2023-11-28T21:46:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.793+0000 7f9a13fff640 1 -- 172.21.15.12:0/517676961 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c0ab7e0 con 0x7f9a1c15b180 2023-11-28T21:46:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.793+0000 7f9a13fff640 1 -- 172.21.15.12:0/517676961 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0abb70 con 0x7f9a1c15b180 2023-11-28T21:46:49.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.793+0000 7f9a233f5640 1 -- 172.21.15.12:0/517676961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15b180 msgr2=0x7f9a1c15b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.793+0000 7f9a233f5640 1 --2- 172.21.15.12:0/517676961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15b180 0x7f9a1c15b5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0097c0 tx=0x7f9a0c01c480 comp rx=0 tx=0).stop 2023-11-28T21:46:49.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 -- 172.21.15.12:0/517676961 shutdown_connections 2023-11-28T21:46:49.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 --2- 172.21.15.12:0/517676961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15b180 0x7f9a1c15b5a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 -- 172.21.15.12:0/517676961 >> 172.21.15.12:0/517676961 conn(0x7f9a1c066360 msgr2=0x7f9a1c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:49.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 -- 172.21.15.12:0/517676961 shutdown_connections 2023-11-28T21:46:49.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 -- 172.21.15.12:0/517676961 wait complete. 2023-11-28T21:46:49.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 Processor -- start 2023-11-28T21:46:49.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 -- start start 2023-11-28T21:46:49.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c1540f0 0x7f9a1c14e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c15b180 0x7f9a1c06fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15d210 0x7f9a1c146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1498c0 con 0x7f9a1c1540f0 2023-11-28T21:46:49.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c149d50 con 0x7f9a1c15d210 2023-11-28T21:46:49.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a233f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c072010 con 0x7f9a1c15b180 2023-11-28T21:46:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a21191640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c1540f0 0x7f9a1c14e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a21191640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c1540f0 0x7f9a1c14e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50570/0 (socket says 172.21.15.12:50570) 2023-11-28T21:46:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a21191640 1 -- 172.21.15.12:0/806849803 learned_addr learned my addr 172.21.15.12:0/806849803 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a21992640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15d210 0x7f9a1c146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a20990640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c15b180 0x7f9a1c06fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a21191640 1 -- 172.21.15.12:0/806849803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c15b180 msgr2=0x7f9a1c06fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a21191640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c15b180 0x7f9a1c06fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a21191640 1 -- 172.21.15.12:0/806849803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15d210 msgr2=0x7f9a1c146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a21191640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15d210 0x7f9a1c146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:49.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a21191640 1 -- 172.21.15.12:0/806849803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c009420 con 0x7f9a1c1540f0 2023-11-28T21:46:49.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a21191640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c1540f0 0x7f9a1c14e9b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800c490 tx=0x7f9a1800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.797+0000 7f9a11ffb640 1 -- 172.21.15.12:0/806849803 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a180052d0 con 0x7f9a1c1540f0 2023-11-28T21:46:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.801+0000 7f9a11ffb640 1 -- 172.21.15.12:0/806849803 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a18005430 con 0x7f9a1c1540f0 2023-11-28T21:46:49.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.801+0000 7f9a11ffb640 1 -- 172.21.15.12:0/806849803 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18012870 con 0x7f9a1c1540f0 2023-11-28T21:46:49.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.801+0000 7f9a233f5640 1 -- 172.21.15.12:0/806849803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c1492b0 con 0x7f9a1c1540f0 2023-11-28T21:46:49.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.801+0000 7f9a233f5640 1 -- 172.21.15.12:0/806849803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c149550 con 0x7f9a1c1540f0 2023-11-28T21:46:49.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.801+0000 7f9a233f5640 1 -- 172.21.15.12:0/806849803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c074f60 con 0x7f9a1c1540f0 2023-11-28T21:46:49.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.801+0000 7f9a11ffb640 1 -- 172.21.15.12:0/806849803 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9a180129d0 con 0x7f9a1c1540f0 2023-11-28T21:46:49.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.801+0000 7f9a11ffb640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a000664a0 0x7f9a00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:49.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.801+0000 7f9a11ffb640 1 -- 172.21.15.12:0/806849803 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f9a18084000 con 0x7f9a1c1540f0 2023-11-28T21:46:49.811 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.801+0000 7f9a20990640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a000664a0 0x7f9a00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:49.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.805+0000 7f9a20990640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a000664a0 0x7f9a00068960 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0af000 tx=0x7f9a0c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:49.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:49.805+0000 7f9a11ffb640 1 -- 172.21.15.12:0/806849803 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a18050ed0 con 0x7f9a1c1540f0 2023-11-28T21:46:50.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.009+0000 7f9a233f5640 1 -- 172.21.15.12:0/806849803 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a1c0c4ac0 con 0x7f9a1c1540f0 2023-11-28T21:46:50.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.009+0000 7f9a11ffb640 1 -- 172.21.15.12:0/806849803 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f9a180544f0 con 0x7f9a1c1540f0 2023-11-28T21:46:50.016 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:50.016 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:46:48.503189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:50.017 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 51 2023-11-28T21:46:50.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 -- 172.21.15.12:0/806849803 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a000664a0 msgr2=0x7f9a00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a000664a0 0x7f9a00068960 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0af000 tx=0x7f9a0c0a7000 comp rx=0 tx=0).stop 2023-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 -- 172.21.15.12:0/806849803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c1540f0 msgr2=0x7f9a1c14e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c1540f0 0x7f9a1c14e9b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800c490 tx=0x7f9a1800c950 comp rx=0 tx=0).stop 2023-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 -- 172.21.15.12:0/806849803 shutdown_connections 2023-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a000664a0 0x7f9a00068960 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15d210 0x7f9a1c146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c15b180 0x7f9a1c06fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 --2- 172.21.15.12:0/806849803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a1c1540f0 0x7f9a1c14e9b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 -- 172.21.15.12:0/806849803 >> 172.21.15.12:0/806849803 conn(0x7f9a1c066360 msgr2=0x7f9a1c0c3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:50.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 -- 172.21.15.12:0/806849803 shutdown_connections 2023-11-28T21:46:50.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:50.013+0000 7f9a233f5640 1 -- 172.21.15.12:0/806849803 wait complete. 2023-11-28T21:46:50.039 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 51, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 51, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6024": { "addr": "172.21.15.12:6839/2705421041", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 2705421041, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 2705421041, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6024, "incarnation": 51, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 48, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:46:48.503189+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 6024 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:46:52.040 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.185+0000 7f46495d1640 1 Processor -- start 2023-11-28T21:46:52.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46495d1640 1 -- start start 2023-11-28T21:46:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46495d1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f464406e300 0x7f4644156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46495d1640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f464406cf40 0x7f4644158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46495d1640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4644067410 con 0x7f4644150c20 2023-11-28T21:46:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46495d1640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464406e090 con 0x7f464406cf40 2023-11-28T21:46:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46495d1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4644157f30 con 0x7f464406e300 2023-11-28T21:46:52.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f4642ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4644150c20 0x7f46440c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:36270/0 (socket says 172.21.15.12:36270) 2023-11-28T21:46:52.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f4642ffd640 1 -- 172.21.15.12:0/2416161823 learned_addr learned my addr 172.21.15.12:0/2416161823 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:46:52.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46437fe640 1 --2- 172.21.15.12:0/2416161823 >> v2:172.21.15.144:3300/0 conn(0x7f464406cf40 0x7f4644158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:52.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f463bfff640 1 --2- 172.21.15.12:0/2416161823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f464406e300 0x7f4644156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46427fc640 1 -- 172.21.15.12:0/2416161823 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730662831 0 0) 0x7f4644067410 con 0x7f4644150c20 2023-11-28T21:46:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46427fc640 1 -- 172.21.15.12:0/2416161823 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4630003530 con 0x7f4644150c20 2023-11-28T21:46:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46437fe640 1 -- 172.21.15.12:0/2416161823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f464406e300 msgr2=0x7f4644156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46427fc640 1 -- 172.21.15.12:0/2416161823 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187009081 0 0) 0x7f4630003530 con 0x7f4644150c20 2023-11-28T21:46:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46437fe640 1 --2- 172.21.15.12:0/2416161823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f464406e300 0x7f4644156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46437fe640 1 -- 172.21.15.12:0/2416161823 >> v1:172.21.15.12:6789/0 conn(0x7f4644150c20 legacy=0x7f46440c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:52.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.189+0000 7f46437fe640 1 -- 172.21.15.12:0/2416161823 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4644151000 con 0x7f464406cf40 2023-11-28T21:46:52.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46437fe640 1 --2- 172.21.15.12:0/2416161823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464406cf40 0x7f4644158410 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f463c009d90 tx=0x7f463c01c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c981a351160f0799 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:52.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46427fc640 1 -- 172.21.15.12:0/2416161823 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634001680 con 0x7f464406cf40 2023-11-28T21:46:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46427fc640 1 -- 172.21.15.12:0/2416161823 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c0a5560 con 0x7f464406cf40 2023-11-28T21:46:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46427fc640 1 -- 172.21.15.12:0/2416161823 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c0a58d0 con 0x7f464406cf40 2023-11-28T21:46:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46495d1640 1 -- 172.21.15.12:0/2416161823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464406cf40 msgr2=0x7f4644158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:52.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46495d1640 1 --2- 172.21.15.12:0/2416161823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464406cf40 0x7f4644158410 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f463c009d90 tx=0x7f463c01c870 comp rx=0 tx=0).stop 2023-11-28T21:46:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46495d1640 1 -- 172.21.15.12:0/2416161823 shutdown_connections 2023-11-28T21:46:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46495d1640 1 --2- 172.21.15.12:0/2416161823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464406cf40 0x7f4644158410 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46495d1640 1 --2- 172.21.15.12:0/2416161823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f464406e300 0x7f4644156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46495d1640 1 -- 172.21.15.12:0/2416161823 >> 172.21.15.12:0/2416161823 conn(0x7f4644066360 msgr2=0x7f4644066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46495d1640 1 -- 172.21.15.12:0/2416161823 shutdown_connections 2023-11-28T21:46:52.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.193+0000 7f46495d1640 1 -- 172.21.15.12:0/2416161823 wait complete. 2023-11-28T21:46:52.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 Processor -- start 2023-11-28T21:46:52.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 -- start start 2023-11-28T21:46:52.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464406cf40 0x7f46440c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f464406e300 0x7f464414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644150c20 0x7f464414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46440c43a0 con 0x7f4644150c20 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464414eb10 con 0x7f464406cf40 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46440c5670 con 0x7f464406e300 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644150c20 0x7f464414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644150c20 0x7f464414df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50572/0 (socket says 172.21.15.12:50572) 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46437fe640 1 -- 172.21.15.12:0/878694503 learned_addr learned my addr 172.21.15.12:0/878694503 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f463bfff640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f464406e300 0x7f464414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f4642ffd640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464406cf40 0x7f46440c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46437fe640 1 -- 172.21.15.12:0/878694503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f464406e300 msgr2=0x7f464414d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:52.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46437fe640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f464406e300 0x7f464414d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46437fe640 1 -- 172.21.15.12:0/878694503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464406cf40 msgr2=0x7f46440c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46437fe640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464406cf40 0x7f46440c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46437fe640 1 -- 172.21.15.12:0/878694503 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c009a70 con 0x7f4644150c20 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46437fe640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644150c20 0x7f464414df10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f463c01af20 tx=0x7f463c019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f463b7fe640 1 -- 172.21.15.12:0/878694503 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c0bd070 con 0x7f4644150c20 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f463b7fe640 1 -- 172.21.15.12:0/878694503 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c0a58e0 con 0x7f4644150c20 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f463b7fe640 1 -- 172.21.15.12:0/878694503 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c0a5c50 con 0x7f4644150c20 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 -- 172.21.15.12:0/878694503 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46440c4070 con 0x7f4644150c20 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 -- 172.21.15.12:0/878694503 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4644156880 con 0x7f4644150c20 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.197+0000 7f46495d1640 1 -- 172.21.15.12:0/878694503 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46440c4510 con 0x7f4644150c20 2023-11-28T21:46:52.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.201+0000 7f463b7fe640 1 -- 172.21.15.12:0/878694503 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f463c004040 con 0x7f4644150c20 2023-11-28T21:46:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.201+0000 7f463b7fe640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4624066470 0x7f4624068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.201+0000 7f463b7fe640 1 -- 172.21.15.12:0/878694503 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f463c11eda0 con 0x7f4644150c20 2023-11-28T21:46:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.201+0000 7f4642ffd640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4624066470 0x7f4624068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:52.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.201+0000 7f4642ffd640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4624066470 0x7f4624068930 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4634001ec0 tx=0x7f46340016e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:52.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.205+0000 7f463b7fe640 1 -- 172.21.15.12:0/878694503 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f463c0ebc70 con 0x7f4644150c20 2023-11-28T21:46:52.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.393+0000 7f46495d1640 1 -- 172.21.15.12:0/878694503 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46440c44e0 con 0x7f4644150c20 2023-11-28T21:46:52.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.393+0000 7f463b7fe640 1 -- 172.21.15.12:0/878694503 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f463c0b2070 con 0x7f4644150c20 2023-11-28T21:46:52.402 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:52.402 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:46:48.503189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:52.403 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 51 2023-11-28T21:46:52.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 -- 172.21.15.12:0/878694503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4624066470 msgr2=0x7f4624068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:52.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4624066470 0x7f4624068930 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4634001ec0 tx=0x7f46340016e0 comp rx=0 tx=0).stop 2023-11-28T21:46:52.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 -- 172.21.15.12:0/878694503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644150c20 msgr2=0x7f464414df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:52.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644150c20 0x7f464414df10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f463c01af20 tx=0x7f463c019610 comp rx=0 tx=0).stop 2023-11-28T21:46:52.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 -- 172.21.15.12:0/878694503 shutdown_connections 2023-11-28T21:46:52.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4624066470 0x7f4624068930 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:52.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4644150c20 0x7f464414df10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:52.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f464406e300 0x7f464414d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:52.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 --2- 172.21.15.12:0/878694503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f464406cf40 0x7f46440c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:52.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 -- 172.21.15.12:0/878694503 >> 172.21.15.12:0/878694503 conn(0x7f4644066360 msgr2=0x7f464406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:52.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 -- 172.21.15.12:0/878694503 shutdown_connections 2023-11-28T21:46:52.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:52.397+0000 7f46495d1640 1 -- 172.21.15.12:0/878694503 wait complete. 2023-11-28T21:46:52.425 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:46:54.427 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:54.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca5f3b640 1 Processor -- start 2023-11-28T21:46:54.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca5f3b640 1 -- start start 2023-11-28T21:46:54.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca5f3b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7ca0153fb0 0x7f7ca00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca5f3b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7ca015a980 0x7f7ca00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca5f3b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca00b3ad0 0x7f7ca00b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca5f3b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca00b2b60 con 0x7f7ca0153fb0 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca5f3b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca015e1b0 con 0x7f7ca015a980 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca5f3b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00b44a0 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca4f39640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7ca0153fb0 0x7f7ca00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca4f39640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7ca0153fb0 0x7f7ca00ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50586/0 (socket says 172.21.15.12:50586) 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca4f39640 1 -- 172.21.15.12:0/3758901031 learned_addr learned my addr 172.21.15.12:0/3758901031 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca573a640 1 --2- 172.21.15.12:0/3758901031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca00b3ad0 0x7f7ca00b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7c9ffff640 1 --2- 172.21.15.12:0/3758901031 >> v2:172.21.15.144:3300/0 conn(0x7f7ca015a980 0x7f7ca00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca4f39640 1 -- 172.21.15.12:0/3758901031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca00b3ad0 msgr2=0x7f7ca00b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca4f39640 1 --2- 172.21.15.12:0/3758901031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca00b3ad0 0x7f7ca00b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca4f39640 1 -- 172.21.15.12:0/3758901031 >> v2:172.21.15.144:3300/0 conn(0x7f7ca015a980 msgr2=0x7f7ca00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:54.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca4f39640 1 --2- 172.21.15.12:0/3758901031 >> v2:172.21.15.144:3300/0 conn(0x7f7ca015a980 0x7f7ca00caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca4f39640 1 -- 172.21.15.12:0/3758901031 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca015d390 con 0x7f7ca0153fb0 2023-11-28T21:46:54.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.569+0000 7f7ca573a640 1 --2- 172.21.15.12:0/3758901031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca00b3ad0 0x7f7ca00b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:46:54.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7ca4f39640 1 --2- 172.21.15.12:0/3758901031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca0153fb0 0x7f7ca00ca4f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00c9fa0 tx=0x7f7c9401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861fef929cd58cf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7c9f7fe640 1 -- 172.21.15.12:0/3758901031 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940aa5f0 con 0x7f7ca0153fb0 2023-11-28T21:46:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7c9f7fe640 1 -- 172.21.15.12:0/3758901031 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c940aa750 con 0x7f7ca0153fb0 2023-11-28T21:46:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7c9f7fe640 1 -- 172.21.15.12:0/3758901031 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940aaae0 con 0x7f7ca0153fb0 2023-11-28T21:46:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3758901031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca0153fb0 msgr2=0x7f7ca00ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7ca5f3b640 1 --2- 172.21.15.12:0/3758901031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca0153fb0 0x7f7ca00ca4f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00c9fa0 tx=0x7f7c9401c4d0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3758901031 shutdown_connections 2023-11-28T21:46:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7ca5f3b640 1 --2- 172.21.15.12:0/3758901031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca00b3ad0 0x7f7ca00b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7ca5f3b640 1 --2- 172.21.15.12:0/3758901031 >> v2:172.21.15.144:3300/0 conn(0x7f7ca015a980 0x7f7ca00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7ca5f3b640 1 --2- 172.21.15.12:0/3758901031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca0153fb0 0x7f7ca00ca4f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3758901031 >> 172.21.15.12:0/3758901031 conn(0x7f7ca0071ce0 msgr2=0x7f7ca00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3758901031 shutdown_connections 2023-11-28T21:46:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.573+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3758901031 wait complete. 2023-11-28T21:46:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca5f3b640 1 Processor -- start 2023-11-28T21:46:54.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca5f3b640 1 -- start start 2023-11-28T21:46:54.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca5f3b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca00b3ad0 0x7f7ca01f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:54.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca5f3b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca0153fb0 0x7f7ca0152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:54.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca5f3b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ca015a980 0x7f7ca015b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca5f3b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca0148e00 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca5f3b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca01f3c90 con 0x7f7ca015a980 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca5f3b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca00c5120 con 0x7f7ca0153fb0 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca4f39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca00b3ad0 0x7f7ca01f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca4f39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca00b3ad0 0x7f7ca01f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50596/0 (socket says 172.21.15.12:50596) 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca4f39640 1 -- 172.21.15.12:0/3351650133 learned_addr learned my addr 172.21.15.12:0/3351650133 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7c9ffff640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca0153fb0 0x7f7ca0152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca573a640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ca015a980 0x7f7ca015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca4f39640 1 -- 172.21.15.12:0/3351650133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca0153fb0 msgr2=0x7f7ca0152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca4f39640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca0153fb0 0x7f7ca0152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca4f39640 1 -- 172.21.15.12:0/3351650133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ca015a980 msgr2=0x7f7ca015b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca4f39640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ca015a980 0x7f7ca015b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca4f39640 1 -- 172.21.15.12:0/3351650133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c940094d0 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.577+0000 7f7ca4f39640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca00b3ad0 0x7f7ca01f3750 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7c9401ab80 tx=0x7f7c9401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:54.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.581+0000 7f7c9d7fa640 1 -- 172.21.15.12:0/3351650133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940aae50 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.581+0000 7f7c9d7fa640 1 -- 172.21.15.12:0/3351650133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c940bd3f0 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.581+0000 7f7c9d7fa640 1 -- 172.21.15.12:0/3351650133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940bd7a0 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.581+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3351650133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca00c4550 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.581+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3351650133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca00c4ad0 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.581+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3351650133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca0074f60 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.581+0000 7f7c9d7fa640 1 -- 172.21.15.12:0/3351650133 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7c940b33f0 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.581+0000 7f7c9d7fa640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7c84066470 0x7f7c84068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.581+0000 7f7c9d7fa640 1 -- 172.21.15.12:0/3351650133 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f7c940a7070 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.581+0000 7f7c9ffff640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7c84066470 0x7f7c84068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.585+0000 7f7c9ffff640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7c84066470 0x7f7c84068930 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7ca015d2c0 tx=0x7f7c8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.585+0000 7f7c9d7fa640 1 -- 172.21.15.12:0/3351650133 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7c940eb7e0 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.789+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3351650133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ca014e040 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.789+0000 7f7c9d7fa640 1 -- 172.21.15.12:0/3351650133 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f7c940eee00 con 0x7f7ca00b3ad0 2023-11-28T21:46:54.797 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:54.797 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:46:48.503189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:54.797 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 51 2023-11-28T21:46:54.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3351650133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7c84066470 msgr2=0x7f7c84068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:54.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7c84066470 0x7f7c84068930 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7ca015d2c0 tx=0x7f7c8c008040 comp rx=0 tx=0).stop 2023-11-28T21:46:54.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3351650133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca00b3ad0 msgr2=0x7f7ca01f3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:54.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca00b3ad0 0x7f7ca01f3750 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7c9401ab80 tx=0x7f7c9401ad10 comp rx=0 tx=0).stop 2023-11-28T21:46:54.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3351650133 shutdown_connections 2023-11-28T21:46:54.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7c84066470 0x7f7c84068930 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ca015a980 0x7f7ca015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ca0153fb0 0x7f7ca0152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 --2- 172.21.15.12:0/3351650133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ca00b3ad0 0x7f7ca01f3750 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:54.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3351650133 >> 172.21.15.12:0/3351650133 conn(0x7f7ca0071ce0 msgr2=0x7f7ca014a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:54.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3351650133 shutdown_connections 2023-11-28T21:46:54.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:54.793+0000 7f7ca5f3b640 1 -- 172.21.15.12:0/3351650133 wait complete. 2023-11-28T21:46:54.828 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:46:56.828 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:56.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1edd972640 1 Processor -- start 2023-11-28T21:46:56.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1edd972640 1 -- start start 2023-11-28T21:46:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1edd972640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed8153e00 0x7f1ed80ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1edd972640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1ed80cba30 0x7f1ed806ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1edd972640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1ed806b1b0 0x7f1ed80caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1edd972640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed806a850 con 0x7f1ed806b1b0 2023-11-28T21:46:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1edd972640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed80b2670 con 0x7f1ed80cba30 2023-11-28T21:46:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1edd972640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80c46d0 con 0x7f1ed8153e00 2023-11-28T21:46:56.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1ed77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1ed806b1b0 0x7f1ed80caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:56.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1ed77fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1ed806b1b0 0x7f1ed80caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43878/0 (socket says 172.21.15.12:43878) 2023-11-28T21:46:56.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1ed77fe640 1 -- 172.21.15.12:0/4258752231 learned_addr learned my addr 172.21.15.12:0/4258752231 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:56.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1ed6ffd640 1 --2- 172.21.15.12:0/4258752231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed8153e00 0x7f1ed80ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:56.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1ed67fc640 1 --2- 172.21.15.12:0/4258752231 >> v2:172.21.15.144:3300/0 conn(0x7f1ed80cba30 0x7f1ed806ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:56.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1ed77fe640 1 -- 172.21.15.12:0/4258752231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed8153e00 msgr2=0x7f1ed80ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:56.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1ed77fe640 1 --2- 172.21.15.12:0/4258752231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed8153e00 0x7f1ed80ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:56.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1ed77fe640 1 -- 172.21.15.12:0/4258752231 >> v2:172.21.15.144:3300/0 conn(0x7f1ed80cba30 msgr2=0x7f1ed806ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:56.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1ed77fe640 1 --2- 172.21.15.12:0/4258752231 >> v2:172.21.15.144:3300/0 conn(0x7f1ed80cba30 0x7f1ed806ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:56.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.977+0000 7f1ed77fe640 1 -- 172.21.15.12:0/4258752231 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed815ab40 con 0x7f1ed806b1b0 2023-11-28T21:46:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1ed77fe640 1 --2- 172.21.15.12:0/4258752231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed806b1b0 0x7f1ed80caec0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1ed806d8a0 tx=0x7f1ed001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea9428579f9f961 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1ed5ffb640 1 -- 172.21.15.12:0/4258752231 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00af400 con 0x7f1ed806b1b0 2023-11-28T21:46:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1ed5ffb640 1 -- 172.21.15.12:0/4258752231 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed00ab050 con 0x7f1ed806b1b0 2023-11-28T21:46:56.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1ed5ffb640 1 -- 172.21.15.12:0/4258752231 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00af780 con 0x7f1ed806b1b0 2023-11-28T21:46:56.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1edd972640 1 -- 172.21.15.12:0/4258752231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed806b1b0 msgr2=0x7f1ed80caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:56.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1edd972640 1 --2- 172.21.15.12:0/4258752231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed806b1b0 0x7f1ed80caec0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1ed806d8a0 tx=0x7f1ed001f1c0 comp rx=0 tx=0).stop 2023-11-28T21:46:56.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1edd972640 1 -- 172.21.15.12:0/4258752231 shutdown_connections 2023-11-28T21:46:56.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1edd972640 1 --2- 172.21.15.12:0/4258752231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed806b1b0 0x7f1ed80caec0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:56.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1edd972640 1 --2- 172.21.15.12:0/4258752231 >> v2:172.21.15.144:3300/0 conn(0x7f1ed80cba30 0x7f1ed806ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:56.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1edd972640 1 --2- 172.21.15.12:0/4258752231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed8153e00 0x7f1ed80ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:56.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1edd972640 1 -- 172.21.15.12:0/4258752231 >> 172.21.15.12:0/4258752231 conn(0x7f1ed80cf4e0 msgr2=0x7f1ed80cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:56.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1edd972640 1 -- 172.21.15.12:0/4258752231 shutdown_connections 2023-11-28T21:46:56.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.981+0000 7f1edd972640 1 -- 172.21.15.12:0/4258752231 wait complete. 2023-11-28T21:46:56.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1edd972640 1 Processor -- start 2023-11-28T21:46:56.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1edd972640 1 -- start start 2023-11-28T21:46:56.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1edd972640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed806b1b0 0x7f1ed80c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:56.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1edd972640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed80cba30 0x7f1ed81e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:56.993 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1edd972640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed8153e00 0x7f1ed8149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1edd972640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed815d070 con 0x7f1ed80cba30 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1edd972640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80c93c0 con 0x7f1ed8153e00 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1edd972640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80c6d20 con 0x7f1ed806b1b0 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed67fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed80cba30 0x7f1ed81e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed67fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed80cba30 0x7f1ed81e68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43890/0 (socket says 172.21.15.12:43890) 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed67fc640 1 -- 172.21.15.12:0/3454719218 learned_addr learned my addr 172.21.15.12:0/3454719218 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed6ffd640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed806b1b0 0x7f1ed80c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed77fe640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed8153e00 0x7f1ed8149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed67fc640 1 -- 172.21.15.12:0/3454719218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed806b1b0 msgr2=0x7f1ed80c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed67fc640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed806b1b0 0x7f1ed80c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed67fc640 1 -- 172.21.15.12:0/3454719218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed8153e00 msgr2=0x7f1ed8149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:56.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed67fc640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed8153e00 0x7f1ed8149d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:56.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed67fc640 1 -- 172.21.15.12:0/3454719218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0009b30 con 0x7f1ed80cba30 2023-11-28T21:46:56.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ed67fc640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed80cba30 0x7f1ed81e68e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00e8b0 tx=0x7f1ecc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:56.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ebffff640 1 -- 172.21.15.12:0/3454719218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00c940 con 0x7f1ed80cba30 2023-11-28T21:46:56.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ebffff640 1 -- 172.21.15.12:0/3454719218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ecc00caa0 con 0x7f1ed80cba30 2023-11-28T21:46:56.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1edd972640 1 -- 172.21.15.12:0/3454719218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed815c130 con 0x7f1ed80cba30 2023-11-28T21:46:56.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1ebffff640 1 -- 172.21.15.12:0/3454719218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00ce30 con 0x7f1ed80cba30 2023-11-28T21:46:56.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.985+0000 7f1edd972640 1 -- 172.21.15.12:0/3454719218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed815c6d0 con 0x7f1ed80cba30 2023-11-28T21:46:56.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.989+0000 7f1edd972640 1 -- 172.21.15.12:0/3454719218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed8074f60 con 0x7f1ed80cba30 2023-11-28T21:46:56.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.989+0000 7f1ebffff640 1 -- 172.21.15.12:0/3454719218 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1ecc013470 con 0x7f1ed80cba30 2023-11-28T21:46:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.993+0000 7f1ebffff640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ec4066470 0x7f1ec4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.993+0000 7f1ebffff640 1 -- 172.21.15.12:0/3454719218 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f1ecc084e90 con 0x7f1ed80cba30 2023-11-28T21:46:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.993+0000 7f1ed6ffd640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ec4066470 0x7f1ec4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.993+0000 7f1ed6ffd640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ec4066470 0x7f1ec4068930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009af0 tx=0x7f1ec8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:56.993+0000 7f1ebffff640 1 -- 172.21.15.12:0/3454719218 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1ecc051d30 con 0x7f1ed80cba30 2023-11-28T21:46:57.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 -- 172.21.15.12:0/3454719218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ed8151410 con 0x7f1ed80cba30 2023-11-28T21:46:57.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1ebffff640 1 -- 172.21.15.12:0/3454719218 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f1ecc021020 con 0x7f1ed80cba30 2023-11-28T21:46:57.196 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:57.196 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:46:48.503189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:57.196 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 51 2023-11-28T21:46:57.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 -- 172.21.15.12:0/3454719218 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ec4066470 msgr2=0x7f1ec4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:57.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ec4066470 0x7f1ec4068930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009af0 tx=0x7f1ec8008040 comp rx=0 tx=0).stop 2023-11-28T21:46:57.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 -- 172.21.15.12:0/3454719218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed80cba30 msgr2=0x7f1ed81e68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:57.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed80cba30 0x7f1ed81e68e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00e8b0 tx=0x7f1ecc00ed70 comp rx=0 tx=0).stop 2023-11-28T21:46:57.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 -- 172.21.15.12:0/3454719218 shutdown_connections 2023-11-28T21:46:57.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ec4066470 0x7f1ec4068930 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:57.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed8153e00 0x7f1ed8149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:57.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1ed80cba30 0x7f1ed81e68e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:57.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 --2- 172.21.15.12:0/3454719218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed806b1b0 0x7f1ed80c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:57.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.189+0000 7f1edd972640 1 -- 172.21.15.12:0/3454719218 >> 172.21.15.12:0/3454719218 conn(0x7f1ed80cf4e0 msgr2=0x7f1ed8152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:57.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.193+0000 7f1edd972640 1 -- 172.21.15.12:0/3454719218 shutdown_connections 2023-11-28T21:46:57.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:57.193+0000 7f1edd972640 1 -- 172.21.15.12:0/3454719218 wait complete. 2023-11-28T21:46:57.218 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:46:59.219 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:46:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5acc3f5640 1 Processor -- start 2023-11-28T21:46:59.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5acc3f5640 1 -- start start 2023-11-28T21:46:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5acc3f5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5ac4154100 0x7f5ac415d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5acc3f5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac415e270 con 0x7f5ac40c9fd0 2023-11-28T21:46:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5acc3f5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac40c4e60 con 0x7f5ac40ca3f0 2023-11-28T21:46:59.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5acc3f5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac41544e0 con 0x7f5ac4154100 2023-11-28T21:46:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac9990640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5ac40c9fd0 0x7f5ac406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34730/0 (socket says 172.21.15.12:34730) 2023-11-28T21:46:59.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac9990640 1 -- 172.21.15.12:0/189253978 learned_addr learned my addr 172.21.15.12:0/189253978 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:46:59.378 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5aca191640 1 --2- 172.21.15.12:0/189253978 >> v2:172.21.15.144:3300/0 conn(0x7f5ac4154100 0x7f5ac415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346212742 0 0) 0x7f5ac415e270 con 0x7f5ac40c9fd0 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ab0003530 con 0x7f5ac40c9fd0 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4268162714 0 0) 0x7f5ab0003530 con 0x7f5ac40c9fd0 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 >> v2:172.21.15.144:3300/0 conn(0x7f5ac4154100 msgr2=0x7f5ac415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 --2- 172.21.15.12:0/189253978 >> v2:172.21.15.144:3300/0 conn(0x7f5ac4154100 0x7f5ac415d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 >> v1:172.21.15.144:6789/0 conn(0x7f5ac40ca3f0 legacy=0x7f5ac40c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac415d9d0 con 0x7f5ac40c9fd0 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f5ab4001680 con 0x7f5ac40c9fd0 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 >> v1:172.21.15.12:6789/0 conn(0x7f5ac40c9fd0 legacy=0x7f5ac406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab00039f0 0x7f5ab0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:59.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab00047d0 0x7f5ab0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab0005190 0x7f5ab000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac41544e0 con 0x7f5ab00047d0 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab000c0c0 con 0x7f5ab0005190 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab000c280 con 0x7f5ab00039f0 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5ac9990640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab00047d0 0x7f5ab0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5aca191640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab00039f0 0x7f5ab0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.369+0000 7f5aca992640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab0005190 0x7f5ab000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5ac9990640 1 -- 172.21.15.12:0/189253978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab00039f0 msgr2=0x7f5ab0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5ac9990640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab00039f0 0x7f5ab0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5ac9990640 1 -- 172.21.15.12:0/189253978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab0005190 msgr2=0x7f5ab000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5ac9990640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab0005190 0x7f5ab000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5ac9990640 1 -- 172.21.15.12:0/189253978 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab0003530 con 0x7f5ab00047d0 2023-11-28T21:46:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5aca992640 1 -- 172.21.15.12:0/189253978 reap_dead start 2023-11-28T21:46:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5ac9990640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab00047d0 0x7f5ab0004c50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400c240 tx=0x7f5ab401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b079f047e9370f06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4005ae0 con 0x7f5ab00047d0 2023-11-28T21:46:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab40ab070 con 0x7f5ab00047d0 2023-11-28T21:46:59.381 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5ac918f640 1 -- 172.21.15.12:0/189253978 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4007070 con 0x7f5ab00047d0 2023-11-28T21:46:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5acc3f5640 1 -- 172.21.15.12:0/189253978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab00047d0 msgr2=0x7f5ab0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.373+0000 7f5acc3f5640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab00047d0 0x7f5ab0004c50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400c240 tx=0x7f5ab401f3b0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 -- 172.21.15.12:0/189253978 shutdown_connections 2023-11-28T21:46:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 --2- 172.21.15.12:0/189253978 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ab00047d0 0x7f5ab0004c50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 -- 172.21.15.12:0/189253978 >> 172.21.15.12:0/189253978 conn(0x7f5ac406c690 msgr2=0x7f5ac4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 -- 172.21.15.12:0/189253978 shutdown_connections 2023-11-28T21:46:59.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 -- 172.21.15.12:0/189253978 wait complete. 2023-11-28T21:46:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 Processor -- start 2023-11-28T21:46:59.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 -- start start 2023-11-28T21:46:59.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab00047d0 0x7f5ac415cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4155390 0x7f5ac4151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ac415c4a0 0x7f5ac4148020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac4143b70 con 0x7f5ac4155390 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac4148e20 con 0x7f5ac415c4a0 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5acc3f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac41ef260 con 0x7f5ab00047d0 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5ac9990640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4155390 0x7f5ac4151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5ac9990640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4155390 0x7f5ac4151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43914/0 (socket says 172.21.15.12:43914) 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5ac9990640 1 -- 172.21.15.12:0/2626230408 learned_addr learned my addr 172.21.15.12:0/2626230408 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5aca191640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab00047d0 0x7f5ac415cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5aca992640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ac415c4a0 0x7f5ac4148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5ac9990640 1 -- 172.21.15.12:0/2626230408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab00047d0 msgr2=0x7f5ac415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:59.386 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5ac9990640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab00047d0 0x7f5ac415cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5ac9990640 1 -- 172.21.15.12:0/2626230408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ac415c4a0 msgr2=0x7f5ac4148020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5ac9990640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ac415c4a0 0x7f5ac4148020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5ac9990640 1 -- 172.21.15.12:0/2626230408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac41ef520 con 0x7f5ac4155390 2023-11-28T21:46:59.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.377+0000 7f5ac9990640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4155390 0x7f5ac4151000 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4001940 tx=0x7f5ab0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.381+0000 7f5abaffd640 1 -- 172.21.15.12:0/2626230408 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4004a70 con 0x7f5ac4155390 2023-11-28T21:46:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.381+0000 7f5abaffd640 1 -- 172.21.15.12:0/2626230408 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4004bd0 con 0x7f5ac4155390 2023-11-28T21:46:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.381+0000 7f5abaffd640 1 -- 172.21.15.12:0/2626230408 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4007070 con 0x7f5ac4155390 2023-11-28T21:46:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.381+0000 7f5acc3f5640 1 -- 172.21.15.12:0/2626230408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac41ef800 con 0x7f5ac4155390 2023-11-28T21:46:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.381+0000 7f5acc3f5640 1 -- 172.21.15.12:0/2626230408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac41efdc0 con 0x7f5ac4155390 2023-11-28T21:46:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.381+0000 7f5acc3f5640 1 -- 172.21.15.12:0/2626230408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac4067c40 con 0x7f5ac4155390 2023-11-28T21:46:59.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.381+0000 7f5abaffd640 1 -- 172.21.15.12:0/2626230408 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5ab40b9030 con 0x7f5ac4155390 2023-11-28T21:46:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.381+0000 7f5abaffd640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa8066470 0x7f5aa8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:46:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.381+0000 7f5abaffd640 1 -- 172.21.15.12:0/2626230408 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f5ab40b7070 con 0x7f5ac4155390 2023-11-28T21:46:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.381+0000 7f5aca191640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa8066470 0x7f5aa8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:46:59.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.385+0000 7f5aca191640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa8066470 0x7f5aa8068930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009c60 tx=0x7f5ac0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:46:59.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.389+0000 7f5abaffd640 1 -- 172.21.15.12:0/2626230408 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ab40f1da0 con 0x7f5ac4155390 2023-11-28T21:46:59.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.577+0000 7f5acc3f5640 1 -- 172.21.15.12:0/2626230408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ac40cceb0 con 0x7f5ac4155390 2023-11-28T21:46:59.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5abaffd640 1 -- 172.21.15.12:0/2626230408 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5718 (secure 0 0 0) 0x7f5ab40f1930 con 0x7f5ac4155390 2023-11-28T21:46:59.587 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:46:59.587 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:46:57.974118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:46:59.587 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 52 2023-11-28T21:46:59.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 -- 172.21.15.12:0/2626230408 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa8066470 msgr2=0x7f5aa8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:59.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa8066470 0x7f5aa8068930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009c60 tx=0x7f5ac0009190 comp rx=0 tx=0).stop 2023-11-28T21:46:59.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 -- 172.21.15.12:0/2626230408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4155390 msgr2=0x7f5ac4151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:46:59.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4155390 0x7f5ac4151000 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4001940 tx=0x7f5ab0005800 comp rx=0 tx=0).stop 2023-11-28T21:46:59.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 -- 172.21.15.12:0/2626230408 shutdown_connections 2023-11-28T21:46:59.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5aa8066470 0x7f5aa8068930 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ac415c4a0 0x7f5ac4148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ac4155390 0x7f5ac4151000 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 --2- 172.21.15.12:0/2626230408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab00047d0 0x7f5ac415cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:46:59.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 -- 172.21.15.12:0/2626230408 >> 172.21.15.12:0/2626230408 conn(0x7f5ac406c690 msgr2=0x7f5ac40ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:46:59.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 -- 172.21.15.12:0/2626230408 shutdown_connections 2023-11-28T21:46:59.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:46:59.581+0000 7f5acc3f5640 1 -- 172.21.15.12:0/2626230408 wait complete. 2023-11-28T21:46:59.611 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:47:01.612 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:47:01.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.761+0000 7f61a8916640 1 Processor -- start 2023-11-28T21:47:01.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.761+0000 7f61a8916640 1 -- start start 2023-11-28T21:47:01.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f61a8916640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a4156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:01.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f61a8916640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f61a406d7d0 0x7f61a406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:01.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f61a8916640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a40c6ce0 con 0x7f61a41500d0 2023-11-28T21:47:01.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f61a8916640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a4157030 con 0x7f61a406d7d0 2023-11-28T21:47:01.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f61a8916640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a40c7170 con 0x7f61a40c27c0 2023-11-28T21:47:01.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f619bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a4156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:01.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f61a259c640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f61a41500d0 0x7f61a40c2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:36368/0 (socket says 172.21.15.12:36368) 2023-11-28T21:47:01.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f619bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a4156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:58396/0 (socket says 172.21.15.12:58396) 2023-11-28T21:47:01.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f619bfff640 1 -- 172.21.15.12:0/1229938173 learned_addr learned my addr 172.21.15.12:0/1229938173 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:47:01.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f61a2d9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f61a406d7d0 0x7f61a406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:01.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f61a1d9b640 1 -- 172.21.15.12:0/1229938173 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303323255 0 0) 0x7f61a40c6ce0 con 0x7f61a41500d0 2023-11-28T21:47:01.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f61a1d9b640 1 -- 172.21.15.12:0/1229938173 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6190003530 con 0x7f61a41500d0 2023-11-28T21:47:01.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f619bfff640 1 -- 172.21.15.12:0/1229938173 >> v2:172.21.15.144:3300/0 conn(0x7f61a406d7d0 msgr2=0x7f61a406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:47:01.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f619bfff640 1 --2- 172.21.15.12:0/1229938173 >> v2:172.21.15.144:3300/0 conn(0x7f61a406d7d0 0x7f61a406dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:01.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f619bfff640 1 -- 172.21.15.12:0/1229938173 >> v1:172.21.15.144:6789/0 conn(0x7f61a41500d0 legacy=0x7f61a40c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:01.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.765+0000 7f619bfff640 1 -- 172.21.15.12:0/1229938173 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a40c2c00 con 0x7f61a40c27c0 2023-11-28T21:47:01.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f619bfff640 1 --2- 172.21.15.12:0/1229938173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a4156af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f618c009870 tx=0x7f618c01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cf0193d1a53debe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:02.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a1d9b640 1 -- 172.21.15.12:0/1229938173 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c0aa5f0 con 0x7f61a40c27c0 2023-11-28T21:47:02.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a1d9b640 1 -- 172.21.15.12:0/1229938173 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c0aa750 con 0x7f61a40c27c0 2023-11-28T21:47:02.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a1d9b640 1 -- 172.21.15.12:0/1229938173 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c0aaac0 con 0x7f61a40c27c0 2023-11-28T21:47:02.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a8916640 1 -- 172.21.15.12:0/1229938173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 msgr2=0x7f61a4156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:02.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a8916640 1 --2- 172.21.15.12:0/1229938173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a4156af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f618c009870 tx=0x7f618c01c510 comp rx=0 tx=0).stop 2023-11-28T21:47:02.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a8916640 1 -- 172.21.15.12:0/1229938173 shutdown_connections 2023-11-28T21:47:02.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a8916640 1 --2- 172.21.15.12:0/1229938173 >> v2:172.21.15.144:3300/0 conn(0x7f61a406d7d0 0x7f61a406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:02.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a8916640 1 --2- 172.21.15.12:0/1229938173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a4156af0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:02.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a8916640 1 -- 172.21.15.12:0/1229938173 >> 172.21.15.12:0/1229938173 conn(0x7f61a4066360 msgr2=0x7f61a4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:02.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a8916640 1 -- 172.21.15.12:0/1229938173 shutdown_connections 2023-11-28T21:47:02.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:01.769+0000 7f61a8916640 1 -- 172.21.15.12:0/1229938173 wait complete. 2023-11-28T21:47:02.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.009+0000 7f61a8916640 1 Processor -- start 2023-11-28T21:47:02.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.009+0000 7f61a8916640 1 -- start start 2023-11-28T21:47:02.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a8916640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a406d7d0 0x7f61a4152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:02.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a8916640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a414bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:02.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a8916640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a41500d0 0x7f61a414c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:02.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a8916640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a414df90 con 0x7f61a41500d0 2023-11-28T21:47:02.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a8916640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a4154e00 con 0x7f61a406d7d0 2023-11-28T21:47:02.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a8916640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a4150960 con 0x7f61a40c27c0 2023-11-28T21:47:02.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a41500d0 0x7f61a414c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:02.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a406d7d0 0x7f61a4152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:02.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f619bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a414bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:02.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a406d7d0 0x7f61a4152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:58898/0 (socket says 172.21.15.12:58898) 2023-11-28T21:47:02.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f619bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a414bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:58400/0 (socket says 172.21.15.12:58400) 2023-11-28T21:47:02.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f619bfff640 1 -- 172.21.15.12:0/430352003 learned_addr learned my addr 172.21.15.12:0/430352003 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:47:02.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a41500d0 0x7f61a414c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43916/0 (socket says 172.21.15.12:43916) 2023-11-28T21:47:02.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a2d9d640 1 -- 172.21.15.12:0/430352003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 msgr2=0x7f61a414bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:02.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a2d9d640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a414bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:02.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a2d9d640 1 -- 172.21.15.12:0/430352003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a406d7d0 msgr2=0x7f61a4152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:02.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a2d9d640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a406d7d0 0x7f61a4152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:02.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a2d9d640 1 -- 172.21.15.12:0/430352003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0094d0 con 0x7f61a41500d0 2023-11-28T21:47:02.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a2d9d640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a41500d0 0x7f61a414c140 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f619c008080 tx=0x7f619c008540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:02.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f619affd640 1 -- 172.21.15.12:0/430352003 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c010040 con 0x7f61a41500d0 2023-11-28T21:47:02.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f619affd640 1 -- 172.21.15.12:0/430352003 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c0165f0 con 0x7f61a41500d0 2023-11-28T21:47:02.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f619affd640 1 -- 172.21.15.12:0/430352003 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c016980 con 0x7f61a41500d0 2023-11-28T21:47:02.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a8916640 1 -- 172.21.15.12:0/430352003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a4155180 con 0x7f61a41500d0 2023-11-28T21:47:02.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.013+0000 7f61a8916640 1 -- 172.21.15.12:0/430352003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a415a3d0 con 0x7f61a41500d0 2023-11-28T21:47:02.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.017+0000 7f61a8916640 1 -- 172.21.15.12:0/430352003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a4074f60 con 0x7f61a41500d0 2023-11-28T21:47:02.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.017+0000 7f619affd640 1 -- 172.21.15.12:0/430352003 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f619c016ae0 con 0x7f61a41500d0 2023-11-28T21:47:02.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.017+0000 7f619affd640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f61840664a0 0x7f6184068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:02.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.017+0000 7f619affd640 1 -- 172.21.15.12:0/430352003 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f619c0846e0 con 0x7f61a41500d0 2023-11-28T21:47:02.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.017+0000 7f61a259c640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f61840664a0 0x7f6184068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:02.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.021+0000 7f61a259c640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f61840664a0 0x7f6184068960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f61940011f0 tx=0x7f6194007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:02.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.021+0000 7f619affd640 1 -- 172.21.15.12:0/430352003 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f619c051640 con 0x7f61a41500d0 2023-11-28T21:47:02.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 -- 172.21.15.12:0/430352003 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61a4155310 con 0x7f61a41500d0 2023-11-28T21:47:02.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f619affd640 1 -- 172.21.15.12:0/430352003 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5718 (secure 0 0 0) 0x7f619c054c60 con 0x7f61a41500d0 2023-11-28T21:47:02.223 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:47:02.223 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:46:57.974118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:47:02.224 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 52 2023-11-28T21:47:02.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 -- 172.21.15.12:0/430352003 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f61840664a0 msgr2=0x7f6184068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:02.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f61840664a0 0x7f6184068960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f61940011f0 tx=0x7f6194007d40 comp rx=0 tx=0).stop 2023-11-28T21:47:02.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 -- 172.21.15.12:0/430352003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a41500d0 msgr2=0x7f61a414c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:02.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a41500d0 0x7f61a414c140 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f619c008080 tx=0x7f619c008540 comp rx=0 tx=0).stop 2023-11-28T21:47:02.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 -- 172.21.15.12:0/430352003 shutdown_connections 2023-11-28T21:47:02.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f61840664a0 0x7f6184068960 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:02.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f61a41500d0 0x7f61a414c140 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:02.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61a40c27c0 0x7f61a414bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:02.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 --2- 172.21.15.12:0/430352003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61a406d7d0 0x7f61a4152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:02.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 -- 172.21.15.12:0/430352003 >> 172.21.15.12:0/430352003 conn(0x7f61a4066360 msgr2=0x7f61a4072270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:02.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 -- 172.21.15.12:0/430352003 shutdown_connections 2023-11-28T21:47:02.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:02.217+0000 7f61a8916640 1 -- 172.21.15.12:0/430352003 wait complete. 2023-11-28T21:47:02.246 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 52, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 52, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6024": { "addr": "172.21.15.12:6839/2705421041", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 2705421041, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 2705421041, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6024, "incarnation": 51, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:reconnect", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 48, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:46:57.974118+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 6024 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:47:04.247 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:47:04.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.393+0000 7efc16ff5640 1 Processor -- start 2023-11-28T21:47:04.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.393+0000 7efc16ff5640 1 -- start start 2023-11-28T21:47:04.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc16ff5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efc10151ed0 0x7efc100c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:04.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc16ff5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc1006e300 0x7efc10072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:04.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc16ff5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc10157b70 con 0x7efc10151af0 2023-11-28T21:47:04.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc16ff5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc100707f0 con 0x7efc10151ed0 2023-11-28T21:47:04.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc16ff5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc100c7b90 con 0x7efc1006e300 2023-11-28T21:47:04.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc14d91640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efc10151af0 0x7efc100c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:34746/0 (socket says 172.21.15.12:34746) 2023-11-28T21:47:04.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc14d91640 1 -- 172.21.15.12:0/773584050 learned_addr learned my addr 172.21.15.12:0/773584050 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:47:04.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc0ffff640 1 --2- 172.21.15.12:0/773584050 >> v2:172.21.15.144:3300/0 conn(0x7efc10151ed0 0x7efc100c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:04.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc15592640 1 --2- 172.21.15.12:0/773584050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc1006e300 0x7efc10072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:04.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc0f7fe640 1 -- 172.21.15.12:0/773584050 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096270199 0 0) 0x7efc10157b70 con 0x7efc10151af0 2023-11-28T21:47:04.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc0f7fe640 1 -- 172.21.15.12:0/773584050 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc00003530 con 0x7efc10151af0 2023-11-28T21:47:04.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc0ffff640 1 -- 172.21.15.12:0/773584050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc1006e300 msgr2=0x7efc10072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:47:04.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc0ffff640 1 --2- 172.21.15.12:0/773584050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc1006e300 0x7efc10072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:04.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc0ffff640 1 -- 172.21.15.12:0/773584050 >> v1:172.21.15.12:6789/0 conn(0x7efc10151af0 legacy=0x7efc100c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:04.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc0f7fe640 1 -- 172.21.15.12:0/773584050 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2751145294 0 0) 0x7efc00003530 con 0x7efc10151af0 2023-11-28T21:47:04.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.397+0000 7efc0ffff640 1 -- 172.21.15.12:0/773584050 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc10157e50 con 0x7efc10151ed0 2023-11-28T21:47:04.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc0ffff640 1 --2- 172.21.15.12:0/773584050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc10151ed0 0x7efc100c7650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0097c0 tx=0x7efbfc01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65cf314d74209994 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:04.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc0f7fe640 1 -- 172.21.15.12:0/773584050 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04001680 con 0x7efc10151ed0 2023-11-28T21:47:04.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc0f7fe640 1 -- 172.21.15.12:0/773584050 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc0a7050 con 0x7efc10151ed0 2023-11-28T21:47:04.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc0f7fe640 1 -- 172.21.15.12:0/773584050 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0ab600 con 0x7efc10151ed0 2023-11-28T21:47:04.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc16ff5640 1 -- 172.21.15.12:0/773584050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc10151ed0 msgr2=0x7efc100c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:04.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc16ff5640 1 --2- 172.21.15.12:0/773584050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc10151ed0 0x7efc100c7650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0097c0 tx=0x7efbfc01c3a0 comp rx=0 tx=0).stop 2023-11-28T21:47:04.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc16ff5640 1 -- 172.21.15.12:0/773584050 shutdown_connections 2023-11-28T21:47:04.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc16ff5640 1 --2- 172.21.15.12:0/773584050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc1006e300 0x7efc10072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:04.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc16ff5640 1 --2- 172.21.15.12:0/773584050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc10151ed0 0x7efc100c7650 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:04.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc16ff5640 1 -- 172.21.15.12:0/773584050 >> 172.21.15.12:0/773584050 conn(0x7efc10066360 msgr2=0x7efc10066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:04.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc16ff5640 1 -- 172.21.15.12:0/773584050 shutdown_connections 2023-11-28T21:47:04.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.401+0000 7efc16ff5640 1 -- 172.21.15.12:0/773584050 wait complete. 2023-11-28T21:47:04.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 Processor -- start 2023-11-28T21:47:04.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 -- start start 2023-11-28T21:47:04.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc1006e300 0x7efc10155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:04.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc10151af0 0x7efc10140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:04.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc10145650 0x7efc101f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:04.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc101f52e0 con 0x7efc10145650 2023-11-28T21:47:04.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc101f0ed0 con 0x7efc10151af0 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc101f4d40 con 0x7efc1006e300 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc15592640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc10145650 0x7efc101f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc15592640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc10145650 0x7efc101f1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43932/0 (socket says 172.21.15.12:43932) 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc15592640 1 -- 172.21.15.12:0/3317305601 learned_addr learned my addr 172.21.15.12:0/3317305601 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc0ffff640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc10151af0 0x7efc10140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc14d91640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc1006e300 0x7efc10155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc15592640 1 -- 172.21.15.12:0/3317305601 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc1006e300 msgr2=0x7efc10155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc15592640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc1006e300 0x7efc10155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc15592640 1 -- 172.21.15.12:0/3317305601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc10151af0 msgr2=0x7efc10140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc15592640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc10151af0 0x7efc10140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc15592640 1 -- 172.21.15.12:0/3317305601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc009420 con 0x7efc10145650 2023-11-28T21:47:04.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc0ffff640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc10151af0 0x7efc10140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:47:04.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc15592640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc10145650 0x7efc101f1690 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7efc0800efc0 tx=0x7efc0800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:04.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc0d7fa640 1 -- 172.21.15.12:0/3317305601 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc08009ca0 con 0x7efc10145650 2023-11-28T21:47:04.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc0d7fa640 1 -- 172.21.15.12:0/3317305601 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc0800f040 con 0x7efc10145650 2023-11-28T21:47:04.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 -- 172.21.15.12:0/3317305601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc101f5000 con 0x7efc10145650 2023-11-28T21:47:04.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc0d7fa640 1 -- 172.21.15.12:0/3317305601 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc080146c0 con 0x7efc10145650 2023-11-28T21:47:04.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 -- 172.21.15.12:0/3317305601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc101f1170 con 0x7efc10145650 2023-11-28T21:47:04.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.405+0000 7efc16ff5640 1 -- 172.21.15.12:0/3317305601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc101f1360 con 0x7efc10145650 2023-11-28T21:47:04.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.409+0000 7efc0d7fa640 1 -- 172.21.15.12:0/3317305601 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7efc08014820 con 0x7efc10145650 2023-11-28T21:47:04.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.409+0000 7efc0d7fa640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efbf40664a0 0x7efbf4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:04.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.409+0000 7efc0d7fa640 1 -- 172.21.15.12:0/3317305601 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7efc08085290 con 0x7efc10145650 2023-11-28T21:47:04.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.409+0000 7efc14d91640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efbf40664a0 0x7efbf4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:04.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.409+0000 7efc14d91640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efbf40664a0 0x7efbf4068960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc04007a10 tx=0x7efc04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:04.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.413+0000 7efc0d7fa640 1 -- 172.21.15.12:0/3317305601 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efc080521e0 con 0x7efc10145650 2023-11-28T21:47:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 -- 172.21.15.12:0/3317305601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc10074f60 con 0x7efc10145650 2023-11-28T21:47:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc0d7fa640 1 -- 172.21.15.12:0/3317305601 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+5715 (secure 0 0 0) 0x7efc08055800 con 0x7efc10145650 2023-11-28T21:47:04.615 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:47:04.615 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:03.773937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:47:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 53 2023-11-28T21:47:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 -- 172.21.15.12:0/3317305601 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efbf40664a0 msgr2=0x7efbf4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efbf40664a0 0x7efbf4068960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc04007a10 tx=0x7efc04009040 comp rx=0 tx=0).stop 2023-11-28T21:47:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 -- 172.21.15.12:0/3317305601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc10145650 msgr2=0x7efc101f1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc10145650 0x7efc101f1690 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7efc0800efc0 tx=0x7efc0800c490 comp rx=0 tx=0).stop 2023-11-28T21:47:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 -- 172.21.15.12:0/3317305601 shutdown_connections 2023-11-28T21:47:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efbf40664a0 0x7efbf4068960 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc10145650 0x7efc101f1690 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc10151af0 0x7efc10140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:04.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 --2- 172.21.15.12:0/3317305601 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc1006e300 0x7efc10155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:04.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 -- 172.21.15.12:0/3317305601 >> 172.21.15.12:0/3317305601 conn(0x7efc10066360 msgr2=0x7efc1014f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:04.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 -- 172.21.15.12:0/3317305601 shutdown_connections 2023-11-28T21:47:04.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:04.609+0000 7efc16ff5640 1 -- 172.21.15.12:0/3317305601 wait complete. 2023-11-28T21:47:04.637 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:47:06.637 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:47:06.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.785+0000 7f1769854640 1 Processor -- start 2023-11-28T21:47:06.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.785+0000 7f1769854640 1 -- start start 2023-11-28T21:47:06.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f1769854640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f176406e300 0x7f1764156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:06.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f1769854640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1764067410 con 0x7f176406cf40 2023-11-28T21:47:06.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f1769854640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176406e090 con 0x7f1764150c20 2023-11-28T21:47:06.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f1769854640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1764157f30 con 0x7f176406e300 2023-11-28T21:47:06.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f175bfff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f176406e300 0x7f1764156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:06.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f17637fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f176406cf40 0x7f1764158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57008/0 (socket says 172.21.15.12:57008) 2023-11-28T21:47:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f17637fe640 1 -- 172.21.15.12:0/817746780 learned_addr learned my addr 172.21.15.12:0/817746780 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:47:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f175bfff640 1 -- 172.21.15.12:0/817746780 >> v1:172.21.15.144:6789/0 conn(0x7f1764150c20 legacy=0x7f17640c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:47:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f175bfff640 1 -- 172.21.15.12:0/817746780 >> v1:172.21.15.12:6789/0 conn(0x7f176406cf40 legacy=0x7f1764158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:06.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f175bfff640 1 -- 172.21.15.12:0/817746780 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1764151000 con 0x7f176406e300 2023-11-28T21:47:06.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.789+0000 7f175bfff640 1 --2- 172.21.15.12:0/817746780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 0x7f1764156910 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f174c0097c0 tx=0x7f174c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff64f60bfd1b1b93 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:06.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f17627fc640 1 -- 172.21.15.12:0/817746780 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c004230 con 0x7f176406e300 2023-11-28T21:47:06.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f17627fc640 1 -- 172.21.15.12:0/817746780 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c0a7050 con 0x7f176406e300 2023-11-28T21:47:06.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f17627fc640 1 -- 172.21.15.12:0/817746780 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c003620 con 0x7f176406e300 2023-11-28T21:47:06.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 -- 172.21.15.12:0/817746780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 msgr2=0x7f1764156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:06.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 --2- 172.21.15.12:0/817746780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 0x7f1764156910 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f174c0097c0 tx=0x7f174c01c440 comp rx=0 tx=0).stop 2023-11-28T21:47:06.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 -- 172.21.15.12:0/817746780 shutdown_connections 2023-11-28T21:47:06.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 --2- 172.21.15.12:0/817746780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 0x7f1764156910 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:06.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 -- 172.21.15.12:0/817746780 >> 172.21.15.12:0/817746780 conn(0x7f1764066360 msgr2=0x7f1764066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:06.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 -- 172.21.15.12:0/817746780 shutdown_connections 2023-11-28T21:47:06.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 -- 172.21.15.12:0/817746780 wait complete. 2023-11-28T21:47:06.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 Processor -- start 2023-11-28T21:47:06.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 -- start start 2023-11-28T21:47:06.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f176406cf40 0x7f17640c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:06.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 0x7f1764145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:06.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1764150c20 0x7f17640715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:06.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640c3ba0 con 0x7f176406e300 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640c3e40 con 0x7f1764150c20 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1769854640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17640c4120 con 0x7f176406cf40 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f175bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 0x7f1764145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f175bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 0x7f1764145650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41976/0 (socket says 172.21.15.12:41976) 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f175bfff640 1 -- 172.21.15.12:0/3574787846 learned_addr learned my addr 172.21.15.12:0/3574787846 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f17637fe640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1764150c20 0x7f17640715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.793+0000 7f1762ffd640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f176406cf40 0x7f17640c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f175bfff640 1 -- 172.21.15.12:0/3574787846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f176406cf40 msgr2=0x7f17640c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f175bfff640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f176406cf40 0x7f17640c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f175bfff640 1 -- 172.21.15.12:0/3574787846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1764150c20 msgr2=0x7f17640715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f175bfff640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1764150c20 0x7f17640715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:06.803 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f175bfff640 1 -- 172.21.15.12:0/3574787846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c009420 con 0x7f176406e300 2023-11-28T21:47:06.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f175bfff640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 0x7f1764145650 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f174c0ae040 tx=0x7f174c0191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:06.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f1760ff9640 1 -- 172.21.15.12:0/3574787846 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c003cf0 con 0x7f176406e300 2023-11-28T21:47:06.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f1760ff9640 1 -- 172.21.15.12:0/3574787846 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c003e50 con 0x7f176406e300 2023-11-28T21:47:06.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f1760ff9640 1 -- 172.21.15.12:0/3574787846 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c0ab660 con 0x7f176406e300 2023-11-28T21:47:06.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f1769854640 1 -- 172.21.15.12:0/3574787846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17640c4400 con 0x7f176406e300 2023-11-28T21:47:06.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f1769854640 1 -- 172.21.15.12:0/3574787846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1764071090 con 0x7f176406e300 2023-11-28T21:47:06.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f1769854640 1 -- 172.21.15.12:0/3574787846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1764071280 con 0x7f176406e300 2023-11-28T21:47:06.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.797+0000 7f1760ff9640 1 -- 172.21.15.12:0/3574787846 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f174c0b2040 con 0x7f176406e300 2023-11-28T21:47:06.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.801+0000 7f1760ff9640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1744066470 0x7f1744068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:06.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.801+0000 7f1760ff9640 1 -- 172.21.15.12:0/3574787846 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f174c11e460 con 0x7f176406e300 2023-11-28T21:47:06.806 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.801+0000 7f1762ffd640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1744066470 0x7f1744068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:06.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.801+0000 7f1762ffd640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1744066470 0x7f1744068930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f175400b810 tx=0x7f1754008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:06.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.805+0000 7f1760ff9640 1 -- 172.21.15.12:0/3574787846 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f174c0eb1f0 con 0x7f176406e300 2023-11-28T21:47:07.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 -- 172.21.15.12:0/3574787846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1764074f60 con 0x7f176406e300 2023-11-28T21:47:07.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1760ff9640 1 -- 172.21.15.12:0/3574787846 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f174c0b0090 con 0x7f176406e300 2023-11-28T21:47:07.003 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:47:07.004 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:03.773937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:47:07.004 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 53 2023-11-28T21:47:07.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 -- 172.21.15.12:0/3574787846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1744066470 msgr2=0x7f1744068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:07.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1744066470 0x7f1744068930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f175400b810 tx=0x7f1754008040 comp rx=0 tx=0).stop 2023-11-28T21:47:07.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 -- 172.21.15.12:0/3574787846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 msgr2=0x7f1764145650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:07.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 0x7f1764145650 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f174c0ae040 tx=0x7f174c0191e0 comp rx=0 tx=0).stop 2023-11-28T21:47:07.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 -- 172.21.15.12:0/3574787846 shutdown_connections 2023-11-28T21:47:07.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1744066470 0x7f1744068930 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:07.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1764150c20 0x7f17640715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:07.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f176406e300 0x7f1764145650 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:07.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 --2- 172.21.15.12:0/3574787846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f176406cf40 0x7f17640c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:07.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 -- 172.21.15.12:0/3574787846 >> 172.21.15.12:0/3574787846 conn(0x7f1764066360 msgr2=0x7f17640c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:07.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 -- 172.21.15.12:0/3574787846 shutdown_connections 2023-11-28T21:47:07.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:06.997+0000 7f1769854640 1 -- 172.21.15.12:0/3574787846 wait complete. 2023-11-28T21:47:07.025 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:47:09.026 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:47:09.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.173+0000 7fe8317f5640 1 Processor -- start 2023-11-28T21:47:09.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.173+0000 7fe8317f5640 1 -- start start 2023-11-28T21:47:09.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.173+0000 7fe8317f5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe82c1500d0 0x7fe82c0c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:09.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.173+0000 7fe8317f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c06d7d0 0x7fe82c06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:09.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.173+0000 7fe8317f5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82c0c6ce0 con 0x7fe82c156af0 2023-11-28T21:47:09.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.173+0000 7fe8317f5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82c0c7170 con 0x7fe82c1500d0 2023-11-28T21:47:09.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.173+0000 7fe8317f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c156f60 con 0x7fe82c06d7d0 2023-11-28T21:47:09.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82affd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe82c1500d0 0x7fe82c0c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:09.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82affd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe82c1500d0 0x7fe82c0c27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41980/0 (socket says 172.21.15.12:41980) 2023-11-28T21:47:09.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82affd640 1 -- 172.21.15.12:0/2722409831 learned_addr learned my addr 172.21.15.12:0/2722409831 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:47:09.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82b7fe640 1 --2- 172.21.15.12:0/2722409831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c06d7d0 0x7fe82c06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:09.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82affd640 1 -- 172.21.15.12:0/2722409831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c06d7d0 msgr2=0x7fe82c06dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:47:09.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82affd640 1 --2- 172.21.15.12:0/2722409831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c06d7d0 0x7fe82c06dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:09.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82affd640 1 -- 172.21.15.12:0/2722409831 >> v1:172.21.15.144:6789/0 conn(0x7fe82c156af0 legacy=0x7fe82c0c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:47:09.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82affd640 1 -- 172.21.15.12:0/2722409831 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c0c5df0 con 0x7fe82c1500d0 2023-11-28T21:47:09.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82affd640 1 --2- 172.21.15.12:0/2722409831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 0x7fe82c0c27c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00aa10 tx=0x7fe81c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e69994a142bbdc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:09.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82a7fc640 1 -- 172.21.15.12:0/2722409831 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c0a5850 con 0x7fe82c1500d0 2023-11-28T21:47:09.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82a7fc640 1 -- 172.21.15.12:0/2722409831 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81c0a59b0 con 0x7fe82c1500d0 2023-11-28T21:47:09.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.177+0000 7fe82a7fc640 1 -- 172.21.15.12:0/2722409831 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c0a5d40 con 0x7fe82c1500d0 2023-11-28T21:47:09.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 -- 172.21.15.12:0/2722409831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 msgr2=0x7fe82c0c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:09.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 --2- 172.21.15.12:0/2722409831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 0x7fe82c0c27c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00aa10 tx=0x7fe81c01c810 comp rx=0 tx=0).stop 2023-11-28T21:47:09.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 -- 172.21.15.12:0/2722409831 shutdown_connections 2023-11-28T21:47:09.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 --2- 172.21.15.12:0/2722409831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c06d7d0 0x7fe82c06dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:09.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 --2- 172.21.15.12:0/2722409831 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 0x7fe82c0c27c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:09.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 -- 172.21.15.12:0/2722409831 >> 172.21.15.12:0/2722409831 conn(0x7fe82c066360 msgr2=0x7fe82c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:09.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 -- 172.21.15.12:0/2722409831 shutdown_connections 2023-11-28T21:47:09.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 -- 172.21.15.12:0/2722409831 wait complete. 2023-11-28T21:47:09.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 Processor -- start 2023-11-28T21:47:09.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 -- start start 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe82c06d7d0 0x7fe82c14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 0x7fe82c146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c156af0 0x7fe82c146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c146060 con 0x7fe82c1500d0 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c152ff0 con 0x7fe82c06d7d0 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe8317f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c159260 con 0x7fe82c156af0 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe823fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 0x7fe82c146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe823fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 0x7fe82c146540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41988/0 (socket says 172.21.15.12:41988) 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe823fff640 1 -- 172.21.15.12:0/4270207490 learned_addr learned my addr 172.21.15.12:0/4270207490 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe82affd640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe82c06d7d0 0x7fe82c14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe82b7fe640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c156af0 0x7fe82c146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:09.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe823fff640 1 -- 172.21.15.12:0/4270207490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c156af0 msgr2=0x7fe82c146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:09.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe823fff640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c156af0 0x7fe82c146a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:09.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe823fff640 1 -- 172.21.15.12:0/4270207490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe82c06d7d0 msgr2=0x7fe82c14e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:09.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe823fff640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe82c06d7d0 0x7fe82c14e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:09.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.181+0000 7fe823fff640 1 -- 172.21.15.12:0/4270207490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c00a670 con 0x7fe82c1500d0 2023-11-28T21:47:09.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe823fff640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 0x7fe82c146540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c500 tx=0x7fe81400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:09.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe8237fe640 1 -- 172.21.15.12:0/4270207490 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814005220 con 0x7fe82c1500d0 2023-11-28T21:47:09.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe8237fe640 1 -- 172.21.15.12:0/4270207490 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814005380 con 0x7fe82c1500d0 2023-11-28T21:47:09.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe8237fe640 1 -- 172.21.15.12:0/4270207490 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814007270 con 0x7fe82c1500d0 2023-11-28T21:47:09.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe8317f5640 1 -- 172.21.15.12:0/4270207490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c145c70 con 0x7fe82c1500d0 2023-11-28T21:47:09.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe8317f5640 1 -- 172.21.15.12:0/4270207490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82c158ca0 con 0x7fe82c1500d0 2023-11-28T21:47:09.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe8317f5640 1 -- 172.21.15.12:0/4270207490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82c145ec0 con 0x7fe82c1500d0 2023-11-28T21:47:09.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe8237fe640 1 -- 172.21.15.12:0/4270207490 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe8140073d0 con 0x7fe82c1500d0 2023-11-28T21:47:09.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe8237fe640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe80c066470 0x7fe80c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:09.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe8237fe640 1 -- 172.21.15.12:0/4270207490 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7fe814083bb0 con 0x7fe82c1500d0 2023-11-28T21:47:09.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.185+0000 7fe82affd640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe80c066470 0x7fe80c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:09.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.189+0000 7fe82affd640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe80c066470 0x7fe80c068930 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0af000 tx=0x7fe81c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:09.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.189+0000 7fe8237fe640 1 -- 172.21.15.12:0/4270207490 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe82c145ec0 con 0x7fe82c1500d0 2023-11-28T21:47:09.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.381+0000 7fe8317f5640 1 -- 172.21.15.12:0/4270207490 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe82c146260 con 0x7fe82c1500d0 2023-11-28T21:47:09.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.381+0000 7fe8237fe640 1 -- 172.21.15.12:0/4270207490 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fe82c146260 con 0x7fe82c1500d0 2023-11-28T21:47:09.390 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:47:09.390 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:03.773937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:47:09.390 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 53 2023-11-28T21:47:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 -- 172.21.15.12:0/4270207490 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe80c066470 msgr2=0x7fe80c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe80c066470 0x7fe80c068930 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0af000 tx=0x7fe81c0a7070 comp rx=0 tx=0).stop 2023-11-28T21:47:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 -- 172.21.15.12:0/4270207490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 msgr2=0x7fe82c146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 0x7fe82c146540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c500 tx=0x7fe81400c9c0 comp rx=0 tx=0).stop 2023-11-28T21:47:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 -- 172.21.15.12:0/4270207490 shutdown_connections 2023-11-28T21:47:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe80c066470 0x7fe80c068930 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:09.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c156af0 0x7fe82c146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:09.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe82c1500d0 0x7fe82c146540 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:09.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 --2- 172.21.15.12:0/4270207490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe82c06d7d0 0x7fe82c14e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:09.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 -- 172.21.15.12:0/4270207490 >> 172.21.15.12:0/4270207490 conn(0x7fe82c066360 msgr2=0x7fe82c14f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:09.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 -- 172.21.15.12:0/4270207490 shutdown_connections 2023-11-28T21:47:09.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:09.385+0000 7fe8317f5640 1 -- 172.21.15.12:0/4270207490 wait complete. 2023-11-28T21:47:09.412 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:47:11.413 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:47:11.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf6c8f5640 1 Processor -- start 2023-11-28T21:47:11.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf6c8f5640 1 -- start start 2023-11-28T21:47:11.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf6c8f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c6f20 0x7faf68156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:11.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf6c8f5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf680c26b0 0x7faf68158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:11.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf6c8f5640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6806b7c0 con 0x7faf680c7300 2023-11-28T21:47:11.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf6c8f5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf6806a8f0 con 0x7faf680c26b0 2023-11-28T21:47:11.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf6c8f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6806abb0 con 0x7faf680c6f20 2023-11-28T21:47:11.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf66ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf680c26b0 0x7faf68158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:11.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf66ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faf680c26b0 0x7faf68158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42004/0 (socket says 172.21.15.12:42004) 2023-11-28T21:47:11.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf66ffd640 1 -- 172.21.15.12:0/1011890840 learned_addr learned my addr 172.21.15.12:0/1011890840 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:47:11.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf677fe640 1 --2- 172.21.15.12:0/1011890840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c6f20 0x7faf68156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf66ffd640 1 -- 172.21.15.12:0/1011890840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c6f20 msgr2=0x7faf68156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:47:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf66ffd640 1 --2- 172.21.15.12:0/1011890840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c6f20 0x7faf68156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf66ffd640 1 -- 172.21.15.12:0/1011890840 >> v1:172.21.15.144:6789/0 conn(0x7faf680c7300 legacy=0x7faf68150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:47:11.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.565+0000 7faf66ffd640 1 -- 172.21.15.12:0/1011890840 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6806ae90 con 0x7faf680c26b0 2023-11-28T21:47:11.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf66ffd640 1 --2- 172.21.15.12:0/1011890840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c26b0 0x7faf68158170 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009870 tx=0x7faf5c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a1efe5e0937641 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:11.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf667fc640 1 -- 172.21.15.12:0/1011890840 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c0aa620 con 0x7faf680c26b0 2023-11-28T21:47:11.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf667fc640 1 -- 172.21.15.12:0/1011890840 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c0aa780 con 0x7faf680c26b0 2023-11-28T21:47:11.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf667fc640 1 -- 172.21.15.12:0/1011890840 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c0aab10 con 0x7faf680c26b0 2023-11-28T21:47:11.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1011890840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c26b0 msgr2=0x7faf68158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:11.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf6c8f5640 1 --2- 172.21.15.12:0/1011890840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c26b0 0x7faf68158170 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009870 tx=0x7faf5c01c500 comp rx=0 tx=0).stop 2023-11-28T21:47:11.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1011890840 shutdown_connections 2023-11-28T21:47:11.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf6c8f5640 1 --2- 172.21.15.12:0/1011890840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c26b0 0x7faf68158170 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:11.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf6c8f5640 1 --2- 172.21.15.12:0/1011890840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c6f20 0x7faf68156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:11.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1011890840 >> 172.21.15.12:0/1011890840 conn(0x7faf68066360 msgr2=0x7faf68066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:11.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1011890840 shutdown_connections 2023-11-28T21:47:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.569+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1011890840 wait complete. 2023-11-28T21:47:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf6c8f5640 1 Processor -- start 2023-11-28T21:47:11.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf6c8f5640 1 -- start start 2023-11-28T21:47:11.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf6c8f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c26b0 0x7faf681f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:11.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf6c8f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c6f20 0x7faf681f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:11.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf6c8f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf680c7300 0x7faf680ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:11.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf6c8f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681f27f0 con 0x7faf680c6f20 2023-11-28T21:47:11.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf6c8f5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6814a990 con 0x7faf680c7300 2023-11-28T21:47:11.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf6c8f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf681f6cb0 con 0x7faf680c26b0 2023-11-28T21:47:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf66ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c6f20 0x7faf681f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf66ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c6f20 0x7faf681f3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42008/0 (socket says 172.21.15.12:42008) 2023-11-28T21:47:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf66ffd640 1 -- 172.21.15.12:0/1874204063 learned_addr learned my addr 172.21.15.12:0/1874204063 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:47:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf677fe640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c26b0 0x7faf681f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf67fff640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf680c7300 0x7faf680ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf66ffd640 1 -- 172.21.15.12:0/1874204063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c26b0 msgr2=0x7faf681f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf66ffd640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c26b0 0x7faf681f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf66ffd640 1 -- 172.21.15.12:0/1874204063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf680c7300 msgr2=0x7faf680ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf66ffd640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf680c7300 0x7faf680ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:11.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf66ffd640 1 -- 172.21.15.12:0/1874204063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0094d0 con 0x7faf680c6f20 2023-11-28T21:47:11.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf67fff640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf680c7300 0x7faf680ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:47:11.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf677fe640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c26b0 0x7faf681f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:47:11.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf66ffd640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c6f20 0x7faf681f3200 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7faf5c01cda0 tx=0x7faf5c01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:11.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf4ffff640 1 -- 172.21.15.12:0/1874204063 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c0bd3f0 con 0x7faf680c6f20 2023-11-28T21:47:11.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf4ffff640 1 -- 172.21.15.12:0/1874204063 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c0bd550 con 0x7faf680c6f20 2023-11-28T21:47:11.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1874204063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf680c2370 con 0x7faf680c6f20 2023-11-28T21:47:11.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf4ffff640 1 -- 172.21.15.12:0/1874204063 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c0bd900 con 0x7faf680c6f20 2023-11-28T21:47:11.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.573+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1874204063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf681f3740 con 0x7faf680c6f20 2023-11-28T21:47:11.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.577+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1874204063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf68067c70 con 0x7faf680c6f20 2023-11-28T21:47:11.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.577+0000 7faf4ffff640 1 -- 172.21.15.12:0/1874204063 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faf5c0b33f0 con 0x7faf680c6f20 2023-11-28T21:47:11.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.577+0000 7faf4ffff640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf54066470 0x7faf54068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:11.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.577+0000 7faf4ffff640 1 -- 172.21.15.12:0/1874204063 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7faf5c11e6b0 con 0x7faf680c6f20 2023-11-28T21:47:11.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.577+0000 7faf677fe640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf54066470 0x7faf54068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:11.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.577+0000 7faf677fe640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf54066470 0x7faf54068930 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faf58007a70 tx=0x7faf58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:11.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.581+0000 7faf4ffff640 1 -- 172.21.15.12:0/1874204063 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faf5c0eb610 con 0x7faf680c6f20 2023-11-28T21:47:11.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.769+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1874204063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf68074f60 con 0x7faf680c6f20 2023-11-28T21:47:11.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf4ffff640 1 -- 172.21.15.12:0/1874204063 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5715 (secure 0 0 0) 0x7faf5c0eec30 con 0x7faf680c6f20 2023-11-28T21:47:11.779 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:47:11.779 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:09.646722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:active","state_seq":21,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:47:11.779 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 54 2023-11-28T21:47:11.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1874204063 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf54066470 msgr2=0x7faf54068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:11.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf54066470 0x7faf54068930 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faf58007a70 tx=0x7faf58008040 comp rx=0 tx=0).stop 2023-11-28T21:47:11.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1874204063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c6f20 msgr2=0x7faf681f3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:11.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c6f20 0x7faf681f3200 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7faf5c01cda0 tx=0x7faf5c01ad40 comp rx=0 tx=0).stop 2023-11-28T21:47:11.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1874204063 shutdown_connections 2023-11-28T21:47:11.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faf54066470 0x7faf54068930 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:11.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf680c7300 0x7faf680ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:11.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf680c6f20 0x7faf681f3200 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:11.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 --2- 172.21.15.12:0/1874204063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf680c26b0 0x7faf681f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:11.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1874204063 >> 172.21.15.12:0/1874204063 conn(0x7faf68066360 msgr2=0x7faf68150610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:11.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1874204063 shutdown_connections 2023-11-28T21:47:11.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:11.773+0000 7faf6c8f5640 1 -- 172.21.15.12:0/1874204063 wait complete. 2023-11-28T21:47:11.800 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=0, replacing gid=5871 2023-11-28T21:47:11.801 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.a 2023-11-28T21:47:15.510 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:47:15.514 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:47:16.557 INFO:tasks.ceph.mds.e.smithi012.stderr:2023-11-28T21:47:16.549+0000 7f5d9c596640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T21:47:18.801 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2023-11-28T21:47:18.802 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T21:47:18.802 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T21:47:18.806 INFO:tasks.ceph.mds.a:Started 2023-11-28T21:47:18.806 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:47:18.903 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-11-28T21:47:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.953+0000 7f5842e79640 1 Processor -- start 2023-11-28T21:47:18.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5842e79640 1 -- start start 2023-11-28T21:47:18.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5842e79640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f583c153ee0 0x7f583c15a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:18.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5842e79640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c0c6350 con 0x7f583c070bd0 2023-11-28T21:47:18.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5842e79640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c0c65d0 con 0x7f583c06a5b0 2023-11-28T21:47:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5842e79640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c0c6b90 con 0x7f583c153ee0 2023-11-28T21:47:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5841416640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f583c070bd0 0x7f583c071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33986/0 (socket says 172.21.15.12:33986) 2023-11-28T21:47:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5840c15640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f583c153ee0 0x7f583c15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:18.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5841416640 1 -- 172.21.15.12:0/2707295407 learned_addr learned my addr 172.21.15.12:0/2707295407 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:47:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f583b7fe640 1 -- 172.21.15.12:0/2707295407 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000039332 0 0) 0x7f583c0c6350 con 0x7f583c070bd0 2023-11-28T21:47:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f583b7fe640 1 -- 172.21.15.12:0/2707295407 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f582c003530 con 0x7f583c070bd0 2023-11-28T21:47:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5840c15640 1 -- 172.21.15.12:0/2707295407 >> v1:172.21.15.144:6789/0 conn(0x7f583c06a5b0 legacy=0x7f583c0c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5840c15640 1 -- 172.21.15.12:0/2707295407 >> v1:172.21.15.12:6789/0 conn(0x7f583c070bd0 legacy=0x7f583c071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.957+0000 7f5840c15640 1 -- 172.21.15.12:0/2707295407 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c0c68b0 con 0x7f583c153ee0 2023-11-28T21:47:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f5840c15640 1 --2- 172.21.15.12:0/2707295407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f583c153ee0 0x7f583c15a680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f58300097c0 tx=0x7f583001ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad091b69adb3178 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f583b7fe640 1 -- 172.21.15.12:0/2707295407 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300a8970 con 0x7f583c153ee0 2023-11-28T21:47:18.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f583b7fe640 1 -- 172.21.15.12:0/2707295407 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58300a8ad0 con 0x7f583c153ee0 2023-11-28T21:47:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f583b7fe640 1 -- 172.21.15.12:0/2707295407 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300a8e60 con 0x7f583c153ee0 2023-11-28T21:47:18.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f5842e79640 1 -- 172.21.15.12:0/2707295407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f583c153ee0 msgr2=0x7f583c15a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f5842e79640 1 --2- 172.21.15.12:0/2707295407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f583c153ee0 0x7f583c15a680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f58300097c0 tx=0x7f583001ec70 comp rx=0 tx=0).stop 2023-11-28T21:47:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f5842e79640 1 -- 172.21.15.12:0/2707295407 shutdown_connections 2023-11-28T21:47:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f5842e79640 1 --2- 172.21.15.12:0/2707295407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f583c153ee0 0x7f583c15a680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f5842e79640 1 -- 172.21.15.12:0/2707295407 >> 172.21.15.12:0/2707295407 conn(0x7f583c0c8450 msgr2=0x7f583c0c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f5842e79640 1 -- 172.21.15.12:0/2707295407 shutdown_connections 2023-11-28T21:47:18.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f5842e79640 1 -- 172.21.15.12:0/2707295407 wait complete. 2023-11-28T21:47:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.961+0000 7f5842e79640 1 Processor -- start 2023-11-28T21:47:18.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5842e79640 1 -- start start 2023-11-28T21:47:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5842e79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f583c06a5b0 0x7f583c1f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5842e79640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f583c070bd0 0x7f583c0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5842e79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c153ee0 0x7f583c159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5842e79640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0c3330 con 0x7f583c153ee0 2023-11-28T21:47:18.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5842e79640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c1f3bd0 con 0x7f583c070bd0 2023-11-28T21:47:18.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5842e79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0c2d70 con 0x7f583c06a5b0 2023-11-28T21:47:18.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5841416640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c153ee0 0x7f583c159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:18.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5841416640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c153ee0 0x7f583c159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57178/0 (socket says 172.21.15.12:57178) 2023-11-28T21:47:18.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5841416640 1 -- 172.21.15.12:0/250554732 learned_addr learned my addr 172.21.15.12:0/250554732 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:47:18.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5840c15640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f583c06a5b0 0x7f583c1f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:18.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f583bfff640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f583c070bd0 0x7f583c0c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:18.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5841416640 1 -- 172.21.15.12:0/250554732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f583c06a5b0 msgr2=0x7f583c1f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:18.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5841416640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f583c06a5b0 0x7f583c1f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:18.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5841416640 1 -- 172.21.15.12:0/250554732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f583c070bd0 msgr2=0x7f583c0c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:18.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5841416640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f583c070bd0 0x7f583c0c4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:18.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5841416640 1 -- 172.21.15.12:0/250554732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5830009420 con 0x7f583c153ee0 2023-11-28T21:47:18.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f583bfff640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f583c070bd0 0x7f583c0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:47:18.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5841416640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c153ee0 0x7f583c159270 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f583400cb40 tx=0x7f583400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:18.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f58397fa640 1 -- 172.21.15.12:0/250554732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583400aca0 con 0x7f583c153ee0 2023-11-28T21:47:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f58397fa640 1 -- 172.21.15.12:0/250554732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583400ae00 con 0x7f583c153ee0 2023-11-28T21:47:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5842e79640 1 -- 172.21.15.12:0/250554732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583c0c3050 con 0x7f583c153ee0 2023-11-28T21:47:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f58397fa640 1 -- 172.21.15.12:0/250554732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834012620 con 0x7f583c153ee0 2023-11-28T21:47:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5842e79640 1 -- 172.21.15.12:0/250554732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c0c3c40 con 0x7f583c153ee0 2023-11-28T21:47:18.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.965+0000 7f5842e79640 1 -- 172.21.15.12:0/250554732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583c074f60 con 0x7f583c153ee0 2023-11-28T21:47:18.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.969+0000 7f58397fa640 1 -- 172.21.15.12:0/250554732 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5834007070 con 0x7f583c153ee0 2023-11-28T21:47:18.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.969+0000 7f58397fa640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58200664a0 0x7f5820068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:18.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.969+0000 7f58397fa640 1 -- 172.21.15.12:0/250554732 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f58340844f0 con 0x7f583c153ee0 2023-11-28T21:47:18.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.969+0000 7f5840c15640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58200664a0 0x7f5820068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:18.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.969+0000 7f5840c15640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58200664a0 0x7f5820068960 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f58300b2000 tx=0x7f58300aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:18.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:18.973+0000 7f58397fa640 1 -- 172.21.15.12:0/250554732 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5834051310 con 0x7f583c153ee0 2023-11-28T21:47:19.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.161+0000 7f5842e79640 1 -- 172.21.15.12:0/250554732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f583c0c6d20 con 0x7f583c153ee0 2023-11-28T21:47:19.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f58397fa640 1 -- 172.21.15.12:0/250554732 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f5834059e60 con 0x7f583c153ee0 2023-11-28T21:47:19.171 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:47:19.171 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:09.646722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:active","state_seq":21,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:47:19.171 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 54 2023-11-28T21:47:19.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 -- 172.21.15.12:0/250554732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58200664a0 msgr2=0x7f5820068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:19.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58200664a0 0x7f5820068960 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f58300b2000 tx=0x7f58300aa070 comp rx=0 tx=0).stop 2023-11-28T21:47:19.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 -- 172.21.15.12:0/250554732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c153ee0 msgr2=0x7f583c159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:19.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c153ee0 0x7f583c159270 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f583400cb40 tx=0x7f583400a060 comp rx=0 tx=0).stop 2023-11-28T21:47:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 -- 172.21.15.12:0/250554732 shutdown_connections 2023-11-28T21:47:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f58200664a0 0x7f5820068960 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f583c153ee0 0x7f583c159270 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f583c070bd0 0x7f583c0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 --2- 172.21.15.12:0/250554732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f583c06a5b0 0x7f583c1f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 -- 172.21.15.12:0/250554732 >> 172.21.15.12:0/250554732 conn(0x7f583c0c8450 msgr2=0x7f583c06b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:19.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 -- 172.21.15.12:0/250554732 shutdown_connections 2023-11-28T21:47:19.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:19.165+0000 7f5842e79640 1 -- 172.21.15.12:0/250554732 wait complete. 2023-11-28T21:47:19.192 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2023-11-28T21:47:21.193 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:47:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.341+0000 7fee4f3b8640 1 Processor -- start 2023-11-28T21:47:21.349 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.341+0000 7fee4f3b8640 1 -- start start 2023-11-28T21:47:21.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4f3b8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee48151af0 0x7fee480c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4f3b8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fee48151ed0 0x7fee480c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4f3b8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee48157b70 con 0x7fee4806e300 2023-11-28T21:47:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4f3b8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee480707f0 con 0x7fee48151ed0 2023-11-28T21:47:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4f3b8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee480c7b90 con 0x7fee48151af0 2023-11-28T21:47:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4c953640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fee48151ed0 0x7fee480c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4d955640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fee4806e300 0x7fee48072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33990/0 (socket says 172.21.15.12:33990) 2023-11-28T21:47:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4d955640 1 -- 172.21.15.12:0/2352441522 learned_addr learned my addr 172.21.15.12:0/2352441522 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:47:21.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4d154640 1 --2- 172.21.15.12:0/2352441522 >> v2:172.21.15.144:3300/0 conn(0x7fee48151af0 0x7fee480c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4c953640 1 -- 172.21.15.12:0/2352441522 >> v2:172.21.15.144:3300/0 conn(0x7fee48151af0 msgr2=0x7fee480c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:47:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4c953640 1 --2- 172.21.15.12:0/2352441522 >> v2:172.21.15.144:3300/0 conn(0x7fee48151af0 0x7fee480c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4c953640 1 -- 172.21.15.12:0/2352441522 >> v1:172.21.15.12:6789/0 conn(0x7fee4806e300 legacy=0x7fee48072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee4c953640 1 -- 172.21.15.12:0/2352441522 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48157e50 con 0x7fee48151ed0 2023-11-28T21:47:21.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.345+0000 7fee3ffff640 1 -- 172.21.15.12:0/2352441522 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035901246 0 0) 0x7fee48157b70 con 0x7fee4806e300 2023-11-28T21:47:21.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4c953640 1 --2- 172.21.15.12:0/2352441522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee48151ed0 0x7fee480c49d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fee480c7d20 tx=0x7fee3801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57157e8cabfa1baf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee3ffff640 1 -- 172.21.15.12:0/2352441522 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380ab600 con 0x7fee48151ed0 2023-11-28T21:47:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee3ffff640 1 -- 172.21.15.12:0/2352441522 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee380a7050 con 0x7fee48151ed0 2023-11-28T21:47:21.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee3ffff640 1 -- 172.21.15.12:0/2352441522 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee380ab980 con 0x7fee48151ed0 2023-11-28T21:47:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4f3b8640 1 -- 172.21.15.12:0/2352441522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee48151ed0 msgr2=0x7fee480c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4f3b8640 1 --2- 172.21.15.12:0/2352441522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee48151ed0 0x7fee480c49d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fee480c7d20 tx=0x7fee3801c460 comp rx=0 tx=0).stop 2023-11-28T21:47:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4f3b8640 1 -- 172.21.15.12:0/2352441522 shutdown_connections 2023-11-28T21:47:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4f3b8640 1 --2- 172.21.15.12:0/2352441522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee48151ed0 0x7fee480c49d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4f3b8640 1 --2- 172.21.15.12:0/2352441522 >> v2:172.21.15.144:3300/0 conn(0x7fee48151af0 0x7fee480c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4f3b8640 1 -- 172.21.15.12:0/2352441522 >> 172.21.15.12:0/2352441522 conn(0x7fee48066360 msgr2=0x7fee48066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4f3b8640 1 -- 172.21.15.12:0/2352441522 shutdown_connections 2023-11-28T21:47:21.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4f3b8640 1 -- 172.21.15.12:0/2352441522 wait complete. 2023-11-28T21:47:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4f3b8640 1 Processor -- start 2023-11-28T21:47:21.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.349+0000 7fee4f3b8640 1 -- start start 2023-11-28T21:47:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4f3b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee4806e300 0x7fee4814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4f3b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48151af0 0x7fee481f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4f3b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee48155470 0x7fee481f1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4f3b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee481451d0 con 0x7fee4806e300 2023-11-28T21:47:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4f3b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee481f46d0 con 0x7fee48151af0 2023-11-28T21:47:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4f3b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee481f43d0 con 0x7fee48155470 2023-11-28T21:47:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4d154640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee4806e300 0x7fee4814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:21.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4d154640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee4806e300 0x7fee4814d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57198/0 (socket says 172.21.15.12:57198) 2023-11-28T21:47:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4d154640 1 -- 172.21.15.12:0/1427161161 learned_addr learned my addr 172.21.15.12:0/1427161161 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:47:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4d955640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee48155470 0x7fee481f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4c953640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48151af0 0x7fee481f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4d154640 1 -- 172.21.15.12:0/1427161161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee48155470 msgr2=0x7fee481f1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4d154640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee48155470 0x7fee481f1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4d154640 1 -- 172.21.15.12:0/1427161161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48151af0 msgr2=0x7fee481f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4d154640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48151af0 0x7fee481f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:21.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4d154640 1 -- 172.21.15.12:0/1427161161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee38009420 con 0x7fee4806e300 2023-11-28T21:47:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4d154640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee4806e300 0x7fee4814d2e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fee4400e960 tx=0x7fee4400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee3dffb640 1 -- 172.21.15.12:0/1427161161 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44009690 con 0x7fee4806e300 2023-11-28T21:47:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee3dffb640 1 -- 172.21.15.12:0/1427161161 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4400ca00 con 0x7fee4806e300 2023-11-28T21:47:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee3dffb640 1 -- 172.21.15.12:0/1427161161 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4400cd70 con 0x7fee4806e300 2023-11-28T21:47:21.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4f3b8640 1 -- 172.21.15.12:0/1427161161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee48141280 con 0x7fee4806e300 2023-11-28T21:47:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.353+0000 7fee4f3b8640 1 -- 172.21.15.12:0/1427161161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee48141820 con 0x7fee4806e300 2023-11-28T21:47:21.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.357+0000 7fee4f3b8640 1 -- 172.21.15.12:0/1427161161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee48074f60 con 0x7fee4806e300 2023-11-28T21:47:21.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.357+0000 7fee3dffb640 1 -- 172.21.15.12:0/1427161161 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fee44013470 con 0x7fee4806e300 2023-11-28T21:47:21.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.357+0000 7fee3dffb640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee2c066470 0x7fee2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:47:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.357+0000 7fee3dffb640 1 -- 172.21.15.12:0/1427161161 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7fee440846f0 con 0x7fee4806e300 2023-11-28T21:47:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.357+0000 7fee4c953640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee2c066470 0x7fee2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:47:21.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.361+0000 7fee4c953640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee2c066470 0x7fee2c068930 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fee380ad000 tx=0x7fee38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:47:21.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.361+0000 7fee3dffb640 1 -- 172.21.15.12:0/1427161161 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee44051530 con 0x7fee4806e300 2023-11-28T21:47:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.557+0000 7fee4f3b8640 1 -- 172.21.15.12:0/1427161161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee4806d380 con 0x7fee4806e300 2023-11-28T21:47:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.557+0000 7fee3dffb640 1 -- 172.21.15.12:0/1427161161 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fee44054b50 con 0x7fee4806e300 2023-11-28T21:47:21.565 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:47:21.565 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:09.646722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:active","state_seq":21,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:47:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 55 2023-11-28T21:47:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 -- 172.21.15.12:0/1427161161 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee2c066470 msgr2=0x7fee2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee2c066470 0x7fee2c068930 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fee380ad000 tx=0x7fee38008040 comp rx=0 tx=0).stop 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 -- 172.21.15.12:0/1427161161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee4806e300 msgr2=0x7fee4814d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee4806e300 0x7fee4814d2e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fee4400e960 tx=0x7fee4400ee20 comp rx=0 tx=0).stop 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 -- 172.21.15.12:0/1427161161 shutdown_connections 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee2c066470 0x7fee2c068930 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee48155470 0x7fee481f1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee48151af0 0x7fee481f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 --2- 172.21.15.12:0/1427161161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee4806e300 0x7fee4814d2e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 -- 172.21.15.12:0/1427161161 >> 172.21.15.12:0/1427161161 conn(0x7fee48066360 msgr2=0x7fee48073a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 -- 172.21.15.12:0/1427161161 shutdown_connections 2023-11-28T21:47:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:47:21.561+0000 7fee4f3b8640 1 -- 172.21.15.12:0/1427161161 wait complete. 2023-11-28T21:47:21.586 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2023-11-28T21:47:21.587 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 79 secs before thrashing 2023-11-28T21:47:45.581 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:47:45.586 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:48:15.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:48:15.701 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:48:40.588 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:48:40.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.732+0000 7f1a0dbdf640 1 Processor -- start 2023-11-28T21:48:40.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.732+0000 7f1a0dbdf640 1 -- start start 2023-11-28T21:48:40.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a0dbdf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806e300 0x7f1a08156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a0dbdf640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a08067410 con 0x7f1a0806cf40 2023-11-28T21:48:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a0dbdf640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0806e090 con 0x7f1a08150c20 2023-11-28T21:48:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a0dbdf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08157f30 con 0x7f1a0806e300 2023-11-28T21:48:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a07fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1a0806cf40 0x7f1a08158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37662/0 (socket says 172.21.15.12:37662) 2023-11-28T21:48:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a07fff640 1 -- 172.21.15.12:0/1746971085 learned_addr learned my addr 172.21.15.12:0/1746971085 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:48:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a06ffd640 1 --2- 172.21.15.12:0/1746971085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806e300 0x7f1a08156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a067fc640 1 -- 172.21.15.12:0/1746971085 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241092860 0 0) 0x7f1a08067410 con 0x7f1a0806cf40 2023-11-28T21:48:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a067fc640 1 -- 172.21.15.12:0/1746971085 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f0003530 con 0x7f1a0806cf40 2023-11-28T21:48:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a067fc640 1 -- 172.21.15.12:0/1746971085 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282167372 0 0) 0x7f1a0806e090 con 0x7f1a08150c20 2023-11-28T21:48:41.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a067fc640 1 -- 172.21.15.12:0/1746971085 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a08067410 con 0x7f1a08150c20 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a067fc640 1 -- 172.21.15.12:0/1746971085 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2163260765 0 0) 0x7f19f0003530 con 0x7f1a0806cf40 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a06ffd640 1 -- 172.21.15.12:0/1746971085 >> v1:172.21.15.144:6789/0 conn(0x7f1a08150c20 legacy=0x7f1a080c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a06ffd640 1 -- 172.21.15.12:0/1746971085 >> v1:172.21.15.12:6789/0 conn(0x7f1a0806cf40 legacy=0x7f1a08158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.736+0000 7f1a06ffd640 1 -- 172.21.15.12:0/1746971085 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a08151000 con 0x7f1a0806e300 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a06ffd640 1 --2- 172.21.15.12:0/1746971085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806e300 0x7f1a08156910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0097f0 tx=0x7f19fc01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b81a7f98c71018d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a067fc640 1 -- 172.21.15.12:0/1746971085 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00001980 con 0x7f1a0806e300 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a067fc640 1 -- 172.21.15.12:0/1746971085 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc0a7050 con 0x7f1a0806e300 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a067fc640 1 -- 172.21.15.12:0/1746971085 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0ab600 con 0x7f1a0806e300 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/1746971085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806e300 msgr2=0x7f1a08156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a0dbdf640 1 --2- 172.21.15.12:0/1746971085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806e300 0x7f1a08156910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0097f0 tx=0x7f19fc01c3e0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/1746971085 shutdown_connections 2023-11-28T21:48:41.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a0dbdf640 1 --2- 172.21.15.12:0/1746971085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806e300 0x7f1a08156910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/1746971085 >> 172.21.15.12:0/1746971085 conn(0x7f1a08066360 msgr2=0x7f1a08066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:41.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/1746971085 shutdown_connections 2023-11-28T21:48:41.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:40.740+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/1746971085 wait complete. 2023-11-28T21:48:41.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a0dbdf640 1 Processor -- start 2023-11-28T21:48:41.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a0dbdf640 1 -- start start 2023-11-28T21:48:41.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a0dbdf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806cf40 0x7f1a08070f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a0dbdf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0806e300 0x7f1a081f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a0dbdf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a08150c20 0x7f1a0806adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a0dbdf640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08160d10 con 0x7f1a0806e300 2023-11-28T21:48:41.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a0dbdf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a081f6aa0 con 0x7f1a08150c20 2023-11-28T21:48:41.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a0dbdf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080715b0 con 0x7f1a0806cf40 2023-11-28T21:48:41.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a06ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0806e300 0x7f1a081f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a06ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0806e300 0x7f1a081f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51638/0 (socket says 172.21.15.12:51638) 2023-11-28T21:48:41.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a06ffd640 1 -- 172.21.15.12:0/3846629534 learned_addr learned my addr 172.21.15.12:0/3846629534 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:41.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a07fff640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a08150c20 0x7f1a0806adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a077fe640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806cf40 0x7f1a08070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a06ffd640 1 -- 172.21.15.12:0/3846629534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806cf40 msgr2=0x7f1a08070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a06ffd640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806cf40 0x7f1a08070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a06ffd640 1 -- 172.21.15.12:0/3846629534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a08150c20 msgr2=0x7f1a0806adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a06ffd640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a08150c20 0x7f1a0806adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a06ffd640 1 -- 172.21.15.12:0/3846629534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc009420 con 0x7f1a0806e300 2023-11-28T21:48:41.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.056+0000 7f1a06ffd640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0806e300 0x7f1a081f6560 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f19fc000c10 tx=0x7f19fc019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:41.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.060+0000 7f19effff640 1 -- 172.21.15.12:0/3846629534 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0b9070 con 0x7f1a0806e300 2023-11-28T21:48:41.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.060+0000 7f19effff640 1 -- 172.21.15.12:0/3846629534 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc0038b0 con 0x7f1a0806e300 2023-11-28T21:48:41.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.060+0000 7f19effff640 1 -- 172.21.15.12:0/3846629534 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc003c20 con 0x7f1a0806e300 2023-11-28T21:48:41.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.060+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/3846629534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a08161ac0 con 0x7f1a0806e300 2023-11-28T21:48:41.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.060+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/3846629534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a081f2a20 con 0x7f1a0806e300 2023-11-28T21:48:41.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.060+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/3846629534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a08074f60 con 0x7f1a0806e300 2023-11-28T21:48:41.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.064+0000 7f19effff640 1 -- 172.21.15.12:0/3846629534 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f19fc0b2040 con 0x7f1a0806e300 2023-11-28T21:48:41.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.064+0000 7f19effff640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19f4066470 0x7f19f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.064+0000 7f19effff640 1 -- 172.21.15.12:0/3846629534 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f19fc11d990 con 0x7f1a0806e300 2023-11-28T21:48:41.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.064+0000 7f1a077fe640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19f4066470 0x7f19f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.064+0000 7f1a077fe640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19f4066470 0x7f19f4068930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f19f8007fd0 tx=0x7f19f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:41.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.072+0000 7f19effff640 1 -- 172.21.15.12:0/3846629534 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f19fc0ea990 con 0x7f1a0806e300 2023-11-28T21:48:41.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.300+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/3846629534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a080c2c30 con 0x7f1a0806e300 2023-11-28T21:48:41.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.300+0000 7f19effff640 1 -- 172.21.15.12:0/3846629534 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f19fc0edfb0 con 0x7f1a0806e300 2023-11-28T21:48:41.305 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:48:41.305 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:09.646722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:active","state_seq":21,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:48:41.305 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 55 2023-11-28T21:48:41.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/3846629534 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19f4066470 msgr2=0x7f19f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19f4066470 0x7f19f4068930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f19f8007fd0 tx=0x7f19f8009040 comp rx=0 tx=0).stop 2023-11-28T21:48:41.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/3846629534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0806e300 msgr2=0x7f1a081f6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0806e300 0x7f1a081f6560 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f19fc000c10 tx=0x7f19fc019180 comp rx=0 tx=0).stop 2023-11-28T21:48:41.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/3846629534 shutdown_connections 2023-11-28T21:48:41.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f19f4066470 0x7f19f4068930 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a08150c20 0x7f1a0806adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a0806e300 0x7f1a081f6560 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 --2- 172.21.15.12:0/3846629534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a0806cf40 0x7f1a08070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/3846629534 >> 172.21.15.12:0/3846629534 conn(0x7f1a08066360 msgr2=0x7f1a080c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:41.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/3846629534 shutdown_connections 2023-11-28T21:48:41.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.304+0000 7f1a0dbdf640 1 -- 172.21.15.12:0/3846629534 wait complete. 2023-11-28T21:48:41.332 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2023-11-28T21:48:41.333 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-11-28T21:48:41.333 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:48:41.354 INFO:tasks.ceph.mds.e:Stopped 2023-11-28T21:48:41.355 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:48:41.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa888d1d640 1 Processor -- start 2023-11-28T21:48:41.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa888d1d640 1 -- start start 2023-11-28T21:48:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa888d1d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa884151af0 0x7fa8840c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa888d1d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa884151ed0 0x7fa8840c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa888d1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa88406e300 0x7fa8840c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa888d1d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8840707f0 con 0x7fa88406e300 2023-11-28T21:48:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa888d1d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8840c7b90 con 0x7fa884151ed0 2023-11-28T21:48:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa888d1d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa884071c20 con 0x7fa884151af0 2023-11-28T21:48:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa882d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa88406e300 0x7fa8840c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa882d9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa88406e300 0x7fa8840c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51652/0 (socket says 172.21.15.12:51652) 2023-11-28T21:48:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa882d9d640 1 -- 172.21.15.12:0/3142148900 learned_addr learned my addr 172.21.15.12:0/3142148900 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:41.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa881d9b640 1 --2- 172.21.15.12:0/3142148900 >> v2:172.21.15.144:3300/0 conn(0x7fa884151ed0 0x7fa8840c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.540+0000 7fa88259c640 1 --2- 172.21.15.12:0/3142148900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa884151af0 0x7fa8840c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa882d9d640 1 -- 172.21.15.12:0/3142148900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa884151af0 msgr2=0x7fa8840c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:41.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa882d9d640 1 --2- 172.21.15.12:0/3142148900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa884151af0 0x7fa8840c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa882d9d640 1 -- 172.21.15.12:0/3142148900 >> v2:172.21.15.144:3300/0 conn(0x7fa884151ed0 msgr2=0x7fa8840c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:41.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa882d9d640 1 --2- 172.21.15.12:0/3142148900 >> v2:172.21.15.144:3300/0 conn(0x7fa884151ed0 0x7fa8840c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa882d9d640 1 -- 172.21.15.12:0/3142148900 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa884157b70 con 0x7fa88406e300 2023-11-28T21:48:41.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa882d9d640 1 --2- 172.21.15.12:0/3142148900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88406e300 0x7fa8840c4f10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa88406c7c0 tx=0x7fa87c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f6e94046bdd591 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:41.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa88159a640 1 -- 172.21.15.12:0/3142148900 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0a5740 con 0x7fa88406e300 2023-11-28T21:48:41.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa88159a640 1 -- 172.21.15.12:0/3142148900 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c0a58a0 con 0x7fa88406e300 2023-11-28T21:48:41.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa88159a640 1 -- 172.21.15.12:0/3142148900 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0a5c30 con 0x7fa88406e300 2023-11-28T21:48:41.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa888d1d640 1 -- 172.21.15.12:0/3142148900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88406e300 msgr2=0x7fa8840c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.544+0000 7fa888d1d640 1 --2- 172.21.15.12:0/3142148900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88406e300 0x7fa8840c4f10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa88406c7c0 tx=0x7fa87c01c9d0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 -- 172.21.15.12:0/3142148900 shutdown_connections 2023-11-28T21:48:41.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 --2- 172.21.15.12:0/3142148900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88406e300 0x7fa8840c4f10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 --2- 172.21.15.12:0/3142148900 >> v2:172.21.15.144:3300/0 conn(0x7fa884151ed0 0x7fa8840c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 --2- 172.21.15.12:0/3142148900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa884151af0 0x7fa8840c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 -- 172.21.15.12:0/3142148900 >> 172.21.15.12:0/3142148900 conn(0x7fa884066360 msgr2=0x7fa884066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:41.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 -- 172.21.15.12:0/3142148900 shutdown_connections 2023-11-28T21:48:41.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 -- 172.21.15.12:0/3142148900 wait complete. 2023-11-28T21:48:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 Processor -- start 2023-11-28T21:48:41.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 -- start start 2023-11-28T21:48:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa88406e300 0x7fa88415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa884151af0 0x7fa8840ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88415f960 0x7fa8841f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88415ae00 con 0x7fa88415f960 2023-11-28T21:48:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8841f63f0 con 0x7fa88406e300 2023-11-28T21:48:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa888d1d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8841f2db0 con 0x7fa884151af0 2023-11-28T21:48:41.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa882d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88415f960 0x7fa8841f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa882d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88415f960 0x7fa8841f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51668/0 (socket says 172.21.15.12:51668) 2023-11-28T21:48:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa88259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa88406e300 0x7fa88415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa882d9d640 1 -- 172.21.15.12:0/3034858296 learned_addr learned my addr 172.21.15.12:0/3034858296 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:41.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa881d9b640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa884151af0 0x7fa8840ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa882d9d640 1 -- 172.21.15.12:0/3034858296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa884151af0 msgr2=0x7fa8840ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa882d9d640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa884151af0 0x7fa8840ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa882d9d640 1 -- 172.21.15.12:0/3034858296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa88406e300 msgr2=0x7fa88415a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa882d9d640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa88406e300 0x7fa88415a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.548+0000 7fa882d9d640 1 -- 172.21.15.12:0/3034858296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c009b30 con 0x7fa88415f960 2023-11-28T21:48:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.552+0000 7fa882d9d640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88415f960 0x7fa8841f6880 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0af040 tx=0x7fa87c019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:41.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.552+0000 7fa872ffd640 1 -- 172.21.15.12:0/3034858296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0b8670 con 0x7fa88415f960 2023-11-28T21:48:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.552+0000 7fa872ffd640 1 -- 172.21.15.12:0/3034858296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c0b87f0 con 0x7fa88415f960 2023-11-28T21:48:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.552+0000 7fa888d1d640 1 -- 172.21.15.12:0/3034858296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8841f3070 con 0x7fa88415f960 2023-11-28T21:48:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.552+0000 7fa872ffd640 1 -- 172.21.15.12:0/3034858296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0b8ba0 con 0x7fa88415f960 2023-11-28T21:48:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.552+0000 7fa888d1d640 1 -- 172.21.15.12:0/3034858296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8841f3630 con 0x7fa88415f960 2023-11-28T21:48:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.552+0000 7fa888d1d640 1 -- 172.21.15.12:0/3034858296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa884074f60 con 0x7fa88415f960 2023-11-28T21:48:41.556 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.552+0000 7fa872ffd640 1 -- 172.21.15.12:0/3034858296 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa87c0b8d00 con 0x7fa88415f960 2023-11-28T21:48:41.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.556+0000 7fa872ffd640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa864066470 0x7fa864068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:41.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.556+0000 7fa872ffd640 1 -- 172.21.15.12:0/3034858296 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7fa87c11f9e0 con 0x7fa88415f960 2023-11-28T21:48:41.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.556+0000 7fa88259c640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa864066470 0x7fa864068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:41.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.556+0000 7fa88259c640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa864066470 0x7fa864068930 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8740045f0 tx=0x7fa874008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:41.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.560+0000 7fa872ffd640 1 -- 172.21.15.12:0/3034858296 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa87c0ec8e0 con 0x7fa88415f960 2023-11-28T21:48:41.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.748+0000 7fa888d1d640 1 -- 172.21.15.12:0/3034858296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa884072c10 con 0x7fa88415f960 2023-11-28T21:48:41.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa872ffd640 1 -- 172.21.15.12:0/3034858296 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+6506 (secure 0 0 0) 0x7fa87c0eff00 con 0x7fa88415f960 2023-11-28T21:48:41.754 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:48:41.755 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:09.646722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:active","state_seq":21,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:48:41.755 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 55 2023-11-28T21:48:41.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 -- 172.21.15.12:0/3034858296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa864066470 msgr2=0x7fa864068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa864066470 0x7fa864068930 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8740045f0 tx=0x7fa874008040 comp rx=0 tx=0).stop 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 -- 172.21.15.12:0/3034858296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88415f960 msgr2=0x7fa8841f6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88415f960 0x7fa8841f6880 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0af040 tx=0x7fa87c019770 comp rx=0 tx=0).stop 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 -- 172.21.15.12:0/3034858296 shutdown_connections 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa864066470 0x7fa864068930 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa88415f960 0x7fa8841f6880 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa884151af0 0x7fa8840ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 --2- 172.21.15.12:0/3034858296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa88406e300 0x7fa88415a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 -- 172.21.15.12:0/3034858296 >> 172.21.15.12:0/3034858296 conn(0x7fa884066360 msgr2=0x7fa88406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 -- 172.21.15.12:0/3034858296 shutdown_connections 2023-11-28T21:48:41.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:41.752+0000 7fa888d1d640 1 -- 172.21.15.12:0/3034858296 wait complete. 2023-11-28T21:48:41.776 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:48:43.777 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:48:43.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.924+0000 7f31cc417640 1 Processor -- start 2023-11-28T21:48:43.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.924+0000 7f31cc417640 1 -- start start 2023-11-28T21:48:43.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31cc417640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c4158b20 0x7f31c4156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:43.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31cc417640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c4067410 con 0x7f31c4150c20 2023-11-28T21:48:43.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31cc417640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31c406e090 con 0x7f31c406e300 2023-11-28T21:48:43.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31cc417640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c4157f30 con 0x7f31c4158b20 2023-11-28T21:48:43.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31ca1b3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f31c4150c20 0x7f31c40c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:37664/0 (socket says 172.21.15.12:37664) 2023-11-28T21:48:43.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31ca1b3640 1 -- 172.21.15.12:0/2360429701 learned_addr learned my addr 172.21.15.12:0/2360429701 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:48:43.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31ca9b4640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c4158b20 0x7f31c4156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:43.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128488054 0 0) 0x7f31c4067410 con 0x7f31c4150c20 2023-11-28T21:48:43.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b0005740 con 0x7f31c4150c20 2023-11-28T21:48:43.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 225626594 0 0) 0x7f31b0005740 con 0x7f31c4150c20 2023-11-28T21:48:43.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c4158b20 msgr2=0x7f31c4156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:43.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c4158b20 0x7f31c4156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:43.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 >> v1:172.21.15.144:6789/0 conn(0x7f31c406e300 legacy=0x7f31c4158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:43.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c4151000 con 0x7f31c4150c20 2023-11-28T21:48:43.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f31c0001680 con 0x7f31c4150c20 2023-11-28T21:48:43.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 >> v1:172.21.15.12:6789/0 conn(0x7f31c4150c20 legacy=0x7f31c40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:43.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c4158b20 0x7f31b0005c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:43.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b00061b0 0x7f31b00069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:43.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b0006f20 0x7f31b00073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:43.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c4157f30 con 0x7f31b00061b0 2023-11-28T21:48:43.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b000dd00 con 0x7f31b0006f20 2023-11-28T21:48:43.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b000dec0 con 0x7f31c4158b20 2023-11-28T21:48:43.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c99b2640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b00061b0 0x7f31b00069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:43.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31ca1b3640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c4158b20 0x7f31b0005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:43.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31ca9b4640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b0006f20 0x7f31b00073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:43.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c99b2640 1 -- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c4158b20 msgr2=0x7f31b0005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:43.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c99b2640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c4158b20 0x7f31b0005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:43.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c99b2640 1 -- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b0006f20 msgr2=0x7f31b00073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:43.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c99b2640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b0006f20 0x7f31b00073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:43.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.928+0000 7f31c99b2640 1 -- 172.21.15.12:0/2360429701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b0005740 con 0x7f31b00061b0 2023-11-28T21:48:43.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31c99b2640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b00061b0 0x7f31b00069e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f31b4001750 tx=0x7f31b401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278b24fb2225cd6a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:43.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b40052d0 con 0x7f31b00061b0 2023-11-28T21:48:43.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b40a7050 con 0x7f31b00061b0 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31c91b1640 1 -- 172.21.15.12:0/2360429701 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4003600 con 0x7f31b00061b0 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31cc417640 1 -- 172.21.15.12:0/2360429701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b00061b0 msgr2=0x7f31b00069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31cc417640 1 --2- 172.21.15.12:0/2360429701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31b00061b0 0x7f31b00069e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f31b4001750 tx=0x7f31b401c300 comp rx=0 tx=0).stop 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31ca9b4640 1 -- 172.21.15.12:0/2360429701 reap_dead start 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31cc417640 1 -- 172.21.15.12:0/2360429701 shutdown_connections 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31cc417640 1 -- 172.21.15.12:0/2360429701 >> 172.21.15.12:0/2360429701 conn(0x7f31c4066360 msgr2=0x7f31c4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31cc417640 1 -- 172.21.15.12:0/2360429701 shutdown_connections 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31cc417640 1 -- 172.21.15.12:0/2360429701 wait complete. 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31cc417640 1 Processor -- start 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.932+0000 7f31cc417640 1 -- start start 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31cc417640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b00061b0 0x7f31c414c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31cc417640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c414fb50 0x7f31c41e2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:43.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31cc417640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c414a5b0 0x7f31c414aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31cc417640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c414ffb0 con 0x7f31c414fb50 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31cc417640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c414af70 con 0x7f31b00061b0 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31cc417640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c41eeed0 con 0x7f31c414a5b0 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31c99b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c414fb50 0x7f31c41e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31c99b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c414fb50 0x7f31c41e2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51680/0 (socket says 172.21.15.12:51680) 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31c99b2640 1 -- 172.21.15.12:0/925935680 learned_addr learned my addr 172.21.15.12:0/925935680 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31ca9b4640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c414a5b0 0x7f31c414aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31ca1b3640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b00061b0 0x7f31c414c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31c99b2640 1 -- 172.21.15.12:0/925935680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c414a5b0 msgr2=0x7f31c414aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31c99b2640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c414a5b0 0x7f31c414aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31c99b2640 1 -- 172.21.15.12:0/925935680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b00061b0 msgr2=0x7f31c414c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:43.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31c99b2640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b00061b0 0x7f31c414c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31c99b2640 1 -- 172.21.15.12:0/925935680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c41ef1b0 con 0x7f31c414fb50 2023-11-28T21:48:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31ca1b3640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b00061b0 0x7f31c414c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:48:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31c99b2640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c414fb50 0x7f31c41e2600 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f31b4005040 tx=0x7f31b0007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31baffd640 1 -- 172.21.15.12:0/925935680 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4007350 con 0x7f31c414fb50 2023-11-28T21:48:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31baffd640 1 -- 172.21.15.12:0/925935680 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b40074b0 con 0x7f31c414fb50 2023-11-28T21:48:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31baffd640 1 -- 172.21.15.12:0/925935680 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b40067e0 con 0x7f31c414fb50 2023-11-28T21:48:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31cc417640 1 -- 172.21.15.12:0/925935680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c41ef490 con 0x7f31c414fb50 2023-11-28T21:48:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31cc417640 1 -- 172.21.15.12:0/925935680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c41efa50 con 0x7f31c414fb50 2023-11-28T21:48:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31cc417640 1 -- 172.21.15.12:0/925935680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c41efc40 con 0x7f31c414fb50 2023-11-28T21:48:43.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.936+0000 7f31baffd640 1 -- 172.21.15.12:0/925935680 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f31b4006940 con 0x7f31c414fb50 2023-11-28T21:48:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.940+0000 7f31baffd640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31a8066470 0x7f31a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.940+0000 7f31baffd640 1 -- 172.21.15.12:0/925935680 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f31b411e870 con 0x7f31c414fb50 2023-11-28T21:48:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.940+0000 7f31ca1b3640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31a8066470 0x7f31a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:43.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.940+0000 7f31ca1b3640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31a8066470 0x7f31a8068930 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f31c0001060 tx=0x7f31c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:43.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:43.944+0000 7f31baffd640 1 -- 172.21.15.12:0/925935680 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f31b40eb6a0 con 0x7f31c414fb50 2023-11-28T21:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.132+0000 7f31cc417640 1 -- 172.21.15.12:0/925935680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31c4074f60 con 0x7f31c414fb50 2023-11-28T21:48:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.132+0000 7f31baffd640 1 -- 172.21.15.12:0/925935680 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f31b40eecc0 con 0x7f31c414fb50 2023-11-28T21:48:44.137 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:48:44.137 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:09.646722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:active","state_seq":21,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:48:44.137 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 55 2023-11-28T21:48:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 -- 172.21.15.12:0/925935680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31a8066470 msgr2=0x7f31a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31a8066470 0x7f31a8068930 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f31c0001060 tx=0x7f31c0009040 comp rx=0 tx=0).stop 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 -- 172.21.15.12:0/925935680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c414fb50 msgr2=0x7f31c41e2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c414fb50 0x7f31c41e2600 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f31b4005040 tx=0x7f31b0007a10 comp rx=0 tx=0).stop 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 -- 172.21.15.12:0/925935680 shutdown_connections 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31a8066470 0x7f31a8068930 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31c414a5b0 0x7f31c414aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31c414fb50 0x7f31c41e2600 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 --2- 172.21.15.12:0/925935680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31b00061b0 0x7f31c414c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 -- 172.21.15.12:0/925935680 >> 172.21.15.12:0/925935680 conn(0x7f31c4066360 msgr2=0x7f31c40c9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 -- 172.21.15.12:0/925935680 shutdown_connections 2023-11-28T21:48:44.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:44.136+0000 7f31cc417640 1 -- 172.21.15.12:0/925935680 wait complete. 2023-11-28T21:48:44.158 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:48:45.775 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:48:45.779 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:48:46.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:48:46.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.308+0000 7f6dd5d80640 1 Processor -- start 2023-11-28T21:48:46.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.308+0000 7f6dd5d80640 1 -- start start 2023-11-28T21:48:46.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dd5d80640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd0151ba0 0x7f6dd01579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:46.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dd5d80640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6dd015b9e0 0x7f6dd006cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:46.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dd5d80640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd015be30 con 0x7f6dd0066fa0 2023-11-28T21:48:46.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dd5d80640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd00c24f0 con 0x7f6dd015b9e0 2023-11-28T21:48:46.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dd5d80640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd015afd0 con 0x7f6dd0151ba0 2023-11-28T21:48:46.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dcf7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd0151ba0 0x7f6dd01579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:46.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dcffff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6dd015b9e0 0x7f6dd006cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:46.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dcffff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6dd015b9e0 0x7f6dd006cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:48190/0 (socket says 172.21.15.12:48190) 2023-11-28T21:48:46.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dcffff640 1 -- 172.21.15.12:0/2030680525 learned_addr learned my addr 172.21.15.12:0/2030680525 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:46.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dcf7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd0151ba0 0x7f6dd01579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:45540/0 (socket says 172.21.15.12:45540) 2023-11-28T21:48:46.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dceffd640 1 --1- 172.21.15.12:0/2030680525 >> v1:172.21.15.144:6789/0 conn(0x7f6dd0066fa0 0x7f6dd0071f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:54212/0 (socket says 172.21.15.12:54212) 2023-11-28T21:48:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dcffff640 1 -- 172.21.15.12:0/2030680525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd0151ba0 msgr2=0x7f6dd01579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dcffff640 1 --2- 172.21.15.12:0/2030680525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd0151ba0 0x7f6dd01579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dcffff640 1 -- 172.21.15.12:0/2030680525 >> v1:172.21.15.144:6789/0 conn(0x7f6dd0066fa0 legacy=0x7f6dd0071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:46.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.312+0000 7f6dcffff640 1 -- 172.21.15.12:0/2030680525 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd006da10 con 0x7f6dd015b9e0 2023-11-28T21:48:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dcffff640 1 --2- 172.21.15.12:0/2030680525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dd015b9e0 0x7f6dd006cd70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009e50 tx=0x7f6dc801c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f460072dc786291 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:46.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dce7fc640 1 -- 172.21.15.12:0/2030680525 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc80a57c0 con 0x7f6dd015b9e0 2023-11-28T21:48:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dce7fc640 1 -- 172.21.15.12:0/2030680525 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc80a5920 con 0x7f6dd015b9e0 2023-11-28T21:48:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dce7fc640 1 -- 172.21.15.12:0/2030680525 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc80a5cb0 con 0x7f6dd015b9e0 2023-11-28T21:48:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dd5d80640 1 -- 172.21.15.12:0/2030680525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dd015b9e0 msgr2=0x7f6dd006cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:46.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dd5d80640 1 --2- 172.21.15.12:0/2030680525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dd015b9e0 0x7f6dd006cd70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009e50 tx=0x7f6dc801c9e0 comp rx=0 tx=0).stop 2023-11-28T21:48:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dd5d80640 1 -- 172.21.15.12:0/2030680525 shutdown_connections 2023-11-28T21:48:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dd5d80640 1 --2- 172.21.15.12:0/2030680525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dd015b9e0 0x7f6dd006cd70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dd5d80640 1 --2- 172.21.15.12:0/2030680525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd0151ba0 0x7f6dd01579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dd5d80640 1 -- 172.21.15.12:0/2030680525 >> 172.21.15.12:0/2030680525 conn(0x7f6dd0066360 msgr2=0x7f6dd0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dd5d80640 1 -- 172.21.15.12:0/2030680525 shutdown_connections 2023-11-28T21:48:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dd5d80640 1 -- 172.21.15.12:0/2030680525 wait complete. 2023-11-28T21:48:46.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dd5d80640 1 Processor -- start 2023-11-28T21:48:46.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.316+0000 7f6dd5d80640 1 -- start start 2023-11-28T21:48:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dd5d80640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dd0066fa0 0x7f6dd00ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:46.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dd5d80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6dd0151ba0 0x7f6dd0142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dd5d80640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd015b9e0 0x7f6dd0143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dd5d80640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01513a0 con 0x7f6dd0151ba0 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dd5d80640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd006fc00 con 0x7f6dd0066fa0 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dd5d80640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd0070470 con 0x7f6dd015b9e0 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dceffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6dd0151ba0 0x7f6dd0142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dceffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6dd0151ba0 0x7f6dd0142e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54646/0 (socket says 172.21.15.12:54646) 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dceffd640 1 -- 172.21.15.12:0/1791371372 learned_addr learned my addr 172.21.15.12:0/1791371372 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dcf7fe640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dd0066fa0 0x7f6dd00ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dcffff640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd015b9e0 0x7f6dd0143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dceffd640 1 -- 172.21.15.12:0/1791371372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd015b9e0 msgr2=0x7f6dd0143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:46.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dceffd640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd015b9e0 0x7f6dd0143370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dceffd640 1 -- 172.21.15.12:0/1791371372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dd0066fa0 msgr2=0x7f6dd00ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dceffd640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dd0066fa0 0x7f6dd00ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dceffd640 1 -- 172.21.15.12:0/1791371372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8009b30 con 0x7f6dd0151ba0 2023-11-28T21:48:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dcffff640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd015b9e0 0x7f6dd0143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:48:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dceffd640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6dd0151ba0 0x7f6dd0142e30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400c370 tx=0x7f6dc400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6db7fff640 1 -- 172.21.15.12:0/1791371372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40074e0 con 0x7f6dd0151ba0 2023-11-28T21:48:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6db7fff640 1 -- 172.21.15.12:0/1791371372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc4007640 con 0x7f6dd0151ba0 2023-11-28T21:48:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6db7fff640 1 -- 172.21.15.12:0/1791371372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4004a30 con 0x7f6dd0151ba0 2023-11-28T21:48:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dd5d80640 1 -- 172.21.15.12:0/1791371372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd0151100 con 0x7f6dd0151ba0 2023-11-28T21:48:46.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dd5d80640 1 -- 172.21.15.12:0/1791371372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd00716e0 con 0x7f6dd0151ba0 2023-11-28T21:48:46.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6dd5d80640 1 -- 172.21.15.12:0/1791371372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd0074f60 con 0x7f6dd0151ba0 2023-11-28T21:48:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.320+0000 7f6db7fff640 1 -- 172.21.15.12:0/1791371372 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6dc4004b90 con 0x7f6dd0151ba0 2023-11-28T21:48:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.324+0000 7f6db7fff640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6dbc0664a0 0x7f6dbc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.324+0000 7f6db7fff640 1 -- 172.21.15.12:0/1791371372 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f6dc4083db0 con 0x7f6dd0151ba0 2023-11-28T21:48:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.324+0000 7f6dcf7fe640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6dbc0664a0 0x7f6dbc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:46.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.324+0000 7f6dcf7fe640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6dbc0664a0 0x7f6dbc068960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009b60 tx=0x7f6dc0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:46.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.328+0000 7f6db7fff640 1 -- 172.21.15.12:0/1791371372 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6dc4088070 con 0x7f6dd0151ba0 2023-11-28T21:48:46.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.512+0000 7f6dd5d80640 1 -- 172.21.15.12:0/1791371372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dd014eb10 con 0x7f6dd0151ba0 2023-11-28T21:48:46.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6db7fff640 1 -- 172.21.15.12:0/1791371372 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f6dc4050bd0 con 0x7f6dd0151ba0 2023-11-28T21:48:46.519 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:48:46.519 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:09.646722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:active","state_seq":21,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:48:46.519 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 55 2023-11-28T21:48:46.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 -- 172.21.15.12:0/1791371372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6dbc0664a0 msgr2=0x7f6dbc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:46.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6dbc0664a0 0x7f6dbc068960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009b60 tx=0x7f6dc0009190 comp rx=0 tx=0).stop 2023-11-28T21:48:46.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 -- 172.21.15.12:0/1791371372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6dd0151ba0 msgr2=0x7f6dd0142e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:46.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6dd0151ba0 0x7f6dd0142e30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400c370 tx=0x7f6dc400c830 comp rx=0 tx=0).stop 2023-11-28T21:48:46.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 -- 172.21.15.12:0/1791371372 shutdown_connections 2023-11-28T21:48:46.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6dbc0664a0 0x7f6dbc068960 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:46.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dd015b9e0 0x7f6dd0143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:46.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6dd0151ba0 0x7f6dd0142e30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:46.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 --2- 172.21.15.12:0/1791371372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dd0066fa0 0x7f6dd00ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:46.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 -- 172.21.15.12:0/1791371372 >> 172.21.15.12:0/1791371372 conn(0x7f6dd0066360 msgr2=0x7f6dd0147aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:46.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 -- 172.21.15.12:0/1791371372 shutdown_connections 2023-11-28T21:48:46.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:46.516+0000 7f6dd5d80640 1 -- 172.21.15.12:0/1791371372 wait complete. 2023-11-28T21:48:46.540 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:48:48.541 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:48:48.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8228c9e640 1 Processor -- start 2023-11-28T21:48:48.696 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8228c9e640 1 -- start start 2023-11-28T21:48:48.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8228c9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415b180 0x7f822415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:48.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8228c9e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f822415d210 0x7f822415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:48.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8228c9e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822406d2a0 con 0x7f82241540f0 2023-11-28T21:48:48.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8228c9e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822406e200 con 0x7f822415d210 2023-11-28T21:48:48.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8228c9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822406d580 con 0x7f822415b180 2023-11-28T21:48:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f822259c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f82241540f0 0x7f82240c3be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45478/0 (socket says 172.21.15.12:45478) 2023-11-28T21:48:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f822259c640 1 -- 172.21.15.12:0/2160473101 learned_addr learned my addr 172.21.15.12:0/2160473101 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:48:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8222d9d640 1 --2- 172.21.15.12:0/2160473101 >> v2:172.21.15.12:3300/0 conn(0x7f822415d210 0x7f822415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8221d9b640 1 --2- 172.21.15.12:0/2160473101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415b180 0x7f822415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:48.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f822159a640 1 -- 172.21.15.12:0/2160473101 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813569743 0 0) 0x7f822406d2a0 con 0x7f82241540f0 2023-11-28T21:48:48.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f822159a640 1 -- 172.21.15.12:0/2160473101 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f820c003530 con 0x7f82241540f0 2023-11-28T21:48:48.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8222d9d640 1 -- 172.21.15.12:0/2160473101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415b180 msgr2=0x7f822415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:48.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.692+0000 7f8222d9d640 1 --2- 172.21.15.12:0/2160473101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415b180 0x7f822415b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:48.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8222d9d640 1 -- 172.21.15.12:0/2160473101 >> v1:172.21.15.12:6789/0 conn(0x7f82241540f0 legacy=0x7f82240c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:48.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8222d9d640 1 -- 172.21.15.12:0/2160473101 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822406d860 con 0x7f822415d210 2023-11-28T21:48:48.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8222d9d640 1 --2- 172.21.15.12:0/2160473101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822415d210 0x7f822415d6a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f821c009d90 tx=0x7f821c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd99f35f560dabe7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:48.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f822159a640 1 -- 172.21.15.12:0/2160473101 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c0a5920 con 0x7f822415d210 2023-11-28T21:48:48.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f822159a640 1 -- 172.21.15.12:0/2160473101 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c0a5a80 con 0x7f822415d210 2023-11-28T21:48:48.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f822159a640 1 -- 172.21.15.12:0/2160473101 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c0a5e10 con 0x7f822415d210 2023-11-28T21:48:48.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8228c9e640 1 -- 172.21.15.12:0/2160473101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822415d210 msgr2=0x7f822415d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:48.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8228c9e640 1 --2- 172.21.15.12:0/2160473101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822415d210 0x7f822415d6a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f821c009d90 tx=0x7f821c01c950 comp rx=0 tx=0).stop 2023-11-28T21:48:48.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8228c9e640 1 -- 172.21.15.12:0/2160473101 shutdown_connections 2023-11-28T21:48:48.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8228c9e640 1 --2- 172.21.15.12:0/2160473101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f822415d210 0x7f822415d6a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:48.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8228c9e640 1 --2- 172.21.15.12:0/2160473101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415b180 0x7f822415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:48.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8228c9e640 1 -- 172.21.15.12:0/2160473101 >> 172.21.15.12:0/2160473101 conn(0x7f8224066360 msgr2=0x7f8224066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:48.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8228c9e640 1 -- 172.21.15.12:0/2160473101 shutdown_connections 2023-11-28T21:48:48.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.696+0000 7f8228c9e640 1 -- 172.21.15.12:0/2160473101 wait complete. 2023-11-28T21:48:48.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8228c9e640 1 Processor -- start 2023-11-28T21:48:48.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8228c9e640 1 -- start start 2023-11-28T21:48:48.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8228c9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82241540f0 0x7f82240c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8228c9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822415b180 0x7f8224142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8228c9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415d210 0x7f8224143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8228c9e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82241f6ae0 con 0x7f82241540f0 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8228c9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82240ca830 con 0x7f822415b180 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8228c9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82241427c0 con 0x7f822415d210 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f822259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82241540f0 0x7f82240c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f822259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82241540f0 0x7f82240c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54660/0 (socket says 172.21.15.12:54660) 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f822259c640 1 -- 172.21.15.12:0/2712591868 learned_addr learned my addr 172.21.15.12:0/2712591868 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8221d9b640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822415b180 0x7f8224142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8222d9d640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415d210 0x7f8224143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:48.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f822259c640 1 -- 172.21.15.12:0/2712591868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415d210 msgr2=0x7f8224143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f822259c640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415d210 0x7f8224143500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f822259c640 1 -- 172.21.15.12:0/2712591868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822415b180 msgr2=0x7f8224142fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f822259c640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822415b180 0x7f8224142fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f822259c640 1 -- 172.21.15.12:0/2712591868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c009a70 con 0x7f82241540f0 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8222d9d640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415d210 0x7f8224143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f822259c640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82241540f0 0x7f82240c3430 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f821400c3c0 tx=0x7f821400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8212ffd640 1 -- 172.21.15.12:0/2712591868 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214005cf0 con 0x7f82241540f0 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8212ffd640 1 -- 172.21.15.12:0/2712591868 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214005e50 con 0x7f82241540f0 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8212ffd640 1 -- 172.21.15.12:0/2712591868 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140072f0 con 0x7f82241540f0 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.700+0000 7f8228c9e640 1 -- 172.21.15.12:0/2712591868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8224142aa0 con 0x7f82241540f0 2023-11-28T21:48:48.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.704+0000 7f8228c9e640 1 -- 172.21.15.12:0/2712591868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82241f6da0 con 0x7f82241540f0 2023-11-28T21:48:48.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.704+0000 7f8228c9e640 1 -- 172.21.15.12:0/2712591868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8224142ce0 con 0x7f82241540f0 2023-11-28T21:48:48.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.704+0000 7f8212ffd640 1 -- 172.21.15.12:0/2712591868 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8214007450 con 0x7f82241540f0 2023-11-28T21:48:48.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.704+0000 7f8212ffd640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f82040664a0 0x7f8204068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.704+0000 7f8212ffd640 1 -- 172.21.15.12:0/2712591868 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f8214083b50 con 0x7f82241540f0 2023-11-28T21:48:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.704+0000 7f8221d9b640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f82040664a0 0x7f8204068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:48.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.704+0000 7f8221d9b640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f82040664a0 0x7f8204068960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f822415db30 tx=0x7f8218008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:48.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.708+0000 7f8212ffd640 1 -- 172.21.15.12:0/2712591868 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8224142ce0 con 0x7f82241540f0 2023-11-28T21:48:48.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 -- 172.21.15.12:0/2712591868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8224142ce0 con 0x7f82241540f0 2023-11-28T21:48:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8212ffd640 1 -- 172.21.15.12:0/2712591868 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f8214017070 con 0x7f82241540f0 2023-11-28T21:48:48.903 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:48:48.903 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:09.646722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:active","state_seq":21,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:48:48.903 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 55 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 -- 172.21.15.12:0/2712591868 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f82040664a0 msgr2=0x7f8204068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f82040664a0 0x7f8204068960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f822415db30 tx=0x7f8218008040 comp rx=0 tx=0).stop 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 -- 172.21.15.12:0/2712591868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82241540f0 msgr2=0x7f82240c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82241540f0 0x7f82240c3430 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f821400c3c0 tx=0x7f821400c880 comp rx=0 tx=0).stop 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 -- 172.21.15.12:0/2712591868 shutdown_connections 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f82040664a0 0x7f8204068960 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822415d210 0x7f8224143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822415b180 0x7f8224142fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 --2- 172.21.15.12:0/2712591868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82241540f0 0x7f82240c3430 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 -- 172.21.15.12:0/2712591868 >> 172.21.15.12:0/2712591868 conn(0x7f8224066360 msgr2=0x7f8224153370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:48.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 -- 172.21.15.12:0/2712591868 shutdown_connections 2023-11-28T21:48:48.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:48.900+0000 7f8228c9e640 1 -- 172.21.15.12:0/2712591868 wait complete. 2023-11-28T21:48:48.924 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:48:50.925 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:48:51.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.076+0000 7f792fac9640 1 Processor -- start 2023-11-28T21:48:51.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.076+0000 7f792fac9640 1 -- start start 2023-11-28T21:48:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792fac9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7928150c20 0x7f7928156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792fac9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f792806e300 0x7f79280c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:51.083 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792fac9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f792806cf40 0x7f79280c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792fac9640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7928067410 con 0x7f7928150c20 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792fac9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792806e090 con 0x7f792806e300 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792fac9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7928157f30 con 0x7f792806cf40 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792d865640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7928150c20 0x7f7928156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792d865640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7928150c20 0x7f7928156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54668/0 (socket says 172.21.15.12:54668) 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792d865640 1 -- 172.21.15.12:0/3863404119 learned_addr learned my addr 172.21.15.12:0/3863404119 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792d064640 1 --2- 172.21.15.12:0/3863404119 >> v2:172.21.15.144:3300/0 conn(0x7f792806e300 0x7f79280c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792e066640 1 --2- 172.21.15.12:0/3863404119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f792806cf40 0x7f79280c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792d865640 1 -- 172.21.15.12:0/3863404119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f792806cf40 msgr2=0x7f79280c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792d865640 1 --2- 172.21.15.12:0/3863404119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f792806cf40 0x7f79280c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792d865640 1 -- 172.21.15.12:0/3863404119 >> v2:172.21.15.144:3300/0 conn(0x7f792806e300 msgr2=0x7f79280c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:51.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792d865640 1 --2- 172.21.15.12:0/3863404119 >> v2:172.21.15.144:3300/0 conn(0x7f792806e300 0x7f79280c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.080+0000 7f792d865640 1 -- 172.21.15.12:0/3863404119 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7928151000 con 0x7f7928150c20 2023-11-28T21:48:51.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792d865640 1 --2- 172.21.15.12:0/3863404119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7928150c20 0x7f7928156910 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f79280c8e60 tx=0x7f792401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3da611ebcd94cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:51.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792c863640 1 -- 172.21.15.12:0/3863404119 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79240ac630 con 0x7f7928150c20 2023-11-28T21:48:51.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792c863640 1 -- 172.21.15.12:0/3863404119 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79240ac790 con 0x7f7928150c20 2023-11-28T21:48:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792c863640 1 -- 172.21.15.12:0/3863404119 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79240acb20 con 0x7f7928150c20 2023-11-28T21:48:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 -- 172.21.15.12:0/3863404119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7928150c20 msgr2=0x7f7928156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 --2- 172.21.15.12:0/3863404119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7928150c20 0x7f7928156910 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f79280c8e60 tx=0x7f792401c500 comp rx=0 tx=0).stop 2023-11-28T21:48:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 -- 172.21.15.12:0/3863404119 shutdown_connections 2023-11-28T21:48:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 --2- 172.21.15.12:0/3863404119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f792806cf40 0x7f79280c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 --2- 172.21.15.12:0/3863404119 >> v2:172.21.15.144:3300/0 conn(0x7f792806e300 0x7f79280c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 --2- 172.21.15.12:0/3863404119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7928150c20 0x7f7928156910 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 -- 172.21.15.12:0/3863404119 >> 172.21.15.12:0/3863404119 conn(0x7f7928066360 msgr2=0x7f7928066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 -- 172.21.15.12:0/3863404119 shutdown_connections 2023-11-28T21:48:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 -- 172.21.15.12:0/3863404119 wait complete. 2023-11-28T21:48:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 Processor -- start 2023-11-28T21:48:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.084+0000 7f792fac9640 1 -- start start 2023-11-28T21:48:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792fac9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f792806cf40 0x7f792814d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792fac9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f792806e300 0x7f792814bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792fac9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7928150c20 0x7f7928146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792fac9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7928070e30 con 0x7f792806e300 2023-11-28T21:48:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792fac9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79280719e0 con 0x7f7928150c20 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792fac9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79281f68c0 con 0x7f792806cf40 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792d064640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f792806e300 0x7f792814bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792d064640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f792806e300 0x7f792814bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54684/0 (socket says 172.21.15.12:54684) 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792d064640 1 -- 172.21.15.12:0/3210309612 learned_addr learned my addr 172.21.15.12:0/3210309612 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792d865640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f792806cf40 0x7f792814d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792e066640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7928150c20 0x7f7928146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792d064640 1 -- 172.21.15.12:0/3210309612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f792806cf40 msgr2=0x7f792814d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792d064640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f792806cf40 0x7f792814d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792d064640 1 -- 172.21.15.12:0/3210309612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7928150c20 msgr2=0x7f7928146c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792d064640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7928150c20 0x7f7928146c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792d064640 1 -- 172.21.15.12:0/3210309612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79240094d0 con 0x7f792806e300 2023-11-28T21:48:51.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792d064640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f792806e300 0x7f792814bac0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f791800d800 tx=0x7f791800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:51.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f791e7fc640 1 -- 172.21.15.12:0/3210309612 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918018070 con 0x7f792806e300 2023-11-28T21:48:51.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f791e7fc640 1 -- 172.21.15.12:0/3210309612 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7918004020 con 0x7f792806e300 2023-11-28T21:48:51.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f791e7fc640 1 -- 172.21.15.12:0/3210309612 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180041d0 con 0x7f792806e300 2023-11-28T21:48:51.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792fac9640 1 -- 172.21.15.12:0/3210309612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792814b8d0 con 0x7f792806e300 2023-11-28T21:48:51.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792fac9640 1 -- 172.21.15.12:0/3210309612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792814b5f0 con 0x7f792806e300 2023-11-28T21:48:51.094 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.088+0000 7f792fac9640 1 -- 172.21.15.12:0/3210309612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7928068740 con 0x7f792806e300 2023-11-28T21:48:51.095 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.092+0000 7f791e7fc640 1 -- 172.21.15.12:0/3210309612 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7918004330 con 0x7f792806e300 2023-11-28T21:48:51.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.092+0000 7f791e7fc640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f790c0664a0 0x7f790c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:51.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.092+0000 7f791e7fc640 1 -- 172.21.15.12:0/3210309612 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7f7918084110 con 0x7f792806e300 2023-11-28T21:48:51.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.092+0000 7f792d865640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f790c0664a0 0x7f790c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:51.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.092+0000 7f792d865640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f790c0664a0 0x7f790c068960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f79240ae000 tx=0x7f7924008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:51.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.096+0000 7f791e7fc640 1 -- 172.21.15.12:0/3210309612 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7918051070 con 0x7f792806e300 2023-11-28T21:48:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.284+0000 7f792fac9640 1 -- 172.21.15.12:0/3210309612 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79281f6a30 con 0x7f792806e300 2023-11-28T21:48:51.290 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f791e7fc640 1 -- 172.21.15.12:0/3210309612 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+6506 (secure 0 0 0) 0x7f7918054690 con 0x7f792806e300 2023-11-28T21:48:51.290 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:48:51.291 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:47:09.646722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":48,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6024},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6024":{"gid":6024,"name":"e","rank":0,"incarnation":51,"state":"up:active","state_seq":21,"addr":"172.21.15.12:6839/2705421041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":2705421041},{"type":"v1","addr":"172.21.15.12:6839","nonce":2705421041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:48:51.291 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 55 2023-11-28T21:48:51.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 -- 172.21.15.12:0/3210309612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f790c0664a0 msgr2=0x7f790c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:51.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f790c0664a0 0x7f790c068960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f79240ae000 tx=0x7f7924008040 comp rx=0 tx=0).stop 2023-11-28T21:48:51.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 -- 172.21.15.12:0/3210309612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f792806e300 msgr2=0x7f792814bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:51.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f792806e300 0x7f792814bac0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f791800d800 tx=0x7f791800dcc0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 -- 172.21.15.12:0/3210309612 shutdown_connections 2023-11-28T21:48:51.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f790c0664a0 0x7f790c068960 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7928150c20 0x7f7928146c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f792806e300 0x7f792814bac0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 --2- 172.21.15.12:0/3210309612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f792806cf40 0x7f792814d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:51.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 -- 172.21.15.12:0/3210309612 >> 172.21.15.12:0/3210309612 conn(0x7f7928066360 msgr2=0x7f79280c5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:51.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 -- 172.21.15.12:0/3210309612 shutdown_connections 2023-11-28T21:48:51.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:51.288+0000 7f792fac9640 1 -- 172.21.15.12:0/3210309612 wait complete. 2023-11-28T21:48:51.312 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:48:53.313 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:48:53.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc106217640 1 Processor -- start 2023-11-28T21:48:53.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc106217640 1 -- start start 2023-11-28T21:48:53.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc106217640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc100151ed0 0x7fc1000c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc106217640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc100157b70 con 0x7fc100151af0 2023-11-28T21:48:53.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc106217640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc100157e50 con 0x7fc10006e300 2023-11-28T21:48:53.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc106217640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1000707f0 con 0x7fc100151ed0 2023-11-28T21:48:53.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc0ff7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc100151af0 0x7fc1000c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45492/0 (socket says 172.21.15.12:45492) 2023-11-28T21:48:53.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc0ff7fe640 1 -- 172.21.15.12:0/3764905491 learned_addr learned my addr 172.21.15.12:0/3764905491 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:48:53.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc0feffd640 1 --2- 172.21.15.12:0/3764905491 >> v2:172.21.15.12:3300/0 conn(0x7fc100151ed0 0x7fc1000c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc0fe7fc640 1 -- 172.21.15.12:0/3764905491 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686276556 0 0) 0x7fc100157b70 con 0x7fc100151af0 2023-11-28T21:48:53.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc0fe7fc640 1 -- 172.21.15.12:0/3764905491 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0e8003530 con 0x7fc100151af0 2023-11-28T21:48:53.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc0feffd640 1 -- 172.21.15.12:0/3764905491 >> v1:172.21.15.144:6789/0 conn(0x7fc10006e300 legacy=0x7fc100072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:53.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc0feffd640 1 -- 172.21.15.12:0/3764905491 >> v1:172.21.15.12:6789/0 conn(0x7fc100151af0 legacy=0x7fc1000c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:53.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.468+0000 7fc0feffd640 1 -- 172.21.15.12:0/3764905491 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc100158130 con 0x7fc100151ed0 2023-11-28T21:48:53.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc0feffd640 1 --2- 172.21.15.12:0/3764905491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100151ed0 0x7fc1000c7650 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40097c0 tx=0x7fc0f401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69a981c448a0ac5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:53.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc0fe7fc640 1 -- 172.21.15.12:0/3764905491 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40ab600 con 0x7fc100151ed0 2023-11-28T21:48:53.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc0fe7fc640 1 -- 172.21.15.12:0/3764905491 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f40a7050 con 0x7fc100151ed0 2023-11-28T21:48:53.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc0fe7fc640 1 -- 172.21.15.12:0/3764905491 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40ab980 con 0x7fc100151ed0 2023-11-28T21:48:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc106217640 1 -- 172.21.15.12:0/3764905491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100151ed0 msgr2=0x7fc1000c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc106217640 1 --2- 172.21.15.12:0/3764905491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100151ed0 0x7fc1000c7650 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40097c0 tx=0x7fc0f401c480 comp rx=0 tx=0).stop 2023-11-28T21:48:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc106217640 1 -- 172.21.15.12:0/3764905491 shutdown_connections 2023-11-28T21:48:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc106217640 1 --2- 172.21.15.12:0/3764905491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100151ed0 0x7fc1000c7650 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc106217640 1 -- 172.21.15.12:0/3764905491 >> 172.21.15.12:0/3764905491 conn(0x7fc100066360 msgr2=0x7fc100066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc106217640 1 -- 172.21.15.12:0/3764905491 shutdown_connections 2023-11-28T21:48:53.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.472+0000 7fc106217640 1 -- 172.21.15.12:0/3764905491 wait complete. 2023-11-28T21:48:53.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc106217640 1 Processor -- start 2023-11-28T21:48:53.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc106217640 1 -- start start 2023-11-28T21:48:53.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc106217640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc10006e300 0x7fc1001fb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc106217640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc100151af0 0x7fc1001f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc106217640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100141170 0x7fc100141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc106217640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc100155620 con 0x7fc100141170 2023-11-28T21:48:53.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc106217640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1001facb0 con 0x7fc10006e300 2023-11-28T21:48:53.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc106217640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc100145f60 con 0x7fc100151af0 2023-11-28T21:48:53.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100141170 0x7fc100141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100141170 0x7fc100141ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54714/0 (socket says 172.21.15.12:54714) 2023-11-28T21:48:53.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0fffff640 1 -- 172.21.15.12:0/1108235130 learned_addr learned my addr 172.21.15.12:0/1108235130 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:53.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0feffd640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc100151af0 0x7fc1001f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.481 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0ff7fe640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc10006e300 0x7fc1001fb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0fffff640 1 -- 172.21.15.12:0/1108235130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc100151af0 msgr2=0x7fc1001f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:53.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0fffff640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc100151af0 0x7fc1001f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0fffff640 1 -- 172.21.15.12:0/1108235130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc10006e300 msgr2=0x7fc1001fb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:53.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0fffff640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc10006e300 0x7fc1001fb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.482 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0fffff640 1 -- 172.21.15.12:0/1108235130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f4009420 con 0x7fc100141170 2023-11-28T21:48:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0fffff640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100141170 0x7fc100141ad0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c7a0 tx=0x7fc0f800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0e7fff640 1 -- 172.21.15.12:0/1108235130 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8003b80 con 0x7fc100141170 2023-11-28T21:48:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0e7fff640 1 -- 172.21.15.12:0/1108235130 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f8003ce0 con 0x7fc100141170 2023-11-28T21:48:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc0e7fff640 1 -- 172.21.15.12:0/1108235130 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800aa30 con 0x7fc100141170 2023-11-28T21:48:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc106217640 1 -- 172.21.15.12:0/1108235130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1001fa660 con 0x7fc100141170 2023-11-28T21:48:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.476+0000 7fc106217640 1 -- 172.21.15.12:0/1108235130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc100157710 con 0x7fc100141170 2023-11-28T21:48:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.480+0000 7fc106217640 1 -- 172.21.15.12:0/1108235130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc100074f60 con 0x7fc100141170 2023-11-28T21:48:53.483 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.480+0000 7fc0e7fff640 1 -- 172.21.15.12:0/1108235130 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc0f800ab90 con 0x7fc100141170 2023-11-28T21:48:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.480+0000 7fc0e7fff640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc0ec0664a0 0x7fc0ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.480+0000 7fc0e7fff640 1 -- 172.21.15.12:0/1108235130 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 7179+0+0 (secure 0 0 0) 0x7fc0f8084180 con 0x7fc100141170 2023-11-28T21:48:53.484 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.480+0000 7fc0ff7fe640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc0ec0664a0 0x7fc0ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.485 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.480+0000 7fc0ff7fe640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc0ec0664a0 0x7fc0ec068960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00011f0 tx=0x7fc0f0007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:53.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.484+0000 7fc0e7fff640 1 -- 172.21.15.12:0/1108235130 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc0f8051050 con 0x7fc100141170 2023-11-28T21:48:53.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.680+0000 7fc106217640 1 -- 172.21.15.12:0/1108235130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc10014b930 con 0x7fc100141170 2023-11-28T21:48:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.680+0000 7fc0e7fff640 1 -- 172.21.15.12:0/1108235130 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5714 (secure 0 0 0) 0x7fc0f8054670 con 0x7fc100141170 2023-11-28T21:48:53.685 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:48:53.685 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:48:53.536021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:48:53.685 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-11-28T21:48:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 -- 172.21.15.12:0/1108235130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc0ec0664a0 msgr2=0x7fc0ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc0ec0664a0 0x7fc0ec068960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00011f0 tx=0x7fc0f0007d40 comp rx=0 tx=0).stop 2023-11-28T21:48:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 -- 172.21.15.12:0/1108235130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100141170 msgr2=0x7fc100141ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100141170 0x7fc100141ad0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c7a0 tx=0x7fc0f800cc60 comp rx=0 tx=0).stop 2023-11-28T21:48:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 -- 172.21.15.12:0/1108235130 shutdown_connections 2023-11-28T21:48:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc0ec0664a0 0x7fc0ec068960 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc100141170 0x7fc100141ad0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc100151af0 0x7fc1001f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 --2- 172.21.15.12:0/1108235130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc10006e300 0x7fc1001fb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 -- 172.21.15.12:0/1108235130 >> 172.21.15.12:0/1108235130 conn(0x7fc100066360 msgr2=0x7fc1001501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 -- 172.21.15.12:0/1108235130 shutdown_connections 2023-11-28T21:48:53.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.684+0000 7fc106217640 1 -- 172.21.15.12:0/1108235130 wait complete. 2023-11-28T21:48:53.707 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-11-28T21:48:53.707 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:48:53.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:48:53.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.848+0000 7f2299cc8640 1 Processor -- start 2023-11-28T21:48:53.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.848+0000 7f2299cc8640 1 -- start start 2023-11-28T21:48:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2299cc8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f229415a980 0x7f22940b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2299cc8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f22940b4010 0x7f22940cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2299cc8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22940b44a0 con 0x7f2294153fb0 2023-11-28T21:48:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2299cc8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22940b2b60 con 0x7f229415a980 2023-11-28T21:48:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2299cc8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f229415e1b0 con 0x7f22940b4010 2023-11-28T21:48:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2293fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f229415a980 0x7f22940b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2293fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f229415a980 0x7f22940b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54718/0 (socket says 172.21.15.12:54718) 2023-11-28T21:48:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2293fff640 1 -- 172.21.15.12:0/2577568939 learned_addr learned my addr 172.21.15.12:0/2577568939 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f22994c7640 1 --2- 172.21.15.12:0/2577568939 >> v2:172.21.15.144:3300/0 conn(0x7f22940b4010 0x7f22940cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2293fff640 1 -- 172.21.15.12:0/2577568939 >> v2:172.21.15.144:3300/0 conn(0x7f22940b4010 msgr2=0x7f22940cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2293fff640 1 --2- 172.21.15.12:0/2577568939 >> v2:172.21.15.144:3300/0 conn(0x7f22940b4010 0x7f22940cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2293fff640 1 -- 172.21.15.12:0/2577568939 >> v1:172.21.15.144:6789/0 conn(0x7f2294153fb0 legacy=0x7f22940ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:53.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.852+0000 7f2293fff640 1 -- 172.21.15.12:0/2577568939 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229415d390 con 0x7f229415a980 2023-11-28T21:48:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2293fff640 1 --2- 172.21.15.12:0/2577568939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f229415a980 0x7f22940b3ad0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2294159cb0 tx=0x7f228001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc55e71b064c3d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f22937fe640 1 -- 172.21.15.12:0/2577568939 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22800a89b0 con 0x7f229415a980 2023-11-28T21:48:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f22937fe640 1 -- 172.21.15.12:0/2577568939 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22800a8b10 con 0x7f229415a980 2023-11-28T21:48:53.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f22937fe640 1 -- 172.21.15.12:0/2577568939 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22800a8ea0 con 0x7f229415a980 2023-11-28T21:48:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2299cc8640 1 -- 172.21.15.12:0/2577568939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f229415a980 msgr2=0x7f22940b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2299cc8640 1 --2- 172.21.15.12:0/2577568939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f229415a980 0x7f22940b3ad0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2294159cb0 tx=0x7f228001ecb0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2299cc8640 1 -- 172.21.15.12:0/2577568939 shutdown_connections 2023-11-28T21:48:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2299cc8640 1 --2- 172.21.15.12:0/2577568939 >> v2:172.21.15.144:3300/0 conn(0x7f22940b4010 0x7f22940cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2299cc8640 1 --2- 172.21.15.12:0/2577568939 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f229415a980 0x7f22940b3ad0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2299cc8640 1 -- 172.21.15.12:0/2577568939 >> 172.21.15.12:0/2577568939 conn(0x7f2294071ce0 msgr2=0x7f22940720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2299cc8640 1 -- 172.21.15.12:0/2577568939 shutdown_connections 2023-11-28T21:48:53.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2299cc8640 1 -- 172.21.15.12:0/2577568939 wait complete. 2023-11-28T21:48:53.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2299cc8640 1 Processor -- start 2023-11-28T21:48:53.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.856+0000 7f2299cc8640 1 -- start start 2023-11-28T21:48:53.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2299cc8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22940b4010 0x7f229414a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2299cc8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2294153fb0 0x7f229415c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2299cc8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229415a980 0x7f229415c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2299cc8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2294159250 con 0x7f22940b4010 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2299cc8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22940c5fa0 con 0x7f2294153fb0 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2299cc8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2294157e60 con 0x7f229415a980 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2298cc6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22940b4010 0x7f229414a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2298cc6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22940b4010 0x7f229414a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54728/0 (socket says 172.21.15.12:54728) 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2298cc6640 1 -- 172.21.15.12:0/2933868504 learned_addr learned my addr 172.21.15.12:0/2933868504 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2293fff640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2294153fb0 0x7f229415c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f22994c7640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229415a980 0x7f229415c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2298cc6640 1 -- 172.21.15.12:0/2933868504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229415a980 msgr2=0x7f229415c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:53.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2298cc6640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229415a980 0x7f229415c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2298cc6640 1 -- 172.21.15.12:0/2933868504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2294153fb0 msgr2=0x7f229415c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:53.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2298cc6640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2294153fb0 0x7f229415c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:53.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2298cc6640 1 -- 172.21.15.12:0/2933868504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22800094d0 con 0x7f22940b4010 2023-11-28T21:48:53.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2298cc6640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22940b4010 0x7f229414a0f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f228800c450 tx=0x7f228800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:53.864 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f22917fa640 1 -- 172.21.15.12:0/2933868504 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288011790 con 0x7f22940b4010 2023-11-28T21:48:53.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f22917fa640 1 -- 172.21.15.12:0/2933868504 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2288011910 con 0x7f22940b4010 2023-11-28T21:48:53.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f22917fa640 1 -- 172.21.15.12:0/2933868504 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288011cc0 con 0x7f22940b4010 2023-11-28T21:48:53.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2299cc8640 1 -- 172.21.15.12:0/2933868504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22940c5c50 con 0x7f22940b4010 2023-11-28T21:48:53.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2299cc8640 1 -- 172.21.15.12:0/2933868504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22941f0420 con 0x7f22940b4010 2023-11-28T21:48:53.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f2299cc8640 1 -- 172.21.15.12:0/2933868504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22940c5de0 con 0x7f22940b4010 2023-11-28T21:48:53.865 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.860+0000 7f22917fa640 1 -- 172.21.15.12:0/2933868504 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f228801c460 con 0x7f22940b4010 2023-11-28T21:48:53.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.864+0000 7f22917fa640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2278066470 0x7f2278068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:53.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.864+0000 7f22917fa640 1 -- 172.21.15.12:0/2933868504 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f2288084400 con 0x7f22940b4010 2023-11-28T21:48:53.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.864+0000 7f2293fff640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2278066470 0x7f2278068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:53.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.864+0000 7f2293fff640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2278066470 0x7f2278068930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f22800b2000 tx=0x7f2280008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:53.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:53.868+0000 7f22917fa640 1 -- 172.21.15.12:0/2933868504 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f22940c5de0 con 0x7f22940b4010 2023-11-28T21:48:54.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.056+0000 7f2299cc8640 1 -- 172.21.15.12:0/2933868504 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22940c5de0 con 0x7f22940b4010 2023-11-28T21:48:54.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.056+0000 7f22917fa640 1 -- 172.21.15.12:0/2933868504 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f22940c5de0 con 0x7f22940b4010 2023-11-28T21:48:54.061 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:48:54.061 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:48:53.536021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:48:54.062 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-11-28T21:48:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 -- 172.21.15.12:0/2933868504 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2278066470 msgr2=0x7f2278068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2278066470 0x7f2278068930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f22800b2000 tx=0x7f2280008040 comp rx=0 tx=0).stop 2023-11-28T21:48:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 -- 172.21.15.12:0/2933868504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22940b4010 msgr2=0x7f229414a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:54.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22940b4010 0x7f229414a0f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f228800c450 tx=0x7f228800c910 comp rx=0 tx=0).stop 2023-11-28T21:48:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 -- 172.21.15.12:0/2933868504 shutdown_connections 2023-11-28T21:48:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2278066470 0x7f2278068930 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229415a980 0x7f229415c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2294153fb0 0x7f229415c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 --2- 172.21.15.12:0/2933868504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22940b4010 0x7f229414a0f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 -- 172.21.15.12:0/2933868504 >> 172.21.15.12:0/2933868504 conn(0x7f2294071ce0 msgr2=0x7f22940b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 -- 172.21.15.12:0/2933868504 shutdown_connections 2023-11-28T21:48:54.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:54.060+0000 7f2299cc8640 1 -- 172.21.15.12:0/2933868504 wait complete. 2023-11-28T21:48:54.084 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 56, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 56, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6177": { "addr": "172.21.15.12:6838/1551825785", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 1551825785, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 1551825785, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6177, "incarnation": 56, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:48:53.536021+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 6177 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:48:56.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:48:56.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefd5c17640 1 Processor -- start 2023-11-28T21:48:56.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefd5c17640 1 -- start start 2023-11-28T21:48:56.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefd5c17640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fefd0158420 0x7fefd006d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:56.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefd5c17640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fefd0161600 0x7fefd0161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:56.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefd5c17640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd00c3460 con 0x7fefd00c38f0 2023-11-28T21:48:56.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefd5c17640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd0158800 con 0x7fefd0158420 2023-11-28T21:48:56.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefd5c17640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd00c52c0 con 0x7fefd0161600 2023-11-28T21:48:56.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefcf7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fefd0158420 0x7fefd006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:56.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefcf7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fefd0158420 0x7fefd006d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51290/0 (socket says 172.21.15.12:51290) 2023-11-28T21:48:56.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefcf7fe640 1 -- 172.21.15.12:0/1144039075 learned_addr learned my addr 172.21.15.12:0/1144039075 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:56.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefcffff640 1 --2- 172.21.15.12:0/1144039075 >> v2:172.21.15.144:3300/0 conn(0x7fefd0161600 0x7fefd0161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:56.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefcf7fe640 1 -- 172.21.15.12:0/1144039075 >> v2:172.21.15.144:3300/0 conn(0x7fefd0161600 msgr2=0x7fefd0161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:56.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefcf7fe640 1 --2- 172.21.15.12:0/1144039075 >> v2:172.21.15.144:3300/0 conn(0x7fefd0161600 0x7fefd0161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:56.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.236+0000 7fefcf7fe640 1 -- 172.21.15.12:0/1144039075 >> v1:172.21.15.144:6789/0 conn(0x7fefd00c38f0 legacy=0x7fefd00c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:56.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefcf7fe640 1 -- 172.21.15.12:0/1144039075 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd00c2b70 con 0x7fefd0158420 2023-11-28T21:48:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefcf7fe640 1 --2- 172.21.15.12:0/1144039075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd0158420 0x7fefd006d950 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fefd00c5510 tx=0x7fefc001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5f8be55836a8a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefce7fc640 1 -- 172.21.15.12:0/1144039075 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc00ab600 con 0x7fefd0158420 2023-11-28T21:48:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefce7fc640 1 -- 172.21.15.12:0/1144039075 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc00a7050 con 0x7fefd0158420 2023-11-28T21:48:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefce7fc640 1 -- 172.21.15.12:0/1144039075 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc00ab980 con 0x7fefd0158420 2023-11-28T21:48:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefd5c17640 1 -- 172.21.15.12:0/1144039075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd0158420 msgr2=0x7fefd006d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefd5c17640 1 --2- 172.21.15.12:0/1144039075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd0158420 0x7fefd006d950 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fefd00c5510 tx=0x7fefc001c4c0 comp rx=0 tx=0).stop 2023-11-28T21:48:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefd5c17640 1 -- 172.21.15.12:0/1144039075 shutdown_connections 2023-11-28T21:48:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefd5c17640 1 --2- 172.21.15.12:0/1144039075 >> v2:172.21.15.144:3300/0 conn(0x7fefd0161600 0x7fefd0161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefd5c17640 1 --2- 172.21.15.12:0/1144039075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd0158420 0x7fefd006d950 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefd5c17640 1 -- 172.21.15.12:0/1144039075 >> 172.21.15.12:0/1144039075 conn(0x7fefd0066360 msgr2=0x7fefd0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefd5c17640 1 -- 172.21.15.12:0/1144039075 shutdown_connections 2023-11-28T21:48:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.240+0000 7fefd5c17640 1 -- 172.21.15.12:0/1144039075 wait complete. 2023-11-28T21:48:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefd5c17640 1 Processor -- start 2023-11-28T21:48:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefd5c17640 1 -- start start 2023-11-28T21:48:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefd5c17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd00c38f0 0x7fefd00c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefd5c17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefd0158420 0x7fefd014dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefd5c17640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefd0161600 0x7fefd01fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefd5c17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd014d800 con 0x7fefd00c38f0 2023-11-28T21:48:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefd5c17640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd01fa040 con 0x7fefd0161600 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefd5c17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd00c9f20 con 0x7fefd0158420 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd00c38f0 0x7fefd00c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd00c38f0 0x7fefd00c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51302/0 (socket says 172.21.15.12:51302) 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcf7fe640 1 -- 172.21.15.12:0/3296393739 learned_addr learned my addr 172.21.15.12:0/3296393739 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefceffd640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefd0158420 0x7fefd014dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcf7fe640 1 -- 172.21.15.12:0/3296393739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefd0158420 msgr2=0x7fefd014dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcffff640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefd0161600 0x7fefd01fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcf7fe640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefd0158420 0x7fefd014dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcf7fe640 1 -- 172.21.15.12:0/3296393739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefd0161600 msgr2=0x7fefd01fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcf7fe640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefd0161600 0x7fefd01fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcf7fe640 1 -- 172.21.15.12:0/3296393739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc00094d0 con 0x7fefd00c38f0 2023-11-28T21:48:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcffff640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefd0161600 0x7fefd01fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:48:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.244+0000 7fefcf7fe640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd00c38f0 0x7fefd00c69e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fefc001a0f0 tx=0x7fefc0019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefb7fff640 1 -- 172.21.15.12:0/3296393739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc00b4420 con 0x7fefd00c38f0 2023-11-28T21:48:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefb7fff640 1 -- 172.21.15.12:0/3296393739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc00b4580 con 0x7fefd00c38f0 2023-11-28T21:48:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefd5c17640 1 -- 172.21.15.12:0/3296393739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd01557c0 con 0x7fefd00c38f0 2023-11-28T21:48:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefb7fff640 1 -- 172.21.15.12:0/3296393739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc00b48f0 con 0x7fefd00c38f0 2023-11-28T21:48:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefd5c17640 1 -- 172.21.15.12:0/3296393739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd00ca4a0 con 0x7fefd00c38f0 2023-11-28T21:48:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefd5c17640 1 -- 172.21.15.12:0/3296393739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd0074f60 con 0x7fefd00c38f0 2023-11-28T21:48:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefb7fff640 1 -- 172.21.15.12:0/3296393739 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fefc00b2070 con 0x7fefd00c38f0 2023-11-28T21:48:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefb7fff640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fefbc066470 0x7fefbc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefb7fff640 1 -- 172.21.15.12:0/3296393739 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fefc011d670 con 0x7fefd00c38f0 2023-11-28T21:48:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefceffd640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fefbc066470 0x7fefbc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.248+0000 7fefceffd640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fefbc066470 0x7fefbc068930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fefd015ef80 tx=0x7fefc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.252+0000 7fefb7fff640 1 -- 172.21.15.12:0/3296393739 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fefc00ea4f0 con 0x7fefd00c38f0 2023-11-28T21:48:56.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.440+0000 7fefd5c17640 1 -- 172.21.15.12:0/3296393739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefd006e4f0 con 0x7fefd00c38f0 2023-11-28T21:48:56.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.440+0000 7fefb7fff640 1 -- 172.21.15.12:0/3296393739 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5714 (secure 0 0 0) 0x7fefc00edb10 con 0x7fefd00c38f0 2023-11-28T21:48:56.445 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:48:56.446 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:48:53.536021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:48:56.446 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-11-28T21:48:56.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 -- 172.21.15.12:0/3296393739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fefbc066470 msgr2=0x7fefbc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:56.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fefbc066470 0x7fefbc068930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fefd015ef80 tx=0x7fefc4009040 comp rx=0 tx=0).stop 2023-11-28T21:48:56.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 -- 172.21.15.12:0/3296393739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd00c38f0 msgr2=0x7fefd00c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:56.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd00c38f0 0x7fefd00c69e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fefc001a0f0 tx=0x7fefc0019260 comp rx=0 tx=0).stop 2023-11-28T21:48:56.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 -- 172.21.15.12:0/3296393739 shutdown_connections 2023-11-28T21:48:56.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fefbc066470 0x7fefbc068930 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:56.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefd0161600 0x7fefd01fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:56.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefd0158420 0x7fefd014dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:56.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 --2- 172.21.15.12:0/3296393739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fefd00c38f0 0x7fefd00c69e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:56.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 -- 172.21.15.12:0/3296393739 >> 172.21.15.12:0/3296393739 conn(0x7fefd0066360 msgr2=0x7fefd00c3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:56.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 -- 172.21.15.12:0/3296393739 shutdown_connections 2023-11-28T21:48:56.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:56.444+0000 7fefd5c17640 1 -- 172.21.15.12:0/3296393739 wait complete. 2023-11-28T21:48:56.467 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:48:58.468 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:48:58.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f316c8c8640 1 Processor -- start 2023-11-28T21:48:58.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f316c8c8640 1 -- start start 2023-11-28T21:48:58.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f316c8c8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f316815a980 0x7f31680b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:58.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f316c8c8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f31680b4010 0x7f31680cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:58.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f316c8c8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316815d390 con 0x7f3168153fb0 2023-11-28T21:48:58.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f316c8c8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31680b2b60 con 0x7f31680b4010 2023-11-28T21:48:58.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f316c8c8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316815e1b0 con 0x7f316815a980 2023-11-28T21:48:58.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f3166ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f316815a980 0x7f31680b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:58.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f3167fff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f31680b4010 0x7f31680cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:58.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f3166ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f316815a980 0x7f31680b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:34346/0 (socket says 172.21.15.12:34346) 2023-11-28T21:48:58.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f3167fff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f31680b4010 0x7f31680cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:56334/0 (socket says 172.21.15.12:56334) 2023-11-28T21:48:58.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f3166ffd640 1 -- 172.21.15.12:0/2575371190 learned_addr learned my addr 172.21.15.12:0/2575371190 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f3167fff640 1 -- 172.21.15.12:0/2575371190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f316815a980 msgr2=0x7f31680b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:48:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f3167fff640 1 --2- 172.21.15.12:0/2575371190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f316815a980 0x7f31680b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f3167fff640 1 -- 172.21.15.12:0/2575371190 >> v1:172.21.15.144:6789/0 conn(0x7f3168153fb0 legacy=0x7f31680ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:58.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.636+0000 7f3167fff640 1 -- 172.21.15.12:0/2575371190 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316815d630 con 0x7f31680b4010 2023-11-28T21:48:58.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f3167fff640 1 --2- 172.21.15.12:0/2575371190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31680b4010 0x7f31680cbde0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3160009d90 tx=0x7f316001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a98382394e9f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:58.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f31667fc640 1 -- 172.21.15.12:0/2575371190 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600a59e0 con 0x7f31680b4010 2023-11-28T21:48:58.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f31667fc640 1 -- 172.21.15.12:0/2575371190 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31600a5b40 con 0x7f31680b4010 2023-11-28T21:48:58.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f31667fc640 1 -- 172.21.15.12:0/2575371190 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600ad440 con 0x7f31680b4010 2023-11-28T21:48:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f316c8c8640 1 -- 172.21.15.12:0/2575371190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31680b4010 msgr2=0x7f31680cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f316c8c8640 1 --2- 172.21.15.12:0/2575371190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31680b4010 0x7f31680cbde0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3160009d90 tx=0x7f316001c920 comp rx=0 tx=0).stop 2023-11-28T21:48:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f316c8c8640 1 -- 172.21.15.12:0/2575371190 shutdown_connections 2023-11-28T21:48:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f316c8c8640 1 --2- 172.21.15.12:0/2575371190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31680b4010 0x7f31680cbde0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:58.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f316c8c8640 1 --2- 172.21.15.12:0/2575371190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f316815a980 0x7f31680b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f316c8c8640 1 -- 172.21.15.12:0/2575371190 >> 172.21.15.12:0/2575371190 conn(0x7f3168071ce0 msgr2=0x7f31680720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f316c8c8640 1 -- 172.21.15.12:0/2575371190 shutdown_connections 2023-11-28T21:48:58.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f316c8c8640 1 -- 172.21.15.12:0/2575371190 wait complete. 2023-11-28T21:48:58.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f316c8c8640 1 Processor -- start 2023-11-28T21:48:58.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.644+0000 7f316c8c8640 1 -- start start 2023-11-28T21:48:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f316c8c8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31680b4010 0x7f3168146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f316c8c8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168153fb0 0x7f3168144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f316c8c8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f316815a980 0x7f3168151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f316c8c8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316815c1e0 con 0x7f3168153fb0 2023-11-28T21:48:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f316c8c8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316815ccd0 con 0x7f316815a980 2023-11-28T21:48:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f316c8c8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31681533f0 con 0x7f31680b4010 2023-11-28T21:48:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3166ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168153fb0 0x7f3168144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3166ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168153fb0 0x7f3168144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51314/0 (socket says 172.21.15.12:51314) 2023-11-28T21:48:58.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3166ffd640 1 -- 172.21.15.12:0/602466858 learned_addr learned my addr 172.21.15.12:0/602466858 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:48:58.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3167fff640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f316815a980 0x7f3168151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:58.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f31677fe640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31680b4010 0x7f3168146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:58.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3166ffd640 1 -- 172.21.15.12:0/602466858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31680b4010 msgr2=0x7f3168146220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:58.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3166ffd640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31680b4010 0x7f3168146220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:58.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3166ffd640 1 -- 172.21.15.12:0/602466858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f316815a980 msgr2=0x7f3168151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:58.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3166ffd640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f316815a980 0x7f3168151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:58.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3166ffd640 1 -- 172.21.15.12:0/602466858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3160009a70 con 0x7f3168153fb0 2023-11-28T21:48:58.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3167fff640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f316815a980 0x7f3168151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:48:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f3166ffd640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168153fb0 0x7f3168144e70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f315c00e8a0 tx=0x7f315c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f314ffff640 1 -- 172.21.15.12:0/602466858 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c0094e0 con 0x7f3168153fb0 2023-11-28T21:48:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f314ffff640 1 -- 172.21.15.12:0/602466858 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c00c940 con 0x7f3168153fb0 2023-11-28T21:48:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f314ffff640 1 -- 172.21.15.12:0/602466858 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c00ccd0 con 0x7f3168153fb0 2023-11-28T21:48:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f316c8c8640 1 -- 172.21.15.12:0/602466858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31681536b0 con 0x7f3168153fb0 2023-11-28T21:48:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f316c8c8640 1 -- 172.21.15.12:0/602466858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316815c710 con 0x7f3168153fb0 2023-11-28T21:48:58.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.648+0000 7f316c8c8640 1 -- 172.21.15.12:0/602466858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31680c6600 con 0x7f3168153fb0 2023-11-28T21:48:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.652+0000 7f314ffff640 1 -- 172.21.15.12:0/602466858 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f315c013470 con 0x7f3168153fb0 2023-11-28T21:48:58.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.652+0000 7f314ffff640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3154066470 0x7f3154068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:48:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.652+0000 7f314ffff640 1 -- 172.21.15.12:0/602466858 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f315c084540 con 0x7f3168153fb0 2023-11-28T21:48:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.652+0000 7f31677fe640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3154066470 0x7f3154068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:48:58.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.652+0000 7f31677fe640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3154066470 0x7f3154068930 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3158001650 tx=0x7f3158009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:48:58.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.656+0000 7f314ffff640 1 -- 172.21.15.12:0/602466858 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f315c051390 con 0x7f3168153fb0 2023-11-28T21:48:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.852+0000 7f316c8c8640 1 -- 172.21.15.12:0/602466858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3168074f60 con 0x7f3168153fb0 2023-11-28T21:48:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.852+0000 7f314ffff640 1 -- 172.21.15.12:0/602466858 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f315c0549b0 con 0x7f3168153fb0 2023-11-28T21:48:58.857 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:48:58.857 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:48:53.536021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:48:58.857 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-11-28T21:48:58.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 -- 172.21.15.12:0/602466858 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3154066470 msgr2=0x7f3154068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:58.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3154066470 0x7f3154068930 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3158001650 tx=0x7f3158009040 comp rx=0 tx=0).stop 2023-11-28T21:48:58.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 -- 172.21.15.12:0/602466858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168153fb0 msgr2=0x7f3168144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:48:58.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168153fb0 0x7f3168144e70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f315c00e8a0 tx=0x7f315c00ed60 comp rx=0 tx=0).stop 2023-11-28T21:48:58.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 -- 172.21.15.12:0/602466858 shutdown_connections 2023-11-28T21:48:58.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3154066470 0x7f3154068930 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:58.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f316815a980 0x7f3168151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:58.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3168153fb0 0x7f3168144e70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:58.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 --2- 172.21.15.12:0/602466858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31680b4010 0x7f3168146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:48:58.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 -- 172.21.15.12:0/602466858 >> 172.21.15.12:0/602466858 conn(0x7f3168071ce0 msgr2=0x7f31680cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:48:58.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 -- 172.21.15.12:0/602466858 shutdown_connections 2023-11-28T21:48:58.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:48:58.856+0000 7f316c8c8640 1 -- 172.21.15.12:0/602466858 wait complete. 2023-11-28T21:48:58.879 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:49:00.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c759e640 1 Processor -- start 2023-11-28T21:49:01.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c759e640 1 -- start start 2023-11-28T21:49:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c8153a40 0x7f06c80ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c759e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f06c80ca7c0 0x7f06c80c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c759e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c815df30 con 0x7f06c80c5bd0 2023-11-28T21:49:01.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c759e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c80c5270 con 0x7f06c80ca7c0 2023-11-28T21:49:01.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80c45f0 con 0x7f06c8153a40 2023-11-28T21:49:01.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c6d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f06c80c5bd0 0x7f06c815d0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:57136/0 (socket says 172.21.15.12:57136) 2023-11-28T21:49:01.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c6d9d640 1 -- 172.21.15.12:0/1185666024 learned_addr learned my addr 172.21.15.12:0/1185666024 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:49:01.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c5d9b640 1 --2- 172.21.15.12:0/1185666024 >> v2:172.21.15.144:3300/0 conn(0x7f06c80ca7c0 0x7f06c80c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:01.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.024+0000 7f06c659c640 1 --2- 172.21.15.12:0/1185666024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c8153a40 0x7f06c80ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:01.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.028+0000 7f06c559a640 1 -- 172.21.15.12:0/1185666024 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493599404 0 0) 0x7f06c815df30 con 0x7f06c80c5bd0 2023-11-28T21:49:01.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.028+0000 7f06c559a640 1 -- 172.21.15.12:0/1185666024 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06b0003530 con 0x7f06c80c5bd0 2023-11-28T21:49:01.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.028+0000 7f06c5d9b640 1 -- 172.21.15.12:0/1185666024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c8153a40 msgr2=0x7f06c80ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:01.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.028+0000 7f06c559a640 1 -- 172.21.15.12:0/1185666024 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2571352025 0 0) 0x7f06b0003530 con 0x7f06c80c5bd0 2023-11-28T21:49:01.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.028+0000 7f06c5d9b640 1 --2- 172.21.15.12:0/1185666024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c8153a40 0x7f06c80ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:01.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.028+0000 7f06c5d9b640 1 -- 172.21.15.12:0/1185666024 >> v1:172.21.15.12:6789/0 conn(0x7f06c80c5bd0 legacy=0x7f06c815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:01.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.028+0000 7f06c5d9b640 1 -- 172.21.15.12:0/1185666024 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c80c48d0 con 0x7f06c80ca7c0 2023-11-28T21:49:01.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.028+0000 7f06c5d9b640 1 --2- 172.21.15.12:0/1185666024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c80ca7c0 0x7f06c80c5690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0097c0 tx=0x7f06bc01eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3add8220dd9f4038 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:01.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c559a640 1 -- 172.21.15.12:0/1185666024 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0001a00 con 0x7f06c80ca7c0 2023-11-28T21:49:01.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c559a640 1 -- 172.21.15.12:0/1185666024 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc0a85d0 con 0x7f06c80ca7c0 2023-11-28T21:49:01.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c559a640 1 -- 172.21.15.12:0/1185666024 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0a8940 con 0x7f06c80ca7c0 2023-11-28T21:49:01.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c759e640 1 -- 172.21.15.12:0/1185666024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c80ca7c0 msgr2=0x7f06c80c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:01.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c759e640 1 --2- 172.21.15.12:0/1185666024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c80ca7c0 0x7f06c80c5690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0097c0 tx=0x7f06bc01eb90 comp rx=0 tx=0).stop 2023-11-28T21:49:01.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c759e640 1 -- 172.21.15.12:0/1185666024 shutdown_connections 2023-11-28T21:49:01.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c759e640 1 --2- 172.21.15.12:0/1185666024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c80ca7c0 0x7f06c80c5690 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:01.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c759e640 1 --2- 172.21.15.12:0/1185666024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c8153a40 0x7f06c80ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:01.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c759e640 1 -- 172.21.15.12:0/1185666024 >> 172.21.15.12:0/1185666024 conn(0x7f06c806bda0 msgr2=0x7f06c806c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:01.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c759e640 1 -- 172.21.15.12:0/1185666024 shutdown_connections 2023-11-28T21:49:01.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c759e640 1 -- 172.21.15.12:0/1185666024 wait complete. 2023-11-28T21:49:01.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.032+0000 7f06c759e640 1 Processor -- start 2023-11-28T21:49:01.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c759e640 1 -- start start 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c80c5bd0 0x7f06c80c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c80ca7c0 0x7f06c8152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c8153a40 0x7f06c814bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c759e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80c7f80 con 0x7f06c80ca7c0 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c80c6c50 con 0x7f06c8153a40 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c8159860 con 0x7f06c80c5bd0 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c80ca7c0 0x7f06c8152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c80ca7c0 0x7f06c8152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51316/0 (socket says 172.21.15.12:51316) 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c5d9b640 1 -- 172.21.15.12:0/1650481748 learned_addr learned my addr 172.21.15.12:0/1650481748 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c6d9d640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c8153a40 0x7f06c814bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:01.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c659c640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c80c5bd0 0x7f06c80c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:01.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c5d9b640 1 -- 172.21.15.12:0/1650481748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c80c5bd0 msgr2=0x7f06c80c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:01.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c5d9b640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c80c5bd0 0x7f06c80c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:01.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c5d9b640 1 -- 172.21.15.12:0/1650481748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c8153a40 msgr2=0x7f06c814bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:01.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c5d9b640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c8153a40 0x7f06c814bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:01.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c5d9b640 1 -- 172.21.15.12:0/1650481748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc009420 con 0x7f06c80ca7c0 2023-11-28T21:49:01.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c5d9b640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c80ca7c0 0x7f06c8152a80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f06bc01d240 tx=0x7f06bc01b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:01.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06b6ffd640 1 -- 172.21.15.12:0/1650481748 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0a9c60 con 0x7f06c80ca7c0 2023-11-28T21:49:01.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06b6ffd640 1 -- 172.21.15.12:0/1650481748 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc0a9dc0 con 0x7f06c80ca7c0 2023-11-28T21:49:01.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c759e640 1 -- 172.21.15.12:0/1650481748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c80c7c30 con 0x7f06c80ca7c0 2023-11-28T21:49:01.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06b6ffd640 1 -- 172.21.15.12:0/1650481748 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc003820 con 0x7f06c80ca7c0 2023-11-28T21:49:01.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.036+0000 7f06c759e640 1 -- 172.21.15.12:0/1650481748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c80c71b0 con 0x7f06c80ca7c0 2023-11-28T21:49:01.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.040+0000 7f06c759e640 1 -- 172.21.15.12:0/1650481748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c80c8130 con 0x7f06c80ca7c0 2023-11-28T21:49:01.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.040+0000 7f06b6ffd640 1 -- 172.21.15.12:0/1650481748 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f06bc0b7040 con 0x7f06c80ca7c0 2023-11-28T21:49:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.040+0000 7f06b6ffd640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f06a80664a0 0x7f06a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.040+0000 7f06b6ffd640 1 -- 172.21.15.12:0/1650481748 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f06bc121970 con 0x7f06c80ca7c0 2023-11-28T21:49:01.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.040+0000 7f06c659c640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f06a80664a0 0x7f06a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:01.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.044+0000 7f06c659c640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f06a80664a0 0x7f06a8068960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009ac0 tx=0x7f06b8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:01.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.044+0000 7f06b6ffd640 1 -- 172.21.15.12:0/1650481748 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f06c80c8130 con 0x7f06c80ca7c0 2023-11-28T21:49:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 -- 172.21.15.12:0/1650481748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06c80c7dc0 con 0x7f06c80ca7c0 2023-11-28T21:49:01.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06b6ffd640 1 -- 172.21.15.12:0/1650481748 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f06c80c7dc0 con 0x7f06c80ca7c0 2023-11-28T21:49:01.247 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:01.247 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:48:53.536021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:01.247 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-11-28T21:49:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 -- 172.21.15.12:0/1650481748 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f06a80664a0 msgr2=0x7f06a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f06a80664a0 0x7f06a8068960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009ac0 tx=0x7f06b8009190 comp rx=0 tx=0).stop 2023-11-28T21:49:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 -- 172.21.15.12:0/1650481748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c80ca7c0 msgr2=0x7f06c8152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c80ca7c0 0x7f06c8152a80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f06bc01d240 tx=0x7f06bc01b930 comp rx=0 tx=0).stop 2023-11-28T21:49:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 -- 172.21.15.12:0/1650481748 shutdown_connections 2023-11-28T21:49:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f06a80664a0 0x7f06a8068960 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c8153a40 0x7f06c814bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:01.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f06c80ca7c0 0x7f06c8152a80 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 --2- 172.21.15.12:0/1650481748 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c80c5bd0 0x7f06c80c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 -- 172.21.15.12:0/1650481748 >> 172.21.15.12:0/1650481748 conn(0x7f06c806bda0 msgr2=0x7f06c80cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 -- 172.21.15.12:0/1650481748 shutdown_connections 2023-11-28T21:49:01.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:01.244+0000 7f06c759e640 1 -- 172.21.15.12:0/1650481748 wait complete. 2023-11-28T21:49:01.268 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:49:03.269 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:03.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c89f65640 1 Processor -- start 2023-11-28T21:49:03.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c89f65640 1 -- start start 2023-11-28T21:49:03.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c89f65640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6c84151af0 0x7f6c840c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:03.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c89f65640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6c8406e300 0x7f6c84072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:03.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c89f65640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c84157b70 con 0x7f6c84151ed0 2023-11-28T21:49:03.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c89f65640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c840707f0 con 0x7f6c8406e300 2023-11-28T21:49:03.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c89f65640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c840c7b90 con 0x7f6c84151af0 2023-11-28T21:49:03.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c83fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6c8406e300 0x7f6c84072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:03.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c83fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f6c8406e300 0x7f6c84072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51324/0 (socket says 172.21.15.12:51324) 2023-11-28T21:49:03.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c83fff640 1 -- 172.21.15.12:0/2936922240 learned_addr learned my addr 172.21.15.12:0/2936922240 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:03.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.412+0000 7f6c837fe640 1 --2- 172.21.15.12:0/2936922240 >> v2:172.21.15.144:3300/0 conn(0x7f6c84151af0 0x7f6c840c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:03.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.416+0000 7f6c83fff640 1 -- 172.21.15.12:0/2936922240 >> v2:172.21.15.144:3300/0 conn(0x7f6c84151af0 msgr2=0x7f6c840c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:03.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.416+0000 7f6c83fff640 1 --2- 172.21.15.12:0/2936922240 >> v2:172.21.15.144:3300/0 conn(0x7f6c84151af0 0x7f6c840c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:03.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.416+0000 7f6c83fff640 1 -- 172.21.15.12:0/2936922240 >> v1:172.21.15.144:6789/0 conn(0x7f6c84151ed0 legacy=0x7f6c840c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:03.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.416+0000 7f6c83fff640 1 -- 172.21.15.12:0/2936922240 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c84157e50 con 0x7f6c8406e300 2023-11-28T21:49:03.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.416+0000 7f6c83fff640 1 --2- 172.21.15.12:0/2936922240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c8406e300 0x7f6c84072070 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6c840c7d20 tx=0x7f6c7c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fda10c5ab9f4218 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:03.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.416+0000 7f6c827fc640 1 -- 172.21.15.12:0/2936922240 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c0a5690 con 0x7f6c8406e300 2023-11-28T21:49:03.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.416+0000 7f6c827fc640 1 -- 172.21.15.12:0/2936922240 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c0a57f0 con 0x7f6c8406e300 2023-11-28T21:49:03.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.416+0000 7f6c827fc640 1 -- 172.21.15.12:0/2936922240 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c0a5b80 con 0x7f6c8406e300 2023-11-28T21:49:03.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 -- 172.21.15.12:0/2936922240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c8406e300 msgr2=0x7f6c84072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:03.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 --2- 172.21.15.12:0/2936922240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c8406e300 0x7f6c84072070 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6c840c7d20 tx=0x7f6c7c01c9d0 comp rx=0 tx=0).stop 2023-11-28T21:49:03.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 -- 172.21.15.12:0/2936922240 shutdown_connections 2023-11-28T21:49:03.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 --2- 172.21.15.12:0/2936922240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c8406e300 0x7f6c84072070 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:03.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 --2- 172.21.15.12:0/2936922240 >> v2:172.21.15.144:3300/0 conn(0x7f6c84151af0 0x7f6c840c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:03.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 -- 172.21.15.12:0/2936922240 >> 172.21.15.12:0/2936922240 conn(0x7f6c84066360 msgr2=0x7f6c84066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:03.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 -- 172.21.15.12:0/2936922240 shutdown_connections 2023-11-28T21:49:03.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 -- 172.21.15.12:0/2936922240 wait complete. 2023-11-28T21:49:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 Processor -- start 2023-11-28T21:49:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 -- start start 2023-11-28T21:49:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c8406e300 0x7f6c8414d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c84151af0 0x7f6c8413acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c8413d980 0x7f6c8413b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8414d180 con 0x7f6c84151af0 2023-11-28T21:49:03.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8414be10 con 0x7f6c8413d980 2023-11-28T21:49:03.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c89f65640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8413cc20 con 0x7f6c8406e300 2023-11-28T21:49:03.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c82ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c84151af0 0x7f6c8413acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:03.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c82ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c84151af0 0x7f6c8413acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51336/0 (socket says 172.21.15.12:51336) 2023-11-28T21:49:03.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c82ffd640 1 -- 172.21.15.12:0/2850298241 learned_addr learned my addr 172.21.15.12:0/2850298241 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:03.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c837fe640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c8406e300 0x7f6c8414d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:03.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.420+0000 7f6c83fff640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c8413d980 0x7f6c8413b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:03.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c82ffd640 1 -- 172.21.15.12:0/2850298241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c8406e300 msgr2=0x7f6c8414d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:03.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c82ffd640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c8406e300 0x7f6c8414d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:03.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c82ffd640 1 -- 172.21.15.12:0/2850298241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c8413d980 msgr2=0x7f6c8413b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:03.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c82ffd640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c8413d980 0x7f6c8413b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:03.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c82ffd640 1 -- 172.21.15.12:0/2850298241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c009b30 con 0x7f6c84151af0 2023-11-28T21:49:03.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c83fff640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c8413d980 0x7f6c8413b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:49:03.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c82ffd640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c84151af0 0x7f6c8413acb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800c3c0 tx=0x7f6c7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:03.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c6bfff640 1 -- 172.21.15.12:0/2850298241 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c780052d0 con 0x7f6c84151af0 2023-11-28T21:49:03.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c6bfff640 1 -- 172.21.15.12:0/2850298241 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c78005430 con 0x7f6c84151af0 2023-11-28T21:49:03.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c89f65640 1 -- 172.21.15.12:0/2850298241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8413c940 con 0x7f6c84151af0 2023-11-28T21:49:03.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c6bfff640 1 -- 172.21.15.12:0/2850298241 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c78003840 con 0x7f6c84151af0 2023-11-28T21:49:03.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c89f65640 1 -- 172.21.15.12:0/2850298241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c840c9440 con 0x7f6c84151af0 2023-11-28T21:49:03.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c89f65640 1 -- 172.21.15.12:0/2850298241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c84074f60 con 0x7f6c84151af0 2023-11-28T21:49:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.424+0000 7f6c6bfff640 1 -- 172.21.15.12:0/2850298241 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6c780039a0 con 0x7f6c84151af0 2023-11-28T21:49:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.428+0000 7f6c6bfff640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c700664a0 0x7f6c70068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.428+0000 7f6c6bfff640 1 -- 172.21.15.12:0/2850298241 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f6c780838f0 con 0x7f6c84151af0 2023-11-28T21:49:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.428+0000 7f6c837fe640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c700664a0 0x7f6c70068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.428+0000 7f6c837fe640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c700664a0 0x7f6c70068960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009b60 tx=0x7f6c74009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.432+0000 7f6c6bfff640 1 -- 172.21.15.12:0/2850298241 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c780507f0 con 0x7f6c84151af0 2023-11-28T21:49:03.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.624+0000 7f6c89f65640 1 -- 172.21.15.12:0/2850298241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c84072c10 con 0x7f6c84151af0 2023-11-28T21:49:03.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.624+0000 7f6c6bfff640 1 -- 172.21.15.12:0/2850298241 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5714 (secure 0 0 0) 0x7f6c78053e10 con 0x7f6c84151af0 2023-11-28T21:49:03.630 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:03.630 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:48:53.536021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:03.631 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 56 2023-11-28T21:49:03.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 -- 172.21.15.12:0/2850298241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c700664a0 msgr2=0x7f6c70068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:03.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c700664a0 0x7f6c70068960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009b60 tx=0x7f6c74009190 comp rx=0 tx=0).stop 2023-11-28T21:49:03.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 -- 172.21.15.12:0/2850298241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c84151af0 msgr2=0x7f6c8413acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:03.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c84151af0 0x7f6c8413acb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800c3c0 tx=0x7f6c7800c880 comp rx=0 tx=0).stop 2023-11-28T21:49:03.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 -- 172.21.15.12:0/2850298241 shutdown_connections 2023-11-28T21:49:03.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6c700664a0 0x7f6c70068960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:03.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c8413d980 0x7f6c8413b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:03.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6c84151af0 0x7f6c8413acb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:03.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 --2- 172.21.15.12:0/2850298241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c8406e300 0x7f6c8414d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:03.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 -- 172.21.15.12:0/2850298241 >> 172.21.15.12:0/2850298241 conn(0x7f6c84066360 msgr2=0x7f6c8414ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:03.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 -- 172.21.15.12:0/2850298241 shutdown_connections 2023-11-28T21:49:03.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:03.628+0000 7f6c89f65640 1 -- 172.21.15.12:0/2850298241 wait complete. 2023-11-28T21:49:03.652 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:49:05.653 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:05.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9342a6b640 1 Processor -- start 2023-11-28T21:49:05.807 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9342a6b640 1 -- start start 2023-11-28T21:49:05.808 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9342a6b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f933c0c27c0 0x7f933c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:05.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9342a6b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f933c06d7d0 0x7f933c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:05.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9342a6b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933c0c6ce0 con 0x7f933c1500d0 2023-11-28T21:49:05.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9342a6b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c06e120 con 0x7f933c06d7d0 2023-11-28T21:49:05.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9342a6b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933c157030 con 0x7f933c0c27c0 2023-11-28T21:49:05.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9341008640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f933c06d7d0 0x7f933c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:05.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9340807640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f933c1500d0 0x7f933c0c2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39792/0 (socket says 172.21.15.12:39792) 2023-11-28T21:49:05.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9340807640 1 -- 172.21.15.12:0/1323711164 learned_addr learned my addr 172.21.15.12:0/1323711164 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:49:05.809 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f933bfff640 1 --2- 172.21.15.12:0/1323711164 >> v2:172.21.15.144:3300/0 conn(0x7f933c0c27c0 0x7f933c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:05.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9341008640 1 -- 172.21.15.12:0/1323711164 >> v2:172.21.15.144:3300/0 conn(0x7f933c0c27c0 msgr2=0x7f933c156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:05.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9341008640 1 --2- 172.21.15.12:0/1323711164 >> v2:172.21.15.144:3300/0 conn(0x7f933c0c27c0 0x7f933c156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:05.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9341008640 1 -- 172.21.15.12:0/1323711164 >> v1:172.21.15.12:6789/0 conn(0x7f933c1500d0 legacy=0x7f933c0c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:05.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f9341008640 1 -- 172.21.15.12:0/1323711164 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c0c5df0 con 0x7f933c06d7d0 2023-11-28T21:49:05.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f933bfff640 1 --2- 172.21.15.12:0/1323711164 >> v2:172.21.15.144:3300/0 conn(0x7f933c0c27c0 0x7f933c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:49:05.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.804+0000 7f933b7fe640 1 -- 172.21.15.12:0/1323711164 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452533769 0 0) 0x7f933c0c6ce0 con 0x7f933c1500d0 2023-11-28T21:49:05.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f9341008640 1 --2- 172.21.15.12:0/1323711164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 0x7f933c06dbe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f933c068bd0 tx=0x7f933401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc6cccc0c460353 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:05.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f933b7fe640 1 -- 172.21.15.12:0/1323711164 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340a57d0 con 0x7f933c06d7d0 2023-11-28T21:49:05.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f933b7fe640 1 -- 172.21.15.12:0/1323711164 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93340a5930 con 0x7f933c06d7d0 2023-11-28T21:49:05.812 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f933b7fe640 1 -- 172.21.15.12:0/1323711164 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340a5cc0 con 0x7f933c06d7d0 2023-11-28T21:49:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f9342a6b640 1 -- 172.21.15.12:0/1323711164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 msgr2=0x7f933c06dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f9342a6b640 1 --2- 172.21.15.12:0/1323711164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 0x7f933c06dbe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f933c068bd0 tx=0x7f933401c910 comp rx=0 tx=0).stop 2023-11-28T21:49:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f9342a6b640 1 -- 172.21.15.12:0/1323711164 shutdown_connections 2023-11-28T21:49:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f9342a6b640 1 --2- 172.21.15.12:0/1323711164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 0x7f933c06dbe0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f9342a6b640 1 --2- 172.21.15.12:0/1323711164 >> v2:172.21.15.144:3300/0 conn(0x7f933c0c27c0 0x7f933c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f9342a6b640 1 -- 172.21.15.12:0/1323711164 >> 172.21.15.12:0/1323711164 conn(0x7f933c066360 msgr2=0x7f933c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f9342a6b640 1 -- 172.21.15.12:0/1323711164 shutdown_connections 2023-11-28T21:49:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.808+0000 7f9342a6b640 1 -- 172.21.15.12:0/1323711164 wait complete. 2023-11-28T21:49:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9342a6b640 1 Processor -- start 2023-11-28T21:49:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9342a6b640 1 -- start start 2023-11-28T21:49:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9342a6b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 0x7f933c1f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9342a6b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f933c0c27c0 0x7f933c14d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9342a6b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f933c1500d0 0x7f933c14e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9342a6b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c1f2aa0 con 0x7f933c06d7d0 2023-11-28T21:49:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9342a6b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c1f21d0 con 0x7f933c1500d0 2023-11-28T21:49:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9342a6b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933c0cb020 con 0x7f933c0c27c0 2023-11-28T21:49:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9340807640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 0x7f933c1f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9340807640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 0x7f933c1f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50000/0 (socket says 172.21.15.12:50000) 2023-11-28T21:49:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9340807640 1 -- 172.21.15.12:0/3842612445 learned_addr learned my addr 172.21.15.12:0/3842612445 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9341008640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f933c1500d0 0x7f933c14e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f933bfff640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f933c0c27c0 0x7f933c14d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9340807640 1 -- 172.21.15.12:0/3842612445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f933c0c27c0 msgr2=0x7f933c14d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9340807640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f933c0c27c0 0x7f933c14d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9340807640 1 -- 172.21.15.12:0/3842612445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f933c1500d0 msgr2=0x7f933c14e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9340807640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f933c1500d0 0x7f933c14e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9340807640 1 -- 172.21.15.12:0/3842612445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334009a70 con 0x7f933c06d7d0 2023-11-28T21:49:05.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9340807640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 0x7f933c1f6150 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f93300011f0 tx=0x7f933000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f93397fa640 1 -- 172.21.15.12:0/3842612445 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300074e0 con 0x7f933c06d7d0 2023-11-28T21:49:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f93397fa640 1 -- 172.21.15.12:0/3842612445 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9330007640 con 0x7f933c06d7d0 2023-11-28T21:49:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f93397fa640 1 -- 172.21.15.12:0/3842612445 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330004a10 con 0x7f933c06d7d0 2023-11-28T21:49:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9342a6b640 1 -- 172.21.15.12:0/3842612445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c1f6690 con 0x7f933c06d7d0 2023-11-28T21:49:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.812+0000 7f9342a6b640 1 -- 172.21.15.12:0/3842612445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c14f7b0 con 0x7f933c06d7d0 2023-11-28T21:49:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.816+0000 7f9342a6b640 1 -- 172.21.15.12:0/3842612445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c074f60 con 0x7f933c06d7d0 2023-11-28T21:49:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.816+0000 7f93397fa640 1 -- 172.21.15.12:0/3842612445 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9330004b70 con 0x7f933c06d7d0 2023-11-28T21:49:05.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.816+0000 7f93397fa640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9320066470 0x7f9320068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.816+0000 7f93397fa640 1 -- 172.21.15.12:0/3842612445 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f9330083550 con 0x7f933c06d7d0 2023-11-28T21:49:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.816+0000 7f933bfff640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9320066470 0x7f9320068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:05.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.816+0000 7f933bfff640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9320066470 0x7f9320068930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f933c0c49b0 tx=0x7f9328008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:05.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:05.820+0000 7f93397fa640 1 -- 172.21.15.12:0/3842612445 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9330050450 con 0x7f933c06d7d0 2023-11-28T21:49:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.012+0000 7f9342a6b640 1 -- 172.21.15.12:0/3842612445 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f933c1f6690 con 0x7f933c06d7d0 2023-11-28T21:49:06.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f93397fa640 1 -- 172.21.15.12:0/3842612445 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5718 (secure 0 0 0) 0x7f933c1f6690 con 0x7f933c06d7d0 2023-11-28T21:49:06.018 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:06.018 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:04.141632+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:reconnect","state_seq":28,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:06.018 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 57 2023-11-28T21:49:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 -- 172.21.15.12:0/3842612445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9320066470 msgr2=0x7f9320068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9320066470 0x7f9320068930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f933c0c49b0 tx=0x7f9328008040 comp rx=0 tx=0).stop 2023-11-28T21:49:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 -- 172.21.15.12:0/3842612445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 msgr2=0x7f933c1f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 0x7f933c1f6150 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f93300011f0 tx=0x7f933000c9b0 comp rx=0 tx=0).stop 2023-11-28T21:49:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 -- 172.21.15.12:0/3842612445 shutdown_connections 2023-11-28T21:49:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9320066470 0x7f9320068930 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f933c1500d0 0x7f933c14e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f933c0c27c0 0x7f933c14d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 --2- 172.21.15.12:0/3842612445 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f933c06d7d0 0x7f933c1f6150 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 -- 172.21.15.12:0/3842612445 >> 172.21.15.12:0/3842612445 conn(0x7f933c066360 msgr2=0x7f933c160140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 -- 172.21.15.12:0/3842612445 shutdown_connections 2023-11-28T21:49:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:06.016+0000 7f9342a6b640 1 -- 172.21.15.12:0/3842612445 wait complete. 2023-11-28T21:49:06.039 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 57, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 57, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6177": { "addr": "172.21.15.12:6838/1551825785", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 1551825785, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 1551825785, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6177, "incarnation": 56, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:reconnect", "state_seq": 28 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:49:04.141632+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 6177 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:49:08.041 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:08.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa981679640 1 Processor -- start 2023-11-28T21:49:08.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa981679640 1 -- start start 2023-11-28T21:49:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa981679640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa97c151af0 0x7fa97c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:08.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa981679640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c151ed0 0x7fa97c0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:08.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa981679640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97c157b70 con 0x7fa97c06e300 2023-11-28T21:49:08.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa981679640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97c0707f0 con 0x7fa97c151af0 2023-11-28T21:49:08.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa981679640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c0c7b90 con 0x7fa97c151ed0 2023-11-28T21:49:08.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa97b7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa97c06e300 0x7fa97c072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39804/0 (socket says 172.21.15.12:39804) 2023-11-28T21:49:08.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa97b7fe640 1 -- 172.21.15.12:0/1181232613 learned_addr learned my addr 172.21.15.12:0/1181232613 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:49:08.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa973fff640 1 --2- 172.21.15.12:0/1181232613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c151ed0 0x7fa97c0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:08.196 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa97affd640 1 --2- 172.21.15.12:0/1181232613 >> v2:172.21.15.144:3300/0 conn(0x7fa97c151af0 0x7fa97c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:08.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa97a7fc640 1 -- 172.21.15.12:0/1181232613 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927354185 0 0) 0x7fa97c157b70 con 0x7fa97c06e300 2023-11-28T21:49:08.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa97a7fc640 1 -- 172.21.15.12:0/1181232613 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa968003530 con 0x7fa97c06e300 2023-11-28T21:49:08.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa973fff640 1 -- 172.21.15.12:0/1181232613 >> v2:172.21.15.144:3300/0 conn(0x7fa97c151af0 msgr2=0x7fa97c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:08.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa97a7fc640 1 -- 172.21.15.12:0/1181232613 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 920640656 0 0) 0x7fa968003530 con 0x7fa97c06e300 2023-11-28T21:49:08.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa973fff640 1 --2- 172.21.15.12:0/1181232613 >> v2:172.21.15.144:3300/0 conn(0x7fa97c151af0 0x7fa97c0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:08.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa973fff640 1 -- 172.21.15.12:0/1181232613 >> v1:172.21.15.12:6789/0 conn(0x7fa97c06e300 legacy=0x7fa97c072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:08.197 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.192+0000 7fa973fff640 1 -- 172.21.15.12:0/1181232613 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c157e50 con 0x7fa97c151ed0 2023-11-28T21:49:08.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa973fff640 1 --2- 172.21.15.12:0/1181232613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c151ed0 0x7fa97c0c49d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa9640097c0 tx=0x7fa96401c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e95546ae15d539c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:08.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa97a7fc640 1 -- 172.21.15.12:0/1181232613 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974001a00 con 0x7fa97c151ed0 2023-11-28T21:49:08.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa97a7fc640 1 -- 172.21.15.12:0/1181232613 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9640a7050 con 0x7fa97c151ed0 2023-11-28T21:49:08.200 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa97a7fc640 1 -- 172.21.15.12:0/1181232613 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640ab600 con 0x7fa97c151ed0 2023-11-28T21:49:08.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa981679640 1 -- 172.21.15.12:0/1181232613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c151ed0 msgr2=0x7fa97c0c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:08.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa981679640 1 --2- 172.21.15.12:0/1181232613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c151ed0 0x7fa97c0c49d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa9640097c0 tx=0x7fa96401c3a0 comp rx=0 tx=0).stop 2023-11-28T21:49:08.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa981679640 1 -- 172.21.15.12:0/1181232613 shutdown_connections 2023-11-28T21:49:08.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa981679640 1 --2- 172.21.15.12:0/1181232613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c151ed0 0x7fa97c0c49d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:08.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa981679640 1 --2- 172.21.15.12:0/1181232613 >> v2:172.21.15.144:3300/0 conn(0x7fa97c151af0 0x7fa97c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:08.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa981679640 1 -- 172.21.15.12:0/1181232613 >> 172.21.15.12:0/1181232613 conn(0x7fa97c066360 msgr2=0x7fa97c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:08.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa981679640 1 -- 172.21.15.12:0/1181232613 shutdown_connections 2023-11-28T21:49:08.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.196+0000 7fa981679640 1 -- 172.21.15.12:0/1181232613 wait complete. 2023-11-28T21:49:08.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 Processor -- start 2023-11-28T21:49:08.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 -- start start 2023-11-28T21:49:08.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c06e300 0x7fa97c1f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:08.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa97c151af0 0x7fa97c1f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:08.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa97c144940 0x7fa97c0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c14feb0 con 0x7fa97c144940 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c1f3530 con 0x7fa97c151af0 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c14bed0 con 0x7fa97c06e300 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa97b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa97c144940 0x7fa97c0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa97b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa97c144940 0x7fa97c0ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50004/0 (socket says 172.21.15.12:50004) 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa97b7fe640 1 -- 172.21.15.12:0/1005394844 learned_addr learned my addr 172.21.15.12:0/1005394844 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa97affd640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c06e300 0x7fa97c1f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa973fff640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa97c151af0 0x7fa97c1f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa97b7fe640 1 -- 172.21.15.12:0/1005394844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c06e300 msgr2=0x7fa97c1f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa97b7fe640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c06e300 0x7fa97c1f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa97b7fe640 1 -- 172.21.15.12:0/1005394844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa97c151af0 msgr2=0x7fa97c1f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:08.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa97b7fe640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa97c151af0 0x7fa97c1f2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:08.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa97b7fe640 1 -- 172.21.15.12:0/1005394844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa964009420 con 0x7fa97c144940 2023-11-28T21:49:08.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa973fff640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa97c151af0 0x7fa97c1f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:49:08.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa97b7fe640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa97c144940 0x7fa97c0ca9b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa97400ccc0 tx=0x7fa97400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:08.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa9737fe640 1 -- 172.21.15.12:0/1005394844 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974012400 con 0x7fa97c144940 2023-11-28T21:49:08.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa9737fe640 1 -- 172.21.15.12:0/1005394844 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa974012560 con 0x7fa97c144940 2023-11-28T21:49:08.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa9737fe640 1 -- 172.21.15.12:0/1005394844 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9740128d0 con 0x7fa97c144940 2023-11-28T21:49:08.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 -- 172.21.15.12:0/1005394844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97c14c1b0 con 0x7fa97c144940 2023-11-28T21:49:08.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 -- 172.21.15.12:0/1005394844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97c1f6600 con 0x7fa97c144940 2023-11-28T21:49:08.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.200+0000 7fa981679640 1 -- 172.21.15.12:0/1005394844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97c074f60 con 0x7fa97c144940 2023-11-28T21:49:08.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.204+0000 7fa9737fe640 1 -- 172.21.15.12:0/1005394844 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa974007070 con 0x7fa97c144940 2023-11-28T21:49:08.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.204+0000 7fa9737fe640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa95c0664a0 0x7fa95c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:08.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.204+0000 7fa9737fe640 1 -- 172.21.15.12:0/1005394844 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fa9740845a0 con 0x7fa97c144940 2023-11-28T21:49:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.204+0000 7fa97affd640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa95c0664a0 0x7fa95c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:08.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.204+0000 7fa97affd640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa95c0664a0 0x7fa95c068960 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009b60 tx=0x7fa96c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:08.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.208+0000 7fa9737fe640 1 -- 172.21.15.12:0/1005394844 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa974051410 con 0x7fa97c144940 2023-11-28T21:49:08.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.396+0000 7fa981679640 1 -- 172.21.15.12:0/1005394844 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa97c072c10 con 0x7fa97c144940 2023-11-28T21:49:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.396+0000 7fa9737fe640 1 -- 172.21.15.12:0/1005394844 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5718 (secure 0 0 0) 0x7fa974054a30 con 0x7fa97c144940 2023-11-28T21:49:08.401 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:08.401 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:04.141632+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:reconnect","state_seq":28,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:08.401 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 57 2023-11-28T21:49:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 -- 172.21.15.12:0/1005394844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa95c0664a0 msgr2=0x7fa95c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa95c0664a0 0x7fa95c068960 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009b60 tx=0x7fa96c009190 comp rx=0 tx=0).stop 2023-11-28T21:49:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 -- 172.21.15.12:0/1005394844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa97c144940 msgr2=0x7fa97c0ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa97c144940 0x7fa97c0ca9b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa97400ccc0 tx=0x7fa97400a430 comp rx=0 tx=0).stop 2023-11-28T21:49:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 -- 172.21.15.12:0/1005394844 shutdown_connections 2023-11-28T21:49:08.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa95c0664a0 0x7fa95c068960 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa97c144940 0x7fa97c0ca9b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa97c151af0 0x7fa97c1f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 --2- 172.21.15.12:0/1005394844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa97c06e300 0x7fa97c1f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 -- 172.21.15.12:0/1005394844 >> 172.21.15.12:0/1005394844 conn(0x7fa97c066360 msgr2=0x7fa97c0c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 -- 172.21.15.12:0/1005394844 shutdown_connections 2023-11-28T21:49:08.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:08.400+0000 7fa981679640 1 -- 172.21.15.12:0/1005394844 wait complete. 2023-11-28T21:49:08.422 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:49:10.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:10.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.572+0000 7fa9fc9aa640 1 Processor -- start 2023-11-28T21:49:10.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.572+0000 7fa9fc9aa640 1 -- start start 2023-11-28T21:49:10.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9fc9aa640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa9f806e300 0x7fa9f8156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:10.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9fc9aa640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f8067410 con 0x7fa9f806cf40 2023-11-28T21:49:10.578 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9fc9aa640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f806e090 con 0x7fa9f8150c20 2023-11-28T21:49:10.579 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9fc9aa640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f8157f30 con 0x7fa9f806e300 2023-11-28T21:49:10.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f6d9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa9f806cf40 0x7fa9f8158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39816/0 (socket says 172.21.15.12:39816) 2023-11-28T21:49:10.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f6d9d640 1 -- 172.21.15.12:0/1218839749 learned_addr learned my addr 172.21.15.12:0/1218839749 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:49:10.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9effff640 1 --2- 172.21.15.12:0/1218839749 >> v2:172.21.15.144:3300/0 conn(0x7fa9f806e300 0x7fa9f8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:10.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038252585 0 0) 0x7fa9f8067410 con 0x7fa9f806cf40 2023-11-28T21:49:10.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9e4003530 con 0x7fa9f806cf40 2023-11-28T21:49:10.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3947045155 0 0) 0x7fa9e4003530 con 0x7fa9f806cf40 2023-11-28T21:49:10.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 >> v2:172.21.15.144:3300/0 conn(0x7fa9f806e300 msgr2=0x7fa9f8156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:10.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 --2- 172.21.15.12:0/1218839749 >> v2:172.21.15.144:3300/0 conn(0x7fa9f806e300 0x7fa9f8156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 >> v1:172.21.15.144:6789/0 conn(0x7fa9f8150c20 legacy=0x7fa9f80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:10.581 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f8151000 con 0x7fa9f806cf40 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fa9f0001a00 con 0x7fa9f806cf40 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 >> v1:172.21.15.12:6789/0 conn(0x7fa9f806cf40 legacy=0x7fa9f8158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9e40039f0 0x7fa9e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9e40047d0 0x7fa9e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9e4005190 0x7fa9e400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8157f30 con 0x7fa9e4005190 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e400c010 con 0x7fa9e40039f0 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e400c1d0 con 0x7fa9e40047d0 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f6d9d640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9e4005190 0x7fa9e400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9effff640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9e40047d0 0x7fa9e4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f659c640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9e40039f0 0x7fa9e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:10.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f6d9d640 1 -- 172.21.15.12:0/1218839749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9e40047d0 msgr2=0x7fa9e4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:10.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f6d9d640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9e40047d0 0x7fa9e4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f6d9d640 1 -- 172.21.15.12:0/1218839749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9e40039f0 msgr2=0x7fa9e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:10.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f6d9d640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9e40039f0 0x7fa9e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f6d9d640 1 -- 172.21.15.12:0/1218839749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e4003530 con 0x7fa9e4005190 2023-11-28T21:49:10.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.576+0000 7fa9f659c640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9e40039f0 0x7fa9e4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:49:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9f6d9d640 1 -- 172.21.15.12:0/1218839749 reap_dead start 2023-11-28T21:49:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9f6d9d640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9e4005190 0x7fa9e400b990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000a830 tx=0x7fa9f001faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945e481f222b19b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00af070 con 0x7fa9e4005190 2023-11-28T21:49:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f00aa4d0 con 0x7fa9e4005190 2023-11-28T21:49:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9f5d9b640 1 -- 172.21.15.12:0/1218839749 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00aa8a0 con 0x7fa9e4005190 2023-11-28T21:49:10.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/1218839749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9e4005190 msgr2=0x7fa9e400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:10.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9fc9aa640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9e4005190 0x7fa9e400b990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000a830 tx=0x7fa9f001faa0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/1218839749 shutdown_connections 2023-11-28T21:49:10.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9fc9aa640 1 --2- 172.21.15.12:0/1218839749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9e4005190 0x7fa9e400b990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/1218839749 >> 172.21.15.12:0/1218839749 conn(0x7fa9f8066360 msgr2=0x7fa9f8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:10.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/1218839749 shutdown_connections 2023-11-28T21:49:10.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.580+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/1218839749 wait complete. 2023-11-28T21:49:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9fc9aa640 1 Processor -- start 2023-11-28T21:49:10.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9fc9aa640 1 -- start start 2023-11-28T21:49:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9fc9aa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9e4005190 0x7fa9f8148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9fc9aa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f81eb640 0x7fa9f8149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9fc9aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9f814a1a0 0x7fa9f814adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9fc9aa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f814a680 con 0x7fa9f814a1a0 2023-11-28T21:49:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9fc9aa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f814a960 con 0x7fa9f81eb640 2023-11-28T21:49:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9fc9aa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f81ebb30 con 0x7fa9e4005190 2023-11-28T21:49:10.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9f6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9f814a1a0 0x7fa9f814adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9f659c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9e4005190 0x7fa9f8148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9f6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9f814a1a0 0x7fa9f814adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50016/0 (socket says 172.21.15.12:50016) 2023-11-28T21:49:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9f6d9d640 1 -- 172.21.15.12:0/4252967810 learned_addr learned my addr 172.21.15.12:0/4252967810 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9effff640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f81eb640 0x7fa9f8149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9f6d9d640 1 -- 172.21.15.12:0/4252967810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9e4005190 msgr2=0x7fa9f8148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9f6d9d640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9e4005190 0x7fa9f8148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9f6d9d640 1 -- 172.21.15.12:0/4252967810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f81eb640 msgr2=0x7fa9f8149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9f6d9d640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f81eb640 0x7fa9f8149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9f6d9d640 1 -- 172.21.15.12:0/4252967810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f81ebe10 con 0x7fa9f814a1a0 2023-11-28T21:49:10.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9effff640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f81eb640 0x7fa9f8149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:49:10.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.584+0000 7fa9f6d9d640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9f814a1a0 0x7fa9f814adf0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0002240 tx=0x7fa9e4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.588+0000 7fa9eeffd640 1 -- 172.21.15.12:0/4252967810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00af040 con 0x7fa9f814a1a0 2023-11-28T21:49:10.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.588+0000 7fa9eeffd640 1 -- 172.21.15.12:0/4252967810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f00019c0 con 0x7fa9f814a1a0 2023-11-28T21:49:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.588+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/4252967810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f81ec0f0 con 0x7fa9f814a1a0 2023-11-28T21:49:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.588+0000 7fa9eeffd640 1 -- 172.21.15.12:0/4252967810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00aade0 con 0x7fa9f814a1a0 2023-11-28T21:49:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.588+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/4252967810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f80c32c0 con 0x7fa9f814a1a0 2023-11-28T21:49:10.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.588+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/4252967810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f80c6130 con 0x7fa9f814a1a0 2023-11-28T21:49:10.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.588+0000 7fa9eeffd640 1 -- 172.21.15.12:0/4252967810 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa9f0004b70 con 0x7fa9f814a1a0 2023-11-28T21:49:10.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.592+0000 7fa9eeffd640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa9d80664a0 0x7fa9d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:10.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.592+0000 7fa9eeffd640 1 -- 172.21.15.12:0/4252967810 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fa9f0121380 con 0x7fa9f814a1a0 2023-11-28T21:49:10.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.592+0000 7fa9f659c640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa9d80664a0 0x7fa9d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:10.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.592+0000 7fa9f659c640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa9d80664a0 0x7fa9d8068960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8001790 tx=0x7fa9e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:10.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.596+0000 7fa9eeffd640 1 -- 172.21.15.12:0/4252967810 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa9f00ee0b0 con 0x7fa9f814a1a0 2023-11-28T21:49:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.784+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/4252967810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9f81ec5a0 con 0x7fa9f814a1a0 2023-11-28T21:49:10.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.784+0000 7fa9eeffd640 1 -- 172.21.15.12:0/4252967810 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fa9f00f16d0 con 0x7fa9f814a1a0 2023-11-28T21:49:10.790 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:10.790 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:09.554071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":30,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:10.790 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 58 2023-11-28T21:49:10.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/4252967810 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa9d80664a0 msgr2=0x7fa9d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:10.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa9d80664a0 0x7fa9d8068960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8001790 tx=0x7fa9e8009040 comp rx=0 tx=0).stop 2023-11-28T21:49:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/4252967810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9f814a1a0 msgr2=0x7fa9f814adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9f814a1a0 0x7fa9f814adf0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0002240 tx=0x7fa9e4005800 comp rx=0 tx=0).stop 2023-11-28T21:49:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/4252967810 shutdown_connections 2023-11-28T21:49:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa9d80664a0 0x7fa9d8068960 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa9f814a1a0 0x7fa9f814adf0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9f81eb640 0x7fa9f8149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 --2- 172.21.15.12:0/4252967810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9e4005190 0x7fa9f8148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/4252967810 >> 172.21.15.12:0/4252967810 conn(0x7fa9f8066360 msgr2=0x7fa9f8155380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/4252967810 shutdown_connections 2023-11-28T21:49:10.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:10.788+0000 7fa9fc9aa640 1 -- 172.21.15.12:0/4252967810 wait complete. 2023-11-28T21:49:10.811 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:49:12.812 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:12.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffac569e640 1 Processor -- start 2023-11-28T21:49:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffac569e640 1 -- start start 2023-11-28T21:49:12.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffac569e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffac0151af0 0x7ffac00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:12.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffac569e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac006e300 0x7ffac0072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:12.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffac569e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffac0157b70 con 0x7ffac0151ed0 2023-11-28T21:49:12.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffac569e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac00707f0 con 0x7ffac0151af0 2023-11-28T21:49:12.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffac569e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac00c7b90 con 0x7ffac006e300 2023-11-28T21:49:12.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffabeffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffac0151af0 0x7ffac00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:12.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffabeffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ffac0151af0 0x7ffac00c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50020/0 (socket says 172.21.15.12:50020) 2023-11-28T21:49:12.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffabeffd640 1 -- 172.21.15.12:0/2921360938 learned_addr learned my addr 172.21.15.12:0/2921360938 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:12.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffabf7fe640 1 --2- 172.21.15.12:0/2921360938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac006e300 0x7ffac0072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:12.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffabeffd640 1 -- 172.21.15.12:0/2921360938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac006e300 msgr2=0x7ffac0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:12.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffabeffd640 1 --2- 172.21.15.12:0/2921360938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac006e300 0x7ffac0072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:12.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffabeffd640 1 -- 172.21.15.12:0/2921360938 >> v1:172.21.15.144:6789/0 conn(0x7ffac0151ed0 legacy=0x7ffac00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:12.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.964+0000 7ffabeffd640 1 -- 172.21.15.12:0/2921360938 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac0157e50 con 0x7ffac0151af0 2023-11-28T21:49:12.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffabeffd640 1 --2- 172.21.15.12:0/2921360938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac0151af0 0x7ffac00c7650 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009870 tx=0x7ffab001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de72b7ab81294ebd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:12.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffabe7fc640 1 -- 172.21.15.12:0/2921360938 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00ab600 con 0x7ffac0151af0 2023-11-28T21:49:12.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffabe7fc640 1 -- 172.21.15.12:0/2921360938 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab00a7050 con 0x7ffac0151af0 2023-11-28T21:49:12.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffabe7fc640 1 -- 172.21.15.12:0/2921360938 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00ab980 con 0x7ffac0151af0 2023-11-28T21:49:12.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffac569e640 1 -- 172.21.15.12:0/2921360938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac0151af0 msgr2=0x7ffac00c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:12.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffac569e640 1 --2- 172.21.15.12:0/2921360938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac0151af0 0x7ffac00c7650 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009870 tx=0x7ffab001c540 comp rx=0 tx=0).stop 2023-11-28T21:49:12.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffac569e640 1 -- 172.21.15.12:0/2921360938 shutdown_connections 2023-11-28T21:49:12.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffac569e640 1 --2- 172.21.15.12:0/2921360938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac006e300 0x7ffac0072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:12.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffac569e640 1 --2- 172.21.15.12:0/2921360938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac0151af0 0x7ffac00c7650 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:12.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffac569e640 1 -- 172.21.15.12:0/2921360938 >> 172.21.15.12:0/2921360938 conn(0x7ffac0066360 msgr2=0x7ffac0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:12.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffac569e640 1 -- 172.21.15.12:0/2921360938 shutdown_connections 2023-11-28T21:49:12.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.968+0000 7ffac569e640 1 -- 172.21.15.12:0/2921360938 wait complete. 2023-11-28T21:49:12.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffac569e640 1 Processor -- start 2023-11-28T21:49:12.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffac569e640 1 -- start start 2023-11-28T21:49:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffac569e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac006e300 0x7ffac0155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffac569e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac0151af0 0x7ffac014cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffac569e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac014c080 0x7ffac013c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffac569e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac0156d30 con 0x7ffac014c080 2023-11-28T21:49:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffac569e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac014d610 con 0x7ffac006e300 2023-11-28T21:49:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffac569e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac01f6690 con 0x7ffac0151af0 2023-11-28T21:49:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac014c080 0x7ffac013c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac014c080 0x7ffac013c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50030/0 (socket says 172.21.15.12:50030) 2023-11-28T21:49:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabf7fe640 1 -- 172.21.15.12:0/1012312326 learned_addr learned my addr 172.21.15.12:0/1012312326 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabeffd640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac006e300 0x7ffac0155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffab7fff640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac0151af0 0x7ffac014cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabf7fe640 1 -- 172.21.15.12:0/1012312326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac0151af0 msgr2=0x7ffac014cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabf7fe640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac0151af0 0x7ffac014cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabf7fe640 1 -- 172.21.15.12:0/1012312326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac006e300 msgr2=0x7ffac0155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabf7fe640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac006e300 0x7ffac0155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabf7fe640 1 -- 172.21.15.12:0/1012312326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab00094d0 con 0x7ffac014c080 2023-11-28T21:49:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabeffd640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac006e300 0x7ffac0155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:49:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffab7fff640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac0151af0 0x7ffac014cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:49:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffabf7fe640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac014c080 0x7ffac013c7f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffab8007e30 tx=0x7ffab800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffab77fe640 1 -- 172.21.15.12:0/1012312326 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8014620 con 0x7ffac014c080 2023-11-28T21:49:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffab77fe640 1 -- 172.21.15.12:0/1012312326 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab800ee90 con 0x7ffac014c080 2023-11-28T21:49:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffab77fe640 1 -- 172.21.15.12:0/1012312326 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab80149c0 con 0x7ffac014c080 2023-11-28T21:49:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.972+0000 7ffac569e640 1 -- 172.21.15.12:0/1012312326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac01f6970 con 0x7ffac014c080 2023-11-28T21:49:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.976+0000 7ffac569e640 1 -- 172.21.15.12:0/1012312326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac01f6f30 con 0x7ffac014c080 2023-11-28T21:49:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.976+0000 7ffac569e640 1 -- 172.21.15.12:0/1012312326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac0155240 con 0x7ffac014c080 2023-11-28T21:49:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.976+0000 7ffab77fe640 1 -- 172.21.15.12:0/1012312326 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ffab8014b20 con 0x7ffac014c080 2023-11-28T21:49:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.976+0000 7ffab77fe640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ffaa0066470 0x7ffaa0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.976+0000 7ffab77fe640 1 -- 172.21.15.12:0/1012312326 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7ffab8084f00 con 0x7ffac014c080 2023-11-28T21:49:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.976+0000 7ffabeffd640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ffaa0066470 0x7ffaa0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:12.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.976+0000 7ffabeffd640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ffaa0066470 0x7ffaa0068930 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffac00c8e60 tx=0x7ffab0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:12.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:12.980+0000 7ffab77fe640 1 -- 172.21.15.12:0/1012312326 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffab8051cc0 con 0x7ffac014c080 2023-11-28T21:49:13.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 -- 172.21.15.12:0/1012312326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffac0074f60 con 0x7ffac014c080 2023-11-28T21:49:13.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffab77fe640 1 -- 172.21.15.12:0/1012312326 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5715 (secure 0 0 0) 0x7ffab80552e0 con 0x7ffac014c080 2023-11-28T21:49:13.175 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:13.175 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:09.554071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":30,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:13.175 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 58 2023-11-28T21:49:13.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 -- 172.21.15.12:0/1012312326 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ffaa0066470 msgr2=0x7ffaa0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:13.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ffaa0066470 0x7ffaa0068930 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffac00c8e60 tx=0x7ffab0008040 comp rx=0 tx=0).stop 2023-11-28T21:49:13.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 -- 172.21.15.12:0/1012312326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac014c080 msgr2=0x7ffac013c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:13.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac014c080 0x7ffac013c7f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffab8007e30 tx=0x7ffab800e4b0 comp rx=0 tx=0).stop 2023-11-28T21:49:13.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 -- 172.21.15.12:0/1012312326 shutdown_connections 2023-11-28T21:49:13.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ffaa0066470 0x7ffaa0068930 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:13.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffac014c080 0x7ffac013c7f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:13.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffac0151af0 0x7ffac014cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:13.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 --2- 172.21.15.12:0/1012312326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffac006e300 0x7ffac0155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:13.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 -- 172.21.15.12:0/1012312326 >> 172.21.15.12:0/1012312326 conn(0x7ffac0066360 msgr2=0x7ffac006cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:13.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 -- 172.21.15.12:0/1012312326 shutdown_connections 2023-11-28T21:49:13.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:13.172+0000 7ffac569e640 1 -- 172.21.15.12:0/1012312326 wait complete. 2023-11-28T21:49:13.196 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:49:15.197 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:15.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.348+0000 7f2a2aa5f640 1 Processor -- start 2023-11-28T21:49:15.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.348+0000 7f2a2aa5f640 1 -- start start 2023-11-28T21:49:15.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.348+0000 7f2a2aa5f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2a2406e300 0x7f2a24156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:15.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.348+0000 7f2a2aa5f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a24067410 con 0x7f2a24150c20 2023-11-28T21:49:15.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.348+0000 7f2a2aa5f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2406e090 con 0x7f2a2406cf40 2023-11-28T21:49:15.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.348+0000 7f2a2aa5f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a24157f30 con 0x7f2a2406e300 2023-11-28T21:49:15.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a23fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2a24150c20 0x7f2a240c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53942/0 (socket says 172.21.15.12:53942) 2023-11-28T21:49:15.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a23fff640 1 -- 172.21.15.12:0/3987614592 learned_addr learned my addr 172.21.15.12:0/3987614592 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:49:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a237fe640 1 --2- 172.21.15.12:0/3987614592 >> v2:172.21.15.12:3300/0 conn(0x7f2a2406e300 0x7f2a24156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a22ffd640 1 -- 172.21.15.12:0/3987614592 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194008455 0 0) 0x7f2a24067410 con 0x7f2a24150c20 2023-11-28T21:49:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a22ffd640 1 -- 172.21.15.12:0/3987614592 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a0c003530 con 0x7f2a24150c20 2023-11-28T21:49:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a237fe640 1 -- 172.21.15.12:0/3987614592 >> v1:172.21.15.144:6789/0 conn(0x7f2a2406cf40 legacy=0x7f2a24158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a237fe640 1 -- 172.21.15.12:0/3987614592 >> v1:172.21.15.12:6789/0 conn(0x7f2a24150c20 legacy=0x7f2a240c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:15.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a237fe640 1 -- 172.21.15.12:0/3987614592 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24151000 con 0x7f2a2406e300 2023-11-28T21:49:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a237fe640 1 --2- 172.21.15.12:0/3987614592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a2406e300 0x7f2a24156910 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2a180097c0 tx=0x7f2a1801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7c3d33833bbe5e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a22ffd640 1 -- 172.21.15.12:0/3987614592 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a180a8050 con 0x7f2a2406e300 2023-11-28T21:49:15.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a22ffd640 1 -- 172.21.15.12:0/3987614592 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a180ac5f0 con 0x7f2a2406e300 2023-11-28T21:49:15.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.352+0000 7f2a22ffd640 1 -- 172.21.15.12:0/3987614592 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a180ac980 con 0x7f2a2406e300 2023-11-28T21:49:15.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3987614592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a2406e300 msgr2=0x7f2a24156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:15.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 --2- 172.21.15.12:0/3987614592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a2406e300 0x7f2a24156910 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2a180097c0 tx=0x7f2a1801c480 comp rx=0 tx=0).stop 2023-11-28T21:49:15.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3987614592 shutdown_connections 2023-11-28T21:49:15.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 --2- 172.21.15.12:0/3987614592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a2406e300 0x7f2a24156910 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:15.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3987614592 >> 172.21.15.12:0/3987614592 conn(0x7f2a24066360 msgr2=0x7f2a24066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:15.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3987614592 shutdown_connections 2023-11-28T21:49:15.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3987614592 wait complete. 2023-11-28T21:49:15.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 Processor -- start 2023-11-28T21:49:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 -- start start 2023-11-28T21:49:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2406cf40 0x7f2a240b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a2406e300 0x7f2a241f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24150c20 0x7f2a2414f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a241f25d0 con 0x7f2a24150c20 2023-11-28T21:49:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a24154280 con 0x7f2a2406cf40 2023-11-28T21:49:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a2aa5f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a241f1670 con 0x7f2a2406e300 2023-11-28T21:49:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a28ffc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24150c20 0x7f2a2414f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a28ffc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24150c20 0x7f2a2414f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55996/0 (socket says 172.21.15.12:55996) 2023-11-28T21:49:15.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a28ffc640 1 -- 172.21.15.12:0/3291949393 learned_addr learned my addr 172.21.15.12:0/3291949393 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a237fe640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a2406e300 0x7f2a241f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.356+0000 7f2a23fff640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2406cf40 0x7f2a240b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a28ffc640 1 -- 172.21.15.12:0/3291949393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a2406e300 msgr2=0x7f2a241f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a28ffc640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a2406e300 0x7f2a241f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a28ffc640 1 -- 172.21.15.12:0/3291949393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2406cf40 msgr2=0x7f2a240b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a28ffc640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2406cf40 0x7f2a240b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a28ffc640 1 -- 172.21.15.12:0/3291949393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a18009420 con 0x7f2a24150c20 2023-11-28T21:49:15.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a28ffc640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24150c20 0x7f2a2414f440 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00c770 tx=0x7f2a1c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a20ff9640 1 -- 172.21.15.12:0/3291949393 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c003b80 con 0x7f2a24150c20 2023-11-28T21:49:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a20ff9640 1 -- 172.21.15.12:0/3291949393 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c003ce0 con 0x7f2a24150c20 2023-11-28T21:49:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3291949393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a241f1950 con 0x7f2a24150c20 2023-11-28T21:49:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a20ff9640 1 -- 172.21.15.12:0/3291949393 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00aa90 con 0x7f2a24150c20 2023-11-28T21:49:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3291949393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a241f1f10 con 0x7f2a24150c20 2023-11-28T21:49:15.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3291949393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a241f2710 con 0x7f2a24150c20 2023-11-28T21:49:15.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.360+0000 7f2a20ff9640 1 -- 172.21.15.12:0/3291949393 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2a1c00abf0 con 0x7f2a24150c20 2023-11-28T21:49:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.364+0000 7f2a20ff9640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a10066470 0x7f2a10068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.364+0000 7f2a20ff9640 1 -- 172.21.15.12:0/3291949393 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f2a1c084300 con 0x7f2a24150c20 2023-11-28T21:49:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.364+0000 7f2a23fff640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a10066470 0x7f2a10068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:15.366 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.364+0000 7f2a23fff640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a10066470 0x7f2a10068930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2a140011f0 tx=0x7f2a14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.368+0000 7f2a20ff9640 1 -- 172.21.15.12:0/3291949393 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2a241f2710 con 0x7f2a24150c20 2023-11-28T21:49:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3291949393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a241f2710 con 0x7f2a24150c20 2023-11-28T21:49:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a20ff9640 1 -- 172.21.15.12:0/3291949393 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f2a241f2710 con 0x7f2a24150c20 2023-11-28T21:49:15.564 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:15.564 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:09.554071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:rejoin","state_seq":30,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 58 2023-11-28T21:49:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3291949393 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a10066470 msgr2=0x7f2a10068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a10066470 0x7f2a10068930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2a140011f0 tx=0x7f2a14009040 comp rx=0 tx=0).stop 2023-11-28T21:49:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3291949393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24150c20 msgr2=0x7f2a2414f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24150c20 0x7f2a2414f440 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00c770 tx=0x7f2a1c00cc30 comp rx=0 tx=0).stop 2023-11-28T21:49:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3291949393 shutdown_connections 2023-11-28T21:49:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a10066470 0x7f2a10068930 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a24150c20 0x7f2a2414f440 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a2406e300 0x7f2a241f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 --2- 172.21.15.12:0/3291949393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a2406cf40 0x7f2a240b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3291949393 >> 172.21.15.12:0/3291949393 conn(0x7f2a24066360 msgr2=0x7f2a24156610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.560+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3291949393 shutdown_connections 2023-11-28T21:49:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:15.564+0000 7f2a2aa5f640 1 -- 172.21.15.12:0/3291949393 wait complete. 2023-11-28T21:49:15.585 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:49:15.830 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:49:15.834 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:49:17.586 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee22617640 1 Processor -- start 2023-11-28T21:49:17.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee22617640 1 -- start start 2023-11-28T21:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee22617640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c154100 0x7fee1c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee22617640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fee1c0c9fd0 0x7fee1c06a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee22617640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1c15e270 con 0x7fee1c0ca3f0 2023-11-28T21:49:17.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee22617640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1c0c4e60 con 0x7fee1c0c9fd0 2023-11-28T21:49:17.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee22617640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c1544e0 con 0x7fee1c154100 2023-11-28T21:49:17.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee1b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fee1c0c9fd0 0x7fee1c06a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:17.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee1b7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fee1c0c9fd0 0x7fee1c06a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56004/0 (socket says 172.21.15.12:56004) 2023-11-28T21:49:17.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee1b7fe640 1 -- 172.21.15.12:0/1017635148 learned_addr learned my addr 172.21.15.12:0/1017635148 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:17.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee1bfff640 1 --2- 172.21.15.12:0/1017635148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c154100 0x7fee1c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee1b7fe640 1 -- 172.21.15.12:0/1017635148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c154100 msgr2=0x7fee1c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee1b7fe640 1 --2- 172.21.15.12:0/1017635148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c154100 0x7fee1c15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee1b7fe640 1 -- 172.21.15.12:0/1017635148 >> v1:172.21.15.144:6789/0 conn(0x7fee1c0ca3f0 legacy=0x7fee1c0c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:17.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.740+0000 7fee1b7fe640 1 -- 172.21.15.12:0/1017635148 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c15d9d0 con 0x7fee1c0c9fd0 2023-11-28T21:49:17.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee1b7fe640 1 --2- 172.21.15.12:0/1017635148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c0c9fd0 0x7fee1c06a5b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fee100098a0 tx=0x7fee1001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c849dfc3b2e4fce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:17.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee1affd640 1 -- 172.21.15.12:0/1017635148 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee100aa650 con 0x7fee1c0c9fd0 2023-11-28T21:49:17.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee1affd640 1 -- 172.21.15.12:0/1017635148 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee100aa7b0 con 0x7fee1c0c9fd0 2023-11-28T21:49:17.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee1affd640 1 -- 172.21.15.12:0/1017635148 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee100aab40 con 0x7fee1c0c9fd0 2023-11-28T21:49:17.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee22617640 1 -- 172.21.15.12:0/1017635148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c0c9fd0 msgr2=0x7fee1c06a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:17.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee22617640 1 --2- 172.21.15.12:0/1017635148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c0c9fd0 0x7fee1c06a5b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fee100098a0 tx=0x7fee1001c4f0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee22617640 1 -- 172.21.15.12:0/1017635148 shutdown_connections 2023-11-28T21:49:17.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee22617640 1 --2- 172.21.15.12:0/1017635148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c0c9fd0 0x7fee1c06a5b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee22617640 1 --2- 172.21.15.12:0/1017635148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c154100 0x7fee1c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee22617640 1 -- 172.21.15.12:0/1017635148 >> 172.21.15.12:0/1017635148 conn(0x7fee1c06c690 msgr2=0x7fee1c066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:17.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee22617640 1 -- 172.21.15.12:0/1017635148 shutdown_connections 2023-11-28T21:49:17.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.744+0000 7fee22617640 1 -- 172.21.15.12:0/1017635148 wait complete. 2023-11-28T21:49:17.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee22617640 1 Processor -- start 2023-11-28T21:49:17.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee22617640 1 -- start start 2023-11-28T21:49:17.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee22617640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0c9fd0 0x7fee1c0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:17.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee22617640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee1c0ca3f0 0x7fee1c145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:17.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee22617640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c154100 0x7fee1c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:17.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee22617640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0c95f0 con 0x7fee1c154100 2023-11-28T21:49:17.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee22617640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0c8a90 con 0x7fee1c0ca3f0 2023-11-28T21:49:17.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee22617640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0c8750 con 0x7fee1c0c9fd0 2023-11-28T21:49:17.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee20bb4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c154100 0x7fee1c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:17.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee20bb4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c154100 0x7fee1c0c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56018/0 (socket says 172.21.15.12:56018) 2023-11-28T21:49:17.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee20bb4640 1 -- 172.21.15.12:0/698956766 learned_addr learned my addr 172.21.15.12:0/698956766 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:17.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee1bfff640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0c9fd0 0x7fee1c0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:17.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee1b7fe640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee1c0ca3f0 0x7fee1c145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:17.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee20bb4640 1 -- 172.21.15.12:0/698956766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0c9fd0 msgr2=0x7fee1c0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:17.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee20bb4640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0c9fd0 0x7fee1c0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee20bb4640 1 -- 172.21.15.12:0/698956766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee1c0ca3f0 msgr2=0x7fee1c145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee20bb4640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee1c0ca3f0 0x7fee1c145280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee20bb4640 1 -- 172.21.15.12:0/698956766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100094d0 con 0x7fee1c154100 2023-11-28T21:49:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee20bb4640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c154100 0x7fee1c0c6980 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c730 tx=0x7fee1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee18ff9640 1 -- 172.21.15.12:0/698956766 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14003b80 con 0x7fee1c154100 2023-11-28T21:49:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee18ff9640 1 -- 172.21.15.12:0/698956766 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee14003ce0 con 0x7fee1c154100 2023-11-28T21:49:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee18ff9640 1 -- 172.21.15.12:0/698956766 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1400aa90 con 0x7fee1c154100 2023-11-28T21:49:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee22617640 1 -- 172.21.15.12:0/698956766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1c1443c0 con 0x7fee1c154100 2023-11-28T21:49:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.748+0000 7fee22617640 1 -- 172.21.15.12:0/698956766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1c144120 con 0x7fee1c154100 2023-11-28T21:49:17.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.752+0000 7fee22617640 1 -- 172.21.15.12:0/698956766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1c067c40 con 0x7fee1c154100 2023-11-28T21:49:17.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.752+0000 7fee18ff9640 1 -- 172.21.15.12:0/698956766 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fee1400abf0 con 0x7fee1c154100 2023-11-28T21:49:17.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.752+0000 7fee18ff9640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee08066470 0x7fee08068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:17.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.752+0000 7fee18ff9640 1 -- 172.21.15.12:0/698956766 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fee140841e0 con 0x7fee1c154100 2023-11-28T21:49:17.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.752+0000 7fee1bfff640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee08066470 0x7fee08068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:17.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.752+0000 7fee1bfff640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee08066470 0x7fee08068930 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fee0c007a60 tx=0x7fee0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:17.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.756+0000 7fee18ff9640 1 -- 172.21.15.12:0/698956766 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee14051050 con 0x7fee1c154100 2023-11-28T21:49:17.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.944+0000 7fee22617640 1 -- 172.21.15.12:0/698956766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee1c0c6780 con 0x7fee1c154100 2023-11-28T21:49:17.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.944+0000 7fee18ff9640 1 -- 172.21.15.12:0/698956766 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5721 (secure 0 0 0) 0x7fee14054670 con 0x7fee1c154100 2023-11-28T21:49:17.949 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:17.949 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:14.692539+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:clientreplay","state_seq":32,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:17.950 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 59 2023-11-28T21:49:17.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 -- 172.21.15.12:0/698956766 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee08066470 msgr2=0x7fee08068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:17.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee08066470 0x7fee08068930 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fee0c007a60 tx=0x7fee0c008040 comp rx=0 tx=0).stop 2023-11-28T21:49:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 -- 172.21.15.12:0/698956766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c154100 msgr2=0x7fee1c0c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c154100 0x7fee1c0c6980 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c730 tx=0x7fee1400cbf0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 -- 172.21.15.12:0/698956766 shutdown_connections 2023-11-28T21:49:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fee08066470 0x7fee08068930 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fee1c154100 0x7fee1c0c6980 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee1c0ca3f0 0x7fee1c145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 --2- 172.21.15.12:0/698956766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0c9fd0 0x7fee1c0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 -- 172.21.15.12:0/698956766 >> 172.21.15.12:0/698956766 conn(0x7fee1c06c690 msgr2=0x7fee1c066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 -- 172.21.15.12:0/698956766 shutdown_connections 2023-11-28T21:49:17.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:17.948+0000 7fee22617640 1 -- 172.21.15.12:0/698956766 wait complete. 2023-11-28T21:49:17.971 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 59, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 59, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6177": { "addr": "172.21.15.12:6838/1551825785", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 1551825785, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 1551825785, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6177, "incarnation": 56, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:clientreplay", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:49:14.692539+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 1, 2 ], "tableserver": 0, "up": { "mds_0": 6177 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:49:19.972 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:20.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.124+0000 7f5b7617d640 1 Processor -- start 2023-11-28T21:49:20.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.124+0000 7f5b7617d640 1 -- start start 2023-11-28T21:49:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7617d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5b700c8a90 0x7f5b700c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7617d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5b70159d20 0x7f5b7015a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7617d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b700c1660 con 0x7f5b700c6580 2023-11-28T21:49:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7617d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b700c93b0 con 0x7f5b700c8a90 2023-11-28T21:49:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7617d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b700c82b0 con 0x7f5b70159d20 2023-11-28T21:49:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7517b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5b700c8a90 0x7f5b700c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7497a640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5b700c6580 0x7f5b700c7a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53948/0 (socket says 172.21.15.12:53948) 2023-11-28T21:49:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7497a640 1 -- 172.21.15.12:0/3061246360 learned_addr learned my addr 172.21.15.12:0/3061246360 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:49:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7597c640 1 --2- 172.21.15.12:0/3061246360 >> v2:172.21.15.144:3300/0 conn(0x7f5b70159d20 0x7f5b7015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b67fff640 1 -- 172.21.15.12:0/3061246360 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960822498 0 0) 0x7f5b700c1660 con 0x7f5b700c6580 2023-11-28T21:49:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b67fff640 1 -- 172.21.15.12:0/3061246360 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b5c003530 con 0x7f5b700c6580 2023-11-28T21:49:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7517b640 1 -- 172.21.15.12:0/3061246360 >> v2:172.21.15.144:3300/0 conn(0x7f5b70159d20 msgr2=0x7f5b7015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7517b640 1 --2- 172.21.15.12:0/3061246360 >> v2:172.21.15.144:3300/0 conn(0x7f5b70159d20 0x7f5b7015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7517b640 1 -- 172.21.15.12:0/3061246360 >> v1:172.21.15.12:6789/0 conn(0x7f5b700c6580 legacy=0x7f5b700c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7517b640 1 -- 172.21.15.12:0/3061246360 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7015a6b0 con 0x7f5b700c8a90 2023-11-28T21:49:20.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.128+0000 7f5b7597c640 1 --2- 172.21.15.12:0/3061246360 >> v2:172.21.15.144:3300/0 conn(0x7f5b70159d20 0x7f5b7015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:49:20.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7517b640 1 --2- 172.21.15.12:0/3061246360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 0x7f5b700c8e70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b7015e190 tx=0x7f5b6c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea7d889e056fcd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:20.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b67fff640 1 -- 172.21.15.12:0/3061246360 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0ab600 con 0x7f5b700c8a90 2023-11-28T21:49:20.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b67fff640 1 -- 172.21.15.12:0/3061246360 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c0a7050 con 0x7f5b700c8a90 2023-11-28T21:49:20.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b67fff640 1 -- 172.21.15.12:0/3061246360 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0ab980 con 0x7f5b700c8a90 2023-11-28T21:49:20.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7617d640 1 -- 172.21.15.12:0/3061246360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 msgr2=0x7f5b700c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:20.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7617d640 1 --2- 172.21.15.12:0/3061246360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 0x7f5b700c8e70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b7015e190 tx=0x7f5b6c01c460 comp rx=0 tx=0).stop 2023-11-28T21:49:20.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7617d640 1 -- 172.21.15.12:0/3061246360 shutdown_connections 2023-11-28T21:49:20.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7617d640 1 --2- 172.21.15.12:0/3061246360 >> v2:172.21.15.144:3300/0 conn(0x7f5b70159d20 0x7f5b7015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:20.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7617d640 1 --2- 172.21.15.12:0/3061246360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 0x7f5b700c8e70 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:20.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7617d640 1 -- 172.21.15.12:0/3061246360 >> 172.21.15.12:0/3061246360 conn(0x7f5b7006e220 msgr2=0x7f5b7006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:20.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7617d640 1 -- 172.21.15.12:0/3061246360 shutdown_connections 2023-11-28T21:49:20.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7617d640 1 -- 172.21.15.12:0/3061246360 wait complete. 2023-11-28T21:49:20.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7617d640 1 Processor -- start 2023-11-28T21:49:20.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.132+0000 7f5b7617d640 1 -- start start 2023-11-28T21:49:20.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7617d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b700c6580 0x7f5b70070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:20.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7617d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 0x7f5b701f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:20.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7617d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b70159d20 0x7f5b701f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7617d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b701f61e0 con 0x7f5b700c8a90 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7617d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b701f66d0 con 0x7f5b700c6580 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7617d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b70152810 con 0x7f5b70159d20 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7497a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 0x7f5b701f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7497a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 0x7f5b701f2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56038/0 (socket says 172.21.15.12:56038) 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7497a640 1 -- 172.21.15.12:0/1931353834 learned_addr learned my addr 172.21.15.12:0/1931353834 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7597c640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b70159d20 0x7f5b701f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7517b640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b700c6580 0x7f5b70070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7497a640 1 -- 172.21.15.12:0/1931353834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b70159d20 msgr2=0x7f5b701f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7497a640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b70159d20 0x7f5b701f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7497a640 1 -- 172.21.15.12:0/1931353834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b700c6580 msgr2=0x7f5b70070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7497a640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b700c6580 0x7f5b70070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:20.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7497a640 1 -- 172.21.15.12:0/1931353834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c009420 con 0x7f5b700c8a90 2023-11-28T21:49:20.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7497a640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 0x7f5b701f2890 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000c4d0 tx=0x7f5b6000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:20.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b65ffb640 1 -- 172.21.15.12:0/1931353834 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60005cf0 con 0x7f5b700c8a90 2023-11-28T21:49:20.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b65ffb640 1 -- 172.21.15.12:0/1931353834 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b60005e50 con 0x7f5b700c8a90 2023-11-28T21:49:20.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b65ffb640 1 -- 172.21.15.12:0/1931353834 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60007270 con 0x7f5b700c8a90 2023-11-28T21:49:20.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7617d640 1 -- 172.21.15.12:0/1931353834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b701f63f0 con 0x7f5b700c8a90 2023-11-28T21:49:20.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7617d640 1 -- 172.21.15.12:0/1931353834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b70152d90 con 0x7f5b700c8a90 2023-11-28T21:49:20.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b7617d640 1 -- 172.21.15.12:0/1931353834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b70074f60 con 0x7f5b700c8a90 2023-11-28T21:49:20.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.136+0000 7f5b65ffb640 1 -- 172.21.15.12:0/1931353834 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5b600073d0 con 0x7f5b700c8a90 2023-11-28T21:49:20.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.140+0000 7f5b65ffb640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b540664a0 0x7f5b54068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:20.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.140+0000 7f5b65ffb640 1 -- 172.21.15.12:0/1931353834 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f5b600839d0 con 0x7f5b700c8a90 2023-11-28T21:49:20.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.140+0000 7f5b7517b640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b540664a0 0x7f5b54068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:20.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.140+0000 7f5b7517b640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b540664a0 0x7f5b54068960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0ad000 tx=0x7f5b6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:20.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.144+0000 7f5b65ffb640 1 -- 172.21.15.12:0/1931353834 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5b600508d0 con 0x7f5b700c8a90 2023-11-28T21:49:20.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.336+0000 7f5b7617d640 1 -- 172.21.15.12:0/1931353834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b70153dd0 con 0x7f5b700c8a90 2023-11-28T21:49:20.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b65ffb640 1 -- 172.21.15.12:0/1931353834 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5721 (secure 0 0 0) 0x7f5b60017090 con 0x7f5b700c8a90 2023-11-28T21:49:20.342 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:20.342 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:14.692539+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:clientreplay","state_seq":32,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:20.343 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 59 2023-11-28T21:49:20.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 -- 172.21.15.12:0/1931353834 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b540664a0 msgr2=0x7f5b54068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:20.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b540664a0 0x7f5b54068960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0ad000 tx=0x7f5b6c008040 comp rx=0 tx=0).stop 2023-11-28T21:49:20.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 -- 172.21.15.12:0/1931353834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 msgr2=0x7f5b701f2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:20.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 0x7f5b701f2890 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000c4d0 tx=0x7f5b6000c990 comp rx=0 tx=0).stop 2023-11-28T21:49:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 -- 172.21.15.12:0/1931353834 shutdown_connections 2023-11-28T21:49:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b540664a0 0x7f5b54068960 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b70159d20 0x7f5b701f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:20.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b700c8a90 0x7f5b701f2890 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:20.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 --2- 172.21.15.12:0/1931353834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b700c6580 0x7f5b70070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:20.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 -- 172.21.15.12:0/1931353834 >> 172.21.15.12:0/1931353834 conn(0x7f5b7006e220 msgr2=0x7f5b7006e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:20.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 -- 172.21.15.12:0/1931353834 shutdown_connections 2023-11-28T21:49:20.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:20.340+0000 7f5b7617d640 1 -- 172.21.15.12:0/1931353834 wait complete. 2023-11-28T21:49:20.363 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:49:22.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd8117f8640 1 Processor -- start 2023-11-28T21:49:22.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd8117f8640 1 -- start start 2023-11-28T21:49:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd8117f8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd80c158b20 0x7fd80c156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd8117f8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80c067410 con 0x7fd80c06e300 2023-11-28T21:49:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd8117f8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80c06e090 con 0x7fd80c150c20 2023-11-28T21:49:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd8117f8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80c157f30 con 0x7fd80c158b20 2023-11-28T21:49:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd803fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd80c06e300 0x7fd80c158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:53958/0 (socket says 172.21.15.12:53958) 2023-11-28T21:49:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd803fff640 1 -- 172.21.15.12:0/306383497 learned_addr learned my addr 172.21.15.12:0/306383497 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:49:22.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd80b7fe640 1 --2- 172.21.15.12:0/306383497 >> v2:172.21.15.12:3300/0 conn(0x7fd80c158b20 0x7fd80c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:22.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd80b7fe640 1 -- 172.21.15.12:0/306383497 >> v1:172.21.15.144:6789/0 conn(0x7fd80c150c20 legacy=0x7fd80c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:22.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd80b7fe640 1 -- 172.21.15.12:0/306383497 >> v1:172.21.15.12:6789/0 conn(0x7fd80c06e300 legacy=0x7fd80c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:22.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd80b7fe640 1 -- 172.21.15.12:0/306383497 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c151000 con 0x7fd80c158b20 2023-11-28T21:49:22.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.516+0000 7fd80a7fc640 1 -- 172.21.15.12:0/306383497 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675150514 0 0) 0x7fd80c067410 con 0x7fd80c06e300 2023-11-28T21:49:22.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd80b7fe640 1 --2- 172.21.15.12:0/306383497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c158b20 0x7fd80c156910 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd804009d90 tx=0x7fd80401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a007fd9c59ad91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd80a7fc640 1 -- 172.21.15.12:0/306383497 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8040a5920 con 0x7fd80c158b20 2023-11-28T21:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd80a7fc640 1 -- 172.21.15.12:0/306383497 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8040a5a80 con 0x7fd80c158b20 2023-11-28T21:49:22.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd80a7fc640 1 -- 172.21.15.12:0/306383497 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8040a5e10 con 0x7fd80c158b20 2023-11-28T21:49:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd8117f8640 1 -- 172.21.15.12:0/306383497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c158b20 msgr2=0x7fd80c156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd8117f8640 1 --2- 172.21.15.12:0/306383497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c158b20 0x7fd80c156910 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd804009d90 tx=0x7fd80401c950 comp rx=0 tx=0).stop 2023-11-28T21:49:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd8117f8640 1 -- 172.21.15.12:0/306383497 shutdown_connections 2023-11-28T21:49:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd8117f8640 1 --2- 172.21.15.12:0/306383497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c158b20 0x7fd80c156910 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd8117f8640 1 -- 172.21.15.12:0/306383497 >> 172.21.15.12:0/306383497 conn(0x7fd80c066360 msgr2=0x7fd80c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:22.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd8117f8640 1 -- 172.21.15.12:0/306383497 shutdown_connections 2023-11-28T21:49:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.520+0000 7fd8117f8640 1 -- 172.21.15.12:0/306383497 wait complete. 2023-11-28T21:49:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8117f8640 1 Processor -- start 2023-11-28T21:49:22.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8117f8640 1 -- start start 2023-11-28T21:49:22.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8117f8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c06e300 0x7fd80c1ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:22.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8117f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd80c150c20 0x7fd80c06af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:22.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8117f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd80c158b20 0x7fd80c151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:22.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8117f8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c1513e0 con 0x7fd80c06e300 2023-11-28T21:49:22.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8117f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c152b60 con 0x7fd80c158b20 2023-11-28T21:49:22.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8117f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c152e20 con 0x7fd80c150c20 2023-11-28T21:49:22.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c06e300 0x7fd80c1ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:22.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c06e300 0x7fd80c1ef3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56050/0 (socket says 172.21.15.12:56050) 2023-11-28T21:49:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80affd640 1 -- 172.21.15.12:0/1501327357 learned_addr learned my addr 172.21.15.12:0/1501327357 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80b7fe640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd80c158b20 0x7fd80c151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd803fff640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd80c150c20 0x7fd80c06af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80affd640 1 -- 172.21.15.12:0/1501327357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd80c150c20 msgr2=0x7fd80c06af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80affd640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd80c150c20 0x7fd80c06af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:22.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80affd640 1 -- 172.21.15.12:0/1501327357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd80c158b20 msgr2=0x7fd80c151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80affd640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd80c158b20 0x7fd80c151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80affd640 1 -- 172.21.15.12:0/1501327357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd804009a70 con 0x7fd80c06e300 2023-11-28T21:49:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80b7fe640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd80c158b20 0x7fd80c151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:49:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd80affd640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c06e300 0x7fd80c1ef3a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c500 tx=0x7fd7fc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8037fe640 1 -- 172.21.15.12:0/1501327357 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc005220 con 0x7fd80c06e300 2023-11-28T21:49:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8037fe640 1 -- 172.21.15.12:0/1501327357 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc005380 con 0x7fd80c06e300 2023-11-28T21:49:22.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8037fe640 1 -- 172.21.15.12:0/1501327357 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc007270 con 0x7fd80c06e300 2023-11-28T21:49:22.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8117f8640 1 -- 172.21.15.12:0/1501327357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80c153100 con 0x7fd80c06e300 2023-11-28T21:49:22.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.524+0000 7fd8117f8640 1 -- 172.21.15.12:0/1501327357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c1536c0 con 0x7fd80c06e300 2023-11-28T21:49:22.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.528+0000 7fd8117f8640 1 -- 172.21.15.12:0/1501327357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80c151520 con 0x7fd80c06e300 2023-11-28T21:49:22.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.528+0000 7fd8037fe640 1 -- 172.21.15.12:0/1501327357 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd7fc0073d0 con 0x7fd80c06e300 2023-11-28T21:49:22.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.528+0000 7fd8037fe640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd7ec066470 0x7fd7ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:22.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.528+0000 7fd8037fe640 1 -- 172.21.15.12:0/1501327357 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fd7fc083e20 con 0x7fd80c06e300 2023-11-28T21:49:22.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.528+0000 7fd803fff640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd7ec066470 0x7fd7ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:22.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.528+0000 7fd803fff640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd7ec066470 0x7fd7ec068930 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd80c152650 tx=0x7fd7f4007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:22.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.532+0000 7fd8037fe640 1 -- 172.21.15.12:0/1501327357 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd80c151520 con 0x7fd80c06e300 2023-11-28T21:49:22.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.724+0000 7fd8117f8640 1 -- 172.21.15.12:0/1501327357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd80c151520 con 0x7fd80c06e300 2023-11-28T21:49:22.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8037fe640 1 -- 172.21.15.12:0/1501327357 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5715 (secure 0 0 0) 0x7fd80c151520 con 0x7fd80c06e300 2023-11-28T21:49:22.730 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:22.730 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:20.980151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:22.731 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 60 2023-11-28T21:49:22.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 -- 172.21.15.12:0/1501327357 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd7ec066470 msgr2=0x7fd7ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:22.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd7ec066470 0x7fd7ec068930 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd80c152650 tx=0x7fd7f4007d10 comp rx=0 tx=0).stop 2023-11-28T21:49:22.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 -- 172.21.15.12:0/1501327357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c06e300 msgr2=0x7fd80c1ef3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:22.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c06e300 0x7fd80c1ef3a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c500 tx=0x7fd7fc00c9c0 comp rx=0 tx=0).stop 2023-11-28T21:49:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 -- 172.21.15.12:0/1501327357 shutdown_connections 2023-11-28T21:49:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd7ec066470 0x7fd7ec068930 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd80c158b20 0x7fd80c151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd80c150c20 0x7fd80c06af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 --2- 172.21.15.12:0/1501327357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd80c06e300 0x7fd80c1ef3a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 -- 172.21.15.12:0/1501327357 >> 172.21.15.12:0/1501327357 conn(0x7fd80c066360 msgr2=0x7fd80c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 -- 172.21.15.12:0/1501327357 shutdown_connections 2023-11-28T21:49:22.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:22.728+0000 7fd8117f8640 1 -- 172.21.15.12:0/1501327357 wait complete. 2023-11-28T21:49:22.751 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=6024 2023-11-28T21:49:22.751 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.e 2023-11-28T21:49:24.752 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-11-28T21:49:24.753 INFO:tasks.ceph.mds.e:Restarting daemon 2023-11-28T21:49:24.753 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-11-28T21:49:24.757 INFO:tasks.ceph.mds.e:Started 2023-11-28T21:49:24.757 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:24.857 INFO:tasks.ceph.mds.e.smithi012.stdout:starting mds.e at 2023-11-28T21:49:24.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.912+0000 7f8bf6f80640 1 Processor -- start 2023-11-28T21:49:24.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.912+0000 7f8bf6f80640 1 -- start start 2023-11-28T21:49:24.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.912+0000 7f8bf6f80640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf006e300 0x7f8bf00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:24.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.912+0000 7f8bf6f80640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf0157b70 con 0x7f8bf0151af0 2023-11-28T21:49:24.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.912+0000 7f8bf6f80640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf0157e50 con 0x7f8bf0151ed0 2023-11-28T21:49:24.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.912+0000 7f8bf6f80640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00707f0 con 0x7f8bf006e300 2023-11-28T21:49:24.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.912+0000 7f8bf4d1c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8bf0151af0 0x7f8bf00c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56236/0 (socket says 172.21.15.12:56236) 2023-11-28T21:49:24.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.912+0000 7f8bf4d1c640 1 -- 172.21.15.12:0/1481605828 learned_addr learned my addr 172.21.15.12:0/1481605828 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:49:24.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.912+0000 7f8bf551d640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf006e300 0x7f8bf00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:24.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132832857 0 0) 0x7f8bf0157b70 con 0x7f8bf0151af0 2023-11-28T21:49:24.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8be0003530 con 0x7f8bf0151af0 2023-11-28T21:49:24.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690161457 0 0) 0x7f8bf0157e50 con 0x7f8bf0151ed0 2023-11-28T21:49:24.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bf0157b70 con 0x7f8bf0151ed0 2023-11-28T21:49:24.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3042090110 0 0) 0x7f8be0003530 con 0x7f8bf0151af0 2023-11-28T21:49:24.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf006e300 msgr2=0x7f8bf00c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:24.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf006e300 0x7f8bf00c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:24.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 >> v1:172.21.15.144:6789/0 conn(0x7f8bf0151ed0 legacy=0x7f8bf0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:24.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf0158130 con 0x7f8bf0151af0 2023-11-28T21:49:24.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f8be4001680 con 0x7f8bf0151af0 2023-11-28T21:49:24.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 >> v1:172.21.15.12:6789/0 conn(0x7f8bf0151af0 legacy=0x7f8bf00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:24.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf006e300 0x7f8be0003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:24.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be0004160 0x7f8be00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be0004d10 0x7f8be0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00707f0 con 0x7f8be0004160 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be000ff30 con 0x7f8be0004d10 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be00100f0 con 0x7f8bf006e300 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8beffff640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be0004160 0x7f8be00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bf4d1c640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf006e300 0x7f8be0003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8bf551d640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be0004d10 0x7f8be0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8beffff640 1 -- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf006e300 msgr2=0x7f8be0003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8beffff640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf006e300 0x7f8be0003bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8beffff640 1 -- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be0004d10 msgr2=0x7f8be0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:24.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8beffff640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be0004d10 0x7f8be0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:24.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.916+0000 7f8beffff640 1 -- 172.21.15.12:0/1481605828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be0003530 con 0x7f8be0004160 2023-11-28T21:49:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8beffff640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be0004160 0x7f8be00047d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00c5910 tx=0x7f8bdc01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c865cfa986b5218b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:24.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0052d0 con 0x7f8be0004160 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bdc005450 con 0x7f8be0004160 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bef7fe640 1 -- 172.21.15.12:0/1481605828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0a88d0 con 0x7f8be0004160 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1481605828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be0004160 msgr2=0x7f8be00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 --2- 172.21.15.12:0/1481605828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be0004160 0x7f8be00047d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00c5910 tx=0x7f8bdc01ec70 comp rx=0 tx=0).stop 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf551d640 1 -- 172.21.15.12:0/1481605828 reap_dead start 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1481605828 shutdown_connections 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1481605828 >> 172.21.15.12:0/1481605828 conn(0x7f8bf0066360 msgr2=0x7f8bf0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1481605828 shutdown_connections 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1481605828 wait complete. 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 Processor -- start 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 -- start start 2023-11-28T21:49:24.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8be0004160 0x7f8bf013d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bf01ebd40 0x7f8bf0155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.920+0000 7f8bf6f80640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf0143d20 0x7f8bf00c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bf6f80640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf0145940 con 0x7f8bf01ebd40 2023-11-28T21:49:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bf6f80640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf01446d0 con 0x7f8bf0143d20 2023-11-28T21:49:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bf6f80640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf01eb050 con 0x7f8be0004160 2023-11-28T21:49:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8beffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bf01ebd40 0x7f8bf0155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8beffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bf01ebd40 0x7f8bf0155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33750/0 (socket says 172.21.15.12:33750) 2023-11-28T21:49:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8beffff640 1 -- 172.21.15.12:0/1022042793 learned_addr learned my addr 172.21.15.12:0/1022042793 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:24.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bf4d1c640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8be0004160 0x7f8bf013d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bf551d640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf0143d20 0x7f8bf00c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8beffff640 1 -- 172.21.15.12:0/1022042793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8be0004160 msgr2=0x7f8bf013d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8beffff640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8be0004160 0x7f8bf013d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8beffff640 1 -- 172.21.15.12:0/1022042793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf0143d20 msgr2=0x7f8bf00c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8beffff640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf0143d20 0x7f8bf00c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8beffff640 1 -- 172.21.15.12:0/1022042793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf013d800 con 0x7f8bf01ebd40 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bf551d640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf0143d20 0x7f8bf00c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8beffff640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bf01ebd40 0x7f8bf0155920 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0068c10 tx=0x7f8be0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bed7fa640 1 -- 172.21.15.12:0/1022042793 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0a8cf0 con 0x7f8bf01ebd40 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bed7fa640 1 -- 172.21.15.12:0/1022042793 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bdc0a8e50 con 0x7f8bf01ebd40 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bed7fa640 1 -- 172.21.15.12:0/1022042793 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0052d0 con 0x7f8bf01ebd40 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1022042793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf01eb8b0 con 0x7f8bf01ebd40 2023-11-28T21:49:24.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1022042793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf01eb260 con 0x7f8bf01ebd40 2023-11-28T21:49:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1022042793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf0074f60 con 0x7f8bf01ebd40 2023-11-28T21:49:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.924+0000 7f8bed7fa640 1 -- 172.21.15.12:0/1022042793 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8bdc0067e0 con 0x7f8bf01ebd40 2023-11-28T21:49:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.928+0000 7f8bed7fa640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8bd40664a0 0x7f8bd4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.928+0000 7f8bed7fa640 1 -- 172.21.15.12:0/1022042793 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f8bdc1211c0 con 0x7f8bf01ebd40 2023-11-28T21:49:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.928+0000 7f8bf4d1c640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8bd40664a0 0x7f8bd4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:24.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.928+0000 7f8bf4d1c640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8bd40664a0 0x7f8bd4068960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8be4000c10 tx=0x7f8be4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:24.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:24.932+0000 7f8bed7fa640 1 -- 172.21.15.12:0/1022042793 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8bdc0ee160 con 0x7f8bf01ebd40 2023-11-28T21:49:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1022042793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bf0074f60 con 0x7f8bf01ebd40 2023-11-28T21:49:25.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bed7fa640 1 -- 172.21.15.12:0/1022042793 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5715 (secure 0 0 0) 0x7f8bdc0f1780 con 0x7f8bf01ebd40 2023-11-28T21:49:25.123 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:25.123 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:20.980151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:25.123 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 60 2023-11-28T21:49:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1022042793 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8bd40664a0 msgr2=0x7f8bd4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8bd40664a0 0x7f8bd4068960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8be4000c10 tx=0x7f8be4009040 comp rx=0 tx=0).stop 2023-11-28T21:49:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1022042793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bf01ebd40 msgr2=0x7f8bf0155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bf01ebd40 0x7f8bf0155920 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0068c10 tx=0x7f8be0005800 comp rx=0 tx=0).stop 2023-11-28T21:49:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1022042793 shutdown_connections 2023-11-28T21:49:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8bd40664a0 0x7f8bd4068960 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:25.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf0143d20 0x7f8bf00c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8bf01ebd40 0x7f8bf0155920 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 --2- 172.21.15.12:0/1022042793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8be0004160 0x7f8bf013d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1022042793 >> 172.21.15.12:0/1022042793 conn(0x7f8bf0066360 msgr2=0x7f8bf014f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1022042793 shutdown_connections 2023-11-28T21:49:25.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:25.120+0000 7f8bf6f80640 1 -- 172.21.15.12:0/1022042793 wait complete. 2023-11-28T21:49:25.144 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-11-28T21:49:27.145 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:49:27.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.288+0000 7f707e079640 1 Processor -- start 2023-11-28T21:49:27.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.288+0000 7f707e079640 1 -- start start 2023-11-28T21:49:27.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f707e079640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7078150c20 0x7f7078156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:27.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f707e079640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7078067410 con 0x7f707806e300 2023-11-28T21:49:27.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f707e079640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707806e090 con 0x7f707806cf40 2023-11-28T21:49:27.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f707e079640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7078157f30 con 0x7f7078150c20 2023-11-28T21:49:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7076ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f707806e300 0x7f70780c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56250/0 (socket says 172.21.15.12:56250) 2023-11-28T21:49:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7076ffd640 1 -- 172.21.15.12:0/3288746718 learned_addr learned my addr 172.21.15.12:0/3288746718 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:49:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70777fe640 1 --2- 172.21.15.12:0/3288746718 >> v2:172.21.15.144:3300/0 conn(0x7f7078150c20 0x7f7078156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083933826 0 0) 0x7f7078067410 con 0x7f707806e300 2023-11-28T21:49:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7060003530 con 0x7f707806e300 2023-11-28T21:49:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 263936757 0 0) 0x7f7060003530 con 0x7f707806e300 2023-11-28T21:49:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 >> v2:172.21.15.144:3300/0 conn(0x7f7078150c20 msgr2=0x7f7078156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 --2- 172.21.15.12:0/3288746718 >> v2:172.21.15.144:3300/0 conn(0x7f7078150c20 0x7f7078156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 >> v1:172.21.15.144:6789/0 conn(0x7f707806cf40 legacy=0x7f7078158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:27.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078151000 con 0x7f707806e300 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f706c001680 con 0x7f707806e300 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 >> v1:172.21.15.12:6789/0 conn(0x7f707806e300 legacy=0x7f70780c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060003a40 0x7f7060003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70600047d0 0x7f7060004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70600051e0 0x7f706000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7078157f30 con 0x7f70600047d0 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706000c110 con 0x7f7060003a40 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706000c2d0 con 0x7f70600051e0 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7076ffd640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70600047d0 0x7f7060004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f70777fe640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060003a40 0x7f7060003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7077fff640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70600051e0 0x7f706000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7076ffd640 1 -- 172.21.15.12:0/3288746718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70600051e0 msgr2=0x7f706000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7076ffd640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70600051e0 0x7f706000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7076ffd640 1 -- 172.21.15.12:0/3288746718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060003a40 msgr2=0x7f7060003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:49:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7076ffd640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7060003a40 0x7f7060003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7076ffd640 1 -- 172.21.15.12:0/3288746718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7060003530 con 0x7f70600047d0 2023-11-28T21:49:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7077fff640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70600051e0 0x7f706000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:49:27.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.292+0000 7f7077fff640 1 -- 172.21.15.12:0/3288746718 reap_dead start 2023-11-28T21:49:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f7076ffd640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70600047d0 0x7f7060004c50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f707806e750 tx=0x7f706c01f290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b0112bc7b0f13e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c0058e0 con 0x7f70600047d0 2023-11-28T21:49:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c005a40 con 0x7f70600047d0 2023-11-28T21:49:27.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f70767fc640 1 -- 172.21.15.12:0/3288746718 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c005e70 con 0x7f70600047d0 2023-11-28T21:49:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f707e079640 1 -- 172.21.15.12:0/3288746718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70600047d0 msgr2=0x7f7060004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f707e079640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70600047d0 0x7f7060004c50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f707806e750 tx=0x7f706c01f290 comp rx=0 tx=0).stop 2023-11-28T21:49:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f707e079640 1 -- 172.21.15.12:0/3288746718 shutdown_connections 2023-11-28T21:49:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f707e079640 1 --2- 172.21.15.12:0/3288746718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f70600047d0 0x7f7060004c50 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f707e079640 1 -- 172.21.15.12:0/3288746718 >> 172.21.15.12:0/3288746718 conn(0x7f7078066360 msgr2=0x7f7078066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f707e079640 1 -- 172.21.15.12:0/3288746718 shutdown_connections 2023-11-28T21:49:27.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.296+0000 7f707e079640 1 -- 172.21.15.12:0/3288746718 wait complete. 2023-11-28T21:49:27.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f707e079640 1 Processor -- start 2023-11-28T21:49:27.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f707e079640 1 -- start start 2023-11-28T21:49:27.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f707e079640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70600047d0 0x7f70780ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:27.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f707e079640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70781edec0 0x7f707814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:27.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f707e079640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f707814c030 0x7f707814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:27.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f707e079640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70780cb0a0 con 0x7f707814c030 2023-11-28T21:49:27.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f707e079640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707814c9f0 con 0x7f70600047d0 2023-11-28T21:49:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f707e079640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70780c90c0 con 0x7f70781edec0 2023-11-28T21:49:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f7077fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f707814c030 0x7f707814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f7077fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f707814c030 0x7f707814c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33768/0 (socket says 172.21.15.12:33768) 2023-11-28T21:49:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f7077fff640 1 -- 172.21.15.12:0/1919364142 learned_addr learned my addr 172.21.15.12:0/1919364142 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:49:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f7076ffd640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70781edec0 0x7f707814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:27.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f70777fe640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70600047d0 0x7f70780ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f7077fff640 1 -- 172.21.15.12:0/1919364142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70781edec0 msgr2=0x7f707814d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f7077fff640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70781edec0 0x7f707814d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f7077fff640 1 -- 172.21.15.12:0/1919364142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70600047d0 msgr2=0x7f70780ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f7077fff640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70600047d0 0x7f70780ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f7077fff640 1 -- 172.21.15.12:0/1919364142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707814bba0 con 0x7f707814c030 2023-11-28T21:49:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f7077fff640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f707814c030 0x7f707814c4b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f707806f890 tx=0x7f707000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f705ffff640 1 -- 172.21.15.12:0/1919364142 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070017070 con 0x7f707814c030 2023-11-28T21:49:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f705ffff640 1 -- 172.21.15.12:0/1919364142 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7070003b90 con 0x7f707814c030 2023-11-28T21:49:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f705ffff640 1 -- 172.21.15.12:0/1919364142 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070003d40 con 0x7f707814c030 2023-11-28T21:49:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f707e079640 1 -- 172.21.15.12:0/1919364142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70780c92f0 con 0x7f707814c030 2023-11-28T21:49:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.300+0000 7f707e079640 1 -- 172.21.15.12:0/1919364142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70780c98b0 con 0x7f707814c030 2023-11-28T21:49:27.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.304+0000 7f707e079640 1 -- 172.21.15.12:0/1919364142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707814b970 con 0x7f707814c030 2023-11-28T21:49:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.304+0000 7f705ffff640 1 -- 172.21.15.12:0/1919364142 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f70700050d0 con 0x7f707814c030 2023-11-28T21:49:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.304+0000 7f705ffff640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7064066470 0x7f7064068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:49:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.304+0000 7f705ffff640 1 -- 172.21.15.12:0/1919364142 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f70700837c0 con 0x7f707814c030 2023-11-28T21:49:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.304+0000 7f70777fe640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7064066470 0x7f7064068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:49:27.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.304+0000 7f70777fe640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7064066470 0x7f7064068930 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7068009c60 tx=0x7f7068009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:49:27.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.308+0000 7f705ffff640 1 -- 172.21.15.12:0/1919364142 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7070050610 con 0x7f707814c030 2023-11-28T21:49:27.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 -- 172.21.15.12:0/1919364142 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7078074f60 con 0x7f707814c030 2023-11-28T21:49:27.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f705ffff640 1 -- 172.21.15.12:0/1919364142 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f7070053c30 con 0x7f707814c030 2023-11-28T21:49:27.511 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:49:27.511 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6330,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:20.980151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:49:27.511 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 61 2023-11-28T21:49:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 -- 172.21.15.12:0/1919364142 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7064066470 msgr2=0x7f7064068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7064066470 0x7f7064068930 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7068009c60 tx=0x7f7068009190 comp rx=0 tx=0).stop 2023-11-28T21:49:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 -- 172.21.15.12:0/1919364142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f707814c030 msgr2=0x7f707814c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:49:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f707814c030 0x7f707814c4b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f707806f890 tx=0x7f707000ead0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 -- 172.21.15.12:0/1919364142 shutdown_connections 2023-11-28T21:49:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7064066470 0x7f7064068930 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f707814c030 0x7f707814c4b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70781edec0 0x7f707814d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 --2- 172.21.15.12:0/1919364142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70600047d0 0x7f70780ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:49:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 -- 172.21.15.12:0/1919364142 >> 172.21.15.12:0/1919364142 conn(0x7f7078066360 msgr2=0x7f7078155e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:49:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 -- 172.21.15.12:0/1919364142 shutdown_connections 2023-11-28T21:49:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:49:27.508+0000 7f707e079640 1 -- 172.21.15.12:0/1919364142 wait complete. 2023-11-28T21:49:27.538 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-11-28T21:49:27.538 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 49 secs before thrashing 2023-11-28T21:49:29.562 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-11-28T21:49:29.560+0000 7faa6b3e1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T21:49:45.885 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:49:45.891 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:50:15.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:50:15.960 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:50:16.540 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:16.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f0ce17640 1 Processor -- start 2023-11-28T21:50:16.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f0ce17640 1 -- start start 2023-11-28T21:50:16.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f0ce17640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1f08151ed0 0x7f1f080c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:16.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f0ce17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f0806e300 0x7f1f08072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:16.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f0ce17640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f08157b70 con 0x7f1f08151af0 2023-11-28T21:50:16.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f0ce17640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f080707f0 con 0x7f1f08151ed0 2023-11-28T21:50:16.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f0ce17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080c7b90 con 0x7f1f0806e300 2023-11-28T21:50:16.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f05d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1f08151ed0 0x7f1f080c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:16.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f05d9b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1f08151ed0 0x7f1f080c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45764/0 (socket says 172.21.15.12:45764) 2023-11-28T21:50:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f05d9b640 1 -- 172.21.15.12:0/1084961329 learned_addr learned my addr 172.21.15.12:0/1084961329 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f06d9d640 1 --2- 172.21.15.12:0/1084961329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f0806e300 0x7f1f08072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f05d9b640 1 -- 172.21.15.12:0/1084961329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f0806e300 msgr2=0x7f1f08072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f05d9b640 1 --2- 172.21.15.12:0/1084961329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f0806e300 0x7f1f08072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f05d9b640 1 -- 172.21.15.12:0/1084961329 >> v1:172.21.15.144:6789/0 conn(0x7f1f08151af0 legacy=0x7f1f080c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:16.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.696+0000 7f1f05d9b640 1 -- 172.21.15.12:0/1084961329 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f08157e50 con 0x7f1f08151ed0 2023-11-28T21:50:16.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f05d9b640 1 --2- 172.21.15.12:0/1084961329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f08151ed0 0x7f1f080c7650 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009870 tx=0x7f1efc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fd30a861180a74 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:16.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0559a640 1 -- 172.21.15.12:0/1084961329 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0ab600 con 0x7f1f08151ed0 2023-11-28T21:50:16.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0559a640 1 -- 172.21.15.12:0/1084961329 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc0a7050 con 0x7f1f08151ed0 2023-11-28T21:50:16.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0559a640 1 -- 172.21.15.12:0/1084961329 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0ab980 con 0x7f1f08151ed0 2023-11-28T21:50:16.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0ce17640 1 -- 172.21.15.12:0/1084961329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f08151ed0 msgr2=0x7f1f080c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:16.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0ce17640 1 --2- 172.21.15.12:0/1084961329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f08151ed0 0x7f1f080c7650 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009870 tx=0x7f1efc01c540 comp rx=0 tx=0).stop 2023-11-28T21:50:16.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0ce17640 1 -- 172.21.15.12:0/1084961329 shutdown_connections 2023-11-28T21:50:16.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0ce17640 1 --2- 172.21.15.12:0/1084961329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f0806e300 0x7f1f08072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:16.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0ce17640 1 --2- 172.21.15.12:0/1084961329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f08151ed0 0x7f1f080c7650 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:16.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0ce17640 1 -- 172.21.15.12:0/1084961329 >> 172.21.15.12:0/1084961329 conn(0x7f1f08066360 msgr2=0x7f1f08066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:16.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0ce17640 1 -- 172.21.15.12:0/1084961329 shutdown_connections 2023-11-28T21:50:16.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.700+0000 7f1f0ce17640 1 -- 172.21.15.12:0/1084961329 wait complete. 2023-11-28T21:50:16.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.704+0000 7f1f0ce17640 1 Processor -- start 2023-11-28T21:50:16.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.704+0000 7f1f0ce17640 1 -- start start 2023-11-28T21:50:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.704+0000 7f1f0ce17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f0806e300 0x7f1f081f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.704+0000 7f1f0ce17640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f08151af0 0x7f1f081f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.704+0000 7f1f0ce17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0806ad20 0x7f1f081404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.704+0000 7f1f0ce17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f08145380 con 0x7f1f0806ad20 2023-11-28T21:50:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.704+0000 7f1f0ce17640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081f32e0 con 0x7f1f08151af0 2023-11-28T21:50:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.704+0000 7f1f0ce17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081f6a00 con 0x7f1f0806e300 2023-11-28T21:50:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f0659c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f0806e300 0x7f1f081f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f05d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f08151af0 0x7f1f081f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f05d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f08151af0 0x7f1f081f2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:50914/0 (socket says 172.21.15.12:50914) 2023-11-28T21:50:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f05d9b640 1 -- 172.21.15.12:0/3892509816 learned_addr learned my addr 172.21.15.12:0/3892509816 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f06d9d640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0806ad20 0x7f1f081404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f05d9b640 1 -- 172.21.15.12:0/3892509816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f0806e300 msgr2=0x7f1f081f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f05d9b640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f0806e300 0x7f1f081f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:16.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f05d9b640 1 -- 172.21.15.12:0/3892509816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0806ad20 msgr2=0x7f1f081404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:16.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f05d9b640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0806ad20 0x7f1f081404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:16.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f05d9b640 1 -- 172.21.15.12:0/3892509816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc0094d0 con 0x7f1f08151af0 2023-11-28T21:50:16.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f05d9b640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f08151af0 0x7f1f081f2da0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1efc01a170 tx=0x7f1efc0192e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:16.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1ef6ffd640 1 -- 172.21.15.12:0/3892509816 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0b4620 con 0x7f1f08151af0 2023-11-28T21:50:16.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f0ce17640 1 -- 172.21.15.12:0/3892509816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f080c9d00 con 0x7f1f08151af0 2023-11-28T21:50:16.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1ef6ffd640 1 -- 172.21.15.12:0/3892509816 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc0b4780 con 0x7f1f08151af0 2023-11-28T21:50:16.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1ef6ffd640 1 -- 172.21.15.12:0/3892509816 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0b4af0 con 0x7f1f08151af0 2023-11-28T21:50:16.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f0ce17640 1 -- 172.21.15.12:0/3892509816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f081f63b0 con 0x7f1f08151af0 2023-11-28T21:50:16.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.708+0000 7f1f0ce17640 1 -- 172.21.15.12:0/3892509816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f081f6bb0 con 0x7f1f08151af0 2023-11-28T21:50:16.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.712+0000 7f1ef6ffd640 1 -- 172.21.15.12:0/3892509816 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1efc0b2070 con 0x7f1f08151af0 2023-11-28T21:50:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.712+0000 7f1ef6ffd640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ee8066470 0x7f1ee8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.712+0000 7f1ef6ffd640 1 -- 172.21.15.12:0/3892509816 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f1efc11d8f0 con 0x7f1f08151af0 2023-11-28T21:50:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.712+0000 7f1f0659c640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ee8066470 0x7f1ee8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:16.718 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.712+0000 7f1f0659c640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ee8066470 0x7f1ee8068930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8007fd0 tx=0x7f1ef8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:16.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.724+0000 7f1ef6ffd640 1 -- 172.21.15.12:0/3892509816 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1efc0ea760 con 0x7f1f08151af0 2023-11-28T21:50:16.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.944+0000 7f1f0ce17640 1 -- 172.21.15.12:0/3892509816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f081f6b70 con 0x7f1f08151af0 2023-11-28T21:50:16.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1ef6ffd640 1 -- 172.21.15.12:0/3892509816 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+6503 (secure 0 0 0) 0x7f1efc0edd80 con 0x7f1f08151af0 2023-11-28T21:50:16.955 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:16.955 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6330,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:49:20.980151+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[1,2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:16.955 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 61 2023-11-28T21:50:16.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 -- 172.21.15.12:0/3892509816 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ee8066470 msgr2=0x7f1ee8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:16.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ee8066470 0x7f1ee8068930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8007fd0 tx=0x7f1ef8009040 comp rx=0 tx=0).stop 2023-11-28T21:50:16.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 -- 172.21.15.12:0/3892509816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f08151af0 msgr2=0x7f1f081f2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:16.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f08151af0 0x7f1f081f2da0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1efc01a170 tx=0x7f1efc0192e0 comp rx=0 tx=0).stop 2023-11-28T21:50:16.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 -- 172.21.15.12:0/3892509816 shutdown_connections 2023-11-28T21:50:16.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1ee8066470 0x7f1ee8068930 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:16.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f0806ad20 0x7f1f081404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:16.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f08151af0 0x7f1f081f2da0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:16.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 --2- 172.21.15.12:0/3892509816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f0806e300 0x7f1f081f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:16.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 -- 172.21.15.12:0/3892509816 >> 172.21.15.12:0/3892509816 conn(0x7f1f08066360 msgr2=0x7f1f08147ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:16.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 -- 172.21.15.12:0/3892509816 shutdown_connections 2023-11-28T21:50:16.958 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:16.952+0000 7f1f0ce17640 1 -- 172.21.15.12:0/3892509816 wait complete. 2023-11-28T21:50:16.982 INFO:tasks.mds_thrash.fs.[cephfs]:thrashing max_mds: 1 -> 2 2023-11-28T21:50:16.983 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2023-11-28T21:50:17.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.176+0000 7f314d565640 1 Processor -- start 2023-11-28T21:50:17.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.176+0000 7f314d565640 1 -- start start 2023-11-28T21:50:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f314d565640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f31480c54a0 0x7f3148072310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f314d565640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31480706f0 con 0x7f3148140ee0 2023-11-28T21:50:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f314d565640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314806fb70 con 0x7f314806dfe0 2023-11-28T21:50:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f314d565640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314806fe50 con 0x7f31480c54a0 2023-11-28T21:50:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f3146ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3148140ee0 0x7f31481583e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35028/0 (socket says 172.21.15.12:35028) 2023-11-28T21:50:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f3146ffd640 1 -- 172.21.15.12:0/2537725994 learned_addr learned my addr 172.21.15.12:0/2537725994 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:50:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f31477fe640 1 --2- 172.21.15.12:0/2537725994 >> v2:172.21.15.12:3300/0 conn(0x7f31480c54a0 0x7f3148072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f31467fc640 1 -- 172.21.15.12:0/2537725994 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957328136 0 0) 0x7f31480706f0 con 0x7f3148140ee0 2023-11-28T21:50:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f31467fc640 1 -- 172.21.15.12:0/2537725994 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3134003530 con 0x7f3148140ee0 2023-11-28T21:50:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f31477fe640 1 -- 172.21.15.12:0/2537725994 >> v1:172.21.15.144:6789/0 conn(0x7f314806dfe0 legacy=0x7f31480c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f31477fe640 1 -- 172.21.15.12:0/2537725994 >> v1:172.21.15.12:6789/0 conn(0x7f3148140ee0 legacy=0x7f31481583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.180+0000 7f31477fe640 1 -- 172.21.15.12:0/2537725994 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3148070130 con 0x7f31480c54a0 2023-11-28T21:50:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f31477fe640 1 --2- 172.21.15.12:0/2537725994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 0x7f3148072310 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3140009d90 tx=0x7f314001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9303f45d879dea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f31467fc640 1 -- 172.21.15.12:0/2537725994 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31400a5900 con 0x7f31480c54a0 2023-11-28T21:50:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f31467fc640 1 -- 172.21.15.12:0/2537725994 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31400a5a60 con 0x7f31480c54a0 2023-11-28T21:50:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f31467fc640 1 -- 172.21.15.12:0/2537725994 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31400a5dd0 con 0x7f31480c54a0 2023-11-28T21:50:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 -- 172.21.15.12:0/2537725994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 msgr2=0x7f3148072310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 --2- 172.21.15.12:0/2537725994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 0x7f3148072310 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3140009d90 tx=0x7f314001c950 comp rx=0 tx=0).stop 2023-11-28T21:50:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 -- 172.21.15.12:0/2537725994 shutdown_connections 2023-11-28T21:50:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 --2- 172.21.15.12:0/2537725994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 0x7f3148072310 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 -- 172.21.15.12:0/2537725994 >> 172.21.15.12:0/2537725994 conn(0x7f314806d4c0 msgr2=0x7f314806d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 -- 172.21.15.12:0/2537725994 shutdown_connections 2023-11-28T21:50:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 -- 172.21.15.12:0/2537725994 wait complete. 2023-11-28T21:50:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 Processor -- start 2023-11-28T21:50:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 -- start start 2023-11-28T21:50:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f314806dfe0 0x7f31480c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 0x7f3148152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3148140ee0 0x7f3148152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3148155080 con 0x7f31480c54a0 2023-11-28T21:50:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31481532c0 con 0x7f3148140ee0 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.184+0000 7f314d565640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31481554a0 con 0x7f314806dfe0 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 0x7f3148152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 0x7f3148152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45788/0 (socket says 172.21.15.12:45788) 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313ffff640 1 -- 172.21.15.12:0/2302671549 learned_addr learned my addr 172.21.15.12:0/2302671549 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f3146ffd640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f314806dfe0 0x7f31480c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f31477fe640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3148140ee0 0x7f3148152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313ffff640 1 -- 172.21.15.12:0/2302671549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f314806dfe0 msgr2=0x7f31480c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313ffff640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f314806dfe0 0x7f31480c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313ffff640 1 -- 172.21.15.12:0/2302671549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3148140ee0 msgr2=0x7f3148152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313ffff640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3148140ee0 0x7f3148152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313ffff640 1 -- 172.21.15.12:0/2302671549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3140009a70 con 0x7f31480c54a0 2023-11-28T21:50:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313ffff640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 0x7f3148152840 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f313000c430 tx=0x7f313000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313f7fe640 1 -- 172.21.15.12:0/2302671549 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31300052d0 con 0x7f31480c54a0 2023-11-28T21:50:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313f7fe640 1 -- 172.21.15.12:0/2302671549 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3130005430 con 0x7f31480c54a0 2023-11-28T21:50:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f314d565640 1 -- 172.21.15.12:0/2302671549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314813ab00 con 0x7f31480c54a0 2023-11-28T21:50:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313f7fe640 1 -- 172.21.15.12:0/2302671549 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130007260 con 0x7f31480c54a0 2023-11-28T21:50:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f314d565640 1 -- 172.21.15.12:0/2302671549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31481549c0 con 0x7f31480c54a0 2023-11-28T21:50:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f314d565640 1 -- 172.21.15.12:0/2302671549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3148067c70 con 0x7f31480c54a0 2023-11-28T21:50:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.188+0000 7f313f7fe640 1 -- 172.21.15.12:0/2302671549 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f31300073c0 con 0x7f31480c54a0 2023-11-28T21:50:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.192+0000 7f313f7fe640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31280664a0 0x7f3128068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.192+0000 7f313f7fe640 1 -- 172.21.15.12:0/2302671549 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f3130083850 con 0x7f31480c54a0 2023-11-28T21:50:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.192+0000 7f3146ffd640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31280664a0 0x7f3128068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.192+0000 7f3146ffd640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31280664a0 0x7f3128068960 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f31380011f0 tx=0x7f3138009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.196+0000 7f313f7fe640 1 -- 172.21.15.12:0/2302671549 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f31300506a0 con 0x7f31480c54a0 2023-11-28T21:50:17.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.388+0000 7f314d565640 1 -- 172.21.15.12:0/2302671549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f31481551c0 con 0x7f31480c54a0 2023-11-28T21:50:17.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f313f7fe640 1 -- 172.21.15.12:0/2302671549 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v62) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3130053cc0 con 0x7f31480c54a0 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 -- 172.21.15.12:0/2302671549 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31280664a0 msgr2=0x7f3128068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31280664a0 0x7f3128068960 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f31380011f0 tx=0x7f3138009040 comp rx=0 tx=0).stop 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 -- 172.21.15.12:0/2302671549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 msgr2=0x7f3148152840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 0x7f3148152840 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f313000c430 tx=0x7f313000c8f0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 -- 172.21.15.12:0/2302671549 shutdown_connections 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f31280664a0 0x7f3128068960 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3148140ee0 0x7f3148152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f31480c54a0 0x7f3148152840 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 --2- 172.21.15.12:0/2302671549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f314806dfe0 0x7f31480c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 -- 172.21.15.12:0/2302671549 >> 172.21.15.12:0/2302671549 conn(0x7f314806d4c0 msgr2=0x7f3148139f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:17.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 -- 172.21.15.12:0/2302671549 shutdown_connections 2023-11-28T21:50:17.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.640+0000 7f314d565640 1 -- 172.21.15.12:0/2302671549 wait complete. 2023-11-28T21:50:17.665 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:50:17.666 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:17.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.808+0000 7f3c8dfaa640 1 Processor -- start 2023-11-28T21:50:17.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.808+0000 7f3c8dfaa640 1 -- start start 2023-11-28T21:50:17.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c8dfaa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c88150c20 0x7f3c88156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c8dfaa640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3c8806e300 0x7f3c880c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c8dfaa640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c88067410 con 0x7f3c8806cf40 2023-11-28T21:50:17.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c8dfaa640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8806e090 con 0x7f3c8806e300 2023-11-28T21:50:17.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c8dfaa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c88157f30 con 0x7f3c88150c20 2023-11-28T21:50:17.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c86ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3c8806e300 0x7f3c880c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c86ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3c8806e300 0x7f3c880c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45794/0 (socket says 172.21.15.12:45794) 2023-11-28T21:50:17.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c86ffd640 1 -- 172.21.15.12:0/1620599592 learned_addr learned my addr 172.21.15.12:0/1620599592 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:17.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c87fff640 1 --1- 172.21.15.12:0/1620599592 >> v1:172.21.15.12:6789/0 conn(0x7f3c8806cf40 0x7f3c88158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35032/0 (socket says 172.21.15.12:35032) 2023-11-28T21:50:17.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c877fe640 1 --2- 172.21.15.12:0/1620599592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c88150c20 0x7f3c88156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c86ffd640 1 -- 172.21.15.12:0/1620599592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c88150c20 msgr2=0x7f3c88156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:17.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c86ffd640 1 --2- 172.21.15.12:0/1620599592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c88150c20 0x7f3c88156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c86ffd640 1 -- 172.21.15.12:0/1620599592 >> v1:172.21.15.12:6789/0 conn(0x7f3c8806cf40 legacy=0x7f3c88158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.812+0000 7f3c86ffd640 1 -- 172.21.15.12:0/1620599592 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c88151000 con 0x7f3c8806e300 2023-11-28T21:50:17.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c86ffd640 1 --2- 172.21.15.12:0/1620599592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 0x7f3c880c6dd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00bb10 tx=0x7f3c7c01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93f0a61ae8b9f7a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:17.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c867fc640 1 -- 172.21.15.12:0/1620599592 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0a7af0 con 0x7f3c8806e300 2023-11-28T21:50:17.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c867fc640 1 -- 172.21.15.12:0/1620599592 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c0a7c50 con 0x7f3c8806e300 2023-11-28T21:50:17.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c867fc640 1 -- 172.21.15.12:0/1620599592 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0af440 con 0x7f3c8806e300 2023-11-28T21:50:17.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/1620599592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 msgr2=0x7f3c880c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c8dfaa640 1 --2- 172.21.15.12:0/1620599592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 0x7f3c880c6dd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00bb10 tx=0x7f3c7c01ec10 comp rx=0 tx=0).stop 2023-11-28T21:50:17.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/1620599592 shutdown_connections 2023-11-28T21:50:17.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c8dfaa640 1 --2- 172.21.15.12:0/1620599592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 0x7f3c880c6dd0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c8dfaa640 1 --2- 172.21.15.12:0/1620599592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c88150c20 0x7f3c88156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/1620599592 >> 172.21.15.12:0/1620599592 conn(0x7f3c88066360 msgr2=0x7f3c88066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:17.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/1620599592 shutdown_connections 2023-11-28T21:50:17.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/1620599592 wait complete. 2023-11-28T21:50:17.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c8dfaa640 1 Processor -- start 2023-11-28T21:50:17.821 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.816+0000 7f3c8dfaa640 1 -- start start 2023-11-28T21:50:17.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c8dfaa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c8806cf40 0x7f3c880c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c8dfaa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 0x7f3c881f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c8dfaa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c88150c20 0x7f3c880c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c8dfaa640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8814bc00 con 0x7f3c8806e300 2023-11-28T21:50:17.823 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c8dfaa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c880c3f50 con 0x7f3c8806cf40 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c8dfaa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c88156470 con 0x7f3c88150c20 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c86ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 0x7f3c881f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c86ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 0x7f3c881f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45798/0 (socket says 172.21.15.12:45798) 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c86ffd640 1 -- 172.21.15.12:0/2973245134 learned_addr learned my addr 172.21.15.12:0/2973245134 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c877fe640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c8806cf40 0x7f3c880c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c87fff640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c88150c20 0x7f3c880c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c86ffd640 1 -- 172.21.15.12:0/2973245134 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c88150c20 msgr2=0x7f3c880c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c86ffd640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c88150c20 0x7f3c880c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c86ffd640 1 -- 172.21.15.12:0/2973245134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c8806cf40 msgr2=0x7f3c880c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c86ffd640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c8806cf40 0x7f3c880c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c86ffd640 1 -- 172.21.15.12:0/2973245134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c00b770 con 0x7f3c8806e300 2023-11-28T21:50:17.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c86ffd640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 0x7f3c881f4380 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00b2c0 tx=0x7f3c7c01b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:17.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c6ffff640 1 -- 172.21.15.12:0/2973245134 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0ba660 con 0x7f3c8806e300 2023-11-28T21:50:17.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c6ffff640 1 -- 172.21.15.12:0/2973245134 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c0ba7c0 con 0x7f3c8806e300 2023-11-28T21:50:17.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c6ffff640 1 -- 172.21.15.12:0/2973245134 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0bab70 con 0x7f3c8806e300 2023-11-28T21:50:17.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/2973245134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c880c3900 con 0x7f3c8806e300 2023-11-28T21:50:17.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/2973245134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c88151ca0 con 0x7f3c8806e300 2023-11-28T21:50:17.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/2973245134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c880c40c0 con 0x7f3c8806e300 2023-11-28T21:50:17.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.820+0000 7f3c6ffff640 1 -- 172.21.15.12:0/2973245134 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3c7c0c2360 con 0x7f3c8806e300 2023-11-28T21:50:17.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.824+0000 7f3c6ffff640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3c74066470 0x7f3c74068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:17.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.824+0000 7f3c6ffff640 1 -- 172.21.15.12:0/2973245134 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f3c7c11f2f0 con 0x7f3c8806e300 2023-11-28T21:50:17.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.824+0000 7f3c877fe640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3c74066470 0x7f3c74068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:17.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.824+0000 7f3c877fe640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3c74066470 0x7f3c74068930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800b930 tx=0x7f3c78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:17.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:17.828+0000 7f3c6ffff640 1 -- 172.21.15.12:0/2973245134 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3c7c0ec1d0 con 0x7f3c8806e300 2023-11-28T21:50:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.016+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/2973245134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c88074f60 con 0x7f3c8806e300 2023-11-28T21:50:18.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.016+0000 7f3c6ffff640 1 -- 172.21.15.12:0/2973245134 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6517 (secure 0 0 0) 0x7f3c7c0ef7f0 con 0x7f3c8806e300 2023-11-28T21:50:18.021 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:18.021 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:17.644233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6177,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:starting","state_seq":1,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:18.021 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 63 2023-11-28T21:50:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/2973245134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3c74066470 msgr2=0x7f3c74068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3c74066470 0x7f3c74068930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800b930 tx=0x7f3c78008040 comp rx=0 tx=0).stop 2023-11-28T21:50:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/2973245134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 msgr2=0x7f3c881f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 0x7f3c881f4380 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00b2c0 tx=0x7f3c7c01b9b0 comp rx=0 tx=0).stop 2023-11-28T21:50:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/2973245134 shutdown_connections 2023-11-28T21:50:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3c74066470 0x7f3c74068930 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c88150c20 0x7f3c880c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:18.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c8806e300 0x7f3c881f4380 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:18.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 --2- 172.21.15.12:0/2973245134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c8806cf40 0x7f3c880c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:18.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/2973245134 >> 172.21.15.12:0/2973245134 conn(0x7f3c88066360 msgr2=0x7f3c88070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:18.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/2973245134 shutdown_connections 2023-11-28T21:50:18.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:18.020+0000 7f3c8dfaa640 1 -- 172.21.15.12:0/2973245134 wait complete. 2023-11-28T21:50:18.043 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 63, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 63, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6177": { "addr": "172.21.15.12:6838/1551825785", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 1551825785, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 1551825785, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6177, "incarnation": 56, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 34 }, "gid_6330": { "addr": "172.21.15.12:6839/754378087", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 754378087, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 754378087, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6330, "incarnation": 63, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:starting", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 49, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:50:17.644233+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6177, "mds_1": 6330 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 27, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:50:20.045 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.200+0000 7f8382380640 1 Processor -- start 2023-11-28T21:50:20.205 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.200+0000 7f8382380640 1 -- start start 2023-11-28T21:50:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f8382380640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c151af0 0x7f837c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f8382380640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f837c06e300 0x7f837c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f8382380640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837c157b70 con 0x7f837c151ed0 2023-11-28T21:50:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f8382380640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837c0707f0 con 0x7f837c06e300 2023-11-28T21:50:20.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f8382380640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0c7b90 con 0x7f837c151af0 2023-11-28T21:50:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f837bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c151af0 0x7f837c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f837bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c151af0 0x7f837c0c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:47426/0 (socket says 172.21.15.12:47426) 2023-11-28T21:50:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f837bfff640 1 -- 172.21.15.12:0/1581578657 learned_addr learned my addr 172.21.15.12:0/1581578657 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f838091d640 1 --2- 172.21.15.12:0/1581578657 >> v2:172.21.15.144:3300/0 conn(0x7f837c06e300 0x7f837c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f837bfff640 1 -- 172.21.15.12:0/1581578657 >> v2:172.21.15.144:3300/0 conn(0x7f837c06e300 msgr2=0x7f837c072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f837bfff640 1 --2- 172.21.15.12:0/1581578657 >> v2:172.21.15.144:3300/0 conn(0x7f837c06e300 0x7f837c072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f837bfff640 1 -- 172.21.15.12:0/1581578657 >> v1:172.21.15.144:6789/0 conn(0x7f837c151ed0 legacy=0x7f837c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:20.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.204+0000 7f837bfff640 1 -- 172.21.15.12:0/1581578657 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c157e50 con 0x7f837c151af0 2023-11-28T21:50:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f837bfff640 1 --2- 172.21.15.12:0/1581578657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c151af0 0x7f837c0c7650 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f836c009870 tx=0x7f836c01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4a3132489f2f2b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:20.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f837affd640 1 -- 172.21.15.12:0/1581578657 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c0ab600 con 0x7f837c151af0 2023-11-28T21:50:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f837affd640 1 -- 172.21.15.12:0/1581578657 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836c0a7050 con 0x7f837c151af0 2023-11-28T21:50:20.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f837affd640 1 -- 172.21.15.12:0/1581578657 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c0ab980 con 0x7f837c151af0 2023-11-28T21:50:20.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f8382380640 1 -- 172.21.15.12:0/1581578657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c151af0 msgr2=0x7f837c0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:20.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f8382380640 1 --2- 172.21.15.12:0/1581578657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c151af0 0x7f837c0c7650 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f836c009870 tx=0x7f836c01c540 comp rx=0 tx=0).stop 2023-11-28T21:50:20.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f8382380640 1 -- 172.21.15.12:0/1581578657 shutdown_connections 2023-11-28T21:50:20.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f8382380640 1 --2- 172.21.15.12:0/1581578657 >> v2:172.21.15.144:3300/0 conn(0x7f837c06e300 0x7f837c072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:20.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f8382380640 1 --2- 172.21.15.12:0/1581578657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c151af0 0x7f837c0c7650 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:20.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f8382380640 1 -- 172.21.15.12:0/1581578657 >> 172.21.15.12:0/1581578657 conn(0x7f837c066360 msgr2=0x7f837c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:20.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f8382380640 1 -- 172.21.15.12:0/1581578657 shutdown_connections 2023-11-28T21:50:20.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f8382380640 1 -- 172.21.15.12:0/1581578657 wait complete. 2023-11-28T21:50:20.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f8382380640 1 Processor -- start 2023-11-28T21:50:20.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.208+0000 7f8382380640 1 -- start start 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8382380640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c06e300 0x7f837c1e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8382380640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f837c151af0 0x7f837c1420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8382380640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f837c14b6d0 0x7f837c142620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8382380640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c1419b0 con 0x7f837c151af0 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8382380640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c141c50 con 0x7f837c14b6d0 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8382380640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0c9810 con 0x7f837c06e300 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f837b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f837c151af0 0x7f837c1420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f837b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f837c151af0 0x7f837c1420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45814/0 (socket says 172.21.15.12:45814) 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f837b7fe640 1 -- 172.21.15.12:0/307562019 learned_addr learned my addr 172.21.15.12:0/307562019 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f837bfff640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c06e300 0x7f837c1e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:20.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f838091d640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f837c14b6d0 0x7f837c142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:20.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f837b7fe640 1 -- 172.21.15.12:0/307562019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c06e300 msgr2=0x7f837c1e9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f837b7fe640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c06e300 0x7f837c1e9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f837b7fe640 1 -- 172.21.15.12:0/307562019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f837c14b6d0 msgr2=0x7f837c142620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f837b7fe640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f837c14b6d0 0x7f837c142620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f837b7fe640 1 -- 172.21.15.12:0/307562019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c0094d0 con 0x7f837c151af0 2023-11-28T21:50:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f837b7fe640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f837c151af0 0x7f837c1420e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f837000c500 tx=0x7f837000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8378ff9640 1 -- 172.21.15.12:0/307562019 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83700052d0 con 0x7f837c151af0 2023-11-28T21:50:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8378ff9640 1 -- 172.21.15.12:0/307562019 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8370005430 con 0x7f837c151af0 2023-11-28T21:50:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8378ff9640 1 -- 172.21.15.12:0/307562019 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370012830 con 0x7f837c151af0 2023-11-28T21:50:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8382380640 1 -- 172.21.15.12:0/307562019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837c0c93f0 con 0x7f837c151af0 2023-11-28T21:50:20.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8382380640 1 -- 172.21.15.12:0/307562019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837c0c9d60 con 0x7f837c151af0 2023-11-28T21:50:20.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.212+0000 7f8382380640 1 -- 172.21.15.12:0/307562019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837c141e00 con 0x7f837c151af0 2023-11-28T21:50:20.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.216+0000 7f8378ff9640 1 -- 172.21.15.12:0/307562019 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8370012990 con 0x7f837c151af0 2023-11-28T21:50:20.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.216+0000 7f8378ff9640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8368066470 0x7f8368068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:20.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.216+0000 7f8378ff9640 1 -- 172.21.15.12:0/307562019 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f8370083f40 con 0x7f837c151af0 2023-11-28T21:50:20.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.216+0000 7f837bfff640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8368066470 0x7f8368068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:20.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.216+0000 7f837bfff640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8368066470 0x7f8368068930 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f837c06c7c0 tx=0x7f836c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:20.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.220+0000 7f8378ff9640 1 -- 172.21.15.12:0/307562019 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f837c141e00 con 0x7f837c151af0 2023-11-28T21:50:20.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 -- 172.21.15.12:0/307562019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f837c141e00 con 0x7f837c151af0 2023-11-28T21:50:20.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8378ff9640 1 -- 172.21.15.12:0/307562019 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6517 (secure 0 0 0) 0x7f837c141e00 con 0x7f837c151af0 2023-11-28T21:50:20.415 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:20.415 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:17.644233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6177,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:starting","state_seq":1,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:20.416 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 63 2023-11-28T21:50:20.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 -- 172.21.15.12:0/307562019 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8368066470 msgr2=0x7f8368068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:20.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8368066470 0x7f8368068930 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f837c06c7c0 tx=0x7f836c008040 comp rx=0 tx=0).stop 2023-11-28T21:50:20.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 -- 172.21.15.12:0/307562019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f837c151af0 msgr2=0x7f837c1420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:20.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f837c151af0 0x7f837c1420e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f837000c500 tx=0x7f837000c9c0 comp rx=0 tx=0).stop 2023-11-28T21:50:20.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 -- 172.21.15.12:0/307562019 shutdown_connections 2023-11-28T21:50:20.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8368066470 0x7f8368068930 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:20.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f837c14b6d0 0x7f837c142620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:20.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f837c151af0 0x7f837c1420e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:20.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 --2- 172.21.15.12:0/307562019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f837c06e300 0x7f837c1e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:20.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 -- 172.21.15.12:0/307562019 >> 172.21.15.12:0/307562019 conn(0x7f837c066360 msgr2=0x7f837c14f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:20.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 -- 172.21.15.12:0/307562019 shutdown_connections 2023-11-28T21:50:20.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:20.412+0000 7f8382380640 1 -- 172.21.15.12:0/307562019 wait complete. 2023-11-28T21:50:20.436 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:50:22.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:22.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd1849c640 1 Processor -- start 2023-11-28T21:50:22.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd1849c640 1 -- start start 2023-11-28T21:50:22.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd1849c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd10151af0 0x7fdd100c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:22.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd1849c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd10157b70 con 0x7fdd1006e300 2023-11-28T21:50:22.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd1849c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd10157e50 con 0x7fdd10151ed0 2023-11-28T21:50:22.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd1849c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100707f0 con 0x7fdd10151af0 2023-11-28T21:50:22.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd16a39640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdd1006e300 0x7fdd10072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35044/0 (socket says 172.21.15.12:35044) 2023-11-28T21:50:22.585 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd16a39640 1 -- 172.21.15.12:0/1199243843 learned_addr learned my addr 172.21.15.12:0/1199243843 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:50:22.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd16238640 1 --2- 172.21.15.12:0/1199243843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd10151af0 0x7fdd100c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:22.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd15236640 1 -- 172.21.15.12:0/1199243843 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605593253 0 0) 0x7fdd10157b70 con 0x7fdd1006e300 2023-11-28T21:50:22.586 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.580+0000 7fdd15236640 1 -- 172.21.15.12:0/1199243843 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcfc003530 con 0x7fdd1006e300 2023-11-28T21:50:22.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd15236640 1 -- 172.21.15.12:0/1199243843 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364679756 0 0) 0x7fdd10157e50 con 0x7fdd10151ed0 2023-11-28T21:50:22.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd15236640 1 -- 172.21.15.12:0/1199243843 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd10157b70 con 0x7fdd10151ed0 2023-11-28T21:50:22.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd16238640 1 -- 172.21.15.12:0/1199243843 >> v1:172.21.15.144:6789/0 conn(0x7fdd10151ed0 legacy=0x7fdd100c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:22.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd16238640 1 -- 172.21.15.12:0/1199243843 >> v1:172.21.15.12:6789/0 conn(0x7fdd1006e300 legacy=0x7fdd10072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:22.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd16238640 1 -- 172.21.15.12:0/1199243843 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd10158130 con 0x7fdd10151af0 2023-11-28T21:50:22.588 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd16238640 1 --2- 172.21.15.12:0/1199243843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd10151af0 0x7fdd100c7650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0097f0 tx=0x7fdd0c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=360efca0cb9c244a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd15236640 1 -- 172.21.15.12:0/1199243843 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0ab600 con 0x7fdd10151af0 2023-11-28T21:50:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd15236640 1 -- 172.21.15.12:0/1199243843 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c0a7050 con 0x7fdd10151af0 2023-11-28T21:50:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd15236640 1 -- 172.21.15.12:0/1199243843 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0ab960 con 0x7fdd10151af0 2023-11-28T21:50:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd1849c640 1 -- 172.21.15.12:0/1199243843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd10151af0 msgr2=0x7fdd100c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.584+0000 7fdd1849c640 1 --2- 172.21.15.12:0/1199243843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd10151af0 0x7fdd100c7650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0097f0 tx=0x7fdd0c01c4c0 comp rx=0 tx=0).stop 2023-11-28T21:50:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 -- 172.21.15.12:0/1199243843 shutdown_connections 2023-11-28T21:50:22.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 --2- 172.21.15.12:0/1199243843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd10151af0 0x7fdd100c7650 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:22.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 -- 172.21.15.12:0/1199243843 >> 172.21.15.12:0/1199243843 conn(0x7fdd10066360 msgr2=0x7fdd10066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:22.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 -- 172.21.15.12:0/1199243843 shutdown_connections 2023-11-28T21:50:22.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 -- 172.21.15.12:0/1199243843 wait complete. 2023-11-28T21:50:22.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 Processor -- start 2023-11-28T21:50:22.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 -- start start 2023-11-28T21:50:22.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd1006e300 0x7fdd100c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:22.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd10151af0 0x7fdd100c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1014cb90 0x7fdd101428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1014d8a0 con 0x7fdd1014cb90 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd101f4db0 con 0x7fdd10151af0 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd1849c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100c4180 con 0x7fdd1006e300 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd16a39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1014cb90 0x7fdd101428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd16a39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1014cb90 0x7fdd101428e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45830/0 (socket says 172.21.15.12:45830) 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd16a39640 1 -- 172.21.15.12:0/3489273361 learned_addr learned my addr 172.21.15.12:0/3489273361 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd16238640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd1006e300 0x7fdd100c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd15a37640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd10151af0 0x7fdd100c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd16a39640 1 -- 172.21.15.12:0/3489273361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd1006e300 msgr2=0x7fdd100c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd16a39640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd1006e300 0x7fdd100c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:22.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd16a39640 1 -- 172.21.15.12:0/3489273361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd10151af0 msgr2=0x7fdd100c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd16a39640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd10151af0 0x7fdd100c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd16a39640 1 -- 172.21.15.12:0/3489273361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c009420 con 0x7fdd1014cb90 2023-11-28T21:50:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd16238640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd1006e300 0x7fdd100c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:50:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.588+0000 7fdd15a37640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd10151af0 0x7fdd100c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:50:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd16a39640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1014cb90 0x7fdd101428e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800cb70 tx=0x7fdd0800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd06ffd640 1 -- 172.21.15.12:0/3489273361 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0800ace0 con 0x7fdd1014cb90 2023-11-28T21:50:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd06ffd640 1 -- 172.21.15.12:0/3489273361 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0800ae40 con 0x7fdd1014cb90 2023-11-28T21:50:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd06ffd640 1 -- 172.21.15.12:0/3489273361 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08012620 con 0x7fdd1014cb90 2023-11-28T21:50:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd1849c640 1 -- 172.21.15.12:0/3489273361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd100c4410 con 0x7fdd1014cb90 2023-11-28T21:50:22.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd1849c640 1 -- 172.21.15.12:0/3489273361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd101f4480 con 0x7fdd1014cb90 2023-11-28T21:50:22.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd1849c640 1 -- 172.21.15.12:0/3489273361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd10074f60 con 0x7fdd1014cb90 2023-11-28T21:50:22.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd06ffd640 1 -- 172.21.15.12:0/3489273361 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdd08007070 con 0x7fdd1014cb90 2023-11-28T21:50:22.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd06ffd640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdcf40664a0 0x7fdcf4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:22.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd06ffd640 1 -- 172.21.15.12:0/3489273361 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fdd080845c0 con 0x7fdd1014cb90 2023-11-28T21:50:22.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.592+0000 7fdd16238640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdcf40664a0 0x7fdcf4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:22.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.596+0000 7fdd16238640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdcf40664a0 0x7fdcf4068960 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ad000 tx=0x7fdd0c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:22.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.596+0000 7fdd06ffd640 1 -- 172.21.15.12:0/3489273361 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdd08054d80 con 0x7fdd1014cb90 2023-11-28T21:50:22.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.796+0000 7fdd1849c640 1 -- 172.21.15.12:0/3489273361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd101f4ef0 con 0x7fdd1014cb90 2023-11-28T21:50:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd06ffd640 1 -- 172.21.15.12:0/3489273361 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+6516 (secure 0 0 0) 0x7fdd08017090 con 0x7fdd1014cb90 2023-11-28T21:50:22.802 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:22.803 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:22.662874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6177,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2023-11-28T21:50:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 -- 172.21.15.12:0/3489273361 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdcf40664a0 msgr2=0x7fdcf4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdcf40664a0 0x7fdcf4068960 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ad000 tx=0x7fdd0c0a6000 comp rx=0 tx=0).stop 2023-11-28T21:50:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 -- 172.21.15.12:0/3489273361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1014cb90 msgr2=0x7fdd101428e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1014cb90 0x7fdd101428e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800cb70 tx=0x7fdd0800a430 comp rx=0 tx=0).stop 2023-11-28T21:50:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 -- 172.21.15.12:0/3489273361 shutdown_connections 2023-11-28T21:50:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdcf40664a0 0x7fdcf4068960 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd1014cb90 0x7fdd101428e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd10151af0 0x7fdd100c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 --2- 172.21.15.12:0/3489273361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd1006e300 0x7fdd100c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 -- 172.21.15.12:0/3489273361 >> 172.21.15.12:0/3489273361 conn(0x7fdd10066360 msgr2=0x7fdd1006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 -- 172.21.15.12:0/3489273361 shutdown_connections 2023-11-28T21:50:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:22.800+0000 7fdd1849c640 1 -- 172.21.15.12:0/3489273361 wait complete. 2023-11-28T21:50:22.823 INFO:tasks.mds_thrash.fs.[cephfs]:mds cluster has 2 alive and active, now stable! 2023-11-28T21:50:22.823 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2023-11-28T21:50:22.824 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T21:50:22.824 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:50:22.842 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T21:50:22.842 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:23.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.028+0000 7f032eb3e640 1 Processor -- start 2023-11-28T21:50:23.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.028+0000 7f032eb3e640 1 -- start start 2023-11-28T21:50:23.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.028+0000 7f032eb3e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f03281500d0 0x7f03280c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:23.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.028+0000 7f032eb3e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03280c6ce0 con 0x7f0328156af0 2023-11-28T21:50:23.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.028+0000 7f032eb3e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0328156f60 con 0x7f032806d7d0 2023-11-28T21:50:23.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.028+0000 7f032eb3e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03280c7170 con 0x7f03281500d0 2023-11-28T21:50:23.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.028+0000 7f0327fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0328156af0 0x7f03280c2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35054/0 (socket says 172.21.15.12:35054) 2023-11-28T21:50:23.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.028+0000 7f0327fff640 1 -- 172.21.15.12:0/2510088427 learned_addr learned my addr 172.21.15.12:0/2510088427 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:50:23.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.028+0000 7f032c8da640 1 --2- 172.21.15.12:0/2510088427 >> v2:172.21.15.144:3300/0 conn(0x7f03281500d0 0x7f03280c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835567987 0 0) 0x7f03280c6ce0 con 0x7f0328156af0 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0318003450 con 0x7f0328156af0 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566195185 0 0) 0x7f0328156f60 con 0x7f032806d7d0 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03280c6ce0 con 0x7f032806d7d0 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 528914464 0 0) 0x7f0318003450 con 0x7f0328156af0 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 >> v2:172.21.15.144:3300/0 conn(0x7f03281500d0 msgr2=0x7f03280c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 --2- 172.21.15.12:0/2510088427 >> v2:172.21.15.144:3300/0 conn(0x7f03281500d0 0x7f03280c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 >> v1:172.21.15.144:6789/0 conn(0x7f032806d7d0 legacy=0x7f03281609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03280c5df0 con 0x7f0328156af0 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f0314001680 con 0x7f0328156af0 2023-11-28T21:50:23.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 >> v1:172.21.15.12:6789/0 conn(0x7f0328156af0 legacy=0x7f03280c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0318003cc0 0x7f0318008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0318008da0 0x7f031800b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f031800b6d0 0x7f031800dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280c7170 con 0x7f0318008da0 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031800e2e0 con 0x7f0318003cc0 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031800e4a0 con 0x7f031800b6d0 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f0327fff640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0318008da0 0x7f031800b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f032c8da640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0318003cc0 0x7f0318008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f032d0db640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f031800b6d0 0x7f031800dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f0327fff640 1 -- 172.21.15.12:0/2510088427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f031800b6d0 msgr2=0x7f031800dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f0327fff640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f031800b6d0 0x7f031800dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f0327fff640 1 -- 172.21.15.12:0/2510088427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0318003cc0 msgr2=0x7f0318008860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:23.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f0327fff640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0318003cc0 0x7f0318008860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f0327fff640 1 -- 172.21.15.12:0/2510088427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318003450 con 0x7f0318008da0 2023-11-28T21:50:23.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.032+0000 7f032d0db640 1 -- 172.21.15.12:0/2510088427 reap_dead start 2023-11-28T21:50:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f0327fff640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0318008da0 0x7f031800b190 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0314007fa0 tx=0x7f031401cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ab126a6fe02716 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314005ae0 con 0x7f0318008da0 2023-11-28T21:50:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0314005c40 con 0x7f0318008da0 2023-11-28T21:50:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f03277fe640 1 -- 172.21.15.12:0/2510088427 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140a5a00 con 0x7f0318008da0 2023-11-28T21:50:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f032eb3e640 1 -- 172.21.15.12:0/2510088427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0318008da0 msgr2=0x7f031800b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f032eb3e640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0318008da0 0x7f031800b190 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0314007fa0 tx=0x7f031401cae0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f032eb3e640 1 -- 172.21.15.12:0/2510088427 shutdown_connections 2023-11-28T21:50:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f032eb3e640 1 --2- 172.21.15.12:0/2510088427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0318008da0 0x7f031800b190 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f032eb3e640 1 -- 172.21.15.12:0/2510088427 >> 172.21.15.12:0/2510088427 conn(0x7f0328066360 msgr2=0x7f0328066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:23.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f032eb3e640 1 -- 172.21.15.12:0/2510088427 shutdown_connections 2023-11-28T21:50:23.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f032eb3e640 1 -- 172.21.15.12:0/2510088427 wait complete. 2023-11-28T21:50:23.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f032eb3e640 1 Processor -- start 2023-11-28T21:50:23.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.036+0000 7f032eb3e640 1 -- start start 2023-11-28T21:50:23.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032eb3e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0318008da0 0x7f0328153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:23.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032eb3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0328153840 0x7f0328071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:23.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032eb3e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f032815a070 0x7f0328072020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:23.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032eb3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280716c0 con 0x7f0328153840 2023-11-28T21:50:23.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032eb3e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0328070db0 con 0x7f032815a070 2023-11-28T21:50:23.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032eb3e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032814a480 con 0x7f0318008da0 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f0327fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0328153840 0x7f0328071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f0327fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0328153840 0x7f0328071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45838/0 (socket says 172.21.15.12:45838) 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f0327fff640 1 -- 172.21.15.12:0/188392749 learned_addr learned my addr 172.21.15.12:0/188392749 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032c8da640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0318008da0 0x7f0328153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032d0db640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f032815a070 0x7f0328072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f0327fff640 1 -- 172.21.15.12:0/188392749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0318008da0 msgr2=0x7f0328153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f0327fff640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0318008da0 0x7f0328153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f0327fff640 1 -- 172.21.15.12:0/188392749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f032815a070 msgr2=0x7f0328072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f0327fff640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f032815a070 0x7f0328072020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f0327fff640 1 -- 172.21.15.12:0/188392749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328155850 con 0x7f0328153840 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032d0db640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f032815a070 0x7f0328072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:50:23.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f0327fff640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0328153840 0x7f0328071ae0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0314007fa0 tx=0x7f0318003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f03257fa640 1 -- 172.21.15.12:0/188392749 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140b4070 con 0x7f0328153840 2023-11-28T21:50:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f03257fa640 1 -- 172.21.15.12:0/188392749 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03140a59f0 con 0x7f0328153840 2023-11-28T21:50:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032eb3e640 1 -- 172.21.15.12:0/188392749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0328155b30 con 0x7f0328153840 2023-11-28T21:50:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f03257fa640 1 -- 172.21.15.12:0/188392749 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140a5d60 con 0x7f0328153840 2023-11-28T21:50:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032eb3e640 1 -- 172.21.15.12:0/188392749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03280712e0 con 0x7f0328153840 2023-11-28T21:50:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f032eb3e640 1 -- 172.21.15.12:0/188392749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0328074f60 con 0x7f0328153840 2023-11-28T21:50:23.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.040+0000 7f03257fa640 1 -- 172.21.15.12:0/188392749 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0314007070 con 0x7f0328153840 2023-11-28T21:50:23.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.044+0000 7f03257fa640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f030c066470 0x7f030c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:23.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.044+0000 7f03257fa640 1 -- 172.21.15.12:0/188392749 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f031411e5d0 con 0x7f0328153840 2023-11-28T21:50:23.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.044+0000 7f032c8da640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f030c066470 0x7f030c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:23.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.044+0000 7f032c8da640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f030c066470 0x7f030c068930 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f031c009b80 tx=0x7f031c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:23.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.048+0000 7f03257fa640 1 -- 172.21.15.12:0/188392749 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03140eb420 con 0x7f0328153840 2023-11-28T21:50:23.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 -- 172.21.15.12:0/188392749 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03280c7320 con 0x7f0328153840 2023-11-28T21:50:23.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f03257fa640 1 -- 172.21.15.12:0/188392749 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+6516 (secure 0 0 0) 0x7f03140eea40 con 0x7f0328153840 2023-11-28T21:50:23.239 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:23.240 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:22.662874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6177,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:23.240 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2023-11-28T21:50:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 -- 172.21.15.12:0/188392749 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f030c066470 msgr2=0x7f030c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f030c066470 0x7f030c068930 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f031c009b80 tx=0x7f031c009190 comp rx=0 tx=0).stop 2023-11-28T21:50:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 -- 172.21.15.12:0/188392749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0328153840 msgr2=0x7f0328071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0328153840 0x7f0328071ae0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0314007fa0 tx=0x7f0318003700 comp rx=0 tx=0).stop 2023-11-28T21:50:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 -- 172.21.15.12:0/188392749 shutdown_connections 2023-11-28T21:50:23.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f030c066470 0x7f030c068930 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f032815a070 0x7f0328072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0328153840 0x7f0328071ae0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 --2- 172.21.15.12:0/188392749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0318008da0 0x7f0328153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 -- 172.21.15.12:0/188392749 >> 172.21.15.12:0/188392749 conn(0x7f0328066360 msgr2=0x7f03280ca790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 -- 172.21.15.12:0/188392749 shutdown_connections 2023-11-28T21:50:23.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:23.236+0000 7f032eb3e640 1 -- 172.21.15.12:0/188392749 wait complete. 2023-11-28T21:50:23.260 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:50:25.261 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:31.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bfb2a640 1 Processor -- start 2023-11-28T21:50:31.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bfb2a640 1 -- start start 2023-11-28T21:50:31.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bfb2a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f10b8151af0 0x7f10b80c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:31.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bfb2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b806e300 0x7f10b8072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:31.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bfb2a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b8157b70 con 0x7f10b8151ed0 2023-11-28T21:50:31.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bfb2a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b80707f0 con 0x7f10b8151af0 2023-11-28T21:50:31.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bfb2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80c7b90 con 0x7f10b806e300 2023-11-28T21:50:31.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10be0c7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b806e300 0x7f10b8072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:31.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bd8c6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f10b8151af0 0x7f10b80c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:31.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10be0c7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b806e300 0x7f10b8072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:41224/0 (socket says 172.21.15.12:41224) 2023-11-28T21:50:31.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10be0c7640 1 -- 172.21.15.12:0/490742950 learned_addr learned my addr 172.21.15.12:0/490742950 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:31.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bd8c6640 1 -- 172.21.15.12:0/490742950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b806e300 msgr2=0x7f10b8072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:31.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bd8c6640 1 --2- 172.21.15.12:0/490742950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b806e300 0x7f10b8072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:31.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bd8c6640 1 -- 172.21.15.12:0/490742950 >> v1:172.21.15.144:6789/0 conn(0x7f10b8151ed0 legacy=0x7f10b80c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:31.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10bd8c6640 1 -- 172.21.15.12:0/490742950 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b8157e50 con 0x7f10b8151af0 2023-11-28T21:50:31.561 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.556+0000 7f10be0c7640 1 --2- 172.21.15.12:0/490742950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b806e300 0x7f10b8072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:50:31.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bd8c6640 1 --2- 172.21.15.12:0/490742950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10b8151af0 0x7f10b80c7650 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f10b40097c0 tx=0x7f10b401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a729daa4bd02f91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:31.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bc8c4640 1 -- 172.21.15.12:0/490742950 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40ab600 con 0x7f10b8151af0 2023-11-28T21:50:31.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bc8c4640 1 -- 172.21.15.12:0/490742950 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b40a7050 con 0x7f10b8151af0 2023-11-28T21:50:31.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bc8c4640 1 -- 172.21.15.12:0/490742950 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40ab980 con 0x7f10b8151af0 2023-11-28T21:50:31.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bfb2a640 1 -- 172.21.15.12:0/490742950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10b8151af0 msgr2=0x7f10b80c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:31.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bfb2a640 1 --2- 172.21.15.12:0/490742950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10b8151af0 0x7f10b80c7650 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f10b40097c0 tx=0x7f10b401c480 comp rx=0 tx=0).stop 2023-11-28T21:50:31.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bfb2a640 1 -- 172.21.15.12:0/490742950 shutdown_connections 2023-11-28T21:50:31.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bfb2a640 1 --2- 172.21.15.12:0/490742950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b806e300 0x7f10b8072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bfb2a640 1 --2- 172.21.15.12:0/490742950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10b8151af0 0x7f10b80c7650 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bfb2a640 1 -- 172.21.15.12:0/490742950 >> 172.21.15.12:0/490742950 conn(0x7f10b8066360 msgr2=0x7f10b8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bfb2a640 1 -- 172.21.15.12:0/490742950 shutdown_connections 2023-11-28T21:50:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.560+0000 7f10bfb2a640 1 -- 172.21.15.12:0/490742950 wait complete. 2023-11-28T21:50:31.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bfb2a640 1 Processor -- start 2023-11-28T21:50:31.567 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bfb2a640 1 -- start start 2023-11-28T21:50:31.570 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bfb2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10b806e300 0x7f10b814b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:31.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bfb2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10b8151af0 0x7f10b81f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:31.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bfb2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b81576d0 0x7f10b81f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:31.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bfb2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81f4ee0 con 0x7f10b806e300 2023-11-28T21:50:31.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bfb2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81499f0 con 0x7f10b8151af0 2023-11-28T21:50:31.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bfb2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b81f4c00 con 0x7f10b81576d0 2023-11-28T21:50:31.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bd8c6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10b806e300 0x7f10b814b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:31.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bd8c6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10b806e300 0x7f10b814b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34132/0 (socket says 172.21.15.12:34132) 2023-11-28T21:50:31.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bd8c6640 1 -- 172.21.15.12:0/2349531287 learned_addr learned my addr 172.21.15.12:0/2349531287 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:31.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bd0c5640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10b8151af0 0x7f10b81f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:31.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10be0c7640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b81576d0 0x7f10b81f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:31.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bd8c6640 1 -- 172.21.15.12:0/2349531287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b81576d0 msgr2=0x7f10b81f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:31.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bd8c6640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b81576d0 0x7f10b81f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:31.572 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bd8c6640 1 -- 172.21.15.12:0/2349531287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10b8151af0 msgr2=0x7f10b81f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:31.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bd8c6640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10b8151af0 0x7f10b81f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:31.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bd8c6640 1 -- 172.21.15.12:0/2349531287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b0009b30 con 0x7f10b806e300 2023-11-28T21:50:31.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bd8c6640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10b806e300 0x7f10b814b570 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f10b401a0b0 tx=0x7f10b4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:31.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10ae7fc640 1 -- 172.21.15.12:0/2349531287 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40b4410 con 0x7f10b806e300 2023-11-28T21:50:31.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10ae7fc640 1 -- 172.21.15.12:0/2349531287 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b40b4570 con 0x7f10b806e300 2023-11-28T21:50:31.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10ae7fc640 1 -- 172.21.15.12:0/2349531287 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40b4920 con 0x7f10b806e300 2023-11-28T21:50:31.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bfb2a640 1 -- 172.21.15.12:0/2349531287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b4009420 con 0x7f10b806e300 2023-11-28T21:50:31.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.564+0000 7f10bfb2a640 1 -- 172.21.15.12:0/2349531287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b8148510 con 0x7f10b806e300 2023-11-28T21:50:31.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.568+0000 7f10bfb2a640 1 -- 172.21.15.12:0/2349531287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b8141860 con 0x7f10b806e300 2023-11-28T21:50:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.568+0000 7f10ae7fc640 1 -- 172.21.15.12:0/2349531287 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f10b40b2040 con 0x7f10b806e300 2023-11-28T21:50:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.568+0000 7f10ae7fc640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f109c0664f0 0x7f109c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.568+0000 7f10ae7fc640 1 -- 172.21.15.12:0/2349531287 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7f10b411e3c0 con 0x7f10b806e300 2023-11-28T21:50:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.568+0000 7f10bd0c5640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f109c0664f0 0x7f109c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:31.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.568+0000 7f10bd0c5640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f109c0664f0 0x7f109c0689b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f10b8143050 tx=0x7f10a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:31.577 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.572+0000 7f10ae7fc640 1 -- 172.21.15.12:0/2349531287 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f10b8141860 con 0x7f10b806e300 2023-11-28T21:50:31.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 -- 172.21.15.12:0/2349531287 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10b8141860 con 0x7f10b806e300 2023-11-28T21:50:31.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10ae7fc640 1 -- 172.21.15.12:0/2349531287 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+6516 (secure 0 0 0) 0x7f10b8141860 con 0x7f10b806e300 2023-11-28T21:50:31.767 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:31.767 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:22.662874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6177,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:31.768 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2023-11-28T21:50:31.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 -- 172.21.15.12:0/2349531287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f109c0664f0 msgr2=0x7f109c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:31.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f109c0664f0 0x7f109c0689b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f10b8143050 tx=0x7f10a8009040 comp rx=0 tx=0).stop 2023-11-28T21:50:31.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 -- 172.21.15.12:0/2349531287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10b806e300 msgr2=0x7f10b814b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:31.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10b806e300 0x7f10b814b570 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f10b401a0b0 tx=0x7f10b4019220 comp rx=0 tx=0).stop 2023-11-28T21:50:31.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 -- 172.21.15.12:0/2349531287 shutdown_connections 2023-11-28T21:50:31.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f109c0664f0 0x7f109c0689b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:31.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b81576d0 0x7f10b81f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:31.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10b8151af0 0x7f10b81f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:31.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 --2- 172.21.15.12:0/2349531287 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f10b806e300 0x7f10b814b570 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:31.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 -- 172.21.15.12:0/2349531287 >> 172.21.15.12:0/2349531287 conn(0x7f10b8066360 msgr2=0x7f10b806cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:31.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 -- 172.21.15.12:0/2349531287 shutdown_connections 2023-11-28T21:50:31.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:31.764+0000 7f10bfb2a640 1 -- 172.21.15.12:0/2349531287 wait complete. 2023-11-28T21:50:31.788 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:50:33.789 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:33.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.940+0000 7fdb34fb8640 1 Processor -- start 2023-11-28T21:50:33.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.940+0000 7fdb34fb8640 1 -- start start 2023-11-28T21:50:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb34fb8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30150c20 0x7fdb30156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb34fb8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb3006e300 0x7fdb300c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb34fb8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb30067410 con 0x7fdb3006cf40 2023-11-28T21:50:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb34fb8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3006e090 con 0x7fdb3006e300 2023-11-28T21:50:33.946 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb34fb8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30157f30 con 0x7fdb30150c20 2023-11-28T21:50:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2ed9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdb3006cf40 0x7fdb30158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55052/0 (socket says 172.21.15.12:55052) 2023-11-28T21:50:33.947 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2ed9d640 1 -- 172.21.15.12:0/2950887928 learned_addr learned my addr 172.21.15.12:0/2950887928 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:50:33.948 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2e59c640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30150c20 0x7fdb30156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb27fff640 1 --2- 172.21.15.12:0/2950887928 >> v2:172.21.15.144:3300/0 conn(0x7fdb3006e300 0x7fdb300c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570170248 0 0) 0x7fdb30067410 con 0x7fdb3006cf40 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb1c005740 con 0x7fdb3006cf40 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1979803516 0 0) 0x7fdb1c005740 con 0x7fdb3006cf40 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30150c20 msgr2=0x7fdb30156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30150c20 0x7fdb30156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 >> v2:172.21.15.144:3300/0 conn(0x7fdb3006e300 msgr2=0x7fdb300c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 --2- 172.21.15.12:0/2950887928 >> v2:172.21.15.144:3300/0 conn(0x7fdb3006e300 0x7fdb300c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb30151000 con 0x7fdb3006cf40 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fdb28001a00 con 0x7fdb3006cf40 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 >> v1:172.21.15.12:6789/0 conn(0x7fdb3006cf40 legacy=0x7fdb30158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:33.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30150c20 0x7fdb1c005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb1c0061d0 0x7fdb1c0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c006f20 0x7fdb1c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3006e090 con 0x7fdb1c0061d0 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30157f30 con 0x7fdb1c006f20 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c0035b0 con 0x7fdb30150c20 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb27fff640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb1c0061d0 0x7fdb1c0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2e59c640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30150c20 0x7fdb1c005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2ed9d640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c006f20 0x7fdb1c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb27fff640 1 -- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30150c20 msgr2=0x7fdb1c005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb27fff640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb30150c20 0x7fdb1c005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb27fff640 1 -- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c006f20 msgr2=0x7fdb1c0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:33.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb27fff640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c006f20 0x7fdb1c0073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb27fff640 1 -- 172.21.15.12:0/2950887928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c005740 con 0x7fdb1c0061d0 2023-11-28T21:50:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.944+0000 7fdb2ed9d640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c006f20 0x7fdb1c0073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:50:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb27fff640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb1c0061d0 0x7fdb1c0069e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009ae0 tx=0x7fdb1801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bd1a1315039b52 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb180a5570 con 0x7fdb1c0061d0 2023-11-28T21:50:33.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb180a56d0 con 0x7fdb1c0061d0 2023-11-28T21:50:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb2dd9b640 1 -- 172.21.15.12:0/2950887928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb180a5a80 con 0x7fdb1c0061d0 2023-11-28T21:50:33.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2950887928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb1c0061d0 msgr2=0x7fdb1c0069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:33.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb34fb8640 1 --2- 172.21.15.12:0/2950887928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb1c0061d0 0x7fdb1c0069e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009ae0 tx=0x7fdb1801ca90 comp rx=0 tx=0).stop 2023-11-28T21:50:33.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb2ed9d640 1 -- 172.21.15.12:0/2950887928 reap_dead start 2023-11-28T21:50:33.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2950887928 shutdown_connections 2023-11-28T21:50:33.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2950887928 >> 172.21.15.12:0/2950887928 conn(0x7fdb30066360 msgr2=0x7fdb30066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:33.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2950887928 shutdown_connections 2023-11-28T21:50:33.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.948+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2950887928 wait complete. 2023-11-28T21:50:33.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 Processor -- start 2023-11-28T21:50:33.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 -- start start 2023-11-28T21:50:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0061d0 0x7fdb301ece90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb301ed3d0 0x7fdb3014f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb3014fef0 0x7fdb301eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30154020 con 0x7fdb301ed3d0 2023-11-28T21:50:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb301521c0 con 0x7fdb1c0061d0 2023-11-28T21:50:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb301ed810 con 0x7fdb3014fef0 2023-11-28T21:50:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb301ed3d0 0x7fdb3014f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb301ed3d0 0x7fdb3014f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34140/0 (socket says 172.21.15.12:34140) 2023-11-28T21:50:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb27fff640 1 -- 172.21.15.12:0/2189143768 learned_addr learned my addr 172.21.15.12:0/2189143768 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:33.956 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb2e59c640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0061d0 0x7fdb301ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb2ed9d640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb3014fef0 0x7fdb301eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb27fff640 1 -- 172.21.15.12:0/2189143768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb3014fef0 msgr2=0x7fdb301eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb27fff640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb3014fef0 0x7fdb301eefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb27fff640 1 -- 172.21.15.12:0/2189143768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0061d0 msgr2=0x7fdb301ece90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb27fff640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0061d0 0x7fdb301ece90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb27fff640 1 -- 172.21.15.12:0/2189143768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb301523f0 con 0x7fdb301ed3d0 2023-11-28T21:50:33.957 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb27fff640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb301ed3d0 0x7fdb3014f9b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009ae0 tx=0x7fdb1c007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb26ffd640 1 -- 172.21.15.12:0/2189143768 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb180be070 con 0x7fdb301ed3d0 2023-11-28T21:50:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb26ffd640 1 -- 172.21.15.12:0/2189143768 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb18004020 con 0x7fdb301ed3d0 2023-11-28T21:50:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb26ffd640 1 -- 172.21.15.12:0/2189143768 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb18004390 con 0x7fdb301ed3d0 2023-11-28T21:50:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2189143768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb301526d0 con 0x7fdb301ed3d0 2023-11-28T21:50:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2189143768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb30152c90 con 0x7fdb301ed3d0 2023-11-28T21:50:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.952+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2189143768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb30074f60 con 0x7fdb301ed3d0 2023-11-28T21:50:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.956+0000 7fdb26ffd640 1 -- 172.21.15.12:0/2189143768 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdb180b4040 con 0x7fdb301ed3d0 2023-11-28T21:50:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.956+0000 7fdb26ffd640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb10066470 0x7fdb10068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:33.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.956+0000 7fdb26ffd640 1 -- 172.21.15.12:0/2189143768 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7fdb1811df70 con 0x7fdb301ed3d0 2023-11-28T21:50:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.956+0000 7fdb2e59c640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb10066470 0x7fdb10068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:33.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.956+0000 7fdb2e59c640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb10066470 0x7fdb10068930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009b00 tx=0x7fdb20009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:33.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:33.960+0000 7fdb26ffd640 1 -- 172.21.15.12:0/2189143768 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb180ead30 con 0x7fdb301ed3d0 2023-11-28T21:50:34.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2189143768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb30154fd0 con 0x7fdb301ed3d0 2023-11-28T21:50:34.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb26ffd640 1 -- 172.21.15.12:0/2189143768 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+6516 (secure 0 0 0) 0x7fdb180ee350 con 0x7fdb301ed3d0 2023-11-28T21:50:34.156 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:34.156 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:22.662874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6177,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:34.156 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2023-11-28T21:50:34.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2189143768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb10066470 msgr2=0x7fdb10068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:34.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb10066470 0x7fdb10068930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009b00 tx=0x7fdb20009190 comp rx=0 tx=0).stop 2023-11-28T21:50:34.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2189143768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb301ed3d0 msgr2=0x7fdb3014f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb301ed3d0 0x7fdb3014f9b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009ae0 tx=0x7fdb1c007a10 comp rx=0 tx=0).stop 2023-11-28T21:50:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2189143768 shutdown_connections 2023-11-28T21:50:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdb10066470 0x7fdb10068930 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb3014fef0 0x7fdb301eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb301ed3d0 0x7fdb3014f9b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 --2- 172.21.15.12:0/2189143768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0061d0 0x7fdb301ece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2189143768 >> 172.21.15.12:0/2189143768 conn(0x7fdb30066360 msgr2=0x7fdb300ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2189143768 shutdown_connections 2023-11-28T21:50:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:34.152+0000 7fdb34fb8640 1 -- 172.21.15.12:0/2189143768 wait complete. 2023-11-28T21:50:34.177 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:50:36.178 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:36.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.327+0000 7faed269c640 1 Processor -- start 2023-11-28T21:50:36.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.327+0000 7faed269c640 1 -- start start 2023-11-28T21:50:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faed269c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faecc1582c0 0x7faecc06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faed269c640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faecc0728b0 con 0x7faecc0c6100 2023-11-28T21:50:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faed269c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faecc0c5c70 con 0x7faecc0c6d50 2023-11-28T21:50:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faed269c640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faecc0c4c50 con 0x7faecc1582c0 2023-11-28T21:50:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faed1e9b640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7faecc0c6100 0x7faecc0c6530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33584/0 (socket says 172.21.15.12:33584) 2023-11-28T21:50:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faed169a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faecc1582c0 0x7faecc06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:36.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faed1e9b640 1 -- 172.21.15.12:0/708496683 learned_addr learned my addr 172.21.15.12:0/708496683 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:50:36.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faed169a640 1 -- 172.21.15.12:0/708496683 >> v1:172.21.15.144:6789/0 conn(0x7faecc0c6d50 legacy=0x7faecc06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:36.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faed169a640 1 -- 172.21.15.12:0/708496683 >> v1:172.21.15.12:6789/0 conn(0x7faecc0c6100 legacy=0x7faecc0c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:36.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faec3fff640 1 -- 172.21.15.12:0/708496683 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793910794 0 0) 0x7faecc0728b0 con 0x7faecc0c6100 2023-11-28T21:50:36.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.331+0000 7faed169a640 1 -- 172.21.15.12:0/708496683 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc15ef00 con 0x7faecc1582c0 2023-11-28T21:50:36.338 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faed169a640 1 --2- 172.21.15.12:0/708496683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc1582c0 0x7faecc06b580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7faec80097c0 tx=0x7faec801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95351857ccd07608 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:36.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faec3fff640 1 -- 172.21.15.12:0/708496683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec80ab600 con 0x7faecc1582c0 2023-11-28T21:50:36.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faec3fff640 1 -- 172.21.15.12:0/708496683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec80a7050 con 0x7faecc1582c0 2023-11-28T21:50:36.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faec3fff640 1 -- 172.21.15.12:0/708496683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec80ab980 con 0x7faecc1582c0 2023-11-28T21:50:36.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faed269c640 1 -- 172.21.15.12:0/708496683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc1582c0 msgr2=0x7faecc06b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:36.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faed269c640 1 --2- 172.21.15.12:0/708496683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc1582c0 0x7faecc06b580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7faec80097c0 tx=0x7faec801c3f0 comp rx=0 tx=0).stop 2023-11-28T21:50:36.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faed269c640 1 -- 172.21.15.12:0/708496683 shutdown_connections 2023-11-28T21:50:36.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faed269c640 1 --2- 172.21.15.12:0/708496683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc1582c0 0x7faecc06b580 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:36.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faed269c640 1 -- 172.21.15.12:0/708496683 >> 172.21.15.12:0/708496683 conn(0x7faecc066360 msgr2=0x7faecc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:36.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faed269c640 1 -- 172.21.15.12:0/708496683 shutdown_connections 2023-11-28T21:50:36.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faed269c640 1 -- 172.21.15.12:0/708496683 wait complete. 2023-11-28T21:50:36.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faed269c640 1 Processor -- start 2023-11-28T21:50:36.341 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.335+0000 7faed269c640 1 -- start start 2023-11-28T21:50:36.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed269c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faecc0c6100 0x7faecc15d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:36.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed269c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc0c6d50 0x7faecc152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:36.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed269c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faecc1582c0 0x7faecc152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:36.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed269c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc1f7940 con 0x7faecc0c6d50 2023-11-28T21:50:36.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed269c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc151650 con 0x7faecc0c6100 2023-11-28T21:50:36.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed269c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faecc151930 con 0x7faecc1582c0 2023-11-28T21:50:36.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed0e99640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc0c6d50 0x7faecc152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed0e99640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc0c6d50 0x7faecc152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56662/0 (socket says 172.21.15.12:56662) 2023-11-28T21:50:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed0e99640 1 -- 172.21.15.12:0/126388160 learned_addr learned my addr 172.21.15.12:0/126388160 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed169a640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faecc0c6100 0x7faecc15d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed1e9b640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faecc1582c0 0x7faecc152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed0e99640 1 -- 172.21.15.12:0/126388160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faecc1582c0 msgr2=0x7faecc152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed0e99640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faecc1582c0 0x7faecc152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed0e99640 1 -- 172.21.15.12:0/126388160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faecc0c6100 msgr2=0x7faecc15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed0e99640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faecc0c6100 0x7faecc15d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:36.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed0e99640 1 -- 172.21.15.12:0/126388160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec8009420 con 0x7faecc0c6d50 2023-11-28T21:50:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed0e99640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc0c6d50 0x7faecc152770 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7faebc00c3c0 tx=0x7faebc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faec1ffb640 1 -- 172.21.15.12:0/126388160 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc0052d0 con 0x7faecc0c6d50 2023-11-28T21:50:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faec1ffb640 1 -- 172.21.15.12:0/126388160 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc005430 con 0x7faecc0c6d50 2023-11-28T21:50:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faec1ffb640 1 -- 172.21.15.12:0/126388160 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc003840 con 0x7faecc0c6d50 2023-11-28T21:50:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed269c640 1 -- 172.21.15.12:0/126388160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faecc151c10 con 0x7faecc0c6d50 2023-11-28T21:50:36.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed269c640 1 -- 172.21.15.12:0/126388160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faecc1521d0 con 0x7faecc0c6d50 2023-11-28T21:50:36.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.339+0000 7faed269c640 1 -- 172.21.15.12:0/126388160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faecc067c70 con 0x7faecc0c6d50 2023-11-28T21:50:36.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.343+0000 7faec1ffb640 1 -- 172.21.15.12:0/126388160 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faebc0039a0 con 0x7faecc0c6d50 2023-11-28T21:50:36.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.343+0000 7faec1ffb640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faeb00664a0 0x7faeb0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:36.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.343+0000 7faec1ffb640 1 -- 172.21.15.12:0/126388160 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7265+0+0 (secure 0 0 0) 0x7faebc083a70 con 0x7faecc0c6d50 2023-11-28T21:50:36.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.343+0000 7faed169a640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faeb00664a0 0x7faeb0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:36.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.343+0000 7faed169a640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faeb00664a0 0x7faeb0068960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faec80ad000 tx=0x7faec80a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:36.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.347+0000 7faec1ffb640 1 -- 172.21.15.12:0/126388160 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faebc050850 con 0x7faecc0c6d50 2023-11-28T21:50:36.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.539+0000 7faed269c640 1 -- 172.21.15.12:0/126388160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faecc1523f0 con 0x7faecc0c6d50 2023-11-28T21:50:36.547 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faec1ffb640 1 -- 172.21.15.12:0/126388160 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+6516 (secure 0 0 0) 0x7faebc017090 con 0x7faecc0c6d50 2023-11-28T21:50:36.547 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:36.547 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5057,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:22.662874+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":49,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6177,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":34,"addr":"172.21.15.12:6838/1551825785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1551825785},{"type":"v1","addr":"172.21.15.12:6838","nonce":1551825785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:36.549 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 64 2023-11-28T21:50:36.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 -- 172.21.15.12:0/126388160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faeb00664a0 msgr2=0x7faeb0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:36.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faeb00664a0 0x7faeb0068960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faec80ad000 tx=0x7faec80a6000 comp rx=0 tx=0).stop 2023-11-28T21:50:36.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 -- 172.21.15.12:0/126388160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc0c6d50 msgr2=0x7faecc152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:36.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc0c6d50 0x7faecc152770 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7faebc00c3c0 tx=0x7faebc00c880 comp rx=0 tx=0).stop 2023-11-28T21:50:36.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 -- 172.21.15.12:0/126388160 shutdown_connections 2023-11-28T21:50:36.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faeb00664a0 0x7faeb0068960 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:36.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faecc1582c0 0x7faecc152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:36.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faecc0c6d50 0x7faecc152770 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:36.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 --2- 172.21.15.12:0/126388160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faecc0c6100 0x7faecc15d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:36.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 -- 172.21.15.12:0/126388160 >> 172.21.15.12:0/126388160 conn(0x7faecc066360 msgr2=0x7faecc0c9250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:36.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 -- 172.21.15.12:0/126388160 shutdown_connections 2023-11-28T21:50:36.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:36.543+0000 7faed269c640 1 -- 172.21.15.12:0/126388160 wait complete. 2023-11-28T21:50:36.567 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2023-11-28T21:50:38.569 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:38.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870f916640 1 Processor -- start 2023-11-28T21:50:38.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870f916640 1 -- start start 2023-11-28T21:50:38.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870f916640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8708151af0 0x7f87080c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:38.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870f916640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f870806e300 0x7f8708072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:38.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870f916640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8708157b70 con 0x7f8708151ed0 2023-11-28T21:50:38.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870f916640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87080707f0 con 0x7f870806e300 2023-11-28T21:50:38.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870f916640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87080c7b90 con 0x7f8708151af0 2023-11-28T21:50:38.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870deb3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f870806e300 0x7f8708072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:38.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870deb3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f870806e300 0x7f8708072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56674/0 (socket says 172.21.15.12:56674) 2023-11-28T21:50:38.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870deb3640 1 -- 172.21.15.12:0/2394852380 learned_addr learned my addr 172.21.15.12:0/2394852380 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:38.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870ceb1640 1 --1- 172.21.15.12:0/2394852380 >> v1:172.21.15.12:6789/0 conn(0x7f8708151ed0 0x7f87080c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33586/0 (socket says 172.21.15.12:33586) 2023-11-28T21:50:38.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870d6b2640 1 --2- 172.21.15.12:0/2394852380 >> v2:172.21.15.144:3300/0 conn(0x7f8708151af0 0x7f87080c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:38.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870deb3640 1 -- 172.21.15.12:0/2394852380 >> v2:172.21.15.144:3300/0 conn(0x7f8708151af0 msgr2=0x7f87080c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:38.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870deb3640 1 --2- 172.21.15.12:0/2394852380 >> v2:172.21.15.144:3300/0 conn(0x7f8708151af0 0x7f87080c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:38.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870deb3640 1 -- 172.21.15.12:0/2394852380 >> v1:172.21.15.12:6789/0 conn(0x7f8708151ed0 legacy=0x7f87080c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:38.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.723+0000 7f870deb3640 1 -- 172.21.15.12:0/2394852380 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8708157e50 con 0x7f870806e300 2023-11-28T21:50:38.732 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f870deb3640 1 --2- 172.21.15.12:0/2394852380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 0x7f8708072070 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f87080c7d20 tx=0x7f870001c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a4eca174c6d429 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:38.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f86fffff640 1 -- 172.21.15.12:0/2394852380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87000a56c0 con 0x7f870806e300 2023-11-28T21:50:38.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f86fffff640 1 -- 172.21.15.12:0/2394852380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87000a5820 con 0x7f870806e300 2023-11-28T21:50:38.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f86fffff640 1 -- 172.21.15.12:0/2394852380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87000a5bb0 con 0x7f870806e300 2023-11-28T21:50:38.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f870f916640 1 -- 172.21.15.12:0/2394852380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 msgr2=0x7f8708072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:38.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f870f916640 1 --2- 172.21.15.12:0/2394852380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 0x7f8708072070 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f87080c7d20 tx=0x7f870001c860 comp rx=0 tx=0).stop 2023-11-28T21:50:38.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f870f916640 1 -- 172.21.15.12:0/2394852380 shutdown_connections 2023-11-28T21:50:38.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f870f916640 1 --2- 172.21.15.12:0/2394852380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 0x7f8708072070 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:38.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f870f916640 1 --2- 172.21.15.12:0/2394852380 >> v2:172.21.15.144:3300/0 conn(0x7f8708151af0 0x7f87080c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:38.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f870f916640 1 -- 172.21.15.12:0/2394852380 >> 172.21.15.12:0/2394852380 conn(0x7f8708066360 msgr2=0x7f8708066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:38.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f870f916640 1 -- 172.21.15.12:0/2394852380 shutdown_connections 2023-11-28T21:50:38.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.727+0000 7f870f916640 1 -- 172.21.15.12:0/2394852380 wait complete. 2023-11-28T21:50:38.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870f916640 1 Processor -- start 2023-11-28T21:50:38.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870f916640 1 -- start start 2023-11-28T21:50:38.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870f916640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 0x7f87081f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:38.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870f916640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8708151af0 0x7f8708155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:38.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870f916640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87081f3480 0x7f8708141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:38.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870f916640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87081f3c60 con 0x7f870806e300 2023-11-28T21:50:38.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870f916640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870814dd00 con 0x7f87081f3480 2023-11-28T21:50:38.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870f916640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87081f39a0 con 0x7f8708151af0 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870d6b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 0x7f87081f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870d6b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 0x7f87081f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56680/0 (socket says 172.21.15.12:56680) 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870d6b2640 1 -- 172.21.15.12:0/4102133773 learned_addr learned my addr 172.21.15.12:0/4102133773 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870deb3640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87081f3480 0x7f8708141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870ceb1640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8708151af0 0x7f8708155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870d6b2640 1 -- 172.21.15.12:0/4102133773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8708151af0 msgr2=0x7f8708155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870d6b2640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8708151af0 0x7f8708155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870d6b2640 1 -- 172.21.15.12:0/4102133773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87081f3480 msgr2=0x7f8708141970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870d6b2640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87081f3480 0x7f8708141970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870d6b2640 1 -- 172.21.15.12:0/4102133773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8700009b30 con 0x7f870806e300 2023-11-28T21:50:38.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870deb3640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87081f3480 0x7f8708141970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:50:38.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870d6b2640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 0x7f87081f2f40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f870400d800 tx=0x7f870400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:38.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f86fdffb640 1 -- 172.21.15.12:0/4102133773 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870400b660 con 0x7f870806e300 2023-11-28T21:50:38.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f86fdffb640 1 -- 172.21.15.12:0/4102133773 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870400b7c0 con 0x7f870806e300 2023-11-28T21:50:38.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f86fdffb640 1 -- 172.21.15.12:0/4102133773 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870400bb30 con 0x7f870806e300 2023-11-28T21:50:38.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870f916640 1 -- 172.21.15.12:0/4102133773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8708155970 con 0x7f870806e300 2023-11-28T21:50:38.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.731+0000 7f870f916640 1 -- 172.21.15.12:0/4102133773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870814d500 con 0x7f870806e300 2023-11-28T21:50:38.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.735+0000 7f870f916640 1 -- 172.21.15.12:0/4102133773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8708074f60 con 0x7f870806e300 2023-11-28T21:50:38.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.735+0000 7f86fdffb640 1 -- 172.21.15.12:0/4102133773 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f870400bc90 con 0x7f870806e300 2023-11-28T21:50:38.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.735+0000 7f86fdffb640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86ec066470 0x7f86ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:38.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.735+0000 7f86fdffb640 1 -- 172.21.15.12:0/4102133773 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f8704084af0 con 0x7f870806e300 2023-11-28T21:50:38.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.735+0000 7f870ceb1640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86ec066470 0x7f86ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:38.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.739+0000 7f870ceb1640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86ec066470 0x7f86ec068930 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f870814d430 tx=0x7f86f800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:38.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.739+0000 7f86fdffb640 1 -- 172.21.15.12:0/4102133773 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8704051860 con 0x7f870806e300 2023-11-28T21:50:38.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.935+0000 7f870f916640 1 -- 172.21.15.12:0/4102133773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87080c5f60 con 0x7f870806e300 2023-11-28T21:50:38.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.935+0000 7f86fdffb640 1 -- 172.21.15.12:0/4102133773 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5724 (secure 0 0 0) 0x7f8704011340 con 0x7f870806e300 2023-11-28T21:50:38.941 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:38.942 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:38.562204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:38.942 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 65 2023-11-28T21:50:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 -- 172.21.15.12:0/4102133773 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86ec066470 msgr2=0x7f86ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86ec066470 0x7f86ec068930 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f870814d430 tx=0x7f86f800b040 comp rx=0 tx=0).stop 2023-11-28T21:50:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 -- 172.21.15.12:0/4102133773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 msgr2=0x7f87081f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 0x7f87081f2f40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f870400d800 tx=0x7f870400dcc0 comp rx=0 tx=0).stop 2023-11-28T21:50:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 -- 172.21.15.12:0/4102133773 shutdown_connections 2023-11-28T21:50:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f86ec066470 0x7f86ec068930 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87081f3480 0x7f8708141970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8708151af0 0x7f8708155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:38.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 --2- 172.21.15.12:0/4102133773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f870806e300 0x7f87081f2f40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 -- 172.21.15.12:0/4102133773 >> 172.21.15.12:0/4102133773 conn(0x7f8708066360 msgr2=0x7f870814f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 -- 172.21.15.12:0/4102133773 shutdown_connections 2023-11-28T21:50:38.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:38.939+0000 7f870f916640 1 -- 172.21.15.12:0/4102133773 wait complete. 2023-11-28T21:50:38.963 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2023-11-28T21:50:38.963 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:50:38.963 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:39.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2147edd640 1 Processor -- start 2023-11-28T21:50:39.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2147edd640 1 -- start start 2023-11-28T21:50:39.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2147edd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2140151af0 0x7f21400c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:39.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2147edd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140151ed0 0x7f21400c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:39.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2147edd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2140157b70 con 0x7f214006e300 2023-11-28T21:50:39.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2147edd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21400707f0 con 0x7f2140151af0 2023-11-28T21:50:39.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2147edd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21400c7b90 con 0x7f2140151ed0 2023-11-28T21:50:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f214647a640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f214006e300 0x7f2140072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33590/0 (socket says 172.21.15.12:33590) 2023-11-28T21:50:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f214647a640 1 -- 172.21.15.12:0/3184160548 learned_addr learned my addr 172.21.15.12:0/3184160548 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:50:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2145478640 1 --2- 172.21.15.12:0/3184160548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140151ed0 0x7f21400c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2145c79640 1 --2- 172.21.15.12:0/3184160548 >> v2:172.21.15.144:3300/0 conn(0x7f2140151af0 0x7f21400c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2144c77640 1 -- 172.21.15.12:0/3184160548 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264206213 0 0) 0x7f2140157b70 con 0x7f214006e300 2023-11-28T21:50:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2144c77640 1 -- 172.21.15.12:0/3184160548 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f212c003530 con 0x7f214006e300 2023-11-28T21:50:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2145478640 1 -- 172.21.15.12:0/3184160548 >> v2:172.21.15.144:3300/0 conn(0x7f2140151af0 msgr2=0x7f21400c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2145478640 1 --2- 172.21.15.12:0/3184160548 >> v2:172.21.15.144:3300/0 conn(0x7f2140151af0 0x7f21400c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2145478640 1 -- 172.21.15.12:0/3184160548 >> v1:172.21.15.12:6789/0 conn(0x7f214006e300 legacy=0x7f2140072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:39.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.103+0000 7f2145478640 1 -- 172.21.15.12:0/3184160548 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2140157e50 con 0x7f2140151ed0 2023-11-28T21:50:39.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2145478640 1 --2- 172.21.15.12:0/3184160548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140151ed0 0x7f21400c49d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f21300097c0 tx=0x7f213001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de1099acaf7781b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:39.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2144c77640 1 -- 172.21.15.12:0/3184160548 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300ab600 con 0x7f2140151ed0 2023-11-28T21:50:39.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2144c77640 1 -- 172.21.15.12:0/3184160548 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21300a7050 con 0x7f2140151ed0 2023-11-28T21:50:39.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2144c77640 1 -- 172.21.15.12:0/3184160548 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300ab960 con 0x7f2140151ed0 2023-11-28T21:50:39.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2147edd640 1 -- 172.21.15.12:0/3184160548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140151ed0 msgr2=0x7f21400c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:39.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2147edd640 1 --2- 172.21.15.12:0/3184160548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140151ed0 0x7f21400c49d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f21300097c0 tx=0x7f213001c480 comp rx=0 tx=0).stop 2023-11-28T21:50:39.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2147edd640 1 -- 172.21.15.12:0/3184160548 shutdown_connections 2023-11-28T21:50:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2147edd640 1 --2- 172.21.15.12:0/3184160548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140151ed0 0x7f21400c49d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2147edd640 1 --2- 172.21.15.12:0/3184160548 >> v2:172.21.15.144:3300/0 conn(0x7f2140151af0 0x7f21400c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2147edd640 1 -- 172.21.15.12:0/3184160548 >> 172.21.15.12:0/3184160548 conn(0x7f2140066360 msgr2=0x7f2140066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2147edd640 1 -- 172.21.15.12:0/3184160548 shutdown_connections 2023-11-28T21:50:39.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.107+0000 7f2147edd640 1 -- 172.21.15.12:0/3184160548 wait complete. 2023-11-28T21:50:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2147edd640 1 Processor -- start 2023-11-28T21:50:39.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2147edd640 1 -- start start 2023-11-28T21:50:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2147edd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214006e300 0x7f214014cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2147edd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2140151af0 0x7f21401410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2147edd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140143c40 0x7f2140141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2147edd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2140141b90 con 0x7f214006e300 2023-11-28T21:50:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2147edd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21401406c0 con 0x7f2140151af0 2023-11-28T21:50:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2147edd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2140141820 con 0x7f2140143c40 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145c79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214006e300 0x7f214014cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145c79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214006e300 0x7f214014cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56684/0 (socket says 172.21.15.12:56684) 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145c79640 1 -- 172.21.15.12:0/3179553595 learned_addr learned my addr 172.21.15.12:0/3179553595 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f214647a640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140143c40 0x7f2140141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145478640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2140151af0 0x7f21401410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145c79640 1 -- 172.21.15.12:0/3179553595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140143c40 msgr2=0x7f2140141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145c79640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140143c40 0x7f2140141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145c79640 1 -- 172.21.15.12:0/3179553595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2140151af0 msgr2=0x7f21401410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145c79640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2140151af0 0x7f21401410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145c79640 1 -- 172.21.15.12:0/3179553595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2130009420 con 0x7f214006e300 2023-11-28T21:50:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f214647a640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140143c40 0x7f2140141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:50:39.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145478640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2140151af0 0x7f21401410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:50:39.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2145c79640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214006e300 0x7f214014cf20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f213c00e930 tx=0x7f213c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:39.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f21367fc640 1 -- 172.21.15.12:0/3179553595 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c009690 con 0x7f214006e300 2023-11-28T21:50:39.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f21367fc640 1 -- 172.21.15.12:0/3179553595 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c00ca00 con 0x7f214006e300 2023-11-28T21:50:39.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f21367fc640 1 -- 172.21.15.12:0/3179553595 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00cd70 con 0x7f214006e300 2023-11-28T21:50:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2147edd640 1 -- 172.21.15.12:0/3179553595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21400cace0 con 0x7f214006e300 2023-11-28T21:50:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.111+0000 7f2147edd640 1 -- 172.21.15.12:0/3179553595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2140144c70 con 0x7f214006e300 2023-11-28T21:50:39.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.115+0000 7f2147edd640 1 -- 172.21.15.12:0/3179553595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2140141e00 con 0x7f214006e300 2023-11-28T21:50:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.115+0000 7f21367fc640 1 -- 172.21.15.12:0/3179553595 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f213c013470 con 0x7f214006e300 2023-11-28T21:50:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.115+0000 7f21367fc640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2124066470 0x7f2124068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.115+0000 7f21367fc640 1 -- 172.21.15.12:0/3179553595 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f213c084860 con 0x7f214006e300 2023-11-28T21:50:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.115+0000 7f2145478640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2124066470 0x7f2124068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:39.121 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.115+0000 7f2145478640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2124066470 0x7f2124068930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f21300ad000 tx=0x7f21300a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:39.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.119+0000 7f21367fc640 1 -- 172.21.15.12:0/3179553595 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f213c051690 con 0x7f214006e300 2023-11-28T21:50:39.315 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.311+0000 7f2147edd640 1 -- 172.21.15.12:0/3179553595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2140074f60 con 0x7f214006e300 2023-11-28T21:50:39.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.311+0000 7f21367fc640 1 -- 172.21.15.12:0/3179553595 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5724 (secure 0 0 0) 0x7f213c054cb0 con 0x7f214006e300 2023-11-28T21:50:39.316 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:39.316 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:38.562204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:39.317 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 65 2023-11-28T21:50:39.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 -- 172.21.15.12:0/3179553595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2124066470 msgr2=0x7f2124068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:39.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2124066470 0x7f2124068930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f21300ad000 tx=0x7f21300a6000 comp rx=0 tx=0).stop 2023-11-28T21:50:39.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 -- 172.21.15.12:0/3179553595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214006e300 msgr2=0x7f214014cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214006e300 0x7f214014cf20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f213c00e930 tx=0x7f213c00edf0 comp rx=0 tx=0).stop 2023-11-28T21:50:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 -- 172.21.15.12:0/3179553595 shutdown_connections 2023-11-28T21:50:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2124066470 0x7f2124068930 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2140143c40 0x7f2140141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2140151af0 0x7f21401410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 --2- 172.21.15.12:0/3179553595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f214006e300 0x7f214014cf20 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 -- 172.21.15.12:0/3179553595 >> 172.21.15.12:0/3179553595 conn(0x7f2140066360 msgr2=0x7f21401454d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 -- 172.21.15.12:0/3179553595 shutdown_connections 2023-11-28T21:50:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:39.315+0000 7f2147edd640 1 -- 172.21.15.12:0/3179553595 wait complete. 2023-11-28T21:50:39.338 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 65, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 65, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5057": { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 65, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_6330": { "addr": "172.21.15.12:6839/754378087", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 754378087, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 754378087, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6330, "incarnation": 63, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:50:38.562204+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 5057, "mds_1": 6330 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:50:41.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:41.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.491+0000 7fac9d92a640 1 Processor -- start 2023-11-28T21:50:41.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.491+0000 7fac9d92a640 1 -- start start 2023-11-28T21:50:41.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac9d92a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac9806e300 0x7fac98156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:41.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac9d92a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9806cf40 0x7fac98158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:41.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac9d92a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac98067410 con 0x7fac98150c20 2023-11-28T21:50:41.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac9d92a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9806e090 con 0x7fac9806e300 2023-11-28T21:50:41.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac9d92a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac98157f30 con 0x7fac9806cf40 2023-11-28T21:50:41.499 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac8ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac9806e300 0x7fac98156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac8ffff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fac9806e300 0x7fac98156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56698/0 (socket says 172.21.15.12:56698) 2023-11-28T21:50:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac8ffff640 1 -- 172.21.15.12:0/2451697339 learned_addr learned my addr 172.21.15.12:0/2451697339 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac977fe640 1 --2- 172.21.15.12:0/2451697339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9806cf40 0x7fac98158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac8ffff640 1 -- 172.21.15.12:0/2451697339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9806cf40 msgr2=0x7fac98158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac8ffff640 1 --2- 172.21.15.12:0/2451697339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9806cf40 0x7fac98158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac8ffff640 1 -- 172.21.15.12:0/2451697339 >> v1:172.21.15.12:6789/0 conn(0x7fac98150c20 legacy=0x7fac980c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac8ffff640 1 -- 172.21.15.12:0/2451697339 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98151000 con 0x7fac9806e300 2023-11-28T21:50:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.495+0000 7fac967fc640 1 -- 172.21.15.12:0/2451697339 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473344081 0 0) 0x7fac98067410 con 0x7fac98150c20 2023-11-28T21:50:41.501 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac8ffff640 1 --2- 172.21.15.12:0/2451697339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 0x7fac98156910 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fac80009870 tx=0x7fac8001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f5e8aa6d25d239 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:41.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac967fc640 1 -- 172.21.15.12:0/2451697339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac800ab650 con 0x7fac9806e300 2023-11-28T21:50:41.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac967fc640 1 -- 172.21.15.12:0/2451697339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac800ab7b0 con 0x7fac9806e300 2023-11-28T21:50:41.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac967fc640 1 -- 172.21.15.12:0/2451697339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac800abb40 con 0x7fac9806e300 2023-11-28T21:50:41.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 -- 172.21.15.12:0/2451697339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 msgr2=0x7fac98156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:41.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 --2- 172.21.15.12:0/2451697339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 0x7fac98156910 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fac80009870 tx=0x7fac8001c540 comp rx=0 tx=0).stop 2023-11-28T21:50:41.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 -- 172.21.15.12:0/2451697339 shutdown_connections 2023-11-28T21:50:41.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 --2- 172.21.15.12:0/2451697339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9806cf40 0x7fac98158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:41.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 --2- 172.21.15.12:0/2451697339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 0x7fac98156910 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:41.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 -- 172.21.15.12:0/2451697339 >> 172.21.15.12:0/2451697339 conn(0x7fac98066360 msgr2=0x7fac98066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:41.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 -- 172.21.15.12:0/2451697339 shutdown_connections 2023-11-28T21:50:41.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 -- 172.21.15.12:0/2451697339 wait complete. 2023-11-28T21:50:41.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 Processor -- start 2023-11-28T21:50:41.505 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 -- start start 2023-11-28T21:50:41.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9806cf40 0x7fac981462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:41.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 0x7fac98155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:41.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac98150c20 0x7fac9814cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:41.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac981f6070 con 0x7fac9806e300 2023-11-28T21:50:41.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9814f680 con 0x7fac98150c20 2023-11-28T21:50:41.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.499+0000 7fac9d92a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9814f9e0 con 0x7fac9806cf40 2023-11-28T21:50:41.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac8ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 0x7fac98155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:41.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac8ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 0x7fac98155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56700/0 (socket says 172.21.15.12:56700) 2023-11-28T21:50:41.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac8ffff640 1 -- 172.21.15.12:0/3206160094 learned_addr learned my addr 172.21.15.12:0/3206160094 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:41.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac96ffd640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9806cf40 0x7fac981462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:41.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac977fe640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac98150c20 0x7fac9814cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:41.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac8ffff640 1 -- 172.21.15.12:0/3206160094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9806cf40 msgr2=0x7fac981462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:41.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac8ffff640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9806cf40 0x7fac981462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:41.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac8ffff640 1 -- 172.21.15.12:0/3206160094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac98150c20 msgr2=0x7fac9814cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:41.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac8ffff640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac98150c20 0x7fac9814cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:41.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac8ffff640 1 -- 172.21.15.12:0/3206160094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac800094d0 con 0x7fac9806e300 2023-11-28T21:50:41.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac977fe640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac98150c20 0x7fac9814cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:50:41.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac8ffff640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 0x7fac98155e40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fac800ad040 tx=0x7fac8001ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac94ff9640 1 -- 172.21.15.12:0/3206160094 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac800abdf0 con 0x7fac9806e300 2023-11-28T21:50:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac94ff9640 1 -- 172.21.15.12:0/3206160094 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac800b8070 con 0x7fac9806e300 2023-11-28T21:50:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac94ff9640 1 -- 172.21.15.12:0/3206160094 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80004220 con 0x7fac9806e300 2023-11-28T21:50:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac9d92a640 1 -- 172.21.15.12:0/3206160094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac981559c0 con 0x7fac9806e300 2023-11-28T21:50:41.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac9d92a640 1 -- 172.21.15.12:0/3206160094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac98150210 con 0x7fac9806e300 2023-11-28T21:50:41.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac9d92a640 1 -- 172.21.15.12:0/3206160094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac98155b60 con 0x7fac9806e300 2023-11-28T21:50:41.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.503+0000 7fac94ff9640 1 -- 172.21.15.12:0/3206160094 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fac80004380 con 0x7fac9806e300 2023-11-28T21:50:41.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.507+0000 7fac94ff9640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac78066470 0x7fac78068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:41.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.507+0000 7fac94ff9640 1 -- 172.21.15.12:0/3206160094 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fac8011e6b0 con 0x7fac9806e300 2023-11-28T21:50:41.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.507+0000 7fac96ffd640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac78066470 0x7fac78068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:41.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.507+0000 7fac96ffd640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac78066470 0x7fac78068930 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac88007fd0 tx=0x7fac88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:41.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.511+0000 7fac94ff9640 1 -- 172.21.15.12:0/3206160094 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fac98155b60 con 0x7fac9806e300 2023-11-28T21:50:41.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 -- 172.21.15.12:0/3206160094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac98074f60 con 0x7fac9806e300 2023-11-28T21:50:41.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac94ff9640 1 -- 172.21.15.12:0/3206160094 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5724 (secure 0 0 0) 0x7fac800eb5d0 con 0x7fac9806e300 2023-11-28T21:50:41.708 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:41.708 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:38.562204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:41.710 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 65 2023-11-28T21:50:41.710 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 -- 172.21.15.12:0/3206160094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac78066470 msgr2=0x7fac78068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:41.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac78066470 0x7fac78068930 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac88007fd0 tx=0x7fac88009040 comp rx=0 tx=0).stop 2023-11-28T21:50:41.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 -- 172.21.15.12:0/3206160094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 msgr2=0x7fac98155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:41.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 0x7fac98155e40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fac800ad040 tx=0x7fac8001ad80 comp rx=0 tx=0).stop 2023-11-28T21:50:41.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 -- 172.21.15.12:0/3206160094 shutdown_connections 2023-11-28T21:50:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fac78066470 0x7fac78068930 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac98150c20 0x7fac9814cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac9806e300 0x7fac98155e40 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 --2- 172.21.15.12:0/3206160094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9806cf40 0x7fac981462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 -- 172.21.15.12:0/3206160094 >> 172.21.15.12:0/3206160094 conn(0x7fac98066360 msgr2=0x7fac98070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 -- 172.21.15.12:0/3206160094 shutdown_connections 2023-11-28T21:50:41.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:41.703+0000 7fac9d92a640 1 -- 172.21.15.12:0/3206160094 wait complete. 2023-11-28T21:50:41.727 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:50:43.729 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:43.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.879+0000 7f40b4a32640 1 Processor -- start 2023-11-28T21:50:43.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.879+0000 7f40b4a32640 1 -- start start 2023-11-28T21:50:43.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40b4a32640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f40b00cac40 0x7f40b006b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:43.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40b4a32640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f40b00cb020 0x7f40b0159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:43.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40b4a32640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b00c4da0 con 0x7f40b00c4980 2023-11-28T21:50:43.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40b4a32640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b015a5e0 con 0x7f40b00cac40 2023-11-28T21:50:43.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40b4a32640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b015a8c0 con 0x7f40b00cb020 2023-11-28T21:50:43.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40ae59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f40b00cac40 0x7f40b006b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:43.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40ae59c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f40b00cac40 0x7f40b006b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56708/0 (socket says 172.21.15.12:56708) 2023-11-28T21:50:43.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40ae59c640 1 -- 172.21.15.12:0/1212473459 learned_addr learned my addr 172.21.15.12:0/1212473459 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:43.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40aed9d640 1 --2- 172.21.15.12:0/1212473459 >> v2:172.21.15.144:3300/0 conn(0x7f40b00cb020 0x7f40b0159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:43.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40ae59c640 1 -- 172.21.15.12:0/1212473459 >> v2:172.21.15.144:3300/0 conn(0x7f40b00cb020 msgr2=0x7f40b0159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:43.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40ae59c640 1 --2- 172.21.15.12:0/1212473459 >> v2:172.21.15.144:3300/0 conn(0x7f40b00cb020 0x7f40b0159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:43.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40ae59c640 1 -- 172.21.15.12:0/1212473459 >> v1:172.21.15.144:6789/0 conn(0x7f40b00c4980 legacy=0x7f40b00ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:43.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.883+0000 7f40ae59c640 1 -- 172.21.15.12:0/1212473459 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b006a650 con 0x7f40b00cac40 2023-11-28T21:50:43.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40ae59c640 1 --2- 172.21.15.12:0/1212473459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00cac40 0x7f40b006b130 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f40b0068a60 tx=0x7f40a001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89dcf8d1ed631f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:43.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40ad59a640 1 -- 172.21.15.12:0/1212473459 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a00a89f0 con 0x7f40b00cac40 2023-11-28T21:50:43.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40ad59a640 1 -- 172.21.15.12:0/1212473459 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a00a8b50 con 0x7f40b00cac40 2023-11-28T21:50:43.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40ad59a640 1 -- 172.21.15.12:0/1212473459 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a00b0440 con 0x7f40b00cac40 2023-11-28T21:50:43.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40b4a32640 1 -- 172.21.15.12:0/1212473459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00cac40 msgr2=0x7f40b006b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:43.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40b4a32640 1 --2- 172.21.15.12:0/1212473459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00cac40 0x7f40b006b130 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f40b0068a60 tx=0x7f40a001ecf0 comp rx=0 tx=0).stop 2023-11-28T21:50:43.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40b4a32640 1 -- 172.21.15.12:0/1212473459 shutdown_connections 2023-11-28T21:50:43.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40b4a32640 1 --2- 172.21.15.12:0/1212473459 >> v2:172.21.15.144:3300/0 conn(0x7f40b00cb020 0x7f40b0159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:43.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40b4a32640 1 --2- 172.21.15.12:0/1212473459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00cac40 0x7f40b006b130 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:43.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40b4a32640 1 -- 172.21.15.12:0/1212473459 >> 172.21.15.12:0/1212473459 conn(0x7f40b00720e0 msgr2=0x7f40b00724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:43.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40b4a32640 1 -- 172.21.15.12:0/1212473459 shutdown_connections 2023-11-28T21:50:43.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40b4a32640 1 -- 172.21.15.12:0/1212473459 wait complete. 2023-11-28T21:50:43.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40b4a32640 1 Processor -- start 2023-11-28T21:50:43.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.887+0000 7f40b4a32640 1 -- start start 2023-11-28T21:50:43.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40b4a32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00c4980 0x7f40b01f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:43.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40b4a32640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b00cac40 0x7f40b01f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:43.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40b4a32640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b00cb020 0x7f40b0152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:43.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40b4a32640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b01f38f0 con 0x7f40b00c4980 2023-11-28T21:50:43.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40b4a32640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b01e34a0 con 0x7f40b00cb020 2023-11-28T21:50:43.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40b4a32640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b01e3200 con 0x7f40b00cac40 2023-11-28T21:50:43.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40ae59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00c4980 0x7f40b01f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:43.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40ae59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00c4980 0x7f40b01f3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:56718/0 (socket says 172.21.15.12:56718) 2023-11-28T21:50:43.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40ae59c640 1 -- 172.21.15.12:0/181608802 learned_addr learned my addr 172.21.15.12:0/181608802 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:43.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40aed9d640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b00cb020 0x7f40b0152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:43.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40add9b640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b00cac40 0x7f40b01f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:43.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40ae59c640 1 -- 172.21.15.12:0/181608802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b00cac40 msgr2=0x7f40b01f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:43.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40ae59c640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b00cac40 0x7f40b01f4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:43.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40ae59c640 1 -- 172.21.15.12:0/181608802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b00cb020 msgr2=0x7f40b0152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:43.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40ae59c640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b00cb020 0x7f40b0152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:43.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40ae59c640 1 -- 172.21.15.12:0/181608802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00094d0 con 0x7f40b00c4980 2023-11-28T21:50:43.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40ae59c640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00c4980 0x7f40b01f3d10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f40a00b3040 tx=0x7f40a001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:43.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f409effd640 1 -- 172.21.15.12:0/181608802 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a00ba600 con 0x7f40b00c4980 2023-11-28T21:50:43.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f409effd640 1 -- 172.21.15.12:0/181608802 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a00ba760 con 0x7f40b00c4980 2023-11-28T21:50:43.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40b4a32640 1 -- 172.21.15.12:0/181608802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b01e4df0 con 0x7f40b00c4980 2023-11-28T21:50:43.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f409effd640 1 -- 172.21.15.12:0/181608802 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a00bab10 con 0x7f40b00c4980 2023-11-28T21:50:43.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40b4a32640 1 -- 172.21.15.12:0/181608802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b01e5090 con 0x7f40b00c4980 2023-11-28T21:50:43.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f40b4a32640 1 -- 172.21.15.12:0/181608802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b0067c70 con 0x7f40b00c4980 2023-11-28T21:50:43.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.891+0000 7f409effd640 1 -- 172.21.15.12:0/181608802 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f40a00bac70 con 0x7f40b00c4980 2023-11-28T21:50:43.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.895+0000 7f409effd640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4090066470 0x7f4090068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:43.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.895+0000 7f409effd640 1 -- 172.21.15.12:0/181608802 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f40a0122970 con 0x7f40b00c4980 2023-11-28T21:50:43.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.895+0000 7f40add9b640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4090066470 0x7f4090068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:43.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.895+0000 7f40add9b640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4090066470 0x7f4090068930 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f40b015ba80 tx=0x7f40a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:43.902 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:43.899+0000 7f409effd640 1 -- 172.21.15.12:0/181608802 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f40a00ef770 con 0x7f40b00c4980 2023-11-28T21:50:44.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 -- 172.21.15.12:0/181608802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40b01f3a30 con 0x7f40b00c4980 2023-11-28T21:50:44.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f409effd640 1 -- 172.21.15.12:0/181608802 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5724 (secure 0 0 0) 0x7f40a00f2d90 con 0x7f40b00c4980 2023-11-28T21:50:44.099 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:44.099 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:38.562204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:44.099 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 65 2023-11-28T21:50:44.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 -- 172.21.15.12:0/181608802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4090066470 msgr2=0x7f4090068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:44.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4090066470 0x7f4090068930 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f40b015ba80 tx=0x7f40a4009040 comp rx=0 tx=0).stop 2023-11-28T21:50:44.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 -- 172.21.15.12:0/181608802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00c4980 msgr2=0x7f40b01f3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:44.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00c4980 0x7f40b01f3d10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f40a00b3040 tx=0x7f40a001ba90 comp rx=0 tx=0).stop 2023-11-28T21:50:44.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 -- 172.21.15.12:0/181608802 shutdown_connections 2023-11-28T21:50:44.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4090066470 0x7f4090068930 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:44.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40b00cb020 0x7f40b0152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:44.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40b00cac40 0x7f40b01f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:44.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 --2- 172.21.15.12:0/181608802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40b00c4980 0x7f40b01f3d10 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:44.102 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 -- 172.21.15.12:0/181608802 >> 172.21.15.12:0/181608802 conn(0x7f40b00720e0 msgr2=0x7f40b00b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:44.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 -- 172.21.15.12:0/181608802 shutdown_connections 2023-11-28T21:50:44.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:44.095+0000 7f40b4a32640 1 -- 172.21.15.12:0/181608802 wait complete. 2023-11-28T21:50:44.121 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:50:46.003 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:50:46.007 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:50:46.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.275+0000 7fe014df8640 1 Processor -- start 2023-11-28T21:50:46.281 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.275+0000 7fe014df8640 1 -- start start 2023-11-28T21:50:46.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe014df8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe010151ed0 0x7fe0100c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:46.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe014df8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe010157b70 con 0x7fe01006e300 2023-11-28T21:50:46.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe014df8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe010157e50 con 0x7fe010151af0 2023-11-28T21:50:46.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe014df8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100707f0 con 0x7fe010151ed0 2023-11-28T21:50:46.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00ed9d640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe01006e300 0x7fe010072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48230/0 (socket says 172.21.15.12:48230) 2023-11-28T21:50:46.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00ed9d640 1 -- 172.21.15.12:0/2630194880 learned_addr learned my addr 172.21.15.12:0/2630194880 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:50:46.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00dd9b640 1 --2- 172.21.15.12:0/2630194880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe010151ed0 0x7fe0100c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00d59a640 1 -- 172.21.15.12:0/2630194880 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634207728 0 0) 0x7fe010157b70 con 0x7fe01006e300 2023-11-28T21:50:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00d59a640 1 -- 172.21.15.12:0/2630194880 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff8003530 con 0x7fe01006e300 2023-11-28T21:50:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00d59a640 1 -- 172.21.15.12:0/2630194880 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679706897 0 0) 0x7fe010157e50 con 0x7fe010151af0 2023-11-28T21:50:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00d59a640 1 -- 172.21.15.12:0/2630194880 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe010157b70 con 0x7fe010151af0 2023-11-28T21:50:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00dd9b640 1 -- 172.21.15.12:0/2630194880 >> v1:172.21.15.144:6789/0 conn(0x7fe010151af0 legacy=0x7fe0100c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:46.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00dd9b640 1 -- 172.21.15.12:0/2630194880 >> v1:172.21.15.12:6789/0 conn(0x7fe01006e300 legacy=0x7fe010072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00d59a640 1 -- 172.21.15.12:0/2630194880 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4287644084 0 0) 0x7fdff8003530 con 0x7fe01006e300 2023-11-28T21:50:46.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.279+0000 7fe00dd9b640 1 -- 172.21.15.12:0/2630194880 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe010158130 con 0x7fe010151ed0 2023-11-28T21:50:46.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe00dd9b640 1 --2- 172.21.15.12:0/2630194880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe010151ed0 0x7fe0100c7650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0040097f0 tx=0x7fe00401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4e521578a4e3089 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:46.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe00d59a640 1 -- 172.21.15.12:0/2630194880 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008001a00 con 0x7fe010151ed0 2023-11-28T21:50:46.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe00d59a640 1 -- 172.21.15.12:0/2630194880 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0040a7050 con 0x7fe010151ed0 2023-11-28T21:50:46.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe00d59a640 1 -- 172.21.15.12:0/2630194880 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0040ab620 con 0x7fe010151ed0 2023-11-28T21:50:46.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe014df8640 1 -- 172.21.15.12:0/2630194880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe010151ed0 msgr2=0x7fe0100c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:46.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe014df8640 1 --2- 172.21.15.12:0/2630194880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe010151ed0 0x7fe0100c7650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0040097f0 tx=0x7fe00401c3e0 comp rx=0 tx=0).stop 2023-11-28T21:50:46.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe014df8640 1 -- 172.21.15.12:0/2630194880 shutdown_connections 2023-11-28T21:50:46.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe014df8640 1 --2- 172.21.15.12:0/2630194880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe010151ed0 0x7fe0100c7650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:46.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe014df8640 1 -- 172.21.15.12:0/2630194880 >> 172.21.15.12:0/2630194880 conn(0x7fe010066360 msgr2=0x7fe010066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:46.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe014df8640 1 -- 172.21.15.12:0/2630194880 shutdown_connections 2023-11-28T21:50:46.288 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe014df8640 1 -- 172.21.15.12:0/2630194880 wait complete. 2023-11-28T21:50:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe014df8640 1 Processor -- start 2023-11-28T21:50:46.289 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.283+0000 7fe014df8640 1 -- start start 2023-11-28T21:50:46.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe014df8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe01006e300 0x7fe0101f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe014df8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe010151af0 0x7fe0101f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe014df8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe010142600 0x7fe01013d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:46.292 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe014df8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe010142b60 con 0x7fe010142600 2023-11-28T21:50:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe014df8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0101f6290 con 0x7fe010151af0 2023-11-28T21:50:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe014df8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01013da30 con 0x7fe01006e300 2023-11-28T21:50:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe010142600 0x7fe01013d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe010142600 0x7fe01013d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34444/0 (socket says 172.21.15.12:34444) 2023-11-28T21:50:46.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00ed9d640 1 -- 172.21.15.12:0/4030029047 learned_addr learned my addr 172.21.15.12:0/4030029047 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00dd9b640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe010151af0 0x7fe0101f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00e59c640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe01006e300 0x7fe0101f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00ed9d640 1 -- 172.21.15.12:0/4030029047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe01006e300 msgr2=0x7fe0101f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:46.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00ed9d640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe01006e300 0x7fe0101f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00ed9d640 1 -- 172.21.15.12:0/4030029047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe010151af0 msgr2=0x7fe0101f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00ed9d640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe010151af0 0x7fe0101f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00ed9d640 1 -- 172.21.15.12:0/4030029047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe004009420 con 0x7fe010142600 2023-11-28T21:50:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe00ed9d640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe010142600 0x7fe01013d4f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe008001f30 tx=0x7fe00800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fdffeffd640 1 -- 172.21.15.12:0/4030029047 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008012400 con 0x7fe010142600 2023-11-28T21:50:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fdffeffd640 1 -- 172.21.15.12:0/4030029047 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe008012560 con 0x7fe010142600 2023-11-28T21:50:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe014df8640 1 -- 172.21.15.12:0/4030029047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe010142f00 con 0x7fe010142600 2023-11-28T21:50:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fdffeffd640 1 -- 172.21.15.12:0/4030029047 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080128f0 con 0x7fe010142600 2023-11-28T21:50:46.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe014df8640 1 -- 172.21.15.12:0/4030029047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0101f64e0 con 0x7fe010142600 2023-11-28T21:50:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fe014df8640 1 -- 172.21.15.12:0/4030029047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe010074f60 con 0x7fe010142600 2023-11-28T21:50:46.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.287+0000 7fdffeffd640 1 -- 172.21.15.12:0/4030029047 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe008012a50 con 0x7fe010142600 2023-11-28T21:50:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.291+0000 7fdffeffd640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdff00664a0 0x7fdff0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.291+0000 7fdffeffd640 1 -- 172.21.15.12:0/4030029047 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fe008084aa0 con 0x7fe010142600 2023-11-28T21:50:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.291+0000 7fe00e59c640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdff00664a0 0x7fdff0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:46.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.291+0000 7fe00e59c640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdff00664a0 0x7fdff0068960 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe0000011f0 tx=0x7fe000009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:46.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.295+0000 7fdffeffd640 1 -- 172.21.15.12:0/4030029047 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe008051950 con 0x7fe010142600 2023-11-28T21:50:46.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.483+0000 7fe014df8640 1 -- 172.21.15.12:0/4030029047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe010070450 con 0x7fe010142600 2023-11-28T21:50:46.489 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.483+0000 7fdffeffd640 1 -- 172.21.15.12:0/4030029047 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5724 (secure 0 0 0) 0x7fe008054f70 con 0x7fe010142600 2023-11-28T21:50:46.489 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:46.490 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:38.562204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:46.490 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 65 2023-11-28T21:50:46.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 -- 172.21.15.12:0/4030029047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdff00664a0 msgr2=0x7fdff0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:46.491 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdff00664a0 0x7fdff0068960 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe0000011f0 tx=0x7fe000009040 comp rx=0 tx=0).stop 2023-11-28T21:50:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 -- 172.21.15.12:0/4030029047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe010142600 msgr2=0x7fe01013d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe010142600 0x7fe01013d4f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe008001f30 tx=0x7fe00800a430 comp rx=0 tx=0).stop 2023-11-28T21:50:46.492 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 -- 172.21.15.12:0/4030029047 shutdown_connections 2023-11-28T21:50:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fdff00664a0 0x7fdff0068960 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:46.493 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe010142600 0x7fe01013d4f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe010151af0 0x7fe0101f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 --2- 172.21.15.12:0/4030029047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe01006e300 0x7fe0101f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:46.494 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 -- 172.21.15.12:0/4030029047 >> 172.21.15.12:0/4030029047 conn(0x7fe010066360 msgr2=0x7fe01014faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 -- 172.21.15.12:0/4030029047 shutdown_connections 2023-11-28T21:50:46.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:46.487+0000 7fe014df8640 1 -- 172.21.15.12:0/4030029047 wait complete. 2023-11-28T21:50:46.511 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:50:48.512 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:48.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.659+0000 7f4fe7716640 1 Processor -- start 2023-11-28T21:50:48.664 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.659+0000 7f4fe7716640 1 -- start start 2023-11-28T21:50:48.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.659+0000 7f4fe7716640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fe0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:48.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.659+0000 7f4fe7716640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe0067410 con 0x7f4fe006cf40 2023-11-28T21:50:48.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.659+0000 7f4fe7716640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe006e090 con 0x7f4fe0150c20 2023-11-28T21:50:48.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.659+0000 7f4fe7716640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0157f30 con 0x7f4fe006e300 2023-11-28T21:50:48.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fe5cb3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4fe006cf40 0x7f4fe0158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48236/0 (socket says 172.21.15.12:48236) 2023-11-28T21:50:48.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fe5cb3640 1 -- 172.21.15.12:0/3961977706 learned_addr learned my addr 172.21.15.12:0/3961977706 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:50:48.666 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fe4cb1640 1 --2- 172.21.15.12:0/3961977706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fe0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fd7fff640 1 -- 172.21.15.12:0/3961977706 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124566002 0 0) 0x7f4fe0067410 con 0x7f4fe006cf40 2023-11-28T21:50:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fd7fff640 1 -- 172.21.15.12:0/3961977706 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fcc005740 con 0x7f4fe006cf40 2023-11-28T21:50:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fe4cb1640 1 -- 172.21.15.12:0/3961977706 >> v1:172.21.15.144:6789/0 conn(0x7f4fe0150c20 legacy=0x7f4fe00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fd7fff640 1 -- 172.21.15.12:0/3961977706 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378921935 0 0) 0x7f4fe006e090 con 0x7f4fe0150c20 2023-11-28T21:50:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fe4cb1640 1 -- 172.21.15.12:0/3961977706 >> v1:172.21.15.12:6789/0 conn(0x7f4fe006cf40 legacy=0x7f4fe0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:48.667 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fe4cb1640 1 -- 172.21.15.12:0/3961977706 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe0151000 con 0x7f4fe006e300 2023-11-28T21:50:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fe4cb1640 1 --2- 172.21.15.12:0/3961977706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fe0156910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00097f0 tx=0x7f4fd001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19dd7adbcfe63f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fd7fff640 1 -- 172.21.15.12:0/3961977706 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00ab600 con 0x7f4fe006e300 2023-11-28T21:50:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fd7fff640 1 -- 172.21.15.12:0/3961977706 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd00a7050 con 0x7f4fe006e300 2023-11-28T21:50:48.669 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.663+0000 7f4fd7fff640 1 -- 172.21.15.12:0/3961977706 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00ab980 con 0x7f4fe006e300 2023-11-28T21:50:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 -- 172.21.15.12:0/3961977706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 msgr2=0x7f4fe0156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:48.670 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 --2- 172.21.15.12:0/3961977706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fe0156910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00097f0 tx=0x7f4fd001c4c0 comp rx=0 tx=0).stop 2023-11-28T21:50:48.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 -- 172.21.15.12:0/3961977706 shutdown_connections 2023-11-28T21:50:48.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 --2- 172.21.15.12:0/3961977706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006e300 0x7f4fe0156910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:48.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 -- 172.21.15.12:0/3961977706 >> 172.21.15.12:0/3961977706 conn(0x7f4fe0066360 msgr2=0x7f4fe0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:48.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 -- 172.21.15.12:0/3961977706 shutdown_connections 2023-11-28T21:50:48.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 -- 172.21.15.12:0/3961977706 wait complete. 2023-11-28T21:50:48.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 Processor -- start 2023-11-28T21:50:48.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 -- start start 2023-11-28T21:50:48.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006cf40 0x7f4fe00724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:48.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe006e300 0x7f4fe014e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:48.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0150c20 0x7f4fe0142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:48.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe014e680 con 0x7f4fe0150c20 2023-11-28T21:50:48.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0071920 con 0x7f4fe006e300 2023-11-28T21:50:48.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe7716640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01487e0 con 0x7f4fe006cf40 2023-11-28T21:50:48.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe5cb3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0150c20 0x7f4fe0142a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:48.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe54b2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006cf40 0x7f4fe00724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:48.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe5cb3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0150c20 0x7f4fe0142a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34456/0 (socket says 172.21.15.12:34456) 2023-11-28T21:50:48.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe5cb3640 1 -- 172.21.15.12:0/3827742240 learned_addr learned my addr 172.21.15.12:0/3827742240 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:48.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe54b2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006cf40 0x7f4fe00724f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:39168/0 (socket says 172.21.15.12:39168) 2023-11-28T21:50:48.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.667+0000 7f4fe4cb1640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe006e300 0x7f4fe014e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:48.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fe5cb3640 1 -- 172.21.15.12:0/3827742240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006cf40 msgr2=0x7f4fe00724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:48.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fe5cb3640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006cf40 0x7f4fe00724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:48.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fe5cb3640 1 -- 172.21.15.12:0/3827742240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe006e300 msgr2=0x7f4fe014e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:48.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fe5cb3640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe006e300 0x7f4fe014e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:48.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fe5cb3640 1 -- 172.21.15.12:0/3827742240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd0009420 con 0x7f4fe0150c20 2023-11-28T21:50:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fe5cb3640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0150c20 0x7f4fe0142a70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800cb40 tx=0x7f4fd800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fd5ffb640 1 -- 172.21.15.12:0/3827742240 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd800aca0 con 0x7f4fe0150c20 2023-11-28T21:50:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fd5ffb640 1 -- 172.21.15.12:0/3827742240 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd800ae00 con 0x7f4fe0150c20 2023-11-28T21:50:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fe7716640 1 -- 172.21.15.12:0/3827742240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe0071640 con 0x7f4fe0150c20 2023-11-28T21:50:48.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fd5ffb640 1 -- 172.21.15.12:0/3827742240 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8012620 con 0x7f4fe0150c20 2023-11-28T21:50:48.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fe7716640 1 -- 172.21.15.12:0/3827742240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe0147d00 con 0x7f4fe0150c20 2023-11-28T21:50:48.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fe7716640 1 -- 172.21.15.12:0/3827742240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe0074f60 con 0x7f4fe0150c20 2023-11-28T21:50:48.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fd5ffb640 1 -- 172.21.15.12:0/3827742240 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4fd8007070 con 0x7f4fe0150c20 2023-11-28T21:50:48.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fd5ffb640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fc4066470 0x7f4fc4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:48.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fd5ffb640 1 -- 172.21.15.12:0/3827742240 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f4fd8084550 con 0x7f4fe0150c20 2023-11-28T21:50:48.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.671+0000 7f4fe54b2640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fc4066470 0x7f4fc4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:48.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.675+0000 7f4fe54b2640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fc4066470 0x7f4fc4068930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc007fd0 tx=0x7f4fdc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:48.681 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.675+0000 7f4fd5ffb640 1 -- 172.21.15.12:0/3827742240 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4fd8051380 con 0x7f4fe0150c20 2023-11-28T21:50:48.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.867+0000 7f4fe7716640 1 -- 172.21.15.12:0/3827742240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fe0157440 con 0x7f4fe0150c20 2023-11-28T21:50:48.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.867+0000 7f4fd5ffb640 1 -- 172.21.15.12:0/3827742240 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5724 (secure 0 0 0) 0x7f4fd8017090 con 0x7f4fe0150c20 2023-11-28T21:50:48.874 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:48.874 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:38.562204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:48.874 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 65 2023-11-28T21:50:48.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 -- 172.21.15.12:0/3827742240 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fc4066470 msgr2=0x7f4fc4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fc4066470 0x7f4fc4068930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc007fd0 tx=0x7f4fdc009040 comp rx=0 tx=0).stop 2023-11-28T21:50:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 -- 172.21.15.12:0/3827742240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0150c20 msgr2=0x7f4fe0142a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0150c20 0x7f4fe0142a70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800cb40 tx=0x7f4fd800a060 comp rx=0 tx=0).stop 2023-11-28T21:50:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 -- 172.21.15.12:0/3827742240 shutdown_connections 2023-11-28T21:50:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4fc4066470 0x7f4fc4068930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fe0150c20 0x7f4fe0142a70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fe006e300 0x7f4fe014e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:48.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 --2- 172.21.15.12:0/3827742240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fe006cf40 0x7f4fe00724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:48.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 -- 172.21.15.12:0/3827742240 >> 172.21.15.12:0/3827742240 conn(0x7f4fe0066360 msgr2=0x7f4fe00ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:48.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 -- 172.21.15.12:0/3827742240 shutdown_connections 2023-11-28T21:50:48.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:48.871+0000 7f4fe7716640 1 -- 172.21.15.12:0/3827742240 wait complete. 2023-11-28T21:50:48.896 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:50:50.897 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:51.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b89d8640 1 Processor -- start 2023-11-28T21:50:51.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b89d8640 1 -- start start 2023-11-28T21:50:51.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b89d8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f36b4151af0 0x7f36b40c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:51.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b89d8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f36b4151ed0 0x7f36b40c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:51.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b89d8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b406e300 0x7f36b40c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:51.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b89d8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b40707f0 con 0x7f36b4151af0 2023-11-28T21:50:51.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b89d8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b40c7b90 con 0x7f36b4151ed0 2023-11-28T21:50:51.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b89d8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4071c20 con 0x7f36b406e300 2023-11-28T21:50:51.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f36b4151af0 0x7f36b40c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f36b4151af0 0x7f36b40c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34472/0 (socket says 172.21.15.12:34472) 2023-11-28T21:50:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b259c640 1 -- 172.21.15.12:0/2178456469 learned_addr learned my addr 172.21.15.12:0/2178456469 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b2d9d640 1 --2- 172.21.15.12:0/2178456469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b406e300 0x7f36b40c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36abfff640 1 --2- 172.21.15.12:0/2178456469 >> v2:172.21.15.144:3300/0 conn(0x7f36b4151ed0 0x7f36b40c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b259c640 1 -- 172.21.15.12:0/2178456469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b406e300 msgr2=0x7f36b40c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b259c640 1 --2- 172.21.15.12:0/2178456469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b406e300 0x7f36b40c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b259c640 1 -- 172.21.15.12:0/2178456469 >> v2:172.21.15.144:3300/0 conn(0x7f36b4151ed0 msgr2=0x7f36b40c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:51.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b259c640 1 --2- 172.21.15.12:0/2178456469 >> v2:172.21.15.144:3300/0 conn(0x7f36b4151ed0 0x7f36b40c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.047+0000 7f36b259c640 1 -- 172.21.15.12:0/2178456469 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4157b70 con 0x7f36b4151af0 2023-11-28T21:50:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.051+0000 7f36b259c640 1 --2- 172.21.15.12:0/2178456469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b4151af0 0x7f36b40c7650 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f36b40c8e60 tx=0x7f36a401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a297d8423390fd0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.051+0000 7f36b1d9b640 1 -- 172.21.15.12:0/2178456469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a40ac410 con 0x7f36b4151af0 2023-11-28T21:50:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.051+0000 7f36b1d9b640 1 -- 172.21.15.12:0/2178456469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a40ac570 con 0x7f36b4151af0 2023-11-28T21:50:51.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.051+0000 7f36b1d9b640 1 -- 172.21.15.12:0/2178456469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a40ac900 con 0x7f36b4151af0 2023-11-28T21:50:51.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.051+0000 7f36b89d8640 1 -- 172.21.15.12:0/2178456469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b4151af0 msgr2=0x7f36b40c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.051+0000 7f36b89d8640 1 --2- 172.21.15.12:0/2178456469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b4151af0 0x7f36b40c7650 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f36b40c8e60 tx=0x7f36a401c500 comp rx=0 tx=0).stop 2023-11-28T21:50:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.051+0000 7f36b89d8640 1 -- 172.21.15.12:0/2178456469 shutdown_connections 2023-11-28T21:50:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.051+0000 7f36b89d8640 1 --2- 172.21.15.12:0/2178456469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b406e300 0x7f36b40c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 --2- 172.21.15.12:0/2178456469 >> v2:172.21.15.144:3300/0 conn(0x7f36b4151ed0 0x7f36b40c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 --2- 172.21.15.12:0/2178456469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b4151af0 0x7f36b40c7650 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 -- 172.21.15.12:0/2178456469 >> 172.21.15.12:0/2178456469 conn(0x7f36b4066360 msgr2=0x7f36b4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:51.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 -- 172.21.15.12:0/2178456469 shutdown_connections 2023-11-28T21:50:51.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 -- 172.21.15.12:0/2178456469 wait complete. 2023-11-28T21:50:51.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 Processor -- start 2023-11-28T21:50:51.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 -- start start 2023-11-28T21:50:51.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b406e300 0x7f36b415a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:51.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b4151af0 0x7f36b41522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:51.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b4152800 0x7f36b41f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:51.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b41f6920 con 0x7f36b406e300 2023-11-28T21:50:51.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4154430 con 0x7f36b4151af0 2023-11-28T21:50:51.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b89d8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b4154710 con 0x7f36b4152800 2023-11-28T21:50:51.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b406e300 0x7f36b415a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:51.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b406e300 0x7f36b415a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34488/0 (socket says 172.21.15.12:34488) 2023-11-28T21:50:51.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b259c640 1 -- 172.21.15.12:0/1268188296 learned_addr learned my addr 172.21.15.12:0/1268188296 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:51.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b2d9d640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b4152800 0x7f36b41f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:51.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36abfff640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b4151af0 0x7f36b41522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:51.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b259c640 1 -- 172.21.15.12:0/1268188296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b4152800 msgr2=0x7f36b41f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:51.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b259c640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b4152800 0x7f36b41f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b259c640 1 -- 172.21.15.12:0/1268188296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b4151af0 msgr2=0x7f36b41522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:51.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b259c640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b4151af0 0x7f36b41522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b259c640 1 -- 172.21.15.12:0/1268188296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a40094d0 con 0x7f36b406e300 2023-11-28T21:50:51.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.055+0000 7f36b259c640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b406e300 0x7f36b415a050 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f36a40af040 tx=0x7f36a40192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:51.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36aaffd640 1 -- 172.21.15.12:0/1268188296 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a40ac410 con 0x7f36b406e300 2023-11-28T21:50:51.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36aaffd640 1 -- 172.21.15.12:0/1268188296 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a40b65f0 con 0x7f36b406e300 2023-11-28T21:50:51.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36aaffd640 1 -- 172.21.15.12:0/1268188296 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a40b69a0 con 0x7f36b406e300 2023-11-28T21:50:51.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36b89d8640 1 -- 172.21.15.12:0/1268188296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b41549f0 con 0x7f36b406e300 2023-11-28T21:50:51.066 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36b89d8640 1 -- 172.21.15.12:0/1268188296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b4154130 con 0x7f36b406e300 2023-11-28T21:50:51.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36b89d8640 1 -- 172.21.15.12:0/1268188296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b4074f60 con 0x7f36b406e300 2023-11-28T21:50:51.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36aaffd640 1 -- 172.21.15.12:0/1268188296 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f36a40b6b00 con 0x7f36b406e300 2023-11-28T21:50:51.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36aaffd640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36940664a0 0x7f3694068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:51.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36aaffd640 1 -- 172.21.15.12:0/1268188296 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f36a411f0f0 con 0x7f36b406e300 2023-11-28T21:50:51.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36abfff640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36940664a0 0x7f3694068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:51.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.059+0000 7f36abfff640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36940664a0 0x7f3694068960 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f36b415aeb0 tx=0x7f369c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:51.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.063+0000 7f36aaffd640 1 -- 172.21.15.12:0/1268188296 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f36a40ebef0 con 0x7f36b406e300 2023-11-28T21:50:51.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.251+0000 7f36b89d8640 1 -- 172.21.15.12:0/1268188296 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36b406d240 con 0x7f36b406e300 2023-11-28T21:50:51.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36aaffd640 1 -- 172.21.15.12:0/1268188296 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5727 (secure 0 0 0) 0x7f36a40b1020 con 0x7f36b406e300 2023-11-28T21:50:51.258 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:51.258 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:50.432978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:resolve","state_seq":157,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:51.258 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 66 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 -- 172.21.15.12:0/1268188296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36940664a0 msgr2=0x7f3694068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36940664a0 0x7f3694068960 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f36b415aeb0 tx=0x7f369c008040 comp rx=0 tx=0).stop 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 -- 172.21.15.12:0/1268188296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b406e300 msgr2=0x7f36b415a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b406e300 0x7f36b415a050 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f36a40af040 tx=0x7f36a40192a0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 -- 172.21.15.12:0/1268188296 shutdown_connections 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f36940664a0 0x7f3694068960 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b4152800 0x7f36b41f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b4151af0 0x7f36b41522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 --2- 172.21.15.12:0/1268188296 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f36b406e300 0x7f36b415a050 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 -- 172.21.15.12:0/1268188296 >> 172.21.15.12:0/1268188296 conn(0x7f36b4066360 msgr2=0x7f36b414dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:51.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 -- 172.21.15.12:0/1268188296 shutdown_connections 2023-11-28T21:50:51.261 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:51.255+0000 7f36b89d8640 1 -- 172.21.15.12:0/1268188296 wait complete. 2023-11-28T21:50:51.281 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 66, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 66, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5057": { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 65, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:resolve", "state_seq": 157 }, "gid_6330": { "addr": "172.21.15.12:6839/754378087", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 754378087, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 754378087, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6330, "incarnation": 63, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:50:50.432978+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 5057, "mds_1": 6330 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:50:53.282 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:53.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9b0529e640 1 Processor -- start 2023-11-28T21:50:53.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9b0529e640 1 -- start start 2023-11-28T21:50:53.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9b0529e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9b00150c20 0x7f9b00156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9b0529e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0006e300 0x7f9b000c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9b0529e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b0006cf40 0x7f9b000c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9b0529e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b00067410 con 0x7f9b00150c20 2023-11-28T21:50:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9b0529e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0006e090 con 0x7f9b0006cf40 2023-11-28T21:50:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9b0529e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b00157f30 con 0x7f9b0006e300 2023-11-28T21:50:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9afeffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9b00150c20 0x7f9b00156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9afeffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9b00150c20 0x7f9b00156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34490/0 (socket says 172.21.15.12:34490) 2023-11-28T21:50:53.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9afeffd640 1 -- 172.21.15.12:0/2006583910 learned_addr learned my addr 172.21.15.12:0/2006583910 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9af7fff640 1 --2- 172.21.15.12:0/2006583910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0006e300 0x7f9b000c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9aff7fe640 1 --2- 172.21.15.12:0/2006583910 >> v2:172.21.15.144:3300/0 conn(0x7f9b0006cf40 0x7f9b000c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9afeffd640 1 -- 172.21.15.12:0/2006583910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0006e300 msgr2=0x7f9b000c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9afeffd640 1 --2- 172.21.15.12:0/2006583910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0006e300 0x7f9b000c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9afeffd640 1 -- 172.21.15.12:0/2006583910 >> v2:172.21.15.144:3300/0 conn(0x7f9b0006cf40 msgr2=0x7f9b000c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9afeffd640 1 --2- 172.21.15.12:0/2006583910 >> v2:172.21.15.144:3300/0 conn(0x7f9b0006cf40 0x7f9b000c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.435+0000 7f9afeffd640 1 -- 172.21.15.12:0/2006583910 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b00151000 con 0x7f9b00150c20 2023-11-28T21:50:53.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.439+0000 7f9afeffd640 1 --2- 172.21.15.12:0/2006583910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b00150c20 0x7f9b00156910 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9b000c7d70 tx=0x7f9af001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f969f128cf0b82d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:53.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.439+0000 7f9afe7fc640 1 -- 172.21.15.12:0/2006583910 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00ac630 con 0x7f9b00150c20 2023-11-28T21:50:53.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.439+0000 7f9afe7fc640 1 -- 172.21.15.12:0/2006583910 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af00ac790 con 0x7f9b00150c20 2023-11-28T21:50:53.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.439+0000 7f9afe7fc640 1 -- 172.21.15.12:0/2006583910 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00acb20 con 0x7f9b00150c20 2023-11-28T21:50:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.439+0000 7f9b0529e640 1 -- 172.21.15.12:0/2006583910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b00150c20 msgr2=0x7f9b00156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.439+0000 7f9b0529e640 1 --2- 172.21.15.12:0/2006583910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b00150c20 0x7f9b00156910 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9b000c7d70 tx=0x7f9af001c500 comp rx=0 tx=0).stop 2023-11-28T21:50:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 -- 172.21.15.12:0/2006583910 shutdown_connections 2023-11-28T21:50:53.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 --2- 172.21.15.12:0/2006583910 >> v2:172.21.15.144:3300/0 conn(0x7f9b0006cf40 0x7f9b000c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 --2- 172.21.15.12:0/2006583910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0006e300 0x7f9b000c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 --2- 172.21.15.12:0/2006583910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b00150c20 0x7f9b00156910 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 -- 172.21.15.12:0/2006583910 >> 172.21.15.12:0/2006583910 conn(0x7f9b00066360 msgr2=0x7f9b00066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 -- 172.21.15.12:0/2006583910 shutdown_connections 2023-11-28T21:50:53.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 -- 172.21.15.12:0/2006583910 wait complete. 2023-11-28T21:50:53.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 Processor -- start 2023-11-28T21:50:53.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 -- start start 2023-11-28T21:50:53.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0006cf40 0x7f9b0014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:53.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0006e300 0x7f9b0014b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:53.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b00150c20 0x7f9b00071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:53.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b001ee410 con 0x7f9b0006cf40 2023-11-28T21:50:53.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b001ee6f0 con 0x7f9b00150c20 2023-11-28T21:50:53.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9b0529e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b001ee9d0 con 0x7f9b0006e300 2023-11-28T21:50:53.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9afeffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0006cf40 0x7f9b0014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:53.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9afeffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0006cf40 0x7f9b0014c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34498/0 (socket says 172.21.15.12:34498) 2023-11-28T21:50:53.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9afeffd640 1 -- 172.21.15.12:0/805092133 learned_addr learned my addr 172.21.15.12:0/805092133 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:53.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9af7fff640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0006e300 0x7f9b0014b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:53.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9aff7fe640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b00150c20 0x7f9b00071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:53.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9afeffd640 1 -- 172.21.15.12:0/805092133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0006e300 msgr2=0x7f9b0014b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:53.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9afeffd640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0006e300 0x7f9b0014b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9afeffd640 1 -- 172.21.15.12:0/805092133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b00150c20 msgr2=0x7f9b00071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:53.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9afeffd640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b00150c20 0x7f9b00071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9afeffd640 1 -- 172.21.15.12:0/805092133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af00094d0 con 0x7f9b0006cf40 2023-11-28T21:50:53.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9afeffd640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0006cf40 0x7f9b0014c8a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9af00af040 tx=0x7f9af00192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.443+0000 7f9af77fe640 1 -- 172.21.15.12:0/805092133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00b7670 con 0x7f9b0006cf40 2023-11-28T21:50:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.447+0000 7f9af77fe640 1 -- 172.21.15.12:0/805092133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af00b77f0 con 0x7f9b0006cf40 2023-11-28T21:50:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.447+0000 7f9af77fe640 1 -- 172.21.15.12:0/805092133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00b7ba0 con 0x7f9b0006cf40 2023-11-28T21:50:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.447+0000 7f9b0529e640 1 -- 172.21.15.12:0/805092133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b001eecb0 con 0x7f9b0006cf40 2023-11-28T21:50:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.447+0000 7f9b0529e640 1 -- 172.21.15.12:0/805092133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b000c97a0 con 0x7f9b0006cf40 2023-11-28T21:50:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.447+0000 7f9b0529e640 1 -- 172.21.15.12:0/805092133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b00074f60 con 0x7f9b0006cf40 2023-11-28T21:50:53.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.447+0000 7f9af77fe640 1 -- 172.21.15.12:0/805092133 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9af00b3070 con 0x7f9b0006cf40 2023-11-28T21:50:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.447+0000 7f9af77fe640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ae0066470 0x7f9ae0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.447+0000 7f9af77fe640 1 -- 172.21.15.12:0/805092133 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f9af011ec60 con 0x7f9b0006cf40 2023-11-28T21:50:53.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.447+0000 7f9af7fff640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ae0066470 0x7f9ae0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:53.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.447+0000 7f9af7fff640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ae0066470 0x7f9ae0068930 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8006aa0 tx=0x7f9ae8006a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:53.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.451+0000 7f9af77fe640 1 -- 172.21.15.12:0/805092133 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9af00ebaf0 con 0x7f9b0006cf40 2023-11-28T21:50:53.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.643+0000 7f9b0529e640 1 -- 172.21.15.12:0/805092133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b000c2c30 con 0x7f9b0006cf40 2023-11-28T21:50:53.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.643+0000 7f9af77fe640 1 -- 172.21.15.12:0/805092133 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5727 (secure 0 0 0) 0x7f9af00b1090 con 0x7f9b0006cf40 2023-11-28T21:50:53.648 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:53.648 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:50.432978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:resolve","state_seq":157,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:53.649 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 66 2023-11-28T21:50:53.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.643+0000 7f9b0529e640 1 -- 172.21.15.12:0/805092133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ae0066470 msgr2=0x7f9ae0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:53.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.643+0000 7f9b0529e640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ae0066470 0x7f9ae0068930 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8006aa0 tx=0x7f9ae8006a30 comp rx=0 tx=0).stop 2023-11-28T21:50:53.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.643+0000 7f9b0529e640 1 -- 172.21.15.12:0/805092133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0006cf40 msgr2=0x7f9b0014c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:53.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.643+0000 7f9b0529e640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0006cf40 0x7f9b0014c8a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9af00af040 tx=0x7f9af00192a0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.647+0000 7f9b0529e640 1 -- 172.21.15.12:0/805092133 shutdown_connections 2023-11-28T21:50:53.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.647+0000 7f9b0529e640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ae0066470 0x7f9ae0068930 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.647+0000 7f9b0529e640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b00150c20 0x7f9b00071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.647+0000 7f9b0529e640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b0006e300 0x7f9b0014b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.647+0000 7f9b0529e640 1 --2- 172.21.15.12:0/805092133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b0006cf40 0x7f9b0014c8a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:53.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.647+0000 7f9b0529e640 1 -- 172.21.15.12:0/805092133 >> 172.21.15.12:0/805092133 conn(0x7f9b00066360 msgr2=0x7f9b000c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:53.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.647+0000 7f9b0529e640 1 -- 172.21.15.12:0/805092133 shutdown_connections 2023-11-28T21:50:53.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:53.647+0000 7f9b0529e640 1 -- 172.21.15.12:0/805092133 wait complete. 2023-11-28T21:50:53.670 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:50:55.671 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:55.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.823+0000 7fcfb0b3b640 1 Processor -- start 2023-11-28T21:50:55.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.823+0000 7fcfb0b3b640 1 -- start start 2023-11-28T21:50:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfb0b3b640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcfac153a20 0x7fcfac06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfb0b3b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac159b90 con 0x7fcfac0caea0 2023-11-28T21:50:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfb0b3b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac06c260 con 0x7fcfac0c4690 2023-11-28T21:50:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfb0b3b640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfac15a4d0 con 0x7fcfac153a20 2023-11-28T21:50:55.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfab7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcfac153a20 0x7fcfac06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:55.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfab7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fcfac153a20 0x7fcfac06bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60342/0 (socket says 172.21.15.12:60342) 2023-11-28T21:50:55.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfabfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fcfac0caea0 0x7fcfac164510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39416/0 (socket says 172.21.15.12:39416) 2023-11-28T21:50:55.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfab7fe640 1 -- 172.21.15.12:0/1802984494 learned_addr learned my addr 172.21.15.12:0/1802984494 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:55.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfab7fe640 1 -- 172.21.15.12:0/1802984494 >> v1:172.21.15.144:6789/0 conn(0x7fcfac0c4690 legacy=0x7fcfac066360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:55.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfab7fe640 1 -- 172.21.15.12:0/1802984494 >> v1:172.21.15.12:6789/0 conn(0x7fcfac0caea0 legacy=0x7fcfac164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:55.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.827+0000 7fcfab7fe640 1 -- 172.21.15.12:0/1802984494 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac06c540 con 0x7fcfac153a20 2023-11-28T21:50:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfab7fe640 1 --2- 172.21.15.12:0/1802984494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac153a20 0x7fcfac06bc20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009870 tx=0x7fcf9c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a09906acbce711 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfaa7fc640 1 -- 172.21.15.12:0/1802984494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0a95f0 con 0x7fcfac153a20 2023-11-28T21:50:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfaa7fc640 1 -- 172.21.15.12:0/1802984494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c0a9750 con 0x7fcfac153a20 2023-11-28T21:50:55.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfaa7fc640 1 -- 172.21.15.12:0/1802984494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0a9ae0 con 0x7fcfac153a20 2023-11-28T21:50:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/1802984494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac153a20 msgr2=0x7fcfac06bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 --2- 172.21.15.12:0/1802984494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac153a20 0x7fcfac06bc20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009870 tx=0x7fcf9c01c510 comp rx=0 tx=0).stop 2023-11-28T21:50:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/1802984494 shutdown_connections 2023-11-28T21:50:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 --2- 172.21.15.12:0/1802984494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac153a20 0x7fcfac06bc20 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/1802984494 >> 172.21.15.12:0/1802984494 conn(0x7fcfac0cf550 msgr2=0x7fcfac0cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/1802984494 shutdown_connections 2023-11-28T21:50:55.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/1802984494 wait complete. 2023-11-28T21:50:55.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 Processor -- start 2023-11-28T21:50:55.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 -- start start 2023-11-28T21:50:55.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcfac0c4690 0x7fcfac0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:55.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.831+0000 7fcfb0b3b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac0caea0 0x7fcfac148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:55.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfb0b3b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac153a20 0x7fcfac1428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:55.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfb0b3b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac148130 con 0x7fcfac0caea0 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfb0b3b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac157f30 con 0x7fcfac153a20 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfb0b3b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac15ad50 con 0x7fcfac0c4690 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfaaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac0caea0 0x7fcfac148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfaaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac0caea0 0x7fcfac148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60348/0 (socket says 172.21.15.12:60348) 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfaaffd640 1 -- 172.21.15.12:0/788246930 learned_addr learned my addr 172.21.15.12:0/788246930 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfab7fe640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcfac0c4690 0x7fcfac0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfabfff640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac153a20 0x7fcfac1428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfaaffd640 1 -- 172.21.15.12:0/788246930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcfac0c4690 msgr2=0x7fcfac0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfaaffd640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcfac0c4690 0x7fcfac0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfaaffd640 1 -- 172.21.15.12:0/788246930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac153a20 msgr2=0x7fcfac1428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfaaffd640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac153a20 0x7fcfac1428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfaaffd640 1 -- 172.21.15.12:0/788246930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c0094d0 con 0x7fcfac0caea0 2023-11-28T21:50:55.840 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfab7fe640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcfac0c4690 0x7fcfac0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:50:55.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfabfff640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac153a20 0x7fcfac1428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:50:55.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfaaffd640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac0caea0 0x7fcfac148530 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c500 tx=0x7fcfa000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:55.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcf93fff640 1 -- 172.21.15.12:0/788246930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0005220 con 0x7fcfac0caea0 2023-11-28T21:50:55.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcf93fff640 1 -- 172.21.15.12:0/788246930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa0005380 con 0x7fcfac0caea0 2023-11-28T21:50:55.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcf93fff640 1 -- 172.21.15.12:0/788246930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0007270 con 0x7fcfac0caea0 2023-11-28T21:50:55.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/788246930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac1471f0 con 0x7fcfac0caea0 2023-11-28T21:50:55.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/788246930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac147790 con 0x7fcfac0caea0 2023-11-28T21:50:55.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/788246930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac074f60 con 0x7fcfac0caea0 2023-11-28T21:50:55.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.835+0000 7fcf93fff640 1 -- 172.21.15.12:0/788246930 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fcfa00073d0 con 0x7fcfac0caea0 2023-11-28T21:50:55.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.839+0000 7fcf93fff640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf980664f0 0x7fcf980689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:55.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.839+0000 7fcf93fff640 1 -- 172.21.15.12:0/788246930 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fcfa0083e00 con 0x7fcfac0caea0 2023-11-28T21:50:55.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.839+0000 7fcfab7fe640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf980664f0 0x7fcf980689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:55.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.839+0000 7fcfab7fe640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf980664f0 0x7fcf980689b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0ae000 tx=0x7fcf9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:55.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:55.843+0000 7fcf93fff640 1 -- 172.21.15.12:0/788246930 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcfa0050c00 con 0x7fcfac0caea0 2023-11-28T21:50:56.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.031+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/788246930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfac0cac80 con 0x7fcfac0caea0 2023-11-28T21:50:56.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.031+0000 7fcf93fff640 1 -- 172.21.15.12:0/788246930 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5727 (secure 0 0 0) 0x7fcfa0054220 con 0x7fcfac0caea0 2023-11-28T21:50:56.037 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:56.038 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:50.432978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:resolve","state_seq":157,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:56.038 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 66 2023-11-28T21:50:56.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/788246930 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf980664f0 msgr2=0x7fcf980689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:56.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf980664f0 0x7fcf980689b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0ae000 tx=0x7fcf9c009040 comp rx=0 tx=0).stop 2023-11-28T21:50:56.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/788246930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac0caea0 msgr2=0x7fcfac148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac0caea0 0x7fcfac148530 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c500 tx=0x7fcfa000c9c0 comp rx=0 tx=0).stop 2023-11-28T21:50:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/788246930 shutdown_connections 2023-11-28T21:50:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fcf980664f0 0x7fcf980689b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcfac153a20 0x7fcfac1428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfac0caea0 0x7fcfac148530 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 --2- 172.21.15.12:0/788246930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcfac0c4690 0x7fcfac0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/788246930 >> 172.21.15.12:0/788246930 conn(0x7fcfac0cf550 msgr2=0x7fcfac158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/788246930 shutdown_connections 2023-11-28T21:50:56.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:56.035+0000 7fcfb0b3b640 1 -- 172.21.15.12:0/788246930 wait complete. 2023-11-28T21:50:56.059 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:50:58.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:50:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fff72640 1 Processor -- start 2023-11-28T21:50:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fff72640 1 -- start start 2023-11-28T21:50:58.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fff72640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f32f80c4000 0x7f32f806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:58.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fff72640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8161600 0x7f32f8161a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:58.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fff72640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f80c3460 con 0x7f32f8158420 2023-11-28T21:50:58.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fff72640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f8158800 con 0x7f32f80c4000 2023-11-28T21:50:58.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fff72640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80c52c0 con 0x7f32f8161600 2023-11-28T21:50:58.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fe50f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8161600 0x7f32f8161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:58.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fd50d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f32f80c4000 0x7f32f806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:58.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fdd0e640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f32f8158420 0x7f32f80c38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:46328/0 (socket says 172.21.15.12:46328) 2023-11-28T21:50:58.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fdd0e640 1 -- 172.21.15.12:0/153902580 learned_addr learned my addr 172.21.15.12:0/153902580 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:50:58.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fcd0c640 1 -- 172.21.15.12:0/153902580 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420179821 0 0) 0x7f32f80c3460 con 0x7f32f8158420 2023-11-28T21:50:58.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.211+0000 7f32fcd0c640 1 -- 172.21.15.12:0/153902580 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e4003530 con 0x7f32f8158420 2023-11-28T21:50:58.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.215+0000 7f32fe50f640 1 -- 172.21.15.12:0/153902580 >> v2:172.21.15.144:3300/0 conn(0x7f32f80c4000 msgr2=0x7f32f806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:50:58.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.215+0000 7f32fe50f640 1 --2- 172.21.15.12:0/153902580 >> v2:172.21.15.144:3300/0 conn(0x7f32f80c4000 0x7f32f806d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:58.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.215+0000 7f32fe50f640 1 -- 172.21.15.12:0/153902580 >> v1:172.21.15.144:6789/0 conn(0x7f32f8158420 legacy=0x7f32f80c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:58.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.215+0000 7f32fe50f640 1 -- 172.21.15.12:0/153902580 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f80c2b70 con 0x7f32f8161600 2023-11-28T21:50:58.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.215+0000 7f32fd50d640 1 --2- 172.21.15.12:0/153902580 >> v2:172.21.15.144:3300/0 conn(0x7f32f80c4000 0x7f32f806d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:50:58.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.215+0000 7f32fe50f640 1 --2- 172.21.15.12:0/153902580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8161600 0x7f32f8161a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009d90 tx=0x7f32f001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb9643c49e2b3a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:58.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.215+0000 7f32fcd0c640 1 -- 172.21.15.12:0/153902580 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f00a5810 con 0x7f32f8161600 2023-11-28T21:50:58.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.215+0000 7f32fcd0c640 1 -- 172.21.15.12:0/153902580 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f00a5970 con 0x7f32f8161600 2023-11-28T21:50:58.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.215+0000 7f32fcd0c640 1 -- 172.21.15.12:0/153902580 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f00a5ce0 con 0x7f32f8161600 2023-11-28T21:50:58.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.215+0000 7f32fff72640 1 -- 172.21.15.12:0/153902580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8161600 msgr2=0x7f32f8161a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:58.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 --2- 172.21.15.12:0/153902580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8161600 0x7f32f8161a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009d90 tx=0x7f32f001c950 comp rx=0 tx=0).stop 2023-11-28T21:50:58.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 -- 172.21.15.12:0/153902580 shutdown_connections 2023-11-28T21:50:58.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 --2- 172.21.15.12:0/153902580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8161600 0x7f32f8161a50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:58.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 --2- 172.21.15.12:0/153902580 >> v2:172.21.15.144:3300/0 conn(0x7f32f80c4000 0x7f32f806d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:58.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 -- 172.21.15.12:0/153902580 >> 172.21.15.12:0/153902580 conn(0x7f32f8066360 msgr2=0x7f32f8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 -- 172.21.15.12:0/153902580 shutdown_connections 2023-11-28T21:50:58.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 -- 172.21.15.12:0/153902580 wait complete. 2023-11-28T21:50:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 Processor -- start 2023-11-28T21:50:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 -- start start 2023-11-28T21:50:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f80c4000 0x7f32f8159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8158420 0x7f32f8152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f8161600 0x7f32f81f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:58.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f81f7550 con 0x7f32f80c4000 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815a430 con 0x7f32f8161600 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fff72640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815b900 con 0x7f32f8158420 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fdd0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f80c4000 0x7f32f8159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fdd0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f80c4000 0x7f32f8159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60360/0 (socket says 172.21.15.12:60360) 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fdd0e640 1 -- 172.21.15.12:0/4228550709 learned_addr learned my addr 172.21.15.12:0/4228550709 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fe50f640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f8161600 0x7f32f81f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fd50d640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8158420 0x7f32f8152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fdd0e640 1 -- 172.21.15.12:0/4228550709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8158420 msgr2=0x7f32f8152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fdd0e640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8158420 0x7f32f8152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fdd0e640 1 -- 172.21.15.12:0/4228550709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f8161600 msgr2=0x7f32f81f9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fdd0e640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f8161600 0x7f32f81f9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fdd0e640 1 -- 172.21.15.12:0/4228550709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e8009420 con 0x7f32f80c4000 2023-11-28T21:50:58.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.219+0000 7f32fdd0e640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f80c4000 0x7f32f8159ef0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c480 tx=0x7f32f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.223+0000 7f32ee7fc640 1 -- 172.21.15.12:0/4228550709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4005cf0 con 0x7f32f80c4000 2023-11-28T21:50:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.223+0000 7f32ee7fc640 1 -- 172.21.15.12:0/4228550709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f4005e50 con 0x7f32f80c4000 2023-11-28T21:50:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.223+0000 7f32ee7fc640 1 -- 172.21.15.12:0/4228550709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4007290 con 0x7f32f80c4000 2023-11-28T21:50:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.223+0000 7f32fff72640 1 -- 172.21.15.12:0/4228550709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f0009a70 con 0x7f32f80c4000 2023-11-28T21:50:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.223+0000 7f32fff72640 1 -- 172.21.15.12:0/4228550709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f815f5e0 con 0x7f32f80c4000 2023-11-28T21:50:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.223+0000 7f32fff72640 1 -- 172.21.15.12:0/4228550709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f8074f60 con 0x7f32f80c4000 2023-11-28T21:50:58.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.223+0000 7f32ee7fc640 1 -- 172.21.15.12:0/4228550709 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f32f40073f0 con 0x7f32f80c4000 2023-11-28T21:50:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.223+0000 7f32ee7fc640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32dc0664a0 0x7f32dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:50:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.223+0000 7f32ee7fc640 1 -- 172.21.15.12:0/4228550709 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f32f4083a40 con 0x7f32f80c4000 2023-11-28T21:50:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.223+0000 7f32fd50d640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32dc0664a0 0x7f32dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:50:58.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.227+0000 7f32fd50d640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32dc0664a0 0x7f32dc068960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f32f8066dd0 tx=0x7f32e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:50:58.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.227+0000 7f32ee7fc640 1 -- 172.21.15.12:0/4228550709 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f32f40508f0 con 0x7f32f80c4000 2023-11-28T21:50:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 -- 172.21.15.12:0/4228550709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32f806fd00 con 0x7f32f80c4000 2023-11-28T21:50:58.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32ee7fc640 1 -- 172.21.15.12:0/4228550709 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5729 (secure 0 0 0) 0x7f32f4017020 con 0x7f32f80c4000 2023-11-28T21:50:58.423 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:50:58.423 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:55.892717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:reconnect","state_seq":159,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:50:58.423 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 67 2023-11-28T21:50:58.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 -- 172.21.15.12:0/4228550709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32dc0664a0 msgr2=0x7f32dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:58.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32dc0664a0 0x7f32dc068960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f32f8066dd0 tx=0x7f32e8008040 comp rx=0 tx=0).stop 2023-11-28T21:50:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 -- 172.21.15.12:0/4228550709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f80c4000 msgr2=0x7f32f8159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:50:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f80c4000 0x7f32f8159ef0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f32f400c480 tx=0x7f32f400c940 comp rx=0 tx=0).stop 2023-11-28T21:50:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 -- 172.21.15.12:0/4228550709 shutdown_connections 2023-11-28T21:50:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f32dc0664a0 0x7f32dc068960 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f8161600 0x7f32f81f9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8158420 0x7f32f8152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 --2- 172.21.15.12:0/4228550709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f80c4000 0x7f32f8159ef0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:50:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 -- 172.21.15.12:0/4228550709 >> 172.21.15.12:0/4228550709 conn(0x7f32f8066360 msgr2=0x7f32f80c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:50:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 -- 172.21.15.12:0/4228550709 shutdown_connections 2023-11-28T21:50:58.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:50:58.419+0000 7f32fff72640 1 -- 172.21.15.12:0/4228550709 wait complete. 2023-11-28T21:50:58.445 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:51:00.445 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:51:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7daef80640 1 Processor -- start 2023-11-28T21:51:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7daef80640 1 -- start start 2023-11-28T21:51:00.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7daef80640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7da806a5b0 0x7f7da80c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:00.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7daef80640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da80c6350 con 0x7f7da8070bd0 2023-11-28T21:51:00.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7daef80640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da80c65d0 con 0x7f7da8153ee0 2023-11-28T21:51:00.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7daef80640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da80c6b90 con 0x7f7da806a5b0 2023-11-28T21:51:00.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7da7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7da806a5b0 0x7f7da80c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:00.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7da7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f7da806a5b0 0x7f7da80c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60368/0 (socket says 172.21.15.12:60368) 2023-11-28T21:51:00.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7da7fff640 1 -- 172.21.15.12:0/235303849 learned_addr learned my addr 172.21.15.12:0/235303849 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:00.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7dad51d640 1 --1- 172.21.15.12:0/235303849 >> v1:172.21.15.12:6789/0 conn(0x7f7da8070bd0 0x7f7da8071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:39432/0 (socket says 172.21.15.12:39432) 2023-11-28T21:51:00.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7da7fff640 1 -- 172.21.15.12:0/235303849 >> v1:172.21.15.144:6789/0 conn(0x7f7da8153ee0 legacy=0x7f7da815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:51:00.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7da7fff640 1 -- 172.21.15.12:0/235303849 >> v1:172.21.15.12:6789/0 conn(0x7f7da8070bd0 legacy=0x7f7da8071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:00.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.599+0000 7f7da7fff640 1 -- 172.21.15.12:0/235303849 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da80c68b0 con 0x7f7da806a5b0 2023-11-28T21:51:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7da7fff640 1 --2- 172.21.15.12:0/235303849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 0x7f7da80c7a80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7d94006fd0 tx=0x7f7d9401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530b9e24f34a017a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:00.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7da77fe640 1 -- 172.21.15.12:0/235303849 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d940a89d0 con 0x7f7da806a5b0 2023-11-28T21:51:00.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7da77fe640 1 -- 172.21.15.12:0/235303849 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d940a8b30 con 0x7f7da806a5b0 2023-11-28T21:51:00.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7da77fe640 1 -- 172.21.15.12:0/235303849 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d940b0440 con 0x7f7da806a5b0 2023-11-28T21:51:00.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7daef80640 1 -- 172.21.15.12:0/235303849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 msgr2=0x7f7da80c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:00.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7daef80640 1 --2- 172.21.15.12:0/235303849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 0x7f7da80c7a80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7d94006fd0 tx=0x7f7d9401ecd0 comp rx=0 tx=0).stop 2023-11-28T21:51:00.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7daef80640 1 -- 172.21.15.12:0/235303849 shutdown_connections 2023-11-28T21:51:00.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7daef80640 1 --2- 172.21.15.12:0/235303849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 0x7f7da80c7a80 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:00.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7daef80640 1 -- 172.21.15.12:0/235303849 >> 172.21.15.12:0/235303849 conn(0x7f7da80c8450 msgr2=0x7f7da80c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:00.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7daef80640 1 -- 172.21.15.12:0/235303849 shutdown_connections 2023-11-28T21:51:00.609 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7daef80640 1 -- 172.21.15.12:0/235303849 wait complete. 2023-11-28T21:51:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7daef80640 1 Processor -- start 2023-11-28T21:51:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.603+0000 7f7daef80640 1 -- start start 2023-11-28T21:51:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7daef80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 0x7f7da81e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:00.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7daef80640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7da8070bd0 0x7f7da8157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7daef80640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7da8153ee0 0x7f7da814fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7daef80640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da8152ac0 con 0x7f7da806a5b0 2023-11-28T21:51:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7daef80640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da81ef970 con 0x7f7da8070bd0 2023-11-28T21:51:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7daef80640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da8152d30 con 0x7f7da8153ee0 2023-11-28T21:51:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7dacd1c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 0x7f7da81e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:00.612 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7dacd1c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 0x7f7da81e42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60384/0 (socket says 172.21.15.12:60384) 2023-11-28T21:51:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7dacd1c640 1 -- 172.21.15.12:0/1976423768 learned_addr learned my addr 172.21.15.12:0/1976423768 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7da7fff640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7da8070bd0 0x7f7da8157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7dad51d640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7da8153ee0 0x7f7da814fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7dacd1c640 1 -- 172.21.15.12:0/1976423768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7da8153ee0 msgr2=0x7f7da814fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7dacd1c640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7da8153ee0 0x7f7da814fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7dacd1c640 1 -- 172.21.15.12:0/1976423768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7da8070bd0 msgr2=0x7f7da8157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:00.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7dacd1c640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7da8070bd0 0x7f7da8157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:00.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7dacd1c640 1 -- 172.21.15.12:0/1976423768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d94009480 con 0x7f7da806a5b0 2023-11-28T21:51:00.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7dacd1c640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 0x7f7da81e42c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00b410 tx=0x7f7d9c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:00.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7da57fa640 1 -- 172.21.15.12:0/1976423768 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c004020 con 0x7f7da806a5b0 2023-11-28T21:51:00.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7da57fa640 1 -- 172.21.15.12:0/1976423768 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c004180 con 0x7f7da806a5b0 2023-11-28T21:51:00.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7daef80640 1 -- 172.21.15.12:0/1976423768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da8153010 con 0x7f7da806a5b0 2023-11-28T21:51:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7da57fa640 1 -- 172.21.15.12:0/1976423768 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c011630 con 0x7f7da806a5b0 2023-11-28T21:51:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7daef80640 1 -- 172.21.15.12:0/1976423768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da81eff10 con 0x7f7da806a5b0 2023-11-28T21:51:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.607+0000 7f7daef80640 1 -- 172.21.15.12:0/1976423768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da8074f60 con 0x7f7da806a5b0 2023-11-28T21:51:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.611+0000 7f7da57fa640 1 -- 172.21.15.12:0/1976423768 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7d9c011790 con 0x7f7da806a5b0 2023-11-28T21:51:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.611+0000 7f7da57fa640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d8c066470 0x7f7d8c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:00.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.611+0000 7f7da57fa640 1 -- 172.21.15.12:0/1976423768 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f7d9c084340 con 0x7f7da806a5b0 2023-11-28T21:51:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.611+0000 7f7da7fff640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d8c066470 0x7f7d8c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:00.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.611+0000 7f7da7fff640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d8c066470 0x7f7d8c068930 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7d940b2000 tx=0x7f7d940aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:00.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.615+0000 7f7da57fa640 1 -- 172.21.15.12:0/1976423768 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7d9c051020 con 0x7f7da806a5b0 2023-11-28T21:51:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 -- 172.21.15.12:0/1976423768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7da8074120 con 0x7f7da806a5b0 2023-11-28T21:51:00.810 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7da57fa640 1 -- 172.21.15.12:0/1976423768 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5729 (secure 0 0 0) 0x7f7d9c015070 con 0x7f7da806a5b0 2023-11-28T21:51:00.811 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:51:00.811 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:50:55.892717+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:reconnect","state_seq":159,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:51:00.811 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 67 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 -- 172.21.15.12:0/1976423768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d8c066470 msgr2=0x7f7d8c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d8c066470 0x7f7d8c068930 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7d940b2000 tx=0x7f7d940aa070 comp rx=0 tx=0).stop 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 -- 172.21.15.12:0/1976423768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 msgr2=0x7f7da81e42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 0x7f7da81e42c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00b410 tx=0x7f7d9c00b8d0 comp rx=0 tx=0).stop 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 -- 172.21.15.12:0/1976423768 shutdown_connections 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7d8c066470 0x7f7d8c068930 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7da8153ee0 0x7f7da814fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7da8070bd0 0x7f7da8157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 --2- 172.21.15.12:0/1976423768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7da806a5b0 0x7f7da81e42c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 -- 172.21.15.12:0/1976423768 >> 172.21.15.12:0/1976423768 conn(0x7f7da80c8450 msgr2=0x7f7da80cad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 -- 172.21.15.12:0/1976423768 shutdown_connections 2023-11-28T21:51:00.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:00.807+0000 7f7daef80640 1 -- 172.21.15.12:0/1976423768 wait complete. 2023-11-28T21:51:00.832 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:51:02.833 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:51:02.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.983+0000 7f65a6b1d640 1 Processor -- start 2023-11-28T21:51:02.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.983+0000 7f65a6b1d640 1 -- start start 2023-11-28T21:51:02.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a6b1d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f65a0150c20 0x7f65a0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:02.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a6b1d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65a006e300 0x7f65a00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:02.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a6b1d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a0067410 con 0x7f65a006cf40 2023-11-28T21:51:02.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a6b1d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a006e090 con 0x7f65a0150c20 2023-11-28T21:51:02.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a6b1d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0157f30 con 0x7f65a006e300 2023-11-28T21:51:02.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a48b9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f65a0150c20 0x7f65a0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:02.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a48b9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f65a0150c20 0x7f65a0156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60402/0 (socket says 172.21.15.12:60402) 2023-11-28T21:51:02.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a48b9640 1 -- 172.21.15.12:0/564732344 learned_addr learned my addr 172.21.15.12:0/564732344 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:02.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f659ffff640 1 --2- 172.21.15.12:0/564732344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65a006e300 0x7f65a00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:02.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a48b9640 1 -- 172.21.15.12:0/564732344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65a006e300 msgr2=0x7f65a00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:51:02.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a48b9640 1 --2- 172.21.15.12:0/564732344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65a006e300 0x7f65a00c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:02.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a48b9640 1 -- 172.21.15.12:0/564732344 >> v1:172.21.15.144:6789/0 conn(0x7f65a006cf40 legacy=0x7f65a0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:51:02.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.987+0000 7f65a48b9640 1 -- 172.21.15.12:0/564732344 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a0151000 con 0x7f65a0150c20 2023-11-28T21:51:02.994 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a48b9640 1 --2- 172.21.15.12:0/564732344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a0150c20 0x7f65a0156910 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6594009870 tx=0x7f659401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2f17f31cdee887 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:02.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f659f7fe640 1 -- 172.21.15.12:0/564732344 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65940ac410 con 0x7f65a0150c20 2023-11-28T21:51:02.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f659f7fe640 1 -- 172.21.15.12:0/564732344 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65940ac570 con 0x7f65a0150c20 2023-11-28T21:51:02.995 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f659f7fe640 1 -- 172.21.15.12:0/564732344 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65940ac900 con 0x7f65a0150c20 2023-11-28T21:51:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a6b1d640 1 -- 172.21.15.12:0/564732344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a0150c20 msgr2=0x7f65a0156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a6b1d640 1 --2- 172.21.15.12:0/564732344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a0150c20 0x7f65a0156910 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6594009870 tx=0x7f659401c500 comp rx=0 tx=0).stop 2023-11-28T21:51:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a6b1d640 1 -- 172.21.15.12:0/564732344 shutdown_connections 2023-11-28T21:51:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a6b1d640 1 --2- 172.21.15.12:0/564732344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65a006e300 0x7f65a00c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a6b1d640 1 --2- 172.21.15.12:0/564732344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a0150c20 0x7f65a0156910 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a6b1d640 1 -- 172.21.15.12:0/564732344 >> 172.21.15.12:0/564732344 conn(0x7f65a0066360 msgr2=0x7f65a0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a6b1d640 1 -- 172.21.15.12:0/564732344 shutdown_connections 2023-11-28T21:51:02.996 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a6b1d640 1 -- 172.21.15.12:0/564732344 wait complete. 2023-11-28T21:51:02.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a6b1d640 1 Processor -- start 2023-11-28T21:51:02.997 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.991+0000 7f65a6b1d640 1 -- start start 2023-11-28T21:51:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a6b1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a006cf40 0x7f65a014dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a6b1d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65a006e300 0x7f65a01f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a6b1d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65a0150c20 0x7f65a01556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a6b1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a014c660 con 0x7f65a006cf40 2023-11-28T21:51:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a6b1d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a014ea00 con 0x7f65a006e300 2023-11-28T21:51:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a6b1d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a014ece0 con 0x7f65a0150c20 2023-11-28T21:51:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a48b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a006cf40 0x7f65a014dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a48b9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a006cf40 0x7f65a014dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60416/0 (socket says 172.21.15.12:60416) 2023-11-28T21:51:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a48b9640 1 -- 172.21.15.12:0/271318737 learned_addr learned my addr 172.21.15.12:0/271318737 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:03.000 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a50ba640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65a0150c20 0x7f65a01556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f659ffff640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65a006e300 0x7f65a01f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a48b9640 1 -- 172.21.15.12:0/271318737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65a0150c20 msgr2=0x7f65a01556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a48b9640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65a0150c20 0x7f65a01556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a48b9640 1 -- 172.21.15.12:0/271318737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65a006e300 msgr2=0x7f65a01f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a48b9640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65a006e300 0x7f65a01f6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a48b9640 1 -- 172.21.15.12:0/271318737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65940094d0 con 0x7f65a006cf40 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a48b9640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a006cf40 0x7f65a014dac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f65940af040 tx=0x7f65940192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f659d7fa640 1 -- 172.21.15.12:0/271318737 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65940ac410 con 0x7f65a006cf40 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f659d7fa640 1 -- 172.21.15.12:0/271318737 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65940b65f0 con 0x7f65a006cf40 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a6b1d640 1 -- 172.21.15.12:0/271318737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a014efc0 con 0x7f65a006cf40 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f659d7fa640 1 -- 172.21.15.12:0/271318737 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65940b69a0 con 0x7f65a006cf40 2023-11-28T21:51:03.001 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a6b1d640 1 -- 172.21.15.12:0/271318737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a014e510 con 0x7f65a006cf40 2023-11-28T21:51:03.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f65a6b1d640 1 -- 172.21.15.12:0/271318737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a014c150 con 0x7f65a006cf40 2023-11-28T21:51:03.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.995+0000 7f659d7fa640 1 -- 172.21.15.12:0/271318737 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f65940b6b00 con 0x7f65a006cf40 2023-11-28T21:51:03.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.999+0000 7f659d7fa640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65840664a0 0x7f6584068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:03.002 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.999+0000 7f659d7fa640 1 -- 172.21.15.12:0/271318737 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f659411ef40 con 0x7f65a006cf40 2023-11-28T21:51:03.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.999+0000 7f659ffff640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65840664a0 0x7f6584068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:03.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:02.999+0000 7f659ffff640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65840664a0 0x7f6584068960 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f65a00c9c80 tx=0x7f658c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:03.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.003+0000 7f659d7fa640 1 -- 172.21.15.12:0/271318737 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f65a014c150 con 0x7f65a006cf40 2023-11-28T21:51:03.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 -- 172.21.15.12:0/271318737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65a014c150 con 0x7f65a006cf40 2023-11-28T21:51:03.199 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f659d7fa640 1 -- 172.21.15.12:0/271318737 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5726 (secure 0 0 0) 0x7f65a014c150 con 0x7f65a006cf40 2023-11-28T21:51:03.200 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:51:03.200 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:01.833532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:rejoin","state_seq":161,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:51:03.200 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 68 2023-11-28T21:51:03.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 -- 172.21.15.12:0/271318737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65840664a0 msgr2=0x7f6584068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:03.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65840664a0 0x7f6584068960 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f65a00c9c80 tx=0x7f658c008040 comp rx=0 tx=0).stop 2023-11-28T21:51:03.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 -- 172.21.15.12:0/271318737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a006cf40 msgr2=0x7f65a014dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:03.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a006cf40 0x7f65a014dac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f65940af040 tx=0x7f65940192a0 comp rx=0 tx=0).stop 2023-11-28T21:51:03.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 -- 172.21.15.12:0/271318737 shutdown_connections 2023-11-28T21:51:03.202 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f65840664a0 0x7f6584068960 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:03.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65a0150c20 0x7f65a01556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:03.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65a006e300 0x7f65a01f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:03.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 --2- 172.21.15.12:0/271318737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65a006cf40 0x7f65a014dac0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:03.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.195+0000 7f65a6b1d640 1 -- 172.21.15.12:0/271318737 >> 172.21.15.12:0/271318737 conn(0x7f65a0066360 msgr2=0x7f65a00c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:03.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.199+0000 7f65a6b1d640 1 -- 172.21.15.12:0/271318737 shutdown_connections 2023-11-28T21:51:03.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:03.199+0000 7f65a6b1d640 1 -- 172.21.15.12:0/271318737 wait complete. 2023-11-28T21:51:03.222 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 68, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 68, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_5057": { "addr": "172.21.15.144:6833/126572048", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 126572048, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 126572048, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5057, "incarnation": 65, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:rejoin", "state_seq": 161 }, "gid_6330": { "addr": "172.21.15.12:6839/754378087", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 754378087, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 754378087, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6330, "incarnation": 63, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 50, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:51:01.833532+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 5057, "mds_1": 6330 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:51:05.222 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:51:05.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.371+0000 7f3d66c46640 1 Processor -- start 2023-11-28T21:51:05.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.371+0000 7f3d66c46640 1 -- start start 2023-11-28T21:51:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d66c46640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3d60151af0 0x7f3d600c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d66c46640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3d60151ed0 0x7f3d600c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d66c46640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d60157b70 con 0x7f3d6006e300 2023-11-28T21:51:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d66c46640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d600707f0 con 0x7f3d60151af0 2023-11-28T21:51:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d66c46640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d600c7b90 con 0x7f3d60151ed0 2023-11-28T21:51:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d649e2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3d60151af0 0x7f3d600c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d649e2640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3d60151af0 0x7f3d600c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54954/0 (socket says 172.21.15.12:54954) 2023-11-28T21:51:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d649e2640 1 -- 172.21.15.12:0/2624760335 learned_addr learned my addr 172.21.15.12:0/2624760335 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:05.379 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d5ffff640 1 --2- 172.21.15.12:0/2624760335 >> v2:172.21.15.144:3300/0 conn(0x7f3d60151ed0 0x7f3d600c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d649e2640 1 -- 172.21.15.12:0/2624760335 >> v2:172.21.15.144:3300/0 conn(0x7f3d60151ed0 msgr2=0x7f3d600c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:51:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d649e2640 1 --2- 172.21.15.12:0/2624760335 >> v2:172.21.15.144:3300/0 conn(0x7f3d60151ed0 0x7f3d600c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d649e2640 1 -- 172.21.15.12:0/2624760335 >> v1:172.21.15.144:6789/0 conn(0x7f3d6006e300 legacy=0x7f3d60072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:05.380 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.375+0000 7f3d649e2640 1 -- 172.21.15.12:0/2624760335 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d60157e50 con 0x7f3d60151af0 2023-11-28T21:51:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d649e2640 1 --2- 172.21.15.12:0/2624760335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60151af0 0x7f3d600c7650 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3d600c8e60 tx=0x7f3d5401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fa505d257d8b60 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d5f7fe640 1 -- 172.21.15.12:0/2624760335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d540ab600 con 0x7f3d60151af0 2023-11-28T21:51:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d5f7fe640 1 -- 172.21.15.12:0/2624760335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d540a7050 con 0x7f3d60151af0 2023-11-28T21:51:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d5f7fe640 1 -- 172.21.15.12:0/2624760335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d540ab980 con 0x7f3d60151af0 2023-11-28T21:51:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d66c46640 1 -- 172.21.15.12:0/2624760335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60151af0 msgr2=0x7f3d600c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:05.383 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d66c46640 1 --2- 172.21.15.12:0/2624760335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60151af0 0x7f3d600c7650 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3d600c8e60 tx=0x7f3d5401c500 comp rx=0 tx=0).stop 2023-11-28T21:51:05.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d66c46640 1 -- 172.21.15.12:0/2624760335 shutdown_connections 2023-11-28T21:51:05.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d66c46640 1 --2- 172.21.15.12:0/2624760335 >> v2:172.21.15.144:3300/0 conn(0x7f3d60151ed0 0x7f3d600c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:05.384 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d66c46640 1 --2- 172.21.15.12:0/2624760335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60151af0 0x7f3d600c7650 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:05.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d66c46640 1 -- 172.21.15.12:0/2624760335 >> 172.21.15.12:0/2624760335 conn(0x7f3d60066360 msgr2=0x7f3d60066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:05.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d66c46640 1 -- 172.21.15.12:0/2624760335 shutdown_connections 2023-11-28T21:51:05.385 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d66c46640 1 -- 172.21.15.12:0/2624760335 wait complete. 2023-11-28T21:51:05.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d66c46640 1 Processor -- start 2023-11-28T21:51:05.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.379+0000 7f3d66c46640 1 -- start start 2023-11-28T21:51:05.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d66c46640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d6006e300 0x7f3d601567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:05.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d66c46640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d60151af0 0x7f3d60156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:05.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d66c46640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60157590 0x7f3d6013fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:05.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d66c46640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600ca4d0 con 0x7f3d60157590 2023-11-28T21:51:05.389 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d66c46640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600c9ef0 con 0x7f3d60151af0 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d66c46640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60155740 con 0x7f3d6006e300 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d651e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60157590 0x7f3d6013fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d651e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60157590 0x7f3d6013fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54960/0 (socket says 172.21.15.12:54960) 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d651e3640 1 -- 172.21.15.12:0/2364825540 learned_addr learned my addr 172.21.15.12:0/2364825540 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d649e2640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d6006e300 0x7f3d601567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d5ffff640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d60151af0 0x7f3d60156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d651e3640 1 -- 172.21.15.12:0/2364825540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d6006e300 msgr2=0x7f3d601567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d651e3640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d6006e300 0x7f3d601567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d651e3640 1 -- 172.21.15.12:0/2364825540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d60151af0 msgr2=0x7f3d60156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d651e3640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d60151af0 0x7f3d60156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d651e3640 1 -- 172.21.15.12:0/2364825540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d540094d0 con 0x7f3d60157590 2023-11-28T21:51:05.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d5ffff640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d60151af0 0x7f3d60156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:51:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d651e3640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60157590 0x7f3d6013fbe0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800c7a0 tx=0x7f3d5800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d5d7fa640 1 -- 172.21.15.12:0/2364825540 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58003b80 con 0x7f3d60157590 2023-11-28T21:51:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d5d7fa640 1 -- 172.21.15.12:0/2364825540 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d58003ce0 con 0x7f3d60157590 2023-11-28T21:51:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d5d7fa640 1 -- 172.21.15.12:0/2364825540 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5800aa30 con 0x7f3d60157590 2023-11-28T21:51:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d66c46640 1 -- 172.21.15.12:0/2364825540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d60155a20 con 0x7f3d60157590 2023-11-28T21:51:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d66c46640 1 -- 172.21.15.12:0/2364825540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d60155fe0 con 0x7f3d60157590 2023-11-28T21:51:05.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d66c46640 1 -- 172.21.15.12:0/2364825540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d60073010 con 0x7f3d60157590 2023-11-28T21:51:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.383+0000 7f3d5d7fa640 1 -- 172.21.15.12:0/2364825540 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3d5800ab90 con 0x7f3d60157590 2023-11-28T21:51:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.387+0000 7f3d5d7fa640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d440664c0 0x7f3d44068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.387+0000 7f3d5d7fa640 1 -- 172.21.15.12:0/2364825540 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f3d580842a0 con 0x7f3d60157590 2023-11-28T21:51:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.387+0000 7f3d649e2640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d440664c0 0x7f3d44068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:05.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.387+0000 7f3d649e2640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d440664c0 0x7f3d44068980 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3d540ad000 tx=0x7f3d54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:05.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.391+0000 7f3d5d7fa640 1 -- 172.21.15.12:0/2364825540 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3d58051080 con 0x7f3d60157590 2023-11-28T21:51:05.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 -- 172.21.15.12:0/2364825540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d60074f60 con 0x7f3d60157590 2023-11-28T21:51:05.587 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d5d7fa640 1 -- 172.21.15.12:0/2364825540 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5726 (secure 0 0 0) 0x7f3d580546a0 con 0x7f3d60157590 2023-11-28T21:51:05.588 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:51:05.588 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:01.833532+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:rejoin","state_seq":161,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:51:05.588 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 68 2023-11-28T21:51:05.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 -- 172.21.15.12:0/2364825540 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d440664c0 msgr2=0x7f3d44068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:05.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d440664c0 0x7f3d44068980 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3d540ad000 tx=0x7f3d54008040 comp rx=0 tx=0).stop 2023-11-28T21:51:05.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 -- 172.21.15.12:0/2364825540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60157590 msgr2=0x7f3d6013fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:05.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60157590 0x7f3d6013fbe0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800c7a0 tx=0x7f3d5800cc60 comp rx=0 tx=0).stop 2023-11-28T21:51:05.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 -- 172.21.15.12:0/2364825540 shutdown_connections 2023-11-28T21:51:05.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f3d440664c0 0x7f3d44068980 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:05.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d60157590 0x7f3d6013fbe0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:05.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d60151af0 0x7f3d60156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:05.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 --2- 172.21.15.12:0/2364825540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d6006e300 0x7f3d601567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:05.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 -- 172.21.15.12:0/2364825540 >> 172.21.15.12:0/2364825540 conn(0x7f3d60066360 msgr2=0x7f3d6006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:05.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 -- 172.21.15.12:0/2364825540 shutdown_connections 2023-11-28T21:51:05.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:05.583+0000 7f3d66c46640 1 -- 172.21.15.12:0/2364825540 wait complete. 2023-11-28T21:51:05.609 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:51:07.611 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:51:07.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.759+0000 7f966b29c640 1 Processor -- start 2023-11-28T21:51:07.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f966b29c640 1 -- start start 2023-11-28T21:51:07.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f966b29c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f966406e300 0x7f9664156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:07.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f966b29c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f966406cf40 0x7f9664158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:07.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f966b29c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9664067410 con 0x7f9664150c20 2023-11-28T21:51:07.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f966b29c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f966406e090 con 0x7f966406cf40 2023-11-28T21:51:07.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f966b29c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9664157f30 con 0x7f966406e300 2023-11-28T21:51:07.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f9668837640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f966406e300 0x7f9664156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:07.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f9669839640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f966406cf40 0x7f9664158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:07.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f9669038640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f9664150c20 0x7f96640c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:44128/0 (socket says 172.21.15.12:44128) 2023-11-28T21:51:07.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f9669038640 1 -- 172.21.15.12:0/1140536387 learned_addr learned my addr 172.21.15.12:0/1140536387 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:51:07.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f965bfff640 1 -- 172.21.15.12:0/1140536387 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048968029 0 0) 0x7f9664067410 con 0x7f9664150c20 2023-11-28T21:51:07.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f965bfff640 1 -- 172.21.15.12:0/1140536387 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9650003530 con 0x7f9664150c20 2023-11-28T21:51:07.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f9668837640 1 -- 172.21.15.12:0/1140536387 >> v2:172.21.15.144:3300/0 conn(0x7f966406cf40 msgr2=0x7f9664158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:51:07.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f9668837640 1 --2- 172.21.15.12:0/1140536387 >> v2:172.21.15.144:3300/0 conn(0x7f966406cf40 0x7f9664158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:07.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f9668837640 1 -- 172.21.15.12:0/1140536387 >> v1:172.21.15.144:6789/0 conn(0x7f9664150c20 legacy=0x7f96640c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:07.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.763+0000 7f9668837640 1 -- 172.21.15.12:0/1140536387 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9664151000 con 0x7f966406e300 2023-11-28T21:51:07.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f9668837640 1 --2- 172.21.15.12:0/1140536387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f966406e300 0x7f9664156910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96540097c0 tx=0x7f965401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d68ce7b89f51d165 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:07.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f965bfff640 1 -- 172.21.15.12:0/1140536387 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540ad640 con 0x7f966406e300 2023-11-28T21:51:07.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f965bfff640 1 -- 172.21.15.12:0/1140536387 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96540ad7a0 con 0x7f966406e300 2023-11-28T21:51:07.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f965bfff640 1 -- 172.21.15.12:0/1140536387 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540adb30 con 0x7f966406e300 2023-11-28T21:51:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f966b29c640 1 -- 172.21.15.12:0/1140536387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f966406e300 msgr2=0x7f9664156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f966b29c640 1 --2- 172.21.15.12:0/1140536387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f966406e300 0x7f9664156910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96540097c0 tx=0x7f965401c480 comp rx=0 tx=0).stop 2023-11-28T21:51:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f966b29c640 1 -- 172.21.15.12:0/1140536387 shutdown_connections 2023-11-28T21:51:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f966b29c640 1 --2- 172.21.15.12:0/1140536387 >> v2:172.21.15.144:3300/0 conn(0x7f966406cf40 0x7f9664158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f966b29c640 1 --2- 172.21.15.12:0/1140536387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f966406e300 0x7f9664156910 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f966b29c640 1 -- 172.21.15.12:0/1140536387 >> 172.21.15.12:0/1140536387 conn(0x7f9664066360 msgr2=0x7f9664066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f966b29c640 1 -- 172.21.15.12:0/1140536387 shutdown_connections 2023-11-28T21:51:07.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.767+0000 7f966b29c640 1 -- 172.21.15.12:0/1140536387 wait complete. 2023-11-28T21:51:07.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f966b29c640 1 Processor -- start 2023-11-28T21:51:07.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f966b29c640 1 -- start start 2023-11-28T21:51:07.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f966b29c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f966406cf40 0x7f966414c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:07.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f966b29c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966406e300 0x7f966414ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:07.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f966b29c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9664150c20 0x7f966414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:07.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f966b29c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966414f990 con 0x7f966406cf40 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f966b29c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96641f18c0 con 0x7f966406e300 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f966b29c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966414fbe0 con 0x7f9664150c20 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9669038640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f966406cf40 0x7f966414c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9669038640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f966406cf40 0x7f966414c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54976/0 (socket says 172.21.15.12:54976) 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9669038640 1 -- 172.21.15.12:0/4280990645 learned_addr learned my addr 172.21.15.12:0/4280990645 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9669839640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9664150c20 0x7f966414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9668837640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966406e300 0x7f966414ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9669038640 1 -- 172.21.15.12:0/4280990645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9664150c20 msgr2=0x7f966414e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9669038640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9664150c20 0x7f966414e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9669038640 1 -- 172.21.15.12:0/4280990645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966406e300 msgr2=0x7f966414ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:07.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9669038640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966406e300 0x7f966414ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:07.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9669038640 1 -- 172.21.15.12:0/4280990645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9654009420 con 0x7f966406cf40 2023-11-28T21:51:07.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9669038640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f966406cf40 0x7f966414c100 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f966000c420 tx=0x7f966000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:07.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9659ffb640 1 -- 172.21.15.12:0/4280990645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660005cf0 con 0x7f966406cf40 2023-11-28T21:51:07.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9659ffb640 1 -- 172.21.15.12:0/4280990645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9660005e50 con 0x7f966406cf40 2023-11-28T21:51:07.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f9659ffb640 1 -- 172.21.15.12:0/4280990645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96600072d0 con 0x7f966406cf40 2023-11-28T21:51:07.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f966b29c640 1 -- 172.21.15.12:0/4280990645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96640c2ef0 con 0x7f966406cf40 2023-11-28T21:51:07.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.771+0000 7f966b29c640 1 -- 172.21.15.12:0/4280990645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966414fec0 con 0x7f966406cf40 2023-11-28T21:51:07.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.775+0000 7f966b29c640 1 -- 172.21.15.12:0/4280990645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96640c3410 con 0x7f966406cf40 2023-11-28T21:51:07.779 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.775+0000 7f9659ffb640 1 -- 172.21.15.12:0/4280990645 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9660007430 con 0x7f966406cf40 2023-11-28T21:51:07.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.775+0000 7f9659ffb640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9648066470 0x7f9648068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:07.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.775+0000 7f9659ffb640 1 -- 172.21.15.12:0/4280990645 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f9660083ba0 con 0x7f966406cf40 2023-11-28T21:51:07.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.775+0000 7f9668837640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9648066470 0x7f9648068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:07.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.775+0000 7f9668837640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9648066470 0x7f9648068930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f96540af000 tx=0x7f9654008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:07.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.779+0000 7f9659ffb640 1 -- 172.21.15.12:0/4280990645 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f96640c3410 con 0x7f966406cf40 2023-11-28T21:51:07.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.967+0000 7f966b29c640 1 -- 172.21.15.12:0/4280990645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96640c3410 con 0x7f966406cf40 2023-11-28T21:51:07.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.967+0000 7f9659ffb640 1 -- 172.21.15.12:0/4280990645 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f96640c3410 con 0x7f966406cf40 2023-11-28T21:51:07.973 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:51:07.973 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:06.983538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:clientreplay","state_seq":163,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:51:07.973 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 69 2023-11-28T21:51:07.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 -- 172.21.15.12:0/4280990645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9648066470 msgr2=0x7f9648068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:07.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9648066470 0x7f9648068930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f96540af000 tx=0x7f9654008040 comp rx=0 tx=0).stop 2023-11-28T21:51:07.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 -- 172.21.15.12:0/4280990645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f966406cf40 msgr2=0x7f966414c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:07.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f966406cf40 0x7f966414c100 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f966000c420 tx=0x7f966000c8e0 comp rx=0 tx=0).stop 2023-11-28T21:51:07.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 -- 172.21.15.12:0/4280990645 shutdown_connections 2023-11-28T21:51:07.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9648066470 0x7f9648068930 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:07.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9664150c20 0x7f966414e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:07.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f966406e300 0x7f966414ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:07.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 --2- 172.21.15.12:0/4280990645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f966406cf40 0x7f966414c100 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:07.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 -- 172.21.15.12:0/4280990645 >> 172.21.15.12:0/4280990645 conn(0x7f9664066360 msgr2=0x7f966406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:07.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 -- 172.21.15.12:0/4280990645 shutdown_connections 2023-11-28T21:51:07.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:07.971+0000 7f966b29c640 1 -- 172.21.15.12:0/4280990645 wait complete. 2023-11-28T21:51:07.994 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:51:09.996 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:51:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.147+0000 7f2fbfa79640 1 Processor -- start 2023-11-28T21:51:10.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.147+0000 7f2fbfa79640 1 -- start start 2023-11-28T21:51:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.147+0000 7f2fbfa79640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fb8158420 0x7f2fb806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.147+0000 7f2fbfa79640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2fb80c38f0 0x7f2fb80c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.147+0000 7f2fbfa79640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb80c3460 con 0x7f2fb815e010 2023-11-28T21:51:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.147+0000 7f2fbfa79640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb8158800 con 0x7f2fb8158420 2023-11-28T21:51:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.147+0000 7f2fbfa79640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb8162610 con 0x7f2fb80c38f0 2023-11-28T21:51:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.147+0000 7f2fbd815640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2fb8158420 0x7f2fb806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.147+0000 7f2fbe016640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2fb815e010 0x7f2fb815e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:56800/0 (socket says 172.21.15.12:56800) 2023-11-28T21:51:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.147+0000 7f2fbe016640 1 -- 172.21.15.12:0/11005337 learned_addr learned my addr 172.21.15.12:0/11005337 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:51:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.151+0000 7f2fbd014640 1 --2- 172.21.15.12:0/11005337 >> v2:172.21.15.144:3300/0 conn(0x7f2fb80c38f0 0x7f2fb80c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.151+0000 7f2fbd815640 1 -- 172.21.15.12:0/11005337 >> v2:172.21.15.144:3300/0 conn(0x7f2fb80c38f0 msgr2=0x7f2fb80c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:51:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.151+0000 7f2fbd815640 1 --2- 172.21.15.12:0/11005337 >> v2:172.21.15.144:3300/0 conn(0x7f2fb80c38f0 0x7f2fb80c3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.151+0000 7f2fbd815640 1 -- 172.21.15.12:0/11005337 >> v1:172.21.15.12:6789/0 conn(0x7f2fb815e010 legacy=0x7f2fb815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.151+0000 7f2fbd815640 1 -- 172.21.15.12:0/11005337 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb80c2b70 con 0x7f2fb8158420 2023-11-28T21:51:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.151+0000 7f2fbc813640 1 -- 172.21.15.12:0/11005337 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673867052 0 0) 0x7f2fb80c3460 con 0x7f2fb815e010 2023-11-28T21:51:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.151+0000 7f2fbd815640 1 --2- 172.21.15.12:0/11005337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8158420 0x7f2fb806d950 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2fb806b930 tx=0x7f2fb401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d01347c803c2d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:10.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.151+0000 7f2fbc813640 1 -- 172.21.15.12:0/11005337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40ab600 con 0x7f2fb8158420 2023-11-28T21:51:10.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.151+0000 7f2fbc813640 1 -- 172.21.15.12:0/11005337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb40a7050 con 0x7f2fb8158420 2023-11-28T21:51:10.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.151+0000 7f2fbc813640 1 -- 172.21.15.12:0/11005337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40ab980 con 0x7f2fb8158420 2023-11-28T21:51:10.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 -- 172.21.15.12:0/11005337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8158420 msgr2=0x7f2fb806d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 --2- 172.21.15.12:0/11005337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8158420 0x7f2fb806d950 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2fb806b930 tx=0x7f2fb401c460 comp rx=0 tx=0).stop 2023-11-28T21:51:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 -- 172.21.15.12:0/11005337 shutdown_connections 2023-11-28T21:51:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 --2- 172.21.15.12:0/11005337 >> v2:172.21.15.144:3300/0 conn(0x7f2fb80c38f0 0x7f2fb80c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 --2- 172.21.15.12:0/11005337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb8158420 0x7f2fb806d950 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 -- 172.21.15.12:0/11005337 >> 172.21.15.12:0/11005337 conn(0x7f2fb8066360 msgr2=0x7f2fb8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 -- 172.21.15.12:0/11005337 shutdown_connections 2023-11-28T21:51:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 -- 172.21.15.12:0/11005337 wait complete. 2023-11-28T21:51:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 Processor -- start 2023-11-28T21:51:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 -- start start 2023-11-28T21:51:10.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb80c38f0 0x7f2fb8153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:10.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fb8158420 0x7f2fb8153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:10.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb815e010 0x7f2fb8159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:10.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb8159000 con 0x7f2fb80c38f0 2023-11-28T21:51:10.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb814fb10 con 0x7f2fb815e010 2023-11-28T21:51:10.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbfa79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb8150b10 con 0x7f2fb8158420 2023-11-28T21:51:10.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbd815640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb80c38f0 0x7f2fb8153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:10.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbd815640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb80c38f0 0x7f2fb8153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54984/0 (socket says 172.21.15.12:54984) 2023-11-28T21:51:10.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbd815640 1 -- 172.21.15.12:0/3122751751 learned_addr learned my addr 172.21.15.12:0/3122751751 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:10.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbd014640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fb8158420 0x7f2fb8153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:10.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.155+0000 7f2fbe016640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb815e010 0x7f2fb8159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbd815640 1 -- 172.21.15.12:0/3122751751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fb8158420 msgr2=0x7f2fb8153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbd815640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fb8158420 0x7f2fb8153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbd815640 1 -- 172.21.15.12:0/3122751751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb815e010 msgr2=0x7f2fb8159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbd815640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb815e010 0x7f2fb8159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbd815640 1 -- 172.21.15.12:0/3122751751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb4009420 con 0x7f2fb80c38f0 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbd014640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fb8158420 0x7f2fb8153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbe016640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb815e010 0x7f2fb8159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbd815640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb80c38f0 0x7f2fb8153fe0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40ae040 tx=0x7f2fb4019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fae7fc640 1 -- 172.21.15.12:0/3122751751 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40b4610 con 0x7f2fb80c38f0 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fae7fc640 1 -- 172.21.15.12:0/3122751751 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb40b4770 con 0x7f2fb80c38f0 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbfa79640 1 -- 172.21.15.12:0/3122751751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb81546b0 con 0x7f2fb80c38f0 2023-11-28T21:51:10.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fae7fc640 1 -- 172.21.15.12:0/3122751751 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40b4b20 con 0x7f2fb80c38f0 2023-11-28T21:51:10.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbfa79640 1 -- 172.21.15.12:0/3122751751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb8150030 con 0x7f2fb80c38f0 2023-11-28T21:51:10.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fbfa79640 1 -- 172.21.15.12:0/3122751751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb8074f60 con 0x7f2fb80c38f0 2023-11-28T21:51:10.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.159+0000 7f2fae7fc640 1 -- 172.21.15.12:0/3122751751 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2fb40b2040 con 0x7f2fb80c38f0 2023-11-28T21:51:10.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.163+0000 7f2fae7fc640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f9c0664a0 0x7f2f9c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:10.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.163+0000 7f2fae7fc640 1 -- 172.21.15.12:0/3122751751 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f2fb411e530 con 0x7f2fb80c38f0 2023-11-28T21:51:10.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.163+0000 7f2fbd014640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f9c0664a0 0x7f2f9c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:10.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.163+0000 7f2fbd014640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f9c0664a0 0x7f2f9c068960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8157d00 tx=0x7f2fa8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:10.170 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.167+0000 7f2fae7fc640 1 -- 172.21.15.12:0/3122751751 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2fb40eb2a0 con 0x7f2fb80c38f0 2023-11-28T21:51:10.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 -- 172.21.15.12:0/3122751751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fb8154a60 con 0x7f2fb80c38f0 2023-11-28T21:51:10.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fae7fc640 1 -- 172.21.15.12:0/3122751751 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f2fb40ee8c0 con 0x7f2fb80c38f0 2023-11-28T21:51:10.359 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:51:10.360 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:06.983538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:clientreplay","state_seq":163,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:51:10.361 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 69 2023-11-28T21:51:10.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 -- 172.21.15.12:0/3122751751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f9c0664a0 msgr2=0x7f2f9c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:10.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f9c0664a0 0x7f2f9c068960 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8157d00 tx=0x7f2fa8008040 comp rx=0 tx=0).stop 2023-11-28T21:51:10.361 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 -- 172.21.15.12:0/3122751751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb80c38f0 msgr2=0x7f2fb8153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:10.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb80c38f0 0x7f2fb8153fe0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40ae040 tx=0x7f2fb4019200 comp rx=0 tx=0).stop 2023-11-28T21:51:10.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 -- 172.21.15.12:0/3122751751 shutdown_connections 2023-11-28T21:51:10.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f9c0664a0 0x7f2f9c068960 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:10.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb815e010 0x7f2fb8159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:10.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fb8158420 0x7f2fb8153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:10.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 --2- 172.21.15.12:0/3122751751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2fb80c38f0 0x7f2fb8153fe0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:10.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 -- 172.21.15.12:0/3122751751 >> 172.21.15.12:0/3122751751 conn(0x7f2fb8066360 msgr2=0x7f2fb815e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:10.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 -- 172.21.15.12:0/3122751751 shutdown_connections 2023-11-28T21:51:10.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:10.355+0000 7f2fbfa79640 1 -- 172.21.15.12:0/3122751751 wait complete. 2023-11-28T21:51:10.380 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:51:12.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:51:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.531+0000 7eff22c5f640 1 Processor -- start 2023-11-28T21:51:12.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.531+0000 7eff22c5f640 1 -- start start 2023-11-28T21:51:12.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff22c5f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff1c150c20 0x7eff1c156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:12.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff22c5f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06e300 0x7eff1c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:12.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff22c5f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1c067410 con 0x7eff1c06cf40 2023-11-28T21:51:12.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff22c5f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1c06e090 con 0x7eff1c150c20 2023-11-28T21:51:12.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff22c5f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c157f30 con 0x7eff1c06e300 2023-11-28T21:51:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff1bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06e300 0x7eff1c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:12.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff209fb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff1c150c20 0x7eff1c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff1bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06e300 0x7eff1c0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:37646/0 (socket says 172.21.15.12:37646) 2023-11-28T21:51:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff1bfff640 1 -- 172.21.15.12:0/3092418953 learned_addr learned my addr 172.21.15.12:0/3092418953 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff209fb640 1 --2- 172.21.15.12:0/3092418953 >> v2:172.21.15.144:3300/0 conn(0x7eff1c150c20 0x7eff1c156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:40280/0 (socket says 172.21.15.12:40280) 2023-11-28T21:51:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff211fc640 1 --1- 172.21.15.12:0/3092418953 >> v1:172.21.15.144:6789/0 conn(0x7eff1c06cf40 0x7eff1c158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:44132/0 (socket says 172.21.15.12:44132) 2023-11-28T21:51:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff209fb640 1 -- 172.21.15.12:0/3092418953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06e300 msgr2=0x7eff1c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:51:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff209fb640 1 --2- 172.21.15.12:0/3092418953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06e300 0x7eff1c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff209fb640 1 -- 172.21.15.12:0/3092418953 >> v1:172.21.15.144:6789/0 conn(0x7eff1c06cf40 legacy=0x7eff1c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff209fb640 1 -- 172.21.15.12:0/3092418953 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c151000 con 0x7eff1c150c20 2023-11-28T21:51:12.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.535+0000 7eff1bfff640 1 --2- 172.21.15.12:0/3092418953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06e300 0x7eff1c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:51:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff209fb640 1 --2- 172.21.15.12:0/3092418953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1c150c20 0x7eff1c156910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7eff1000ba60 tx=0x7eff1001eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63918df8c3bae1b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff1b7fe640 1 -- 172.21.15.12:0/3092418953 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff100a7a30 con 0x7eff1c150c20 2023-11-28T21:51:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff1b7fe640 1 -- 172.21.15.12:0/3092418953 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff100a7b90 con 0x7eff1c150c20 2023-11-28T21:51:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff1b7fe640 1 -- 172.21.15.12:0/3092418953 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff100af440 con 0x7eff1c150c20 2023-11-28T21:51:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff22c5f640 1 -- 172.21.15.12:0/3092418953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1c150c20 msgr2=0x7eff1c156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:12.544 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff22c5f640 1 --2- 172.21.15.12:0/3092418953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1c150c20 0x7eff1c156910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7eff1000ba60 tx=0x7eff1001eb50 comp rx=0 tx=0).stop 2023-11-28T21:51:12.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff22c5f640 1 -- 172.21.15.12:0/3092418953 shutdown_connections 2023-11-28T21:51:12.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff22c5f640 1 --2- 172.21.15.12:0/3092418953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06e300 0x7eff1c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:12.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff22c5f640 1 --2- 172.21.15.12:0/3092418953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1c150c20 0x7eff1c156910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:12.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff22c5f640 1 -- 172.21.15.12:0/3092418953 >> 172.21.15.12:0/3092418953 conn(0x7eff1c066360 msgr2=0x7eff1c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:12.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff22c5f640 1 -- 172.21.15.12:0/3092418953 shutdown_connections 2023-11-28T21:51:12.545 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff22c5f640 1 -- 172.21.15.12:0/3092418953 wait complete. 2023-11-28T21:51:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff22c5f640 1 Processor -- start 2023-11-28T21:51:12.546 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.539+0000 7eff22c5f640 1 -- start start 2023-11-28T21:51:12.548 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff22c5f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06cf40 0x7eff1c1f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff22c5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1c06e300 0x7eff1c156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff22c5f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff1c150c20 0x7eff1c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff22c5f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c1f72d0 con 0x7eff1c150c20 2023-11-28T21:51:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff22c5f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c153f50 con 0x7eff1c06e300 2023-11-28T21:51:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff22c5f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c1f6ff0 con 0x7eff1c06cf40 2023-11-28T21:51:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff211fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff1c150c20 0x7eff1c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff209fb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06cf40 0x7eff1c1f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:12.549 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff211fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff1c150c20 0x7eff1c156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54990/0 (socket says 172.21.15.12:54990) 2023-11-28T21:51:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff211fc640 1 -- 172.21.15.12:0/3790561626 learned_addr learned my addr 172.21.15.12:0/3790561626 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff1bfff640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1c06e300 0x7eff1c156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff211fc640 1 -- 172.21.15.12:0/3790561626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06cf40 msgr2=0x7eff1c1f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff211fc640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06cf40 0x7eff1c1f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff211fc640 1 -- 172.21.15.12:0/3790561626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1c06e300 msgr2=0x7eff1c156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff211fc640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1c06e300 0x7eff1c156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff211fc640 1 -- 172.21.15.12:0/3790561626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff080094d0 con 0x7eff1c150c20 2023-11-28T21:51:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff211fc640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff1c150c20 0x7eff1c156ad0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7eff1400c7a0 tx=0x7eff1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff197fa640 1 -- 172.21.15.12:0/3790561626 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400a640 con 0x7eff1c150c20 2023-11-28T21:51:12.550 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff197fa640 1 -- 172.21.15.12:0/3790561626 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1400a7a0 con 0x7eff1c150c20 2023-11-28T21:51:12.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff22c5f640 1 -- 172.21.15.12:0/3790561626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1000b6c0 con 0x7eff1c150c20 2023-11-28T21:51:12.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff197fa640 1 -- 172.21.15.12:0/3790561626 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400ab10 con 0x7eff1c150c20 2023-11-28T21:51:12.551 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff22c5f640 1 -- 172.21.15.12:0/3790561626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c154910 con 0x7eff1c150c20 2023-11-28T21:51:12.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.543+0000 7eff22c5f640 1 -- 172.21.15.12:0/3790561626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c074f60 con 0x7eff1c150c20 2023-11-28T21:51:12.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.547+0000 7eff197fa640 1 -- 172.21.15.12:0/3790561626 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7eff1400ac70 con 0x7eff1c150c20 2023-11-28T21:51:12.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.547+0000 7eff197fa640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7eff000664a0 0x7eff00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:12.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.547+0000 7eff197fa640 1 -- 172.21.15.12:0/3790561626 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7eff14083c50 con 0x7eff1c150c20 2023-11-28T21:51:12.553 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.547+0000 7eff209fb640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7eff000664a0 0x7eff00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:12.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.547+0000 7eff209fb640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7eff000664a0 0x7eff00068960 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7eff100b1000 tx=0x7eff10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:12.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.551+0000 7eff197fa640 1 -- 172.21.15.12:0/3790561626 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7eff14089020 con 0x7eff1c150c20 2023-11-28T21:51:12.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.739+0000 7eff22c5f640 1 -- 172.21.15.12:0/3790561626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff1c1562f0 con 0x7eff1c150c20 2023-11-28T21:51:12.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff197fa640 1 -- 172.21.15.12:0/3790561626 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+5732 (secure 0 0 0) 0x7eff14017020 con 0x7eff1c150c20 2023-11-28T21:51:12.747 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:51:12.747 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:06.983538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:clientreplay","state_seq":163,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:51:12.747 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 69 2023-11-28T21:51:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 -- 172.21.15.12:0/3790561626 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7eff000664a0 msgr2=0x7eff00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7eff000664a0 0x7eff00068960 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7eff100b1000 tx=0x7eff10008040 comp rx=0 tx=0).stop 2023-11-28T21:51:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 -- 172.21.15.12:0/3790561626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff1c150c20 msgr2=0x7eff1c156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff1c150c20 0x7eff1c156ad0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7eff1400c7a0 tx=0x7eff1400cc60 comp rx=0 tx=0).stop 2023-11-28T21:51:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 -- 172.21.15.12:0/3790561626 shutdown_connections 2023-11-28T21:51:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7eff000664a0 0x7eff00068960 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff1c150c20 0x7eff1c156ad0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1c06e300 0x7eff1c156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 --2- 172.21.15.12:0/3790561626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff1c06cf40 0x7eff1c1f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 -- 172.21.15.12:0/3790561626 >> 172.21.15.12:0/3790561626 conn(0x7eff1c066360 msgr2=0x7eff1c0c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 -- 172.21.15.12:0/3790561626 shutdown_connections 2023-11-28T21:51:12.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:12.743+0000 7eff22c5f640 1 -- 172.21.15.12:0/3790561626 wait complete. 2023-11-28T21:51:12.768 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:51:14.769 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:51:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.919+0000 7ff52669e640 1 Processor -- start 2023-11-28T21:51:14.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff52669e640 1 -- start start 2023-11-28T21:51:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff52669e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff520150c20 0x7ff520156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff52669e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff52006e300 0x7ff5200c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff52669e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff52006cf40 0x7ff5200c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:14.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff52669e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff520067410 con 0x7ff52006e300 2023-11-28T21:51:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff52669e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52006e090 con 0x7ff520150c20 2023-11-28T21:51:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff52669e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff520157f30 con 0x7ff52006cf40 2023-11-28T21:51:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff51f7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff52006e300 0x7ff5200c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff51f7fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff52006e300 0x7ff5200c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52466/0 (socket says 172.21.15.12:52466) 2023-11-28T21:51:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff51f7fe640 1 -- 172.21.15.12:0/1676185191 learned_addr learned my addr 172.21.15.12:0/1676185191 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff524c3b640 1 --2- 172.21.15.12:0/1676185191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff52006cf40 0x7ff5200c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:14.928 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff51ffff640 1 --2- 172.21.15.12:0/1676185191 >> v2:172.21.15.144:3300/0 conn(0x7ff520150c20 0x7ff520156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff51f7fe640 1 -- 172.21.15.12:0/1676185191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff52006cf40 msgr2=0x7ff5200c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:51:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff51f7fe640 1 --2- 172.21.15.12:0/1676185191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff52006cf40 0x7ff5200c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff51f7fe640 1 -- 172.21.15.12:0/1676185191 >> v2:172.21.15.144:3300/0 conn(0x7ff520150c20 msgr2=0x7ff520156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:51:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff51f7fe640 1 --2- 172.21.15.12:0/1676185191 >> v2:172.21.15.144:3300/0 conn(0x7ff520150c20 0x7ff520156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff51f7fe640 1 -- 172.21.15.12:0/1676185191 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520151000 con 0x7ff52006e300 2023-11-28T21:51:14.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.923+0000 7ff51ffff640 1 --2- 172.21.15.12:0/1676185191 >> v2:172.21.15.144:3300/0 conn(0x7ff520150c20 0x7ff520156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:51:14.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff51f7fe640 1 --2- 172.21.15.12:0/1676185191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006e300 0x7ff5200c6dd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff52006c6a0 tx=0x7ff51401ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818847062dbeea25 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff51effd640 1 -- 172.21.15.12:0/1676185191 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5140a7b70 con 0x7ff52006e300 2023-11-28T21:51:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff51effd640 1 -- 172.21.15.12:0/1676185191 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5140a7cd0 con 0x7ff52006e300 2023-11-28T21:51:14.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff51effd640 1 -- 172.21.15.12:0/1676185191 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5140af660 con 0x7ff52006e300 2023-11-28T21:51:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff52669e640 1 -- 172.21.15.12:0/1676185191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006e300 msgr2=0x7ff5200c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff52669e640 1 --2- 172.21.15.12:0/1676185191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006e300 0x7ff5200c6dd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff52006c6a0 tx=0x7ff51401ebd0 comp rx=0 tx=0).stop 2023-11-28T21:51:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff52669e640 1 -- 172.21.15.12:0/1676185191 shutdown_connections 2023-11-28T21:51:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff52669e640 1 --2- 172.21.15.12:0/1676185191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff52006cf40 0x7ff5200c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff52669e640 1 --2- 172.21.15.12:0/1676185191 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006e300 0x7ff5200c6dd0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff52669e640 1 --2- 172.21.15.12:0/1676185191 >> v2:172.21.15.144:3300/0 conn(0x7ff520150c20 0x7ff520156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff52669e640 1 -- 172.21.15.12:0/1676185191 >> 172.21.15.12:0/1676185191 conn(0x7ff520066360 msgr2=0x7ff520066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff52669e640 1 -- 172.21.15.12:0/1676185191 shutdown_connections 2023-11-28T21:51:14.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff52669e640 1 -- 172.21.15.12:0/1676185191 wait complete. 2023-11-28T21:51:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.927+0000 7ff52669e640 1 Processor -- start 2023-11-28T21:51:14.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff52669e640 1 -- start start 2023-11-28T21:51:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff52669e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006cf40 0x7ff5201f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff52669e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff52006e300 0x7ff5201f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:14.935 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff52669e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff520150c20 0x7ff5200cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff52669e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520147800 con 0x7ff52006cf40 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff52669e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52014c990 con 0x7ff520150c20 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff52669e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52014bf80 con 0x7ff52006e300 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006cf40 0x7ff5201f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006cf40 0x7ff5201f2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52478/0 (socket says 172.21.15.12:52478) 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51ffff640 1 -- 172.21.15.12:0/1654563954 learned_addr learned my addr 172.21.15.12:0/1654563954 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51f7fe640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff52006e300 0x7ff5201f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff524c3b640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff520150c20 0x7ff5200cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51ffff640 1 -- 172.21.15.12:0/1654563954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff52006e300 msgr2=0x7ff5201f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51ffff640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff52006e300 0x7ff5201f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51ffff640 1 -- 172.21.15.12:0/1654563954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff520150c20 msgr2=0x7ff5200cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:14.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51ffff640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff520150c20 0x7ff5200cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51ffff640 1 -- 172.21.15.12:0/1654563954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51400b770 con 0x7ff52006cf40 2023-11-28T21:51:14.937 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff524c3b640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff520150c20 0x7ff5200cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:51:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51ffff640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006cf40 0x7ff5201f2e80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff51000d800 tx=0x7ff51000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51cff9640 1 -- 172.21.15.12:0/1654563954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100096c0 con 0x7ff52006cf40 2023-11-28T21:51:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51cff9640 1 -- 172.21.15.12:0/1654563954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff510009820 con 0x7ff52006cf40 2023-11-28T21:51:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff52669e640 1 -- 172.21.15.12:0/1654563954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52014c260 con 0x7ff52006cf40 2023-11-28T21:51:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff51cff9640 1 -- 172.21.15.12:0/1654563954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510009bd0 con 0x7ff52006cf40 2023-11-28T21:51:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff52669e640 1 -- 172.21.15.12:0/1654563954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5201f6970 con 0x7ff52006cf40 2023-11-28T21:51:14.938 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.931+0000 7ff52669e640 1 -- 172.21.15.12:0/1654563954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520074f60 con 0x7ff52006cf40 2023-11-28T21:51:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.935+0000 7ff51cff9640 1 -- 172.21.15.12:0/1654563954 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff510009d30 con 0x7ff52006cf40 2023-11-28T21:51:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.935+0000 7ff51cff9640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff50c066470 0x7ff50c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.935+0000 7ff51cff9640 1 -- 172.21.15.12:0/1654563954 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7ff510084b80 con 0x7ff52006cf40 2023-11-28T21:51:14.939 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.935+0000 7ff51f7fe640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff50c066470 0x7ff50c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:14.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.935+0000 7ff51f7fe640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff50c066470 0x7ff50c068930 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff5140b1000 tx=0x7ff514008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:14.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:14.939+0000 7ff51cff9640 1 -- 172.21.15.12:0/1654563954 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff510051980 con 0x7ff52006cf40 2023-11-28T21:51:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 -- 172.21.15.12:0/1654563954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff52014c3f0 con 0x7ff52006cf40 2023-11-28T21:51:15.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff51cff9640 1 -- 172.21.15.12:0/1654563954 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5726 (secure 0 0 0) 0x7ff510054fa0 con 0x7ff52006cf40 2023-11-28T21:51:15.135 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:51:15.135 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:13.111258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:active","state_seq":165,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:51:15.135 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 70 2023-11-28T21:51:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 -- 172.21.15.12:0/1654563954 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff50c066470 msgr2=0x7ff50c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:15.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff50c066470 0x7ff50c068930 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff5140b1000 tx=0x7ff514008040 comp rx=0 tx=0).stop 2023-11-28T21:51:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 -- 172.21.15.12:0/1654563954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006cf40 msgr2=0x7ff5201f2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006cf40 0x7ff5201f2e80 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff51000d800 tx=0x7ff51000dcc0 comp rx=0 tx=0).stop 2023-11-28T21:51:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 -- 172.21.15.12:0/1654563954 shutdown_connections 2023-11-28T21:51:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff50c066470 0x7ff50c068930 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff520150c20 0x7ff5200cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff52006e300 0x7ff5201f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 --2- 172.21.15.12:0/1654563954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff52006cf40 0x7ff5201f2e80 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 -- 172.21.15.12:0/1654563954 >> 172.21.15.12:0/1654563954 conn(0x7ff520066360 msgr2=0x7ff520157200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 -- 172.21.15.12:0/1654563954 shutdown_connections 2023-11-28T21:51:15.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:15.131+0000 7ff52669e640 1 -- 172.21.15.12:0/1654563954 wait complete. 2023-11-28T21:51:15.156 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=6177 2023-11-28T21:51:15.156 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.a 2023-11-28T21:51:16.065 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:51:16.069 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:51:19.522 INFO:tasks.ceph.mds.f.smithi144.stderr:2023-11-28T21:51:19.522+0000 7f9d71eff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T21:51:19.522 INFO:tasks.ceph.mds.f.smithi144.stderr:2023-11-28T21:51:19.522+0000 7f9d71eff640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T21:51:22.157 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2023-11-28T21:51:22.157 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T21:51:22.157 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T21:51:22.161 INFO:tasks.ceph.mds.a:Started 2023-11-28T21:51:22.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:51:22.259 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-11-28T21:51:22.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5bb8abb640 1 Processor -- start 2023-11-28T21:51:22.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5bb8abb640 1 -- start start 2023-11-28T21:51:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5bb8abb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5bb41540f0 0x7f5bb40c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5bb8abb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5bb415d210 0x7f5bb415d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5bb8abb640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb406d2a0 con 0x7f5bb415b180 2023-11-28T21:51:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5bb8abb640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb406e200 con 0x7f5bb41540f0 2023-11-28T21:51:22.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5bb8abb640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb406d580 con 0x7f5bb415d210 2023-11-28T21:51:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5bb259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5bb41540f0 0x7f5bb40c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5babfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5bb415b180 0x7f5bb415b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:35734/0 (socket says 172.21.15.12:35734) 2023-11-28T21:51:22.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5babfff640 1 -- 172.21.15.12:0/1798261567 learned_addr learned my addr 172.21.15.12:0/1798261567 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:51:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5bb259c640 1 -- 172.21.15.12:0/1798261567 >> v2:172.21.15.144:3300/0 conn(0x7f5bb415d210 msgr2=0x7f5bb415d640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T21:51:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.331+0000 7f5bb259c640 1 --2- 172.21.15.12:0/1798261567 >> v2:172.21.15.144:3300/0 conn(0x7f5bb415d210 0x7f5bb415d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.335+0000 7f5bb259c640 1 -- 172.21.15.12:0/1798261567 >> v1:172.21.15.12:6789/0 conn(0x7f5bb415b180 legacy=0x7f5bb415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.335+0000 7f5bb259c640 1 -- 172.21.15.12:0/1798261567 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb406d860 con 0x7f5bb41540f0 2023-11-28T21:51:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb259c640 1 --2- 172.21.15.12:0/1798261567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 0x7f5bb40c3be0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40c9170 tx=0x7f5ba401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bc98c7ccb59868 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb1d9b640 1 -- 172.21.15.12:0/1798261567 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40ad660 con 0x7f5bb41540f0 2023-11-28T21:51:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb1d9b640 1 -- 172.21.15.12:0/1798261567 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba40ad7c0 con 0x7f5bb41540f0 2023-11-28T21:51:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb1d9b640 1 -- 172.21.15.12:0/1798261567 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40adb50 con 0x7f5bb41540f0 2023-11-28T21:51:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1798261567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 msgr2=0x7f5bb40c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb8abb640 1 --2- 172.21.15.12:0/1798261567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 0x7f5bb40c3be0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40c9170 tx=0x7f5ba401c470 comp rx=0 tx=0).stop 2023-11-28T21:51:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1798261567 shutdown_connections 2023-11-28T21:51:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb8abb640 1 --2- 172.21.15.12:0/1798261567 >> v2:172.21.15.144:3300/0 conn(0x7f5bb415d210 0x7f5bb415d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb8abb640 1 --2- 172.21.15.12:0/1798261567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 0x7f5bb40c3be0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1798261567 >> 172.21.15.12:0/1798261567 conn(0x7f5bb4066360 msgr2=0x7f5bb4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1798261567 shutdown_connections 2023-11-28T21:51:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.339+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1798261567 wait complete. 2023-11-28T21:51:22.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.343+0000 7f5bb8abb640 1 Processor -- start 2023-11-28T21:51:22.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.343+0000 7f5bb8abb640 1 -- start start 2023-11-28T21:51:22.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.343+0000 7f5bb8abb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 0x7f5bb40716e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:22.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.343+0000 7f5bb8abb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb415b180 0x7f5bb4146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:22.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.343+0000 7f5bb8abb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb415d210 0x7f5bb4147270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:22.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.343+0000 7f5bb8abb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb41f2fc0 con 0x7f5bb41540f0 2023-11-28T21:51:22.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.343+0000 7f5bb8abb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb41f6680 con 0x7f5bb415d210 2023-11-28T21:51:22.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.343+0000 7f5bb8abb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb41f3450 con 0x7f5bb415b180 2023-11-28T21:51:22.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5bb259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 0x7f5bb40716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:22.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5bb259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 0x7f5bb40716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52520/0 (socket says 172.21.15.12:52520) 2023-11-28T21:51:22.350 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5bb259c640 1 -- 172.21.15.12:0/1281546815 learned_addr learned my addr 172.21.15.12:0/1281546815 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:51:22.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5babfff640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb415b180 0x7f5bb4146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:22.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5bb2d9d640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb415d210 0x7f5bb4147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:22.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5babfff640 1 -- 172.21.15.12:0/1281546815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb415d210 msgr2=0x7f5bb4147270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:22.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5babfff640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb415d210 0x7f5bb4147270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:22.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5babfff640 1 -- 172.21.15.12:0/1281546815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 msgr2=0x7f5bb40716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5babfff640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 0x7f5bb40716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5babfff640 1 -- 172.21.15.12:0/1281546815 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba4009420 con 0x7f5bb415b180 2023-11-28T21:51:22.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5bb259c640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 0x7f5bb40716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:51:22.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.347+0000 7f5babfff640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb415b180 0x7f5bb4146d30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00c430 tx=0x7f5b9c00c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:22.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.351+0000 7f5baaffd640 1 -- 172.21.15.12:0/1281546815 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c005cf0 con 0x7f5bb415b180 2023-11-28T21:51:22.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.351+0000 7f5baaffd640 1 -- 172.21.15.12:0/1281546815 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c005e50 con 0x7f5bb415b180 2023-11-28T21:51:22.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.351+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1281546815 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb41f6940 con 0x7f5bb415b180 2023-11-28T21:51:22.354 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.351+0000 7f5baaffd640 1 -- 172.21.15.12:0/1281546815 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0072d0 con 0x7f5bb415b180 2023-11-28T21:51:22.355 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.351+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1281546815 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb41f2970 con 0x7f5bb415b180 2023-11-28T21:51:22.356 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.351+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1281546815 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb41f3170 con 0x7f5bb415b180 2023-11-28T21:51:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.351+0000 7f5baaffd640 1 -- 172.21.15.12:0/1281546815 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5b9c007430 con 0x7f5bb415b180 2023-11-28T21:51:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.355+0000 7f5baaffd640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b940664a0 0x7f5b94068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:51:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.355+0000 7f5baaffd640 1 -- 172.21.15.12:0/1281546815 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f5b9c083900 con 0x7f5bb415b180 2023-11-28T21:51:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.355+0000 7f5bb259c640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b940664a0 0x7f5b94068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:51:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.355+0000 7f5bb259c640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b940664a0 0x7f5b94068960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5ba401c4b0 tx=0x7f5ba4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:51:22.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.371+0000 7f5baaffd640 1 -- 172.21.15.12:0/1281546815 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5bb41f3170 con 0x7f5bb415b180 2023-11-28T21:51:22.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.607+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1281546815 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bb41f3130 con 0x7f5bb415b180 2023-11-28T21:51:22.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.607+0000 7f5baaffd640 1 -- 172.21.15.12:0/1281546815 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+6517 (secure 0 0 0) 0x7f5b9c0507b0 con 0x7f5bb415b180 2023-11-28T21:51:22.613 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:51:22.613 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:13.111258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:active","state_seq":165,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:51:22.613 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 71 2023-11-28T21:51:22.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1281546815 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b940664a0 msgr2=0x7f5b94068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:22.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b940664a0 0x7f5b94068960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5ba401c4b0 tx=0x7f5ba4008040 comp rx=0 tx=0).stop 2023-11-28T21:51:22.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1281546815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb415b180 msgr2=0x7f5bb4146d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:51:22.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb415b180 0x7f5bb4146d30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00c430 tx=0x7f5b9c00c8f0 comp rx=0 tx=0).stop 2023-11-28T21:51:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1281546815 shutdown_connections 2023-11-28T21:51:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5b940664a0 0x7f5b94068960 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bb415d210 0x7f5bb4147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bb415b180 0x7f5bb4146d30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 --2- 172.21.15.12:0/1281546815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5bb41540f0 0x7f5bb40716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:51:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1281546815 >> 172.21.15.12:0/1281546815 conn(0x7f5bb4066360 msgr2=0x7f5bb4066740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:51:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1281546815 shutdown_connections 2023-11-28T21:51:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:51:22.611+0000 7f5bb8abb640 1 -- 172.21.15.12:0/1281546815 wait complete. 2023-11-28T21:51:22.634 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2023-11-28T21:51:22.635 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 76 secs before thrashing 2023-11-28T21:51:37.734 INFO:tasks.workunit.client.0.smithi012.stderr:+ timeout 3600 git clone https://git.ceph.com/ceph.git 2023-11-28T21:51:37.739 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into 'ceph'... 2023-11-28T21:51:46.130 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:51:46.134 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:52:16.344 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:52:16.348 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:52:38.636 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:39.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.018+0000 7fbba1780640 1 Processor -- start 2023-11-28T21:52:39.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.018+0000 7fbba1780640 1 -- start start 2023-11-28T21:52:39.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.018+0000 7fbba1780640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 0x7fbb9c06d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.018+0000 7fbba1780640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9c0c3460 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.018+0000 7fbba1780640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9c0c2b70 con 0x7fbb9c161600 2023-11-28T21:52:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.018+0000 7fbba1780640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c158800 con 0x7fbb9c158420 2023-11-28T21:52:39.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.018+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 0x7fbb9c06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.018+0000 7fbb9a7fc640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbb9c0c38f0 0x7fbb9c0c3d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:44178/0 (socket says 172.21.15.12:44178) 2023-11-28T21:52:39.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.018+0000 7fbb9a7fc640 1 -- 172.21.15.12:0/1763049946 learned_addr learned my addr 172.21.15.12:0/1763049946 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:52:39.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.022+0000 7fbb9affd640 1 -- 172.21.15.12:0/1763049946 >> v1:172.21.15.144:6789/0 conn(0x7fbb9c161600 legacy=0x7fbb9c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.022+0000 7fbb9affd640 1 -- 172.21.15.12:0/1763049946 >> v1:172.21.15.12:6789/0 conn(0x7fbb9c0c38f0 legacy=0x7fbb9c0c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.022+0000 7fbb9affd640 1 -- 172.21.15.12:0/1763049946 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c06e090 con 0x7fbb9c158420 2023-11-28T21:52:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.026+0000 7fbb9affd640 1 --2- 172.21.15.12:0/1763049946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 0x7fbb9c06d950 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0097f0 tx=0x7fbb8c01c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9edf08eb8e0bc6ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.026+0000 7fbb99ffb640 1 -- 172.21.15.12:0/1763049946 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0ab600 con 0x7fbb9c158420 2023-11-28T21:52:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.026+0000 7fbb99ffb640 1 -- 172.21.15.12:0/1763049946 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c0a7050 con 0x7fbb9c158420 2023-11-28T21:52:39.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.026+0000 7fbb99ffb640 1 -- 172.21.15.12:0/1763049946 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0ab980 con 0x7fbb9c158420 2023-11-28T21:52:39.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.026+0000 7fbba1780640 1 -- 172.21.15.12:0/1763049946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 msgr2=0x7fbb9c06d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.026+0000 7fbba1780640 1 --2- 172.21.15.12:0/1763049946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 0x7fbb9c06d950 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0097f0 tx=0x7fbb8c01c500 comp rx=0 tx=0).stop 2023-11-28T21:52:39.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.030+0000 7fbba1780640 1 -- 172.21.15.12:0/1763049946 shutdown_connections 2023-11-28T21:52:39.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.030+0000 7fbba1780640 1 --2- 172.21.15.12:0/1763049946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 0x7fbb9c06d950 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.030+0000 7fbba1780640 1 -- 172.21.15.12:0/1763049946 >> 172.21.15.12:0/1763049946 conn(0x7fbb9c066360 msgr2=0x7fbb9c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:39.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.030+0000 7fbba1780640 1 -- 172.21.15.12:0/1763049946 shutdown_connections 2023-11-28T21:52:39.035 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.030+0000 7fbba1780640 1 -- 172.21.15.12:0/1763049946 wait complete. 2023-11-28T21:52:39.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.030+0000 7fbba1780640 1 Processor -- start 2023-11-28T21:52:39.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.030+0000 7fbba1780640 1 -- start start 2023-11-28T21:52:39.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbba1780640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c0c38f0 0x7fbb9c14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbba1780640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 0x7fbb9c070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.038 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbba1780640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9c161600 0x7fbb9c147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbba1780640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c15afc0 con 0x7fbb9c161600 2023-11-28T21:52:39.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbba1780640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c1594d0 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.039 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbba1780640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c1597b0 con 0x7fbb9c158420 2023-11-28T21:52:39.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c0c38f0 0x7fbb9c14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c0c38f0 0x7fbb9c14d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:36034/0 (socket says 172.21.15.12:36034) 2023-11-28T21:52:39.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9affd640 1 -- 172.21.15.12:0/2012648984 learned_addr learned my addr 172.21.15.12:0/2012648984 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:39.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9a7fc640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 0x7fbb9c070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9b7fe640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9c161600 0x7fbb9c147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.040 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9affd640 1 -- 172.21.15.12:0/2012648984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 msgr2=0x7fbb9c070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9affd640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 0x7fbb9c070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9affd640 1 -- 172.21.15.12:0/2012648984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9c161600 msgr2=0x7fbb9c147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9affd640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9c161600 0x7fbb9c147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.041 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9affd640 1 -- 172.21.15.12:0/2012648984 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c009420 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb9affd640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c0c38f0 0x7fbb9c14d080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0097f0 tx=0x7fbb8c0192a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:39.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb837fe640 1 -- 172.21.15.12:0/2012648984 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0b4640 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb837fe640 1 -- 172.21.15.12:0/2012648984 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c0b47c0 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbb837fe640 1 -- 172.21.15.12:0/2012648984 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0b4b70 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.042 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbba1780640 1 -- 172.21.15.12:0/2012648984 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9c15b1c0 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.034+0000 7fbba1780640 1 -- 172.21.15.12:0/2012648984 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9c14b000 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.042+0000 7fbb837fe640 1 -- 172.21.15.12:0/2012648984 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fbb8c0b2040 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.042+0000 7fbb837fe640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbb88066540 0x7fbb88068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.042+0000 7fbb837fe640 1 -- 172.21.15.12:0/2012648984 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fbb8c11e720 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.042+0000 7fbb9a7fc640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbb88066540 0x7fbb88068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.046+0000 7fbba1780640 1 -- 172.21.15.12:0/2012648984 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9c074f60 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.046+0000 7fbb9a7fc640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbb88066540 0x7fbb88068a00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c153110 tx=0x7fbb90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.054+0000 7fbb837fe640 1 -- 172.21.15.12:0/2012648984 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbb8c0eb480 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.614+0000 7fbba1780640 1 -- 172.21.15.12:0/2012648984 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb9c161cd0 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.618+0000 7fbb837fe640 1 -- 172.21.15.12:0/2012648984 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6518 (secure 0 0 0) 0x7fbb8c0eeaa0 con 0x7fbb9c0c38f0 2023-11-28T21:52:39.624 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:39.624 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:53.213081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:active","state_seq":165,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:39.624 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-11-28T21:52:39.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.618+0000 7fbba1780640 1 -- 172.21.15.12:0/2012648984 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbb88066540 msgr2=0x7fbb88068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.618+0000 7fbba1780640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbb88066540 0x7fbb88068a00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c153110 tx=0x7fbb90009040 comp rx=0 tx=0).stop 2023-11-28T21:52:39.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.618+0000 7fbba1780640 1 -- 172.21.15.12:0/2012648984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c0c38f0 msgr2=0x7fbb9c14d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.618+0000 7fbba1780640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c0c38f0 0x7fbb9c14d080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0097f0 tx=0x7fbb8c0192a0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.622+0000 7fbba1780640 1 -- 172.21.15.12:0/2012648984 shutdown_connections 2023-11-28T21:52:39.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.622+0000 7fbba1780640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbb88066540 0x7fbb88068a00 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c153110 tx=0x7fbb90009040 comp rx=0 tx=0).stop 2023-11-28T21:52:39.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.622+0000 7fbba1780640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb9c161600 0x7fbb9c147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.622+0000 7fbba1780640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb9c158420 0x7fbb9c070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.622+0000 7fbba1780640 1 --2- 172.21.15.12:0/2012648984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c0c38f0 0x7fbb9c14d080 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.622+0000 7fbba1780640 1 -- 172.21.15.12:0/2012648984 >> 172.21.15.12:0/2012648984 conn(0x7fbb9c066360 msgr2=0x7fbb9c0c92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:39.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.622+0000 7fbba1780640 1 -- 172.21.15.12:0/2012648984 shutdown_connections 2023-11-28T21:52:39.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.622+0000 7fbba1780640 1 -- 172.21.15.12:0/2012648984 wait complete. 2023-11-28T21:52:39.646 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2023-11-28T21:52:39.647 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-11-28T21:52:39.647 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:52:39.669 INFO:tasks.ceph.mds.f:Stopped 2023-11-28T21:52:39.670 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:39.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.846+0000 7f56d97f8640 1 Processor -- start 2023-11-28T21:52:39.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.846+0000 7f56d97f8640 1 -- start start 2023-11-28T21:52:39.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d97f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d415d320 0x7f56d406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d97f8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f56d406aaf0 0x7f56d40c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d97f8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d415e270 con 0x7f56d4154100 2023-11-28T21:52:39.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d97f8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d40c4e60 con 0x7f56d406aaf0 2023-11-28T21:52:39.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d97f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d415d740 con 0x7f56d415d320 2023-11-28T21:52:39.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d27fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d415d320 0x7f56d406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d2ffd640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f56d4154100 0x7f56d40c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:40364/0 (socket says 172.21.15.12:40364) 2023-11-28T21:52:39.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d27fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d415d320 0x7f56d406a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:40592/0 (socket says 172.21.15.12:40592) 2023-11-28T21:52:39.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d27fc640 1 -- 172.21.15.12:0/3598125644 learned_addr learned my addr 172.21.15.12:0/3598125644 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:39.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d37fe640 1 --2- 172.21.15.12:0/3598125644 >> v2:172.21.15.12:3300/0 conn(0x7f56d406aaf0 0x7f56d40c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d1ffb640 1 -- 172.21.15.12:0/3598125644 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845222126 0 0) 0x7f56d415e270 con 0x7f56d4154100 2023-11-28T21:52:39.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d1ffb640 1 -- 172.21.15.12:0/3598125644 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc003530 con 0x7f56d4154100 2023-11-28T21:52:39.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d27fc640 1 -- 172.21.15.12:0/3598125644 >> v2:172.21.15.12:3300/0 conn(0x7f56d406aaf0 msgr2=0x7f56d40c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:39.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d27fc640 1 --2- 172.21.15.12:0/3598125644 >> v2:172.21.15.12:3300/0 conn(0x7f56d406aaf0 0x7f56d40c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d27fc640 1 -- 172.21.15.12:0/3598125644 >> v1:172.21.15.144:6789/0 conn(0x7f56d4154100 legacy=0x7f56d40c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d27fc640 1 -- 172.21.15.12:0/3598125644 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d41544e0 con 0x7f56d415d320 2023-11-28T21:52:39.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.850+0000 7f56d37fe640 1 --2- 172.21.15.12:0/3598125644 >> v2:172.21.15.12:3300/0 conn(0x7f56d406aaf0 0x7f56d40c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:52:39.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.854+0000 7f56d27fc640 1 --2- 172.21.15.12:0/3598125644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d415d320 0x7f56d406a5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009870 tx=0x7f56c801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1222f6b4a25a01ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:39.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.858+0000 7f56d1ffb640 1 -- 172.21.15.12:0/3598125644 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80aa670 con 0x7f56d415d320 2023-11-28T21:52:39.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.858+0000 7f56d1ffb640 1 -- 172.21.15.12:0/3598125644 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c80aa7d0 con 0x7f56d415d320 2023-11-28T21:52:39.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.858+0000 7f56d1ffb640 1 -- 172.21.15.12:0/3598125644 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80aab40 con 0x7f56d415d320 2023-11-28T21:52:39.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.858+0000 7f56d97f8640 1 -- 172.21.15.12:0/3598125644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d415d320 msgr2=0x7f56d406a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.863 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.858+0000 7f56d97f8640 1 --2- 172.21.15.12:0/3598125644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d415d320 0x7f56d406a5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009870 tx=0x7f56c801c530 comp rx=0 tx=0).stop 2023-11-28T21:52:39.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.862+0000 7f56d97f8640 1 -- 172.21.15.12:0/3598125644 shutdown_connections 2023-11-28T21:52:39.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.862+0000 7f56d97f8640 1 --2- 172.21.15.12:0/3598125644 >> v2:172.21.15.12:3300/0 conn(0x7f56d406aaf0 0x7f56d40c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.862+0000 7f56d97f8640 1 --2- 172.21.15.12:0/3598125644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d415d320 0x7f56d406a5b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.862+0000 7f56d97f8640 1 -- 172.21.15.12:0/3598125644 >> 172.21.15.12:0/3598125644 conn(0x7f56d406c690 msgr2=0x7f56d4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:39.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.862+0000 7f56d97f8640 1 -- 172.21.15.12:0/3598125644 shutdown_connections 2023-11-28T21:52:39.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.862+0000 7f56d97f8640 1 -- 172.21.15.12:0/3598125644 wait complete. 2023-11-28T21:52:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d97f8640 1 Processor -- start 2023-11-28T21:52:39.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d97f8640 1 -- start start 2023-11-28T21:52:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d97f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d406aaf0 0x7f56d41f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d97f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d4154100 0x7f56d415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d97f8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56d415d320 0x7f56d4149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d97f8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d4148c80 con 0x7f56d415d320 2023-11-28T21:52:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d97f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d41480b0 con 0x7f56d406aaf0 2023-11-28T21:52:39.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d97f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d4148350 con 0x7f56d4154100 2023-11-28T21:52:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d406aaf0 0x7f56d41f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d406aaf0 0x7f56d41f3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:36050/0 (socket says 172.21.15.12:36050) 2023-11-28T21:52:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d2ffd640 1 -- 172.21.15.12:0/3192001674 learned_addr learned my addr 172.21.15.12:0/3192001674 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d37fe640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56d415d320 0x7f56d4149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.866+0000 7f56d27fc640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d4154100 0x7f56d415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56d2ffd640 1 -- 172.21.15.12:0/3192001674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d4154100 msgr2=0x7f56d415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56d2ffd640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d4154100 0x7f56d415c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56d2ffd640 1 -- 172.21.15.12:0/3192001674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56d415d320 msgr2=0x7f56d4149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56d2ffd640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56d415d320 0x7f56d4149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:39.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56d2ffd640 1 -- 172.21.15.12:0/3192001674 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc009a70 con 0x7f56d406aaf0 2023-11-28T21:52:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56d2ffd640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d406aaf0 0x7f56d41f3010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f56c400c390 tx=0x7f56c400c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56bb7fe640 1 -- 172.21.15.12:0/3192001674 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40074e0 con 0x7f56d406aaf0 2023-11-28T21:52:39.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56bb7fe640 1 -- 172.21.15.12:0/3192001674 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4007640 con 0x7f56d406aaf0 2023-11-28T21:52:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56bb7fe640 1 -- 172.21.15.12:0/3192001674 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4004a10 con 0x7f56d406aaf0 2023-11-28T21:52:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56d97f8640 1 -- 172.21.15.12:0/3192001674 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c80094d0 con 0x7f56d406aaf0 2023-11-28T21:52:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.870+0000 7f56d97f8640 1 -- 172.21.15.12:0/3192001674 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d4148910 con 0x7f56d406aaf0 2023-11-28T21:52:39.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.874+0000 7f56d97f8640 1 -- 172.21.15.12:0/3192001674 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d4067c40 con 0x7f56d406aaf0 2023-11-28T21:52:39.886 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.882+0000 7f56bb7fe640 1 -- 172.21.15.12:0/3192001674 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f56c4004b70 con 0x7f56d406aaf0 2023-11-28T21:52:39.887 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.882+0000 7f56bb7fe640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f56c00664c0 0x7f56c0068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.882+0000 7f56bb7fe640 1 -- 172.21.15.12:0/3192001674 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f56c4084620 con 0x7f56d406aaf0 2023-11-28T21:52:39.888 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.882+0000 7f56d27fc640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f56c00664c0 0x7f56c0068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:39.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.882+0000 7f56d27fc640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f56c00664c0 0x7f56c0068980 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f56c80ae000 tx=0x7f56c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:39.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:39.886+0000 7f56bb7fe640 1 -- 172.21.15.12:0/3192001674 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f56c4051380 con 0x7f56d406aaf0 2023-11-28T21:52:40.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.122+0000 7f56d97f8640 1 -- 172.21.15.12:0/3192001674 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56d4074f60 con 0x7f56d406aaf0 2023-11-28T21:52:40.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56bb7fe640 1 -- 172.21.15.12:0/3192001674 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6518 (secure 0 0 0) 0x7f56c40549a0 con 0x7f56d406aaf0 2023-11-28T21:52:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:40.130 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:53.213081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:active","state_seq":165,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:40.130 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 -- 172.21.15.12:0/3192001674 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f56c00664c0 msgr2=0x7f56c0068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f56c00664c0 0x7f56c0068980 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f56c80ae000 tx=0x7f56c8009040 comp rx=0 tx=0).stop 2023-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 -- 172.21.15.12:0/3192001674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d406aaf0 msgr2=0x7f56d41f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d406aaf0 0x7f56d41f3010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f56c400c390 tx=0x7f56c400c790 comp rx=0 tx=0).stop 2023-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 -- 172.21.15.12:0/3192001674 shutdown_connections 2023-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f56c00664c0 0x7f56c0068980 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f56d415d320 0x7f56d4149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56d4154100 0x7f56d415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:40.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 --2- 172.21.15.12:0/3192001674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d406aaf0 0x7f56d41f3010 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 -- 172.21.15.12:0/3192001674 >> 172.21.15.12:0/3192001674 conn(0x7f56d406c690 msgr2=0x7f56d4147540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 -- 172.21.15.12:0/3192001674 shutdown_connections 2023-11-28T21:52:40.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:40.126+0000 7f56d97f8640 1 -- 172.21.15.12:0/3192001674 wait complete. 2023-11-28T21:52:40.155 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-11-28T21:52:42.156 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f064a0ad640 1 Processor -- start 2023-11-28T21:52:42.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f064a0ad640 1 -- start start 2023-11-28T21:52:42.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f064a0ad640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0644153a40 0x7f06440ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:42.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f064a0ad640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 0x7f064415d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:42.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f064a0ad640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064415df30 con 0x7f06440ca7c0 2023-11-28T21:52:42.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f064a0ad640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06440c5270 con 0x7f0644153a40 2023-11-28T21:52:42.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f064a0ad640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06440c45f0 con 0x7f06440c5da0 2023-11-28T21:52:42.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f06490ab640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0644153a40 0x7f06440ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:42.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f06490ab640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0644153a40 0x7f06440ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37668/0 (socket says 172.21.15.12:37668) 2023-11-28T21:52:42.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f06490ab640 1 -- 172.21.15.12:0/3099713520 learned_addr learned my addr 172.21.15.12:0/3099713520 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:42.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f06498ac640 1 --2- 172.21.15.12:0/3099713520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 0x7f064415d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:42.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f06490ab640 1 -- 172.21.15.12:0/3099713520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 msgr2=0x7f064415d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:42.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f06490ab640 1 --2- 172.21.15.12:0/3099713520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 0x7f064415d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:42.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f06490ab640 1 -- 172.21.15.12:0/3099713520 >> v1:172.21.15.144:6789/0 conn(0x7f06440ca7c0 legacy=0x7f06440c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:42.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.390+0000 7f06490ab640 1 -- 172.21.15.12:0/3099713520 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440c48d0 con 0x7f0644153a40 2023-11-28T21:52:42.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f06490ab640 1 --2- 172.21.15.12:0/3099713520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0644153a40 0x7f06440ca280 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0640009870 tx=0x7f064001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af768fb663d6883 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:42.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f063bfff640 1 -- 172.21.15.12:0/3099713520 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06400ac640 con 0x7f0644153a40 2023-11-28T21:52:42.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f063bfff640 1 -- 172.21.15.12:0/3099713520 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06400ac7a0 con 0x7f0644153a40 2023-11-28T21:52:42.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f063bfff640 1 -- 172.21.15.12:0/3099713520 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06400acb30 con 0x7f0644153a40 2023-11-28T21:52:42.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f064a0ad640 1 -- 172.21.15.12:0/3099713520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0644153a40 msgr2=0x7f06440ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:42.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f064a0ad640 1 --2- 172.21.15.12:0/3099713520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0644153a40 0x7f06440ca280 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0640009870 tx=0x7f064001c510 comp rx=0 tx=0).stop 2023-11-28T21:52:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f064a0ad640 1 -- 172.21.15.12:0/3099713520 shutdown_connections 2023-11-28T21:52:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f064a0ad640 1 --2- 172.21.15.12:0/3099713520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 0x7f064415d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f064a0ad640 1 --2- 172.21.15.12:0/3099713520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0644153a40 0x7f06440ca280 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f064a0ad640 1 -- 172.21.15.12:0/3099713520 >> 172.21.15.12:0/3099713520 conn(0x7f064406bda0 msgr2=0x7f064406c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f064a0ad640 1 -- 172.21.15.12:0/3099713520 shutdown_connections 2023-11-28T21:52:42.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.398+0000 7f064a0ad640 1 -- 172.21.15.12:0/3099713520 wait complete. 2023-11-28T21:52:42.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f064a0ad640 1 Processor -- start 2023-11-28T21:52:42.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f064a0ad640 1 -- start start 2023-11-28T21:52:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f064a0ad640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 0x7f0644152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f064a0ad640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06440ca7c0 0x7f0644148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f064a0ad640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0644153a40 0x7f06441495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f064a0ad640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441f65d0 con 0x7f0644153a40 2023-11-28T21:52:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f064a0ad640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441f6070 con 0x7f06440ca7c0 2023-11-28T21:52:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f064a0ad640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06440c7040 con 0x7f06440c5da0 2023-11-28T21:52:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f06490ab640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 0x7f0644152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f06498ac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0644153a40 0x7f06441495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f06490ab640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 0x7f0644152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:40624/0 (socket says 172.21.15.12:40624) 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f06490ab640 1 -- 172.21.15.12:0/3704590982 learned_addr learned my addr 172.21.15.12:0/3704590982 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.402+0000 7f06488aa640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06440ca7c0 0x7f0644148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f06490ab640 1 -- 172.21.15.12:0/3704590982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06440ca7c0 msgr2=0x7f0644148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f06490ab640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06440ca7c0 0x7f0644148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f06490ab640 1 -- 172.21.15.12:0/3704590982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0644153a40 msgr2=0x7f06441495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f06490ab640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0644153a40 0x7f06441495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f06490ab640 1 -- 172.21.15.12:0/3704590982 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400094d0 con 0x7f06440c5da0 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f06498ac640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0644153a40 0x7f06441495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f06490ab640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 0x7f0644152a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f06400af040 tx=0x7f06400192b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f0639ffb640 1 -- 172.21.15.12:0/3704590982 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06400b9070 con 0x7f06440c5da0 2023-11-28T21:52:42.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f0639ffb640 1 -- 172.21.15.12:0/3704590982 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06400b5440 con 0x7f06440c5da0 2023-11-28T21:52:42.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f0639ffb640 1 -- 172.21.15.12:0/3704590982 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06400b57f0 con 0x7f06440c5da0 2023-11-28T21:52:42.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f064a0ad640 1 -- 172.21.15.12:0/3704590982 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06441f62f0 con 0x7f06440c5da0 2023-11-28T21:52:42.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f064a0ad640 1 -- 172.21.15.12:0/3704590982 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064415c020 con 0x7f06440c5da0 2023-11-28T21:52:42.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.406+0000 7f064a0ad640 1 -- 172.21.15.12:0/3704590982 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0644074f60 con 0x7f06440c5da0 2023-11-28T21:52:42.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.410+0000 7f0639ffb640 1 -- 172.21.15.12:0/3704590982 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f06400b3070 con 0x7f06440c5da0 2023-11-28T21:52:42.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.410+0000 7f0639ffb640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0628066470 0x7f0628068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:42.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.410+0000 7f0639ffb640 1 -- 172.21.15.12:0/3704590982 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f064011e7d0 con 0x7f06440c5da0 2023-11-28T21:52:42.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.410+0000 7f06488aa640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0628066470 0x7f0628068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.414+0000 7f06488aa640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0628066470 0x7f0628068930 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f06441488c0 tx=0x7f0634009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:42.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.422+0000 7f0639ffb640 1 -- 172.21.15.12:0/3704590982 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f06400eb670 con 0x7f06440c5da0 2023-11-28T21:52:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.810+0000 7f064a0ad640 1 -- 172.21.15.12:0/3704590982 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0644151050 con 0x7f06440c5da0 2023-11-28T21:52:42.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.810+0000 7f0639ffb640 1 -- 172.21.15.12:0/3704590982 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6518 (secure 0 0 0) 0x7f06400b1020 con 0x7f06440c5da0 2023-11-28T21:52:42.816 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:42.816 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:53.213081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:active","state_seq":165,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:42.816 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-11-28T21:52:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 -- 172.21.15.12:0/3704590982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0628066470 msgr2=0x7f0628068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0628066470 0x7f0628068930 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f06441488c0 tx=0x7f0634009040 comp rx=0 tx=0).stop 2023-11-28T21:52:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 -- 172.21.15.12:0/3704590982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 msgr2=0x7f0644152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 0x7f0644152a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f06400af040 tx=0x7f06400192b0 comp rx=0 tx=0).stop 2023-11-28T21:52:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 -- 172.21.15.12:0/3704590982 shutdown_connections 2023-11-28T21:52:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0628066470 0x7f0628068930 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0644153a40 0x7f06441495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06440ca7c0 0x7f0644148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 --2- 172.21.15.12:0/3704590982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06440c5da0 0x7f0644152a30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 -- 172.21.15.12:0/3704590982 >> 172.21.15.12:0/3704590982 conn(0x7f064406bda0 msgr2=0x7f06440cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 -- 172.21.15.12:0/3704590982 shutdown_connections 2023-11-28T21:52:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:42.814+0000 7f064a0ad640 1 -- 172.21.15.12:0/3704590982 wait complete. 2023-11-28T21:52:42.850 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-11-28T21:52:44.851 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:44.998+0000 7fae28d6f640 1 Processor -- start 2023-11-28T21:52:45.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:44.998+0000 7fae28d6f640 1 -- start start 2023-11-28T21:52:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae28d6f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae24153fb0 0x7fae240ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae28d6f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae240caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae28d6f640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2415d390 con 0x7fae2415a980 2023-11-28T21:52:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae28d6f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae240b2b60 con 0x7fae24153fb0 2023-11-28T21:52:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae28d6f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2415e1b0 con 0x7fae240b41e0 2023-11-28T21:52:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae23fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae240caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae237fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae24153fb0 0x7fae240ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae23fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae240caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:37976/0 (socket says 172.21.15.12:37976) 2023-11-28T21:52:45.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae23fff640 1 -- 172.21.15.12:0/496253889 learned_addr learned my addr 172.21.15.12:0/496253889 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:45.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae23fff640 1 -- 172.21.15.12:0/496253889 >> v2:172.21.15.144:3300/0 conn(0x7fae24153fb0 msgr2=0x7fae240ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:45.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae23fff640 1 --2- 172.21.15.12:0/496253889 >> v2:172.21.15.144:3300/0 conn(0x7fae24153fb0 0x7fae240ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:45.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae23fff640 1 -- 172.21.15.12:0/496253889 >> v1:172.21.15.144:6789/0 conn(0x7fae2415a980 legacy=0x7fae240b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:45.008 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.002+0000 7fae23fff640 1 -- 172.21.15.12:0/496253889 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2415d630 con 0x7fae240b41e0 2023-11-28T21:52:45.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.006+0000 7fae23fff640 1 --2- 172.21.15.12:0/496253889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae240caa30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009e50 tx=0x7fae1c01ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37b15b30daef2b97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:45.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.006+0000 7fae227fc640 1 -- 172.21.15.12:0/496253889 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0a58d0 con 0x7fae240b41e0 2023-11-28T21:52:45.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.006+0000 7fae227fc640 1 -- 172.21.15.12:0/496253889 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c0a5a30 con 0x7fae240b41e0 2023-11-28T21:52:45.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.006+0000 7fae227fc640 1 -- 172.21.15.12:0/496253889 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0a5e00 con 0x7fae240b41e0 2023-11-28T21:52:45.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.006+0000 7fae28d6f640 1 -- 172.21.15.12:0/496253889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 msgr2=0x7fae240caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:45.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.006+0000 7fae28d6f640 1 --2- 172.21.15.12:0/496253889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae240caa30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009e50 tx=0x7fae1c01ca00 comp rx=0 tx=0).stop 2023-11-28T21:52:45.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.006+0000 7fae28d6f640 1 -- 172.21.15.12:0/496253889 shutdown_connections 2023-11-28T21:52:45.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.006+0000 7fae28d6f640 1 --2- 172.21.15.12:0/496253889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae240caa30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:45.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.006+0000 7fae28d6f640 1 --2- 172.21.15.12:0/496253889 >> v2:172.21.15.144:3300/0 conn(0x7fae24153fb0 0x7fae240ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:45.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.006+0000 7fae28d6f640 1 -- 172.21.15.12:0/496253889 >> 172.21.15.12:0/496253889 conn(0x7fae24071ce0 msgr2=0x7fae240720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:45.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae28d6f640 1 -- 172.21.15.12:0/496253889 shutdown_connections 2023-11-28T21:52:45.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae28d6f640 1 -- 172.21.15.12:0/496253889 wait complete. 2023-11-28T21:52:45.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae28d6f640 1 Processor -- start 2023-11-28T21:52:45.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae28d6f640 1 -- start start 2023-11-28T21:52:45.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae28d6f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae2414a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:45.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae28d6f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae24153fb0 0x7fae24158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:45.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae28d6f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae2415a980 0x7fae241590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:45.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae28d6f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae24155630 con 0x7fae24153fb0 2023-11-28T21:52:45.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae28d6f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae24154600 con 0x7fae2415a980 2023-11-28T21:52:45.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae28d6f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae241f31e0 con 0x7fae240b41e0 2023-11-28T21:52:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae22ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae24153fb0 0x7fae24158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae237fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae2414a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae237fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae2414a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:37992/0 (socket says 172.21.15.12:37992) 2023-11-28T21:52:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae237fe640 1 -- 172.21.15.12:0/606674958 learned_addr learned my addr 172.21.15.12:0/606674958 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae23fff640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae2415a980 0x7fae241590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae22ffd640 1 -- 172.21.15.12:0/606674958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 msgr2=0x7fae2414a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae22ffd640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae2414a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:45.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae22ffd640 1 -- 172.21.15.12:0/606674958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae2415a980 msgr2=0x7fae241590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae22ffd640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae2415a980 0x7fae241590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.010+0000 7fae22ffd640 1 -- 172.21.15.12:0/606674958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c009b30 con 0x7fae24153fb0 2023-11-28T21:52:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.014+0000 7fae22ffd640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae24153fb0 0x7fae24158b70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fae240cd360 tx=0x7fae1800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.014+0000 7fae0bfff640 1 -- 172.21.15.12:0/606674958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180052d0 con 0x7fae24153fb0 2023-11-28T21:52:45.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.014+0000 7fae0bfff640 1 -- 172.21.15.12:0/606674958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae18005430 con 0x7fae24153fb0 2023-11-28T21:52:45.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.014+0000 7fae0bfff640 1 -- 172.21.15.12:0/606674958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18003840 con 0x7fae24153fb0 2023-11-28T21:52:45.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.014+0000 7fae28d6f640 1 -- 172.21.15.12:0/606674958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae24154850 con 0x7fae24153fb0 2023-11-28T21:52:45.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.014+0000 7fae28d6f640 1 -- 172.21.15.12:0/606674958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae24154e10 con 0x7fae24153fb0 2023-11-28T21:52:45.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.014+0000 7fae28d6f640 1 -- 172.21.15.12:0/606674958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae24074f60 con 0x7fae24153fb0 2023-11-28T21:52:45.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.018+0000 7fae0bfff640 1 -- 172.21.15.12:0/606674958 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fae180039a0 con 0x7fae24153fb0 2023-11-28T21:52:45.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.018+0000 7fae0bfff640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae100664a0 0x7fae10068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.018+0000 7fae0bfff640 1 -- 172.21.15.12:0/606674958 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fae18083830 con 0x7fae24153fb0 2023-11-28T21:52:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.018+0000 7fae237fe640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae100664a0 0x7fae10068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:45.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.018+0000 7fae237fe640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae100664a0 0x7fae10068960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fae14009c20 tx=0x7fae14009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:45.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.022+0000 7fae0bfff640 1 -- 172.21.15.12:0/606674958 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae18050780 con 0x7fae24153fb0 2023-11-28T21:52:45.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.214+0000 7fae28d6f640 1 -- 172.21.15.12:0/606674958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae24159f70 con 0x7fae24153fb0 2023-11-28T21:52:45.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.214+0000 7fae0bfff640 1 -- 172.21.15.12:0/606674958 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6518 (secure 0 0 0) 0x7fae18053da0 con 0x7fae24153fb0 2023-11-28T21:52:45.220 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:45.220 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:53.213081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:active","state_seq":165,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:45.220 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-11-28T21:52:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 -- 172.21.15.12:0/606674958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae100664a0 msgr2=0x7fae10068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae100664a0 0x7fae10068960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fae14009c20 tx=0x7fae14009190 comp rx=0 tx=0).stop 2023-11-28T21:52:45.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 -- 172.21.15.12:0/606674958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae24153fb0 msgr2=0x7fae24158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:45.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae24153fb0 0x7fae24158b70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fae240cd360 tx=0x7fae1800c820 comp rx=0 tx=0).stop 2023-11-28T21:52:45.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 -- 172.21.15.12:0/606674958 shutdown_connections 2023-11-28T21:52:45.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fae100664a0 0x7fae10068960 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:45.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae2415a980 0x7fae241590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:45.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae24153fb0 0x7fae24158b70 secure :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fae240cd360 tx=0x7fae1800c820 comp rx=0 tx=0).stop 2023-11-28T21:52:45.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 --2- 172.21.15.12:0/606674958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae240b41e0 0x7fae2414a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:45.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 -- 172.21.15.12:0/606674958 >> 172.21.15.12:0/606674958 conn(0x7fae24071ce0 msgr2=0x7fae24152cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:45.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 -- 172.21.15.12:0/606674958 shutdown_connections 2023-11-28T21:52:45.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:45.218+0000 7fae28d6f640 1 -- 172.21.15.12:0/606674958 wait complete. 2023-11-28T21:52:45.243 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-11-28T21:52:46.440 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:52:46.444 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:52:47.244 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe6236d6640 1 Processor -- start 2023-11-28T21:52:47.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe6236d6640 1 -- start start 2023-11-28T21:52:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe6236d6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe61c06b130 0x7fe61c0c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe6236d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c0cb020 0x7fe61c159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe6236d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61c15a5e0 con 0x7fe61c0cac40 2023-11-28T21:52:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe6236d6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61c06b550 con 0x7fe61c06b130 2023-11-28T21:52:47.400 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe6236d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c15a8c0 con 0x7fe61c0cb020 2023-11-28T21:52:47.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe621472640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe61c0cac40 0x7fe61c0ca030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42780/0 (socket says 172.21.15.12:42780) 2023-11-28T21:52:47.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe621472640 1 -- 172.21.15.12:0/3198248538 learned_addr learned my addr 172.21.15.12:0/3198248538 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:52:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe621c73640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c0cb020 0x7fe61c159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe620c71640 1 --2- 172.21.15.12:0/3198248538 >> v2:172.21.15.144:3300/0 conn(0x7fe61c06b130 0x7fe61c0c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893098453 0 0) 0x7fe61c15a5e0 con 0x7fe61c0cac40 2023-11-28T21:52:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe608003530 con 0x7fe61c0cac40 2023-11-28T21:52:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2676574167 0 0) 0x7fe608003530 con 0x7fe61c0cac40 2023-11-28T21:52:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c0cb020 msgr2=0x7fe61c159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:47.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c0cb020 0x7fe61c159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 >> v2:172.21.15.144:3300/0 conn(0x7fe61c06b130 msgr2=0x7fe61c0c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 --2- 172.21.15.12:0/3198248538 >> v2:172.21.15.144:3300/0 conn(0x7fe61c06b130 0x7fe61c0c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c06a650 con 0x7fe61c0cac40 2023-11-28T21:52:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fe618001680 con 0x7fe61c0cac40 2023-11-28T21:52:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 >> v1:172.21.15.12:6789/0 conn(0x7fe61c0cac40 legacy=0x7fe61c0ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c0cb020 0x7fe6080039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608003f70 0x7fe6080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:47.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe608004d10 0x7fe608005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c06b550 con 0x7fe608004d10 2023-11-28T21:52:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c15a8c0 con 0x7fe608003f70 2023-11-28T21:52:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60800ba40 con 0x7fe61c0cb020 2023-11-28T21:52:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe621c73640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe608004d10 0x7fe608005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe621472640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c0cb020 0x7fe6080039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:47.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.394+0000 7fe620c71640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608003f70 0x7fe6080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe621c73640 1 -- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c0cb020 msgr2=0x7fe6080039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe621c73640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c0cb020 0x7fe6080039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe621c73640 1 -- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608003f70 msgr2=0x7fe6080047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe621c73640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608003f70 0x7fe6080047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe621c73640 1 -- 172.21.15.12:0/3198248538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608003530 con 0x7fe608004d10 2023-11-28T21:52:47.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe621472640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c0cb020 0x7fe6080039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:52:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe621c73640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe608004d10 0x7fe608005190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe614009fc0 tx=0x7fe61401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61aea5cd9b55c258 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140a5c70 con 0x7fe608004d10 2023-11-28T21:52:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61400f040 con 0x7fe608004d10 2023-11-28T21:52:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe613fff640 1 -- 172.21.15.12:0/3198248538 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140ad440 con 0x7fe608004d10 2023-11-28T21:52:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe6236d6640 1 -- 172.21.15.12:0/3198248538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe608004d10 msgr2=0x7fe608005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:47.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe6236d6640 1 --2- 172.21.15.12:0/3198248538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe608004d10 0x7fe608005190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe614009fc0 tx=0x7fe61401cc60 comp rx=0 tx=0).stop 2023-11-28T21:52:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe621c73640 1 -- 172.21.15.12:0/3198248538 reap_dead start 2023-11-28T21:52:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe6236d6640 1 -- 172.21.15.12:0/3198248538 shutdown_connections 2023-11-28T21:52:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.398+0000 7fe6236d6640 1 -- 172.21.15.12:0/3198248538 >> 172.21.15.12:0/3198248538 conn(0x7fe61c0720e0 msgr2=0x7fe61c0724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe6236d6640 1 -- 172.21.15.12:0/3198248538 shutdown_connections 2023-11-28T21:52:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe6236d6640 1 -- 172.21.15.12:0/3198248538 wait complete. 2023-11-28T21:52:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe6236d6640 1 Processor -- start 2023-11-28T21:52:47.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe6236d6640 1 -- start start 2023-11-28T21:52:47.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe6236d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608004d10 0x7fe61c14d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:47.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe6236d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c14fb80 0x7fe61c15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:47.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe6236d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe61c1f3180 0x7fe61c150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:47.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe6236d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c15c210 con 0x7fe61c1f3180 2023-11-28T21:52:47.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe6236d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c154660 con 0x7fe608004d10 2023-11-28T21:52:47.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe6236d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c154e30 con 0x7fe61c14fb80 2023-11-28T21:52:47.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe621c73640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe61c1f3180 0x7fe61c150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:47.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe621c73640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe61c1f3180 0x7fe61c150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59580/0 (socket says 172.21.15.12:59580) 2023-11-28T21:52:47.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe621c73640 1 -- 172.21.15.12:0/3754806447 learned_addr learned my addr 172.21.15.12:0/3754806447 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:47.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe621472640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608004d10 0x7fe61c14d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:47.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe620c71640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c14fb80 0x7fe61c15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe621c73640 1 -- 172.21.15.12:0/3754806447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c14fb80 msgr2=0x7fe61c15af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe621c73640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c14fb80 0x7fe61c15af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe621c73640 1 -- 172.21.15.12:0/3754806447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608004d10 msgr2=0x7fe61c14d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe621c73640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608004d10 0x7fe61c14d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe621c73640 1 -- 172.21.15.12:0/3754806447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c156760 con 0x7fe61c1f3180 2023-11-28T21:52:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe621c73640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe61c1f3180 0x7fe61c150030 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe6140b0040 tx=0x7fe614009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.402+0000 7fe611ffb640 1 -- 172.21.15.12:0/3754806447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140bc070 con 0x7fe61c1f3180 2023-11-28T21:52:47.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.406+0000 7fe611ffb640 1 -- 172.21.15.12:0/3754806447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6140039a0 con 0x7fe61c1f3180 2023-11-28T21:52:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.406+0000 7fe611ffb640 1 -- 172.21.15.12:0/3754806447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614003b30 con 0x7fe61c1f3180 2023-11-28T21:52:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.406+0000 7fe6236d6640 1 -- 172.21.15.12:0/3754806447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c154870 con 0x7fe61c1f3180 2023-11-28T21:52:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.406+0000 7fe6236d6640 1 -- 172.21.15.12:0/3754806447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c155c80 con 0x7fe61c1f3180 2023-11-28T21:52:47.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.406+0000 7fe6236d6640 1 -- 172.21.15.12:0/3754806447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61c067c70 con 0x7fe61c1f3180 2023-11-28T21:52:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.406+0000 7fe611ffb640 1 -- 172.21.15.12:0/3754806447 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe6140b4040 con 0x7fe61c1f3180 2023-11-28T21:52:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.406+0000 7fe611ffb640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe600066470 0x7fe600068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.406+0000 7fe611ffb640 1 -- 172.21.15.12:0/3754806447 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7fe61411e700 con 0x7fe61c1f3180 2023-11-28T21:52:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.406+0000 7fe621472640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe600066470 0x7fe600068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:47.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.406+0000 7fe621472640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe600066470 0x7fe600068930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe618005c80 tx=0x7fe618007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:47.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.410+0000 7fe611ffb640 1 -- 172.21.15.12:0/3754806447 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe6140eb490 con 0x7fe61c1f3180 2023-11-28T21:52:47.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.598+0000 7fe6236d6640 1 -- 172.21.15.12:0/3754806447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe61c074f60 con 0x7fe61c1f3180 2023-11-28T21:52:47.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.598+0000 7fe611ffb640 1 -- 172.21.15.12:0/3754806447 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6518 (secure 0 0 0) 0x7fe6140b2090 con 0x7fe61c1f3180 2023-11-28T21:52:47.605 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:47.605 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:53.213081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:active","state_seq":165,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:47.605 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-11-28T21:52:47.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 -- 172.21.15.12:0/3754806447 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe600066470 msgr2=0x7fe600068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:47.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe600066470 0x7fe600068930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe618005c80 tx=0x7fe618007040 comp rx=0 tx=0).stop 2023-11-28T21:52:47.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 -- 172.21.15.12:0/3754806447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe61c1f3180 msgr2=0x7fe61c150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:47.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe61c1f3180 0x7fe61c150030 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe6140b0040 tx=0x7fe614009750 comp rx=0 tx=0).stop 2023-11-28T21:52:47.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 -- 172.21.15.12:0/3754806447 shutdown_connections 2023-11-28T21:52:47.607 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe600066470 0x7fe600068930 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe61c1f3180 0x7fe61c150030 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe61c14fb80 0x7fe61c15af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 --2- 172.21.15.12:0/3754806447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608004d10 0x7fe61c14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 -- 172.21.15.12:0/3754806447 >> 172.21.15.12:0/3754806447 conn(0x7fe61c0720e0 msgr2=0x7fe61c0cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 -- 172.21.15.12:0/3754806447 shutdown_connections 2023-11-28T21:52:47.608 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:47.602+0000 7fe6236d6640 1 -- 172.21.15.12:0/3754806447 wait complete. 2023-11-28T21:52:47.626 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-11-28T21:52:49.627 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:49.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d78a9e640 1 Processor -- start 2023-11-28T21:52:49.780 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d78a9e640 1 -- start start 2023-11-28T21:52:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d78a9e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2d741500d0 0x7f2d740c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d78a9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d74156af0 0x7f2d740c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:49.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d78a9e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2d740c3500 0x7f2d7406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d78a9e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d740c7170 con 0x7f2d741500d0 2023-11-28T21:52:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d78a9e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d740c6ce0 con 0x7f2d740c3500 2023-11-28T21:52:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d78a9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740c5df0 con 0x7f2d74156af0 2023-11-28T21:52:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d7259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2d741500d0 0x7f2d740c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d7259c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2d741500d0 0x7f2d740c27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59592/0 (socket says 172.21.15.12:59592) 2023-11-28T21:52:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.774+0000 7f2d7259c640 1 -- 172.21.15.12:0/1438120931 learned_addr learned my addr 172.21.15.12:0/1438120931 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:49.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d72d9d640 1 --2- 172.21.15.12:0/1438120931 >> v2:172.21.15.144:3300/0 conn(0x7f2d740c3500 0x7f2d7406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d71d9b640 1 --2- 172.21.15.12:0/1438120931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d74156af0 0x7f2d740c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d7259c640 1 -- 172.21.15.12:0/1438120931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d74156af0 msgr2=0x7f2d740c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d7259c640 1 --2- 172.21.15.12:0/1438120931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d74156af0 0x7f2d740c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d7259c640 1 -- 172.21.15.12:0/1438120931 >> v2:172.21.15.144:3300/0 conn(0x7f2d740c3500 msgr2=0x7f2d7406d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d7259c640 1 --2- 172.21.15.12:0/1438120931 >> v2:172.21.15.144:3300/0 conn(0x7f2d740c3500 0x7f2d7406d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d7259c640 1 -- 172.21.15.12:0/1438120931 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d740c60d0 con 0x7f2d741500d0 2023-11-28T21:52:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d7259c640 1 --2- 172.21.15.12:0/1438120931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 0x7f2d740c27c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2d74074360 tx=0x7f2d6401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418aa692e55267ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d7159a640 1 -- 172.21.15.12:0/1438120931 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640ab600 con 0x7f2d741500d0 2023-11-28T21:52:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d7159a640 1 -- 172.21.15.12:0/1438120931 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d640a7050 con 0x7f2d741500d0 2023-11-28T21:52:49.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.778+0000 7f2d7159a640 1 -- 172.21.15.12:0/1438120931 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640ab980 con 0x7f2d741500d0 2023-11-28T21:52:49.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1438120931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 msgr2=0x7f2d740c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 --2- 172.21.15.12:0/1438120931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 0x7f2d740c27c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2d74074360 tx=0x7f2d6401c440 comp rx=0 tx=0).stop 2023-11-28T21:52:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1438120931 shutdown_connections 2023-11-28T21:52:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 --2- 172.21.15.12:0/1438120931 >> v2:172.21.15.144:3300/0 conn(0x7f2d740c3500 0x7f2d7406d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 --2- 172.21.15.12:0/1438120931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d74156af0 0x7f2d740c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 --2- 172.21.15.12:0/1438120931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 0x7f2d740c27c0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1438120931 >> 172.21.15.12:0/1438120931 conn(0x7f2d74066360 msgr2=0x7f2d74066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:49.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1438120931 shutdown_connections 2023-11-28T21:52:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1438120931 wait complete. 2023-11-28T21:52:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 Processor -- start 2023-11-28T21:52:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 -- start start 2023-11-28T21:52:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d740c3500 0x7f2d741f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 0x7f2d7414aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d74156af0 0x7f2d7414b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d74070bb0 con 0x7f2d741500d0 2023-11-28T21:52:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741f65f0 con 0x7f2d74156af0 2023-11-28T21:52:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d78a9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741f6310 con 0x7f2d740c3500 2023-11-28T21:52:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d71d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 0x7f2d7414aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d71d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 0x7f2d7414aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59596/0 (socket says 172.21.15.12:59596) 2023-11-28T21:52:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d71d9b640 1 -- 172.21.15.12:0/1863206048 learned_addr learned my addr 172.21.15.12:0/1863206048 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d7259c640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d740c3500 0x7f2d741f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d72d9d640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d74156af0 0x7f2d7414b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d71d9b640 1 -- 172.21.15.12:0/1863206048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d740c3500 msgr2=0x7f2d741f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d71d9b640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d740c3500 0x7f2d741f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d71d9b640 1 -- 172.21.15.12:0/1863206048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d74156af0 msgr2=0x7f2d7414b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d71d9b640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d74156af0 0x7f2d7414b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d71d9b640 1 -- 172.21.15.12:0/1863206048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640094d0 con 0x7f2d741500d0 2023-11-28T21:52:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.782+0000 7f2d7259c640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d740c3500 0x7f2d741f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:52:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d71d9b640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 0x7f2d7414aae0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800d800 tx=0x7f2d6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d62ffd640 1 -- 172.21.15.12:0/1863206048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800b6c0 con 0x7f2d741500d0 2023-11-28T21:52:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d62ffd640 1 -- 172.21.15.12:0/1863206048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6800b820 con 0x7f2d741500d0 2023-11-28T21:52:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d62ffd640 1 -- 172.21.15.12:0/1863206048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800bb90 con 0x7f2d741500d0 2023-11-28T21:52:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1863206048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d74158420 con 0x7f2d741500d0 2023-11-28T21:52:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1863206048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d74155a00 con 0x7f2d741500d0 2023-11-28T21:52:49.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1863206048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d74074f60 con 0x7f2d741500d0 2023-11-28T21:52:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d62ffd640 1 -- 172.21.15.12:0/1863206048 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2d6800bcf0 con 0x7f2d741500d0 2023-11-28T21:52:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d62ffd640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 0x7f2d54068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d62ffd640 1 -- 172.21.15.12:0/1863206048 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7f2d68084a10 con 0x7f2d741500d0 2023-11-28T21:52:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.786+0000 7f2d7259c640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 0x7f2d54068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:49.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.790+0000 7f2d7259c640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 0x7f2d54068930 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2d640ad000 tx=0x7f2d64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.790+0000 7f2d62ffd640 1 -- 172.21.15.12:0/1863206048 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d68051820 con 0x7f2d741500d0 2023-11-28T21:52:49.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.978+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1863206048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d740c7320 con 0x7f2d741500d0 2023-11-28T21:52:49.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d62ffd640 1 -- 172.21.15.12:0/1863206048 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6518 (secure 0 0 0) 0x7f2d68054e40 con 0x7f2d741500d0 2023-11-28T21:52:49.987 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:49.987 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:53.213081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:active","state_seq":165,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:49.988 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-11-28T21:52:49.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1863206048 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 msgr2=0x7f2d54068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:49.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 0x7f2d54068930 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2d640ad000 tx=0x7f2d64008040 comp rx=0 tx=0).stop 2023-11-28T21:52:49.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1863206048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 msgr2=0x7f2d7414aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:49.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 0x7f2d7414aae0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800d800 tx=0x7f2d6800dcc0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1863206048 shutdown_connections 2023-11-28T21:52:49.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2d54066470 0x7f2d54068930 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d74156af0 0x7f2d7414b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d741500d0 0x7f2d7414aae0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 --2- 172.21.15.12:0/1863206048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d740c3500 0x7f2d741f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1863206048 >> 172.21.15.12:0/1863206048 conn(0x7f2d74066360 msgr2=0x7f2d7414f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1863206048 shutdown_connections 2023-11-28T21:52:49.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:49.982+0000 7f2d78a9e640 1 -- 172.21.15.12:0/1863206048 wait complete. 2023-11-28T21:52:50.008 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-11-28T21:52:52.009 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f8a9b640 1 Processor -- start 2023-11-28T21:52:52.166 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f8a9b640 1 -- start start 2023-11-28T21:52:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f8a9b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 0x7ff7f40b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f8a9b640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff7f40b4010 0x7ff7f40cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f8a9b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f415d390 con 0x7ff7f4153fb0 2023-11-28T21:52:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f8a9b640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f40b2b60 con 0x7ff7f40b4010 2023-11-28T21:52:52.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f8a9b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f415e1b0 con 0x7ff7f415a980 2023-11-28T21:52:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f2ffd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 0x7ff7f40b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f37fe640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7ff7f4153fb0 0x7ff7f40ca4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:51964/0 (socket says 172.21.15.12:51964) 2023-11-28T21:52:52.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f37fe640 1 -- 172.21.15.12:0/1880230717 learned_addr learned my addr 172.21.15.12:0/1880230717 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:52:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f3fff640 1 --2- 172.21.15.12:0/1880230717 >> v2:172.21.15.144:3300/0 conn(0x7ff7f40b4010 0x7ff7f40cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f2ffd640 1 -- 172.21.15.12:0/1880230717 >> v2:172.21.15.144:3300/0 conn(0x7ff7f40b4010 msgr2=0x7ff7f40cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f2ffd640 1 --2- 172.21.15.12:0/1880230717 >> v2:172.21.15.144:3300/0 conn(0x7ff7f40b4010 0x7ff7f40cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f2ffd640 1 -- 172.21.15.12:0/1880230717 >> v1:172.21.15.144:6789/0 conn(0x7ff7f4153fb0 legacy=0x7ff7f40ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f2ffd640 1 -- 172.21.15.12:0/1880230717 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f415d630 con 0x7ff7f415a980 2023-11-28T21:52:52.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.162+0000 7ff7f3fff640 1 --2- 172.21.15.12:0/1880230717 >> v2:172.21.15.144:3300/0 conn(0x7ff7f40b4010 0x7ff7f40cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:52:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f2ffd640 1 --2- 172.21.15.12:0/1880230717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 0x7ff7f40b3ad0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80097c0 tx=0x7ff7e801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b87681adf6b9f59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:52.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f27fc640 1 -- 172.21.15.12:0/1880230717 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80ab650 con 0x7ff7f415a980 2023-11-28T21:52:52.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f27fc640 1 -- 172.21.15.12:0/1880230717 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e80ab7b0 con 0x7ff7f415a980 2023-11-28T21:52:52.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f27fc640 1 -- 172.21.15.12:0/1880230717 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80abb40 con 0x7ff7f415a980 2023-11-28T21:52:52.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/1880230717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 msgr2=0x7ff7f40b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:52.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f8a9b640 1 --2- 172.21.15.12:0/1880230717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 0x7ff7f40b3ad0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80097c0 tx=0x7ff7e801c450 comp rx=0 tx=0).stop 2023-11-28T21:52:52.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/1880230717 shutdown_connections 2023-11-28T21:52:52.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f8a9b640 1 --2- 172.21.15.12:0/1880230717 >> v2:172.21.15.144:3300/0 conn(0x7ff7f40b4010 0x7ff7f40cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:52.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f8a9b640 1 --2- 172.21.15.12:0/1880230717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 0x7ff7f40b3ad0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:52.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/1880230717 >> 172.21.15.12:0/1880230717 conn(0x7ff7f4071ce0 msgr2=0x7ff7f40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:52.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/1880230717 shutdown_connections 2023-11-28T21:52:52.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.166+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/1880230717 wait complete. 2023-11-28T21:52:52.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f8a9b640 1 Processor -- start 2023-11-28T21:52:52.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f8a9b640 1 -- start start 2023-11-28T21:52:52.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f8a9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7f40b4010 0x7ff7f40c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:52.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f8a9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f4153fb0 0x7ff7f4141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f8a9b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 0x7ff7f415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f8a9b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f415b820 con 0x7ff7f40b4010 2023-11-28T21:52:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f8a9b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f40c5940 con 0x7ff7f4153fb0 2023-11-28T21:52:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f8a9b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4141dd0 con 0x7ff7f415a980 2023-11-28T21:52:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7f40b4010 0x7ff7f40c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7f40b4010 0x7ff7f40c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59606/0 (socket says 172.21.15.12:59606) 2023-11-28T21:52:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f37fe640 1 -- 172.21.15.12:0/399757109 learned_addr learned my addr 172.21.15.12:0/399757109 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f2ffd640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f4153fb0 0x7ff7f4141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:52.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f3fff640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 0x7ff7f415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:52.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f37fe640 1 -- 172.21.15.12:0/399757109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 msgr2=0x7ff7f415c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:52.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f37fe640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 0x7ff7f415c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:52.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f37fe640 1 -- 172.21.15.12:0/399757109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f4153fb0 msgr2=0x7ff7f4141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:52.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f37fe640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f4153fb0 0x7ff7f4141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f37fe640 1 -- 172.21.15.12:0/399757109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec009a70 con 0x7ff7f40b4010 2023-11-28T21:52:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f37fe640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7f40b4010 0x7ff7f40c5400 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400c450 tx=0x7ff7e400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7dbfff640 1 -- 172.21.15.12:0/399757109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4005cf0 con 0x7ff7f40b4010 2023-11-28T21:52:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7dbfff640 1 -- 172.21.15.12:0/399757109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e4005e50 con 0x7ff7f40b4010 2023-11-28T21:52:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7dbfff640 1 -- 172.21.15.12:0/399757109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40072f0 con 0x7ff7f40b4010 2023-11-28T21:52:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/399757109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e8009420 con 0x7ff7f40b4010 2023-11-28T21:52:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.170+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/399757109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f40c5bb0 con 0x7ff7f40b4010 2023-11-28T21:52:52.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.174+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/399757109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f4074f60 con 0x7ff7f40b4010 2023-11-28T21:52:52.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.174+0000 7ff7dbfff640 1 -- 172.21.15.12:0/399757109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff7e4007450 con 0x7ff7f40b4010 2023-11-28T21:52:52.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.174+0000 7ff7dbfff640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 0x7ff7e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:52.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.174+0000 7ff7dbfff640 1 -- 172.21.15.12:0/399757109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7351+0+0 (secure 0 0 0) 0x7ff7e4083a50 con 0x7ff7f40b4010 2023-11-28T21:52:52.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.174+0000 7ff7f2ffd640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 0x7ff7e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:52.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.174+0000 7ff7f2ffd640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 0x7ff7e0068930 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80af000 tx=0x7ff7e80b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:52.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.178+0000 7ff7dbfff640 1 -- 172.21.15.12:0/399757109 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff7e4050870 con 0x7ff7f40b4010 2023-11-28T21:52:52.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.366+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/399757109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7f40cb810 con 0x7ff7f40b4010 2023-11-28T21:52:52.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.366+0000 7ff7dbfff640 1 -- 172.21.15.12:0/399757109 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6518 (secure 0 0 0) 0x7ff7e4017090 con 0x7ff7f40b4010 2023-11-28T21:52:52.374 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:52.374 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:51:53.213081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":50,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5057,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_5057":{"gid":5057,"name":"f","rank":0,"incarnation":65,"state":"up:active","state_seq":165,"addr":"172.21.15.144:6833/126572048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":126572048},{"type":"v1","addr":"172.21.15.144:6833","nonce":126572048}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:52.374 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 72 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/399757109 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 msgr2=0x7ff7e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 0x7ff7e0068930 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80af000 tx=0x7ff7e80b6040 comp rx=0 tx=0).stop 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/399757109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7f40b4010 msgr2=0x7ff7f40c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7f40b4010 0x7ff7f40c5400 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400c450 tx=0x7ff7e400c850 comp rx=0 tx=0).stop 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/399757109 shutdown_connections 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 0x7ff7e0068930 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7f415a980 0x7ff7f415c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7f4153fb0 0x7ff7f4141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 --2- 172.21.15.12:0/399757109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7f40b4010 0x7ff7f40c5400 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/399757109 >> 172.21.15.12:0/399757109 conn(0x7ff7f4071ce0 msgr2=0x7ff7f40b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/399757109 shutdown_connections 2023-11-28T21:52:52.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:52.370+0000 7ff7f8a9b640 1 -- 172.21.15.12:0/399757109 wait complete. 2023-11-28T21:52:52.395 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-11-28T21:52:54.397 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:54.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.546+0000 7fa09a957640 1 Processor -- start 2023-11-28T21:52:54.552 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.546+0000 7fa09a957640 1 -- start start 2023-11-28T21:52:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.546+0000 7fa09a957640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa094151af0 0x7fa0940c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.546+0000 7fa09a957640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa09406e300 0x7fa094072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.546+0000 7fa09a957640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa094157b70 con 0x7fa094151ed0 2023-11-28T21:52:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.546+0000 7fa09a957640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0940707f0 con 0x7fa09406e300 2023-11-28T21:52:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.546+0000 7fa09a957640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0940c7b90 con 0x7fa094151af0 2023-11-28T21:52:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa0937fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa094151ed0 0x7fa0940c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:42790/0 (socket says 172.21.15.12:42790) 2023-11-28T21:52:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa0937fe640 1 -- 172.21.15.12:0/3161317352 learned_addr learned my addr 172.21.15.12:0/3161317352 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:52:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa098ef4640 1 --2- 172.21.15.12:0/3161317352 >> v2:172.21.15.12:3300/0 conn(0x7fa09406e300 0x7fa094072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.554 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa093fff640 1 --2- 172.21.15.12:0/3161317352 >> v2:172.21.15.144:3300/0 conn(0x7fa094151af0 0x7fa0940c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa092ffd640 1 -- 172.21.15.12:0/3161317352 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929949671 0 0) 0x7fa094157b70 con 0x7fa094151ed0 2023-11-28T21:52:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa092ffd640 1 -- 172.21.15.12:0/3161317352 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa07c003530 con 0x7fa094151ed0 2023-11-28T21:52:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa098ef4640 1 -- 172.21.15.12:0/3161317352 >> v2:172.21.15.144:3300/0 conn(0x7fa094151af0 msgr2=0x7fa0940c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa098ef4640 1 --2- 172.21.15.12:0/3161317352 >> v2:172.21.15.144:3300/0 conn(0x7fa094151af0 0x7fa0940c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa098ef4640 1 -- 172.21.15.12:0/3161317352 >> v1:172.21.15.12:6789/0 conn(0x7fa094151ed0 legacy=0x7fa0940c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.555 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa098ef4640 1 -- 172.21.15.12:0/3161317352 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa094157e50 con 0x7fa09406e300 2023-11-28T21:52:54.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa098ef4640 1 --2- 172.21.15.12:0/3161317352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa09406e300 0x7fa094072070 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa0940c7d20 tx=0x7fa08c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e381c0ce292ee10c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:54.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa092ffd640 1 -- 172.21.15.12:0/3161317352 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0a57d0 con 0x7fa09406e300 2023-11-28T21:52:54.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa092ffd640 1 -- 172.21.15.12:0/3161317352 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c0a5930 con 0x7fa09406e300 2023-11-28T21:52:54.557 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.550+0000 7fa092ffd640 1 -- 172.21.15.12:0/3161317352 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0a5cc0 con 0x7fa09406e300 2023-11-28T21:52:54.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 -- 172.21.15.12:0/3161317352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa09406e300 msgr2=0x7fa094072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.558 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 --2- 172.21.15.12:0/3161317352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa09406e300 0x7fa094072070 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa0940c7d20 tx=0x7fa08c01c910 comp rx=0 tx=0).stop 2023-11-28T21:52:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 -- 172.21.15.12:0/3161317352 shutdown_connections 2023-11-28T21:52:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 --2- 172.21.15.12:0/3161317352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa09406e300 0x7fa094072070 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 --2- 172.21.15.12:0/3161317352 >> v2:172.21.15.144:3300/0 conn(0x7fa094151af0 0x7fa0940c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 -- 172.21.15.12:0/3161317352 >> 172.21.15.12:0/3161317352 conn(0x7fa094066360 msgr2=0x7fa094066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 -- 172.21.15.12:0/3161317352 shutdown_connections 2023-11-28T21:52:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 -- 172.21.15.12:0/3161317352 wait complete. 2023-11-28T21:52:54.559 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 Processor -- start 2023-11-28T21:52:54.560 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 -- start start 2023-11-28T21:52:54.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09406e300 0x7fa0940c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.562 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa094151af0 0x7fa094156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa094154b10 0x7fa09413eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0941422a0 con 0x7fa094151af0 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09414f980 con 0x7fa09406e300 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa09a957640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940c4430 con 0x7fa094154b10 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa0937fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa094151af0 0x7fa094156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa0937fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa094151af0 0x7fa094156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59636/0 (socket says 172.21.15.12:59636) 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa0937fe640 1 -- 172.21.15.12:0/3457217657 learned_addr learned my addr 172.21.15.12:0/3457217657 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa098ef4640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa094154b10 0x7fa09413eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.554+0000 7fa093fff640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09406e300 0x7fa0940c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa0937fe640 1 -- 172.21.15.12:0/3457217657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa094154b10 msgr2=0x7fa09413eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa0937fe640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa094154b10 0x7fa09413eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa0937fe640 1 -- 172.21.15.12:0/3457217657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09406e300 msgr2=0x7fa0940c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.563 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa0937fe640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09406e300 0x7fa0940c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa0937fe640 1 -- 172.21.15.12:0/3457217657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c009a70 con 0x7fa094151af0 2023-11-28T21:52:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa098ef4640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa094154b10 0x7fa09413eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:52:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa093fff640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09406e300 0x7fa0940c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:52:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa0937fe640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa094151af0 0x7fa094156ef0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c430 tx=0x7fa08800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa090ff9640 1 -- 172.21.15.12:0/3457217657 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088005cf0 con 0x7fa094151af0 2023-11-28T21:52:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa090ff9640 1 -- 172.21.15.12:0/3457217657 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa088005e70 con 0x7fa094151af0 2023-11-28T21:52:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa090ff9640 1 -- 172.21.15.12:0/3457217657 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0880072d0 con 0x7fa094151af0 2023-11-28T21:52:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa09a957640 1 -- 172.21.15.12:0/3457217657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0941f3820 con 0x7fa094151af0 2023-11-28T21:52:54.564 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa09a957640 1 -- 172.21.15.12:0/3457217657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa094142480 con 0x7fa094151af0 2023-11-28T21:52:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa09a957640 1 -- 172.21.15.12:0/3457217657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0941f3cd0 con 0x7fa094151af0 2023-11-28T21:52:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa090ff9640 1 -- 172.21.15.12:0/3457217657 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa088007430 con 0x7fa094151af0 2023-11-28T21:52:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa090ff9640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa080066470 0x7fa080068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa090ff9640 1 -- 172.21.15.12:0/3457217657 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fa0880838f0 con 0x7fa094151af0 2023-11-28T21:52:54.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.558+0000 7fa093fff640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa080066470 0x7fa080068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.562+0000 7fa093fff640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa080066470 0x7fa080068930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa084009bb0 tx=0x7fa084008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:54.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.562+0000 7fa090ff9640 1 -- 172.21.15.12:0/3457217657 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa088050580 con 0x7fa094151af0 2023-11-28T21:52:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.754+0000 7fa09a957640 1 -- 172.21.15.12:0/3457217657 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa09406c920 con 0x7fa094151af0 2023-11-28T21:52:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.754+0000 7fa090ff9640 1 -- 172.21.15.12:0/3457217657 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5724 (secure 0 0 0) 0x7fa088017090 con 0x7fa094151af0 2023-11-28T21:52:54.761 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:54.761 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:52:53.611931+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 73 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 -- 172.21.15.12:0/3457217657 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa080066470 msgr2=0x7fa080068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa080066470 0x7fa080068930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa084009bb0 tx=0x7fa084008040 comp rx=0 tx=0).stop 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 -- 172.21.15.12:0/3457217657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa094151af0 msgr2=0x7fa094156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa094151af0 0x7fa094156ef0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c430 tx=0x7fa08800c8f0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 -- 172.21.15.12:0/3457217657 shutdown_connections 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa080066470 0x7fa080068930 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa094154b10 0x7fa09413eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa094151af0 0x7fa094156ef0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 --2- 172.21.15.12:0/3457217657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09406e300 0x7fa0940c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 -- 172.21.15.12:0/3457217657 >> 172.21.15.12:0/3457217657 conn(0x7fa094066360 msgr2=0x7fa09406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 -- 172.21.15.12:0/3457217657 shutdown_connections 2023-11-28T21:52:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.758+0000 7fa09a957640 1 -- 172.21.15.12:0/3457217657 wait complete. 2023-11-28T21:52:54.782 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2023-11-28T21:52:54.782 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:52:54.782 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e4432640 1 Processor -- start 2023-11-28T21:52:54.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e4432640 1 -- start start 2023-11-28T21:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e4432640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd0dc06e300 0x7fd0dc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e4432640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc157b70 con 0x7fd0dc151ed0 2023-11-28T21:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e4432640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc157e50 con 0x7fd0dc151af0 2023-11-28T21:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e4432640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0dc0707f0 con 0x7fd0dc06e300 2023-11-28T21:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e19cd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd0dc151ed0 0x7fd0dc072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55284/0 (socket says 172.21.15.12:55284) 2023-11-28T21:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e19cd640 1 -- 172.21.15.12:0/1392578821 learned_addr learned my addr 172.21.15.12:0/1392578821 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e29cf640 1 --2- 172.21.15.12:0/1392578821 >> v2:172.21.15.144:3300/0 conn(0x7fd0dc06e300 0x7fd0dc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532807408 0 0) 0x7fd0dc157b70 con 0x7fd0dc151ed0 2023-11-28T21:52:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0c8003530 con 0x7fd0dc151ed0 2023-11-28T21:52:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1423255343 0 0) 0x7fd0c8003530 con 0x7fd0dc151ed0 2023-11-28T21:52:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 >> v2:172.21.15.144:3300/0 conn(0x7fd0dc06e300 msgr2=0x7fd0dc0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e11cc640 1 --2- 172.21.15.12:0/1392578821 >> v2:172.21.15.144:3300/0 conn(0x7fd0dc06e300 0x7fd0dc0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 >> v1:172.21.15.144:6789/0 conn(0x7fd0dc151af0 legacy=0x7fd0dc0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc158130 con 0x7fd0dc151ed0 2023-11-28T21:52:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fd0cc001680 con 0x7fd0dc151ed0 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 >> v1:172.21.15.12:6789/0 conn(0x7fd0dc151ed0 legacy=0x7fd0dc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.922+0000 7fd0e11cc640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0c8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e11cc640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c80047d0 0x7fd0c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e11cc640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c80051e0 0x7fd0c800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc0707f0 con 0x7fd0c8003a40 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c800c110 con 0x7fd0c80051e0 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c800c2d0 con 0x7fd0c80047d0 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e21ce640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0c8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e19cd640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c80047d0 0x7fd0c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e29cf640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c80051e0 0x7fd0c800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e21ce640 1 -- 172.21.15.12:0/1392578821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c80047d0 msgr2=0x7fd0c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e21ce640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c80047d0 0x7fd0c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e21ce640 1 -- 172.21.15.12:0/1392578821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c80051e0 msgr2=0x7fd0c800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e21ce640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c80051e0 0x7fd0c800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e21ce640 1 -- 172.21.15.12:0/1392578821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c8003530 con 0x7fd0c8003a40 2023-11-28T21:52:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e29cf640 1 -- 172.21.15.12:0/1392578821 reap_dead start 2023-11-28T21:52:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e21ce640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0c8003e60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8001780 tx=0x7fd0d801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8488992035d9f6bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80052d0 con 0x7fd0c8003a40 2023-11-28T21:52:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d80a7050 con 0x7fd0c8003a40 2023-11-28T21:52:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.926+0000 7fd0e11cc640 1 -- 172.21.15.12:0/1392578821 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8003600 con 0x7fd0c8003a40 2023-11-28T21:52:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 -- 172.21.15.12:0/1392578821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 msgr2=0x7fd0c8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0c8003e60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8001780 tx=0x7fd0d801c3e0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 -- 172.21.15.12:0/1392578821 shutdown_connections 2023-11-28T21:52:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 --2- 172.21.15.12:0/1392578821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0c8003e60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 -- 172.21.15.12:0/1392578821 >> 172.21.15.12:0/1392578821 conn(0x7fd0dc066360 msgr2=0x7fd0dc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 -- 172.21.15.12:0/1392578821 shutdown_connections 2023-11-28T21:52:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 -- 172.21.15.12:0/1392578821 wait complete. 2023-11-28T21:52:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 Processor -- start 2023-11-28T21:52:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 -- start start 2023-11-28T21:52:54.940 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0dc15a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0dc15aab0 0x7fd0dc147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0dc15ba70 0x7fd0dc148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc159810 con 0x7fd0c8003a40 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc148540 con 0x7fd0dc15aab0 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e4432640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc14a810 con 0x7fd0dc15ba70 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e21ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0dc15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e21ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0dc15a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50994/0 (socket says 172.21.15.12:50994) 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.930+0000 7fd0e21ce640 1 -- 172.21.15.12:0/4130119163 learned_addr learned my addr 172.21.15.12:0/4130119163 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e29cf640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0dc15ba70 0x7fd0dc148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e19cd640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0dc15aab0 0x7fd0dc147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e21ce640 1 -- 172.21.15.12:0/4130119163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0dc15ba70 msgr2=0x7fd0dc148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.941 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e21ce640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0dc15ba70 0x7fd0dc148000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e21ce640 1 -- 172.21.15.12:0/4130119163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0dc15aab0 msgr2=0x7fd0dc147ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:54.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e21ce640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0dc15aab0 0x7fd0dc147ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:54.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e21ce640 1 -- 172.21.15.12:0/4130119163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc1f2e50 con 0x7fd0c8003a40 2023-11-28T21:52:54.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e21ce640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0dc15a570 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8000c10 tx=0x7fd0c8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:54.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0d2ffd640 1 -- 172.21.15.12:0/4130119163 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80072f0 con 0x7fd0c8003a40 2023-11-28T21:52:54.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0d2ffd640 1 -- 172.21.15.12:0/4130119163 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d8007450 con 0x7fd0c8003a40 2023-11-28T21:52:54.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0d2ffd640 1 -- 172.21.15.12:0/4130119163 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80067e0 con 0x7fd0c8003a40 2023-11-28T21:52:54.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e4432640 1 -- 172.21.15.12:0/4130119163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc159c30 con 0x7fd0c8003a40 2023-11-28T21:52:54.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e4432640 1 -- 172.21.15.12:0/4130119163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc158b90 con 0x7fd0c8003a40 2023-11-28T21:52:54.942 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e4432640 1 -- 172.21.15.12:0/4130119163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc074f60 con 0x7fd0c8003a40 2023-11-28T21:52:54.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0d2ffd640 1 -- 172.21.15.12:0/4130119163 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd0d8006940 con 0x7fd0c8003a40 2023-11-28T21:52:54.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0d2ffd640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd0c0066470 0x7fd0c0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:54.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0d2ffd640 1 -- 172.21.15.12:0/4130119163 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fd0d811e740 con 0x7fd0c8003a40 2023-11-28T21:52:54.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.934+0000 7fd0e19cd640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd0c0066470 0x7fd0c0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:54.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.938+0000 7fd0e19cd640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd0c0066470 0x7fd0c0068930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc066f30 tx=0x7fd0cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:54.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:54.938+0000 7fd0d2ffd640 1 -- 172.21.15.12:0/4130119163 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd0d80eb3d0 con 0x7fd0c8003a40 2023-11-28T21:52:55.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 -- 172.21.15.12:0/4130119163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0dc072c10 con 0x7fd0c8003a40 2023-11-28T21:52:55.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0d2ffd640 1 -- 172.21.15.12:0/4130119163 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5724 (secure 0 0 0) 0x7fd0d80ee9f0 con 0x7fd0c8003a40 2023-11-28T21:52:55.135 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:55.135 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:52:53.611931+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:55.136 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 73 2023-11-28T21:52:55.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 -- 172.21.15.12:0/4130119163 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd0c0066470 msgr2=0x7fd0c0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:55.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd0c0066470 0x7fd0c0068930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc066f30 tx=0x7fd0cc009040 comp rx=0 tx=0).stop 2023-11-28T21:52:55.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 -- 172.21.15.12:0/4130119163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 msgr2=0x7fd0dc15a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:55.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0dc15a570 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8000c10 tx=0x7fd0c8005800 comp rx=0 tx=0).stop 2023-11-28T21:52:55.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 -- 172.21.15.12:0/4130119163 shutdown_connections 2023-11-28T21:52:55.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd0c0066470 0x7fd0c0068930 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:55.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0dc15ba70 0x7fd0dc148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:55.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0dc15aab0 0x7fd0dc147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:55.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 --2- 172.21.15.12:0/4130119163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8003a40 0x7fd0dc15a570 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:55.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 -- 172.21.15.12:0/4130119163 >> 172.21.15.12:0/4130119163 conn(0x7fd0dc066360 msgr2=0x7fd0dc0c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:55.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 -- 172.21.15.12:0/4130119163 shutdown_connections 2023-11-28T21:52:55.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:55.130+0000 7fd0e4432640 1 -- 172.21.15.12:0/4130119163 wait complete. 2023-11-28T21:52:55.157 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 73, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 73, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6330": { "addr": "172.21.15.12:6839/754378087", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 754378087, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 754378087, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6330, "incarnation": 63, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 16 }, "gid_6513": { "addr": "172.21.15.12:6838/2286739028", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2286739028, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 2286739028, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6513, "incarnation": 73, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:52:53.611931+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6513, "mds_1": 6330 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:52:57.158 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:57.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.306+0000 7f26a9f16640 1 Processor -- start 2023-11-28T21:52:57.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.306+0000 7f26a9f16640 1 -- start start 2023-11-28T21:52:57.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a9f16640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f26a406e300 0x7f26a4156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:57.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a9f16640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406cf40 0x7f26a4158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:57.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a9f16640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a4067410 con 0x7f26a4150c20 2023-11-28T21:52:57.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a9f16640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a406e090 con 0x7f26a406e300 2023-11-28T21:52:57.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a9f16640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a4157f30 con 0x7f26a406cf40 2023-11-28T21:52:57.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a37fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f26a4150c20 0x7f26a40c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55298/0 (socket says 172.21.15.12:55298) 2023-11-28T21:52:57.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a37fe640 1 -- 172.21.15.12:0/263903478 learned_addr learned my addr 172.21.15.12:0/263903478 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:52:57.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a3fff640 1 --2- 172.21.15.12:0/263903478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406cf40 0x7f26a4158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:57.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a2ffd640 1 --2- 172.21.15.12:0/263903478 >> v2:172.21.15.144:3300/0 conn(0x7f26a406e300 0x7f26a4156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:57.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a27fc640 1 -- 172.21.15.12:0/263903478 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046149573 0 0) 0x7f26a4067410 con 0x7f26a4150c20 2023-11-28T21:52:57.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a27fc640 1 -- 172.21.15.12:0/263903478 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f268c003530 con 0x7f26a4150c20 2023-11-28T21:52:57.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a27fc640 1 -- 172.21.15.12:0/263903478 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1318400892 0 0) 0x7f268c003530 con 0x7f26a4150c20 2023-11-28T21:52:57.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a3fff640 1 -- 172.21.15.12:0/263903478 >> v2:172.21.15.144:3300/0 conn(0x7f26a406e300 msgr2=0x7f26a4156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:52:57.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a3fff640 1 --2- 172.21.15.12:0/263903478 >> v2:172.21.15.144:3300/0 conn(0x7f26a406e300 0x7f26a4156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:57.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a3fff640 1 -- 172.21.15.12:0/263903478 >> v1:172.21.15.12:6789/0 conn(0x7f26a4150c20 legacy=0x7f26a40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:57.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.310+0000 7f26a3fff640 1 -- 172.21.15.12:0/263903478 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a4151000 con 0x7f26a406cf40 2023-11-28T21:52:57.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a3fff640 1 --2- 172.21.15.12:0/263903478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406cf40 0x7f26a4158410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f269c009d90 tx=0x7f269c01ea80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39edd5fcf34110da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:57.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a27fc640 1 -- 172.21.15.12:0/263903478 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694001680 con 0x7f26a406cf40 2023-11-28T21:52:57.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a27fc640 1 -- 172.21.15.12:0/263903478 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c0a7680 con 0x7f26a406cf40 2023-11-28T21:52:57.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a27fc640 1 -- 172.21.15.12:0/263903478 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c0a7a10 con 0x7f26a406cf40 2023-11-28T21:52:57.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a9f16640 1 -- 172.21.15.12:0/263903478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406cf40 msgr2=0x7f26a4158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:57.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a9f16640 1 --2- 172.21.15.12:0/263903478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406cf40 0x7f26a4158410 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f269c009d90 tx=0x7f269c01ea80 comp rx=0 tx=0).stop 2023-11-28T21:52:57.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a9f16640 1 -- 172.21.15.12:0/263903478 shutdown_connections 2023-11-28T21:52:57.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a9f16640 1 --2- 172.21.15.12:0/263903478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406cf40 0x7f26a4158410 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:57.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a9f16640 1 --2- 172.21.15.12:0/263903478 >> v2:172.21.15.144:3300/0 conn(0x7f26a406e300 0x7f26a4156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:57.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a9f16640 1 -- 172.21.15.12:0/263903478 >> 172.21.15.12:0/263903478 conn(0x7f26a4066360 msgr2=0x7f26a4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:57.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a9f16640 1 -- 172.21.15.12:0/263903478 shutdown_connections 2023-11-28T21:52:57.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.314+0000 7f26a9f16640 1 -- 172.21.15.12:0/263903478 wait complete. 2023-11-28T21:52:57.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a9f16640 1 Processor -- start 2023-11-28T21:52:57.322 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a9f16640 1 -- start start 2023-11-28T21:52:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a9f16640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406cf40 0x7f26a414ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:57.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a9f16640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406e300 0x7f26a41465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a9f16640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a4150c20 0x7f26a41e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a9f16640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a414fbb0 con 0x7f26a406cf40 2023-11-28T21:52:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a9f16640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a414ea80 con 0x7f26a4150c20 2023-11-28T21:52:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a9f16640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40c4e80 con 0x7f26a406e300 2023-11-28T21:52:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406cf40 0x7f26a414ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406cf40 0x7f26a414ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51006/0 (socket says 172.21.15.12:51006) 2023-11-28T21:52:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a37fe640 1 -- 172.21.15.12:0/4174934464 learned_addr learned my addr 172.21.15.12:0/4174934464 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a2ffd640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406e300 0x7f26a41465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a3fff640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a4150c20 0x7f26a41e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:57.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a37fe640 1 -- 172.21.15.12:0/4174934464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406e300 msgr2=0x7f26a41465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a37fe640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406e300 0x7f26a41465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a37fe640 1 -- 172.21.15.12:0/4174934464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a4150c20 msgr2=0x7f26a41e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a37fe640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a4150c20 0x7f26a41e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a37fe640 1 -- 172.21.15.12:0/4174934464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c009a70 con 0x7f26a406cf40 2023-11-28T21:52:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a37fe640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406cf40 0x7f26a414ef10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f269400c750 tx=0x7f269400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f268bfff640 1 -- 172.21.15.12:0/4174934464 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694005ae0 con 0x7f26a406cf40 2023-11-28T21:52:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f268bfff640 1 -- 172.21.15.12:0/4174934464 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2694005c40 con 0x7f26a406cf40 2023-11-28T21:52:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f268bfff640 1 -- 172.21.15.12:0/4174934464 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694007070 con 0x7f26a406cf40 2023-11-28T21:52:57.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a9f16640 1 -- 172.21.15.12:0/4174934464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a414fde0 con 0x7f26a406cf40 2023-11-28T21:52:57.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.318+0000 7f26a9f16640 1 -- 172.21.15.12:0/4174934464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a4150080 con 0x7f26a406cf40 2023-11-28T21:52:57.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.322+0000 7f26a9f16640 1 -- 172.21.15.12:0/4174934464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a4074f60 con 0x7f26a406cf40 2023-11-28T21:52:57.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.322+0000 7f268bfff640 1 -- 172.21.15.12:0/4174934464 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f26940071f0 con 0x7f26a406cf40 2023-11-28T21:52:57.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.322+0000 7f268bfff640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2690066470 0x7f2690068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:57.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.322+0000 7f268bfff640 1 -- 172.21.15.12:0/4174934464 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f2694084200 con 0x7f26a406cf40 2023-11-28T21:52:57.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.322+0000 7f26a2ffd640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2690066470 0x7f2690068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:57.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.322+0000 7f26a2ffd640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2690066470 0x7f2690068930 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f26a41472a0 tx=0x7f2698009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:57.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.326+0000 7f268bfff640 1 -- 172.21.15.12:0/4174934464 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2694051060 con 0x7f26a406cf40 2023-11-28T21:52:57.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 -- 172.21.15.12:0/4174934464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26a4160450 con 0x7f26a406cf40 2023-11-28T21:52:57.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f268bfff640 1 -- 172.21.15.12:0/4174934464 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5724 (secure 0 0 0) 0x7f2694054680 con 0x7f26a406cf40 2023-11-28T21:52:57.523 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:57.523 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:52:53.611931+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:57.523 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 73 2023-11-28T21:52:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 -- 172.21.15.12:0/4174934464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2690066470 msgr2=0x7f2690068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2690066470 0x7f2690068930 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f26a41472a0 tx=0x7f2698009190 comp rx=0 tx=0).stop 2023-11-28T21:52:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 -- 172.21.15.12:0/4174934464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406cf40 msgr2=0x7f26a414ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406cf40 0x7f26a414ef10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f269400c750 tx=0x7f269400cc10 comp rx=0 tx=0).stop 2023-11-28T21:52:57.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 -- 172.21.15.12:0/4174934464 shutdown_connections 2023-11-28T21:52:57.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2690066470 0x7f2690068930 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:57.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a4150c20 0x7f26a41e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:57.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406e300 0x7f26a41465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:57.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 --2- 172.21.15.12:0/4174934464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406cf40 0x7f26a414ef10 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:57.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 -- 172.21.15.12:0/4174934464 >> 172.21.15.12:0/4174934464 conn(0x7f26a4066360 msgr2=0x7f26a40c5170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:57.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 -- 172.21.15.12:0/4174934464 shutdown_connections 2023-11-28T21:52:57.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:57.518+0000 7f26a9f16640 1 -- 172.21.15.12:0/4174934464 wait complete. 2023-11-28T21:52:57.545 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:52:59.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:52:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2fc5a640 1 Processor -- start 2023-11-28T21:52:59.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2fc5a640 1 -- start start 2023-11-28T21:52:59.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2fc5a640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4e2815a910 0x7f4e28070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2fc5a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2806f050 con 0x7f4e28143070 2023-11-28T21:52:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2fc5a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2806f580 con 0x7f4e280710e0 2023-11-28T21:52:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2fc5a640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2815e330 con 0x7f4e2815a910 2023-11-28T21:52:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2d1f5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4e2815a910 0x7f4e28070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2d1f5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4e2815a910 0x7f4e28070ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51010/0 (socket says 172.21.15.12:51010) 2023-11-28T21:52:59.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2d1f5640 1 -- 172.21.15.12:0/2940492158 learned_addr learned my addr 172.21.15.12:0/2940492158 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:59.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2d1f5640 1 -- 172.21.15.12:0/2940492158 >> v1:172.21.15.144:6789/0 conn(0x7f4e280710e0 legacy=0x7f4e280714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:59.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2d1f5640 1 -- 172.21.15.12:0/2940492158 >> v1:172.21.15.12:6789/0 conn(0x7f4e28143070 legacy=0x7f4e2815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:59.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2d1f5640 1 -- 172.21.15.12:0/2940492158 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2806f2a0 con 0x7f4e2815a910 2023-11-28T21:52:59.717 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.710+0000 7f4e2c9f4640 1 -- 172.21.15.12:0/2940492158 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162093135 0 0) 0x7f4e2806f050 con 0x7f4e28143070 2023-11-28T21:52:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2d1f5640 1 --2- 172.21.15.12:0/2940492158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e2815a910 0x7f4e28070ba0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009870 tx=0x7f4e1801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa1a14b0842afb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2c9f4640 1 -- 172.21.15.12:0/2940492158 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e180aa670 con 0x7f4e2815a910 2023-11-28T21:52:59.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2c9f4640 1 -- 172.21.15.12:0/2940492158 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e180aa7d0 con 0x7f4e2815a910 2023-11-28T21:52:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2c9f4640 1 -- 172.21.15.12:0/2940492158 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e180aab60 con 0x7f4e2815a910 2023-11-28T21:52:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/2940492158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e2815a910 msgr2=0x7f4e28070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2fc5a640 1 --2- 172.21.15.12:0/2940492158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e2815a910 0x7f4e28070ba0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009870 tx=0x7f4e1801c510 comp rx=0 tx=0).stop 2023-11-28T21:52:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/2940492158 shutdown_connections 2023-11-28T21:52:59.720 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2fc5a640 1 --2- 172.21.15.12:0/2940492158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e2815a910 0x7f4e28070ba0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:59.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/2940492158 >> 172.21.15.12:0/2940492158 conn(0x7f4e2806b6c0 msgr2=0x7f4e2806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:59.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/2940492158 shutdown_connections 2023-11-28T21:52:59.721 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.714+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/2940492158 wait complete. 2023-11-28T21:52:59.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2fc5a640 1 Processor -- start 2023-11-28T21:52:59.722 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2fc5a640 1 -- start start 2023-11-28T21:52:59.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2fc5a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e280710e0 0x7f4e281562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:59.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2fc5a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e28143070 0x7f4e281f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:59.724 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2fc5a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e2815a910 0x7f4e28154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2fc5a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280c7b90 con 0x7f4e28143070 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2fc5a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280cb350 con 0x7f4e280710e0 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2fc5a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e28155e40 con 0x7f4e2815a910 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2d1f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e28143070 0x7f4e281f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2d1f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e28143070 0x7f4e281f3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51022/0 (socket says 172.21.15.12:51022) 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2d1f5640 1 -- 172.21.15.12:0/832514802 learned_addr learned my addr 172.21.15.12:0/832514802 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2d9f6640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e280710e0 0x7f4e281562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2e1f7640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e2815a910 0x7f4e28154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2d1f5640 1 -- 172.21.15.12:0/832514802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e2815a910 msgr2=0x7f4e28154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2d1f5640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e2815a910 0x7f4e28154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2d1f5640 1 -- 172.21.15.12:0/832514802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e280710e0 msgr2=0x7f4e281562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:59.725 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2d1f5640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e280710e0 0x7f4e281562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:59.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2d1f5640 1 -- 172.21.15.12:0/832514802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e180094d0 con 0x7f4e28143070 2023-11-28T21:52:59.726 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2d1f5640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e28143070 0x7f4e281f3ab0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4e180ac040 tx=0x7f4e1801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e1e7fc640 1 -- 172.21.15.12:0/832514802 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e180aae90 con 0x7f4e28143070 2023-11-28T21:52:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e1e7fc640 1 -- 172.21.15.12:0/832514802 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e180b7070 con 0x7f4e28143070 2023-11-28T21:52:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e1e7fc640 1 -- 172.21.15.12:0/832514802 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18004270 con 0x7f4e28143070 2023-11-28T21:52:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/832514802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e281f2970 con 0x7f4e28143070 2023-11-28T21:52:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.718+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/832514802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e28155510 con 0x7f4e28143070 2023-11-28T21:52:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.722+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/832514802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e28155fb0 con 0x7f4e28143070 2023-11-28T21:52:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.722+0000 7f4e1e7fc640 1 -- 172.21.15.12:0/832514802 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4e180043d0 con 0x7f4e28143070 2023-11-28T21:52:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.722+0000 7f4e1e7fc640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e0c066470 0x7f4e0c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:52:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.722+0000 7f4e1e7fc640 1 -- 172.21.15.12:0/832514802 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f4e1811ebb0 con 0x7f4e28143070 2023-11-28T21:52:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.722+0000 7f4e2d9f6640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e0c066470 0x7f4e0c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:52:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.722+0000 7f4e2d9f6640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e0c066470 0x7f4e0c068930 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4e24000f70 tx=0x7f4e24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:52:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.726+0000 7f4e1e7fc640 1 -- 172.21.15.12:0/832514802 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e180eb980 con 0x7f4e28143070 2023-11-28T21:52:59.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/832514802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e28074f60 con 0x7f4e28143070 2023-11-28T21:52:59.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e1e7fc640 1 -- 172.21.15.12:0/832514802 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5724 (secure 0 0 0) 0x7f4e180eefa0 con 0x7f4e28143070 2023-11-28T21:52:59.927 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:52:59.927 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:52:53.611931+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:52:59.927 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 73 2023-11-28T21:52:59.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/832514802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e0c066470 msgr2=0x7f4e0c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:59.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e0c066470 0x7f4e0c068930 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4e24000f70 tx=0x7f4e24009040 comp rx=0 tx=0).stop 2023-11-28T21:52:59.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/832514802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e28143070 msgr2=0x7f4e281f3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:52:59.929 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e28143070 0x7f4e281f3ab0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4e180ac040 tx=0x7f4e1801ad50 comp rx=0 tx=0).stop 2023-11-28T21:52:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/832514802 shutdown_connections 2023-11-28T21:52:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4e0c066470 0x7f4e0c068930 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e2815a910 0x7f4e28154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e28143070 0x7f4e281f3ab0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 --2- 172.21.15.12:0/832514802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e280710e0 0x7f4e281562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:52:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/832514802 >> 172.21.15.12:0/832514802 conn(0x7f4e2806b6c0 msgr2=0x7f4e2806baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:52:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/832514802 shutdown_connections 2023-11-28T21:52:59.930 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:52:59.922+0000 7f4e2fc5a640 1 -- 172.21.15.12:0/832514802 wait complete. 2023-11-28T21:52:59.949 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:53:01.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:02.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7689cf3640 1 Processor -- start 2023-11-28T21:53:02.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7689cf3640 1 -- start start 2023-11-28T21:53:02.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7689cf3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840ca1d0 0x7f768415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:02.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7689cf3640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f768415b1a0 0x7f768415b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:02.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7689cf3640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768415bb70 con 0x7f768406a810 2023-11-28T21:53:02.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7689cf3640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768415acf0 con 0x7f768415b1a0 2023-11-28T21:53:02.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7689cf3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76840c5330 con 0x7f76840ca1d0 2023-11-28T21:53:02.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7683fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840ca1d0 0x7f768415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:02.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7683fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840ca1d0 0x7f768415a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:54514/0 (socket says 172.21.15.12:54514) 2023-11-28T21:53:02.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7683fff640 1 -- 172.21.15.12:0/4283652111 learned_addr learned my addr 172.21.15.12:0/4283652111 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:02.100 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f76894f2640 1 --2- 172.21.15.12:0/4283652111 >> v2:172.21.15.144:3300/0 conn(0x7f768415b1a0 0x7f768415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:02.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7683fff640 1 -- 172.21.15.12:0/4283652111 >> v2:172.21.15.144:3300/0 conn(0x7f768415b1a0 msgr2=0x7f768415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:02.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7683fff640 1 --2- 172.21.15.12:0/4283652111 >> v2:172.21.15.144:3300/0 conn(0x7f768415b1a0 0x7f768415b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:02.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7683fff640 1 -- 172.21.15.12:0/4283652111 >> v1:172.21.15.144:6789/0 conn(0x7f768406a810 legacy=0x7f768406abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:02.101 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.094+0000 7f7683fff640 1 -- 172.21.15.12:0/4283652111 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76840c4480 con 0x7f76840ca1d0 2023-11-28T21:53:02.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f7683fff640 1 --2- 172.21.15.12:0/4283652111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840ca1d0 0x7f768415a7b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7670009870 tx=0x7f767001ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53def99a582f92ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:02.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f76837fe640 1 -- 172.21.15.12:0/4283652111 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76700a8a30 con 0x7f76840ca1d0 2023-11-28T21:53:02.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f76837fe640 1 -- 172.21.15.12:0/4283652111 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76700a8b90 con 0x7f76840ca1d0 2023-11-28T21:53:02.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f76837fe640 1 -- 172.21.15.12:0/4283652111 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76700b0440 con 0x7f76840ca1d0 2023-11-28T21:53:02.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f7689cf3640 1 -- 172.21.15.12:0/4283652111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840ca1d0 msgr2=0x7f768415a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:02.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f7689cf3640 1 --2- 172.21.15.12:0/4283652111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840ca1d0 0x7f768415a7b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7670009870 tx=0x7f767001ed30 comp rx=0 tx=0).stop 2023-11-28T21:53:02.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f7689cf3640 1 -- 172.21.15.12:0/4283652111 shutdown_connections 2023-11-28T21:53:02.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f7689cf3640 1 --2- 172.21.15.12:0/4283652111 >> v2:172.21.15.144:3300/0 conn(0x7f768415b1a0 0x7f768415b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:02.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f7689cf3640 1 --2- 172.21.15.12:0/4283652111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76840ca1d0 0x7f768415a7b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:02.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f7689cf3640 1 -- 172.21.15.12:0/4283652111 >> 172.21.15.12:0/4283652111 conn(0x7f768406bd60 msgr2=0x7f768406c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:02.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f7689cf3640 1 -- 172.21.15.12:0/4283652111 shutdown_connections 2023-11-28T21:53:02.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.098+0000 7f7689cf3640 1 -- 172.21.15.12:0/4283652111 wait complete. 2023-11-28T21:53:02.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7689cf3640 1 Processor -- start 2023-11-28T21:53:02.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7689cf3640 1 -- start start 2023-11-28T21:53:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7689cf3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768406a810 0x7f76841470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7689cf3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76840ca1d0 0x7f7684158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7689cf3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768415b1a0 0x7f76841506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7689cf3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76841ef890 con 0x7f768406a810 2023-11-28T21:53:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7689cf3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76841f2300 con 0x7f76840ca1d0 2023-11-28T21:53:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7689cf3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76841f25e0 con 0x7f768415b1a0 2023-11-28T21:53:02.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7688cf1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768406a810 0x7f76841470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7688cf1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768406a810 0x7f76841470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51026/0 (socket says 172.21.15.12:51026) 2023-11-28T21:53:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.102+0000 7f7688cf1640 1 -- 172.21.15.12:0/937001606 learned_addr learned my addr 172.21.15.12:0/937001606 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f7683fff640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76840ca1d0 0x7f7684158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f76894f2640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768415b1a0 0x7f76841506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f7688cf1640 1 -- 172.21.15.12:0/937001606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768415b1a0 msgr2=0x7f76841506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f7688cf1640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768415b1a0 0x7f76841506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f7688cf1640 1 -- 172.21.15.12:0/937001606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76840ca1d0 msgr2=0x7f7684158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:02.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f7688cf1640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76840ca1d0 0x7f7684158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f7688cf1640 1 -- 172.21.15.12:0/937001606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76700094d0 con 0x7f768406a810 2023-11-28T21:53:02.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f7688cf1640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768406a810 0x7f76841470d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f767800c500 tx=0x7f767800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:02.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f76817fa640 1 -- 172.21.15.12:0/937001606 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678005220 con 0x7f768406a810 2023-11-28T21:53:02.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f76817fa640 1 -- 172.21.15.12:0/937001606 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7678005380 con 0x7f768406a810 2023-11-28T21:53:02.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f76817fa640 1 -- 172.21.15.12:0/937001606 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678007270 con 0x7f768406a810 2023-11-28T21:53:02.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f7689cf3640 1 -- 172.21.15.12:0/937001606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76841f28c0 con 0x7f768406a810 2023-11-28T21:53:02.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f7689cf3640 1 -- 172.21.15.12:0/937001606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76841f30c0 con 0x7f768406a810 2023-11-28T21:53:02.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.106+0000 7f7689cf3640 1 -- 172.21.15.12:0/937001606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76841f2de0 con 0x7f768406a810 2023-11-28T21:53:02.113 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.110+0000 7f76817fa640 1 -- 172.21.15.12:0/937001606 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f76780073d0 con 0x7f768406a810 2023-11-28T21:53:02.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.110+0000 7f76817fa640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76680664a0 0x7f7668068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:02.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.110+0000 7f76817fa640 1 -- 172.21.15.12:0/937001606 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f7678083c80 con 0x7f768406a810 2023-11-28T21:53:02.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.110+0000 7f7683fff640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76680664a0 0x7f7668068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:02.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.110+0000 7f7683fff640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76680664a0 0x7f7668068960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f76700b2000 tx=0x7f7670009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:02.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.118+0000 7f76817fa640 1 -- 172.21.15.12:0/937001606 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f76841f2de0 con 0x7f768406a810 2023-11-28T21:53:02.347 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.342+0000 7f7689cf3640 1 -- 172.21.15.12:0/937001606 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76841f2de0 con 0x7f768406a810 2023-11-28T21:53:02.348 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.342+0000 7f76817fa640 1 -- 172.21.15.12:0/937001606 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5724 (secure 0 0 0) 0x7f76841f2de0 con 0x7f768406a810 2023-11-28T21:53:02.349 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:02.349 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:52:53.611931+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:02.350 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 73 2023-11-28T21:53:02.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.342+0000 7f7689cf3640 1 -- 172.21.15.12:0/937001606 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76680664a0 msgr2=0x7f7668068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:02.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.342+0000 7f7689cf3640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76680664a0 0x7f7668068960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f76700b2000 tx=0x7f7670009040 comp rx=0 tx=0).stop 2023-11-28T21:53:02.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.346+0000 7f7689cf3640 1 -- 172.21.15.12:0/937001606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768406a810 msgr2=0x7f76841470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:02.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.346+0000 7f7689cf3640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768406a810 0x7f76841470d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f767800c500 tx=0x7f767800c9c0 comp rx=0 tx=0).stop 2023-11-28T21:53:02.351 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.346+0000 7f7689cf3640 1 -- 172.21.15.12:0/937001606 shutdown_connections 2023-11-28T21:53:02.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.346+0000 7f7689cf3640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f76680664a0 0x7f7668068960 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:02.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.346+0000 7f7689cf3640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f768415b1a0 0x7f76841506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:02.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.346+0000 7f7689cf3640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76840ca1d0 0x7f7684158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:02.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.346+0000 7f7689cf3640 1 --2- 172.21.15.12:0/937001606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f768406a810 0x7f76841470d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:02.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.346+0000 7f7689cf3640 1 -- 172.21.15.12:0/937001606 >> 172.21.15.12:0/937001606 conn(0x7f768406bd60 msgr2=0x7f76840b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:02.352 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.346+0000 7f7689cf3640 1 -- 172.21.15.12:0/937001606 shutdown_connections 2023-11-28T21:53:02.353 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:02.346+0000 7f7689cf3640 1 -- 172.21.15.12:0/937001606 wait complete. 2023-11-28T21:53:02.371 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:53:04.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.510+0000 7fd187a17640 1 Processor -- start 2023-11-28T21:53:04.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd187a17640 1 -- start start 2023-11-28T21:53:04.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd187a17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1800712b0 0x7fd1801657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:04.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd187a17640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18006f050 con 0x7fd180143070 2023-11-28T21:53:04.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd187a17640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18006f580 con 0x7fd18015a910 2023-11-28T21:53:04.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd187a17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18015e330 con 0x7fd1800712b0 2023-11-28T21:53:04.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd1857b3640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd180143070 0x7fd18015b340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:55330/0 (socket says 172.21.15.12:55330) 2023-11-28T21:53:04.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd1857b3640 1 -- 172.21.15.12:0/2989799851 learned_addr learned my addr 172.21.15.12:0/2989799851 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:53:04.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd185fb4640 1 --2- 172.21.15.12:0/2989799851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1800712b0 0x7fd1801657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:04.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd177fff640 1 -- 172.21.15.12:0/2989799851 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624162132 0 0) 0x7fd18006f050 con 0x7fd180143070 2023-11-28T21:53:04.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd177fff640 1 -- 172.21.15.12:0/2989799851 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd16c003530 con 0x7fd180143070 2023-11-28T21:53:04.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd177fff640 1 -- 172.21.15.12:0/2989799851 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423031725 0 0) 0x7fd18006f580 con 0x7fd18015a910 2023-11-28T21:53:04.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd177fff640 1 -- 172.21.15.12:0/2989799851 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd18006f050 con 0x7fd18015a910 2023-11-28T21:53:04.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd185fb4640 1 -- 172.21.15.12:0/2989799851 >> v1:172.21.15.144:6789/0 conn(0x7fd18015a910 legacy=0x7fd180070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:04.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd185fb4640 1 -- 172.21.15.12:0/2989799851 >> v1:172.21.15.12:6789/0 conn(0x7fd180143070 legacy=0x7fd18015b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:04.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd177fff640 1 -- 172.21.15.12:0/2989799851 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1482039361 0 0) 0x7fd16c003530 con 0x7fd180143070 2023-11-28T21:53:04.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.514+0000 7fd185fb4640 1 -- 172.21.15.12:0/2989799851 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18006f2a0 con 0x7fd1800712b0 2023-11-28T21:53:04.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd185fb4640 1 --2- 172.21.15.12:0/2989799851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1800712b0 0x7fd1801657e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd178009dc0 tx=0x7fd17801c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=334bbcfc959a8e67 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:04.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd177fff640 1 -- 172.21.15.12:0/2989799851 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c001680 con 0x7fd1800712b0 2023-11-28T21:53:04.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd177fff640 1 -- 172.21.15.12:0/2989799851 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1780a56a0 con 0x7fd1800712b0 2023-11-28T21:53:04.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd177fff640 1 -- 172.21.15.12:0/2989799851 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1780a5a10 con 0x7fd1800712b0 2023-11-28T21:53:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd187a17640 1 -- 172.21.15.12:0/2989799851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1800712b0 msgr2=0x7fd1801657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd187a17640 1 --2- 172.21.15.12:0/2989799851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1800712b0 0x7fd1801657e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd178009dc0 tx=0x7fd17801c860 comp rx=0 tx=0).stop 2023-11-28T21:53:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd187a17640 1 -- 172.21.15.12:0/2989799851 shutdown_connections 2023-11-28T21:53:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd187a17640 1 --2- 172.21.15.12:0/2989799851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1800712b0 0x7fd1801657e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd187a17640 1 -- 172.21.15.12:0/2989799851 >> 172.21.15.12:0/2989799851 conn(0x7fd18006b6c0 msgr2=0x7fd18006bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd187a17640 1 -- 172.21.15.12:0/2989799851 shutdown_connections 2023-11-28T21:53:04.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.518+0000 7fd187a17640 1 -- 172.21.15.12:0/2989799851 wait complete. 2023-11-28T21:53:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd187a17640 1 Processor -- start 2023-11-28T21:53:04.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd187a17640 1 -- start start 2023-11-28T21:53:04.527 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd187a17640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1800712b0 0x7fd1801562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd187a17640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd180143070 0x7fd1800ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd187a17640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18015a910 0x7fd180156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd187a17640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1800c3840 con 0x7fd1800712b0 2023-11-28T21:53:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd187a17640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801560f0 con 0x7fd18015a910 2023-11-28T21:53:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd187a17640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd180156830 con 0x7fd180143070 2023-11-28T21:53:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd1857b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1800712b0 0x7fd1801562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd1857b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1800712b0 0x7fd1801562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51030/0 (socket says 172.21.15.12:51030) 2023-11-28T21:53:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd1857b3640 1 -- 172.21.15.12:0/814835846 learned_addr learned my addr 172.21.15.12:0/814835846 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd185fb4640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18015a910 0x7fd180156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:04.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd184fb2640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd180143070 0x7fd1800ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd1857b3640 1 -- 172.21.15.12:0/814835846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd180143070 msgr2=0x7fd1800ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd1857b3640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd180143070 0x7fd1800ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd1857b3640 1 -- 172.21.15.12:0/814835846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18015a910 msgr2=0x7fd180156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd1857b3640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18015a910 0x7fd180156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd1857b3640 1 -- 172.21.15.12:0/814835846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd178009a70 con 0x7fd1800712b0 2023-11-28T21:53:04.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd1857b3640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1800712b0 0x7fd1801562f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c750 tx=0x7fd17c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd175ffb640 1 -- 172.21.15.12:0/814835846 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c005af0 con 0x7fd1800712b0 2023-11-28T21:53:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd175ffb640 1 -- 172.21.15.12:0/814835846 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c005c50 con 0x7fd1800712b0 2023-11-28T21:53:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd175ffb640 1 -- 172.21.15.12:0/814835846 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c007070 con 0x7fd1800712b0 2023-11-28T21:53:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd187a17640 1 -- 172.21.15.12:0/814835846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1800c3d50 con 0x7fd1800712b0 2023-11-28T21:53:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.522+0000 7fd187a17640 1 -- 172.21.15.12:0/814835846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1801557c0 con 0x7fd1800712b0 2023-11-28T21:53:04.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.526+0000 7fd187a17640 1 -- 172.21.15.12:0/814835846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd180074f60 con 0x7fd1800712b0 2023-11-28T21:53:04.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.526+0000 7fd175ffb640 1 -- 172.21.15.12:0/814835846 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd17c0071d0 con 0x7fd1800712b0 2023-11-28T21:53:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.526+0000 7fd175ffb640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd164066470 0x7fd164068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.526+0000 7fd175ffb640 1 -- 172.21.15.12:0/814835846 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fd17c083d10 con 0x7fd1800712b0 2023-11-28T21:53:04.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.526+0000 7fd184fb2640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd164066470 0x7fd164068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:04.534 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.530+0000 7fd184fb2640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd164066470 0x7fd164068930 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd1801427e0 tx=0x7fd170009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:04.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.530+0000 7fd175ffb640 1 -- 172.21.15.12:0/814835846 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd17c054580 con 0x7fd1800712b0 2023-11-28T21:53:04.730 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 -- 172.21.15.12:0/814835846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1800c7d60 con 0x7fd1800712b0 2023-11-28T21:53:04.731 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd175ffb640 1 -- 172.21.15.12:0/814835846 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5724 (secure 0 0 0) 0x7fd17c017020 con 0x7fd1800712b0 2023-11-28T21:53:04.731 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:04.731 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:52:53.611931+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:04.731 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 73 2023-11-28T21:53:04.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 -- 172.21.15.12:0/814835846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd164066470 msgr2=0x7fd164068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:04.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd164066470 0x7fd164068930 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd1801427e0 tx=0x7fd170009040 comp rx=0 tx=0).stop 2023-11-28T21:53:04.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 -- 172.21.15.12:0/814835846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1800712b0 msgr2=0x7fd1801562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:04.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1800712b0 0x7fd1801562f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c750 tx=0x7fd17c00cc10 comp rx=0 tx=0).stop 2023-11-28T21:53:04.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 -- 172.21.15.12:0/814835846 shutdown_connections 2023-11-28T21:53:04.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd164066470 0x7fd164068930 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:04.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18015a910 0x7fd180156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:04.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd180143070 0x7fd1800ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:04.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 --2- 172.21.15.12:0/814835846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1800712b0 0x7fd1801562f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:04.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 -- 172.21.15.12:0/814835846 >> 172.21.15.12:0/814835846 conn(0x7fd18006b6c0 msgr2=0x7fd1800c6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:04.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 -- 172.21.15.12:0/814835846 shutdown_connections 2023-11-28T21:53:04.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:04.726+0000 7fd187a17640 1 -- 172.21.15.12:0/814835846 wait complete. 2023-11-28T21:53:04.752 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:53:06.754 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:06.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe167db8640 1 Processor -- start 2023-11-28T21:53:06.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe167db8640 1 -- start start 2023-11-28T21:53:06.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe167db8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe160151af0 0x7fe1600c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:06.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe167db8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe160151ed0 0x7fe1600c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:06.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe167db8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe16006e300 0x7fe1600c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:06.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe167db8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1600707f0 con 0x7fe160151ed0 2023-11-28T21:53:06.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe167db8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1600c7b90 con 0x7fe160151af0 2023-11-28T21:53:06.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe167db8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe160071c20 con 0x7fe16006e300 2023-11-28T21:53:06.910 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe165353640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe160151ed0 0x7fe1600c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:06.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe165353640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe160151ed0 0x7fe1600c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47832/0 (socket says 172.21.15.12:47832) 2023-11-28T21:53:06.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe165353640 1 -- 172.21.15.12:0/86021927 learned_addr learned my addr 172.21.15.12:0/86021927 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:06.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe166355640 1 --2- 172.21.15.12:0/86021927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe16006e300 0x7fe1600c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:06.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe165b54640 1 --2- 172.21.15.12:0/86021927 >> v2:172.21.15.144:3300/0 conn(0x7fe160151af0 0x7fe1600c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:06.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe165353640 1 -- 172.21.15.12:0/86021927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe16006e300 msgr2=0x7fe1600c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:06.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe165353640 1 --2- 172.21.15.12:0/86021927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe16006e300 0x7fe1600c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:06.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe165353640 1 -- 172.21.15.12:0/86021927 >> v2:172.21.15.144:3300/0 conn(0x7fe160151af0 msgr2=0x7fe1600c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:06.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe165353640 1 --2- 172.21.15.12:0/86021927 >> v2:172.21.15.144:3300/0 conn(0x7fe160151af0 0x7fe1600c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:06.911 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.902+0000 7fe165353640 1 -- 172.21.15.12:0/86021927 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe160157b70 con 0x7fe160151ed0 2023-11-28T21:53:06.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe165353640 1 --2- 172.21.15.12:0/86021927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160151ed0 0x7fe1600c49d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe1600c7d20 tx=0x7fe15001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf33d1e8099aa0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:06.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe164b52640 1 -- 172.21.15.12:0/86021927 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1500ac410 con 0x7fe160151ed0 2023-11-28T21:53:06.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe164b52640 1 -- 172.21.15.12:0/86021927 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1500ac570 con 0x7fe160151ed0 2023-11-28T21:53:06.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe164b52640 1 -- 172.21.15.12:0/86021927 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1500ac900 con 0x7fe160151ed0 2023-11-28T21:53:06.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe167db8640 1 -- 172.21.15.12:0/86021927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160151ed0 msgr2=0x7fe1600c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:06.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe167db8640 1 --2- 172.21.15.12:0/86021927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160151ed0 0x7fe1600c49d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe1600c7d20 tx=0x7fe15001c500 comp rx=0 tx=0).stop 2023-11-28T21:53:06.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe167db8640 1 -- 172.21.15.12:0/86021927 shutdown_connections 2023-11-28T21:53:06.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe167db8640 1 --2- 172.21.15.12:0/86021927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe16006e300 0x7fe1600c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:06.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe167db8640 1 --2- 172.21.15.12:0/86021927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160151ed0 0x7fe1600c49d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:06.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe167db8640 1 --2- 172.21.15.12:0/86021927 >> v2:172.21.15.144:3300/0 conn(0x7fe160151af0 0x7fe1600c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:06.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe167db8640 1 -- 172.21.15.12:0/86021927 >> 172.21.15.12:0/86021927 conn(0x7fe160066360 msgr2=0x7fe160066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:06.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe167db8640 1 -- 172.21.15.12:0/86021927 shutdown_connections 2023-11-28T21:53:06.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.906+0000 7fe167db8640 1 -- 172.21.15.12:0/86021927 wait complete. 2023-11-28T21:53:06.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe167db8640 1 Processor -- start 2023-11-28T21:53:06.915 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe167db8640 1 -- start start 2023-11-28T21:53:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe167db8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe16006e300 0x7fe160158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe167db8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe160151af0 0x7fe1601465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe167db8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160147f80 0x7fe16015a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe167db8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe160146160 con 0x7fe160147f80 2023-11-28T21:53:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe167db8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1601f10c0 con 0x7fe160151af0 2023-11-28T21:53:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe167db8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1601f1380 con 0x7fe16006e300 2023-11-28T21:53:06.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe166355640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160147f80 0x7fe16015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:06.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe166355640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160147f80 0x7fe16015a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47846/0 (socket says 172.21.15.12:47846) 2023-11-28T21:53:06.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe166355640 1 -- 172.21.15.12:0/715199455 learned_addr learned my addr 172.21.15.12:0/715199455 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:06.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe165b54640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe16006e300 0x7fe160158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:06.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe165353640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe160151af0 0x7fe1601465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:06.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe166355640 1 -- 172.21.15.12:0/715199455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe16006e300 msgr2=0x7fe160158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:06.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe166355640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe16006e300 0x7fe160158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:06.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe166355640 1 -- 172.21.15.12:0/715199455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe160151af0 msgr2=0x7fe1601465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:06.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe166355640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe160151af0 0x7fe1601465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:06.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe166355640 1 -- 172.21.15.12:0/715199455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500094d0 con 0x7fe160147f80 2023-11-28T21:53:06.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe166355640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160147f80 0x7fe16015a680 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe158007e30 tx=0x7fe15800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:06.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe1567fc640 1 -- 172.21.15.12:0/715199455 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158014420 con 0x7fe160147f80 2023-11-28T21:53:06.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe1567fc640 1 -- 172.21.15.12:0/715199455 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158014580 con 0x7fe160147f80 2023-11-28T21:53:06.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe1567fc640 1 -- 172.21.15.12:0/715199455 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158014930 con 0x7fe160147f80 2023-11-28T21:53:06.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe167db8640 1 -- 172.21.15.12:0/715199455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1601f1660 con 0x7fe160147f80 2023-11-28T21:53:06.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.910+0000 7fe167db8640 1 -- 172.21.15.12:0/715199455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe160145df0 con 0x7fe160147f80 2023-11-28T21:53:06.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.914+0000 7fe167db8640 1 -- 172.21.15.12:0/715199455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe160146310 con 0x7fe160147f80 2023-11-28T21:53:06.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.914+0000 7fe1567fc640 1 -- 172.21.15.12:0/715199455 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe158014a90 con 0x7fe160147f80 2023-11-28T21:53:06.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.914+0000 7fe1567fc640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1440664a0 0x7fe144068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:06.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.914+0000 7fe1567fc640 1 -- 172.21.15.12:0/715199455 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fe158084d70 con 0x7fe160147f80 2023-11-28T21:53:06.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.914+0000 7fe165b54640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1440664a0 0x7fe144068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:06.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.914+0000 7fe165b54640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1440664a0 0x7fe144068960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe15c007a60 tx=0x7fe15c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:06.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:06.918+0000 7fe1567fc640 1 -- 172.21.15.12:0/715199455 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe160146310 con 0x7fe160147f80 2023-11-28T21:53:07.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 -- 172.21.15.12:0/715199455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe160146310 con 0x7fe160147f80 2023-11-28T21:53:07.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe1567fc640 1 -- 172.21.15.12:0/715199455 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5726 (secure 0 0 0) 0x7fe160146310 con 0x7fe160147f80 2023-11-28T21:53:07.114 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:07.115 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:53:05.093208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:resolve","state_seq":27,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:07.115 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 74 2023-11-28T21:53:07.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 -- 172.21.15.12:0/715199455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1440664a0 msgr2=0x7fe144068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:07.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1440664a0 0x7fe144068960 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe15c007a60 tx=0x7fe15c008040 comp rx=0 tx=0).stop 2023-11-28T21:53:07.116 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 -- 172.21.15.12:0/715199455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160147f80 msgr2=0x7fe16015a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160147f80 0x7fe16015a680 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe158007e30 tx=0x7fe15800e4b0 comp rx=0 tx=0).stop 2023-11-28T21:53:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 -- 172.21.15.12:0/715199455 shutdown_connections 2023-11-28T21:53:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe1440664a0 0x7fe144068960 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe160147f80 0x7fe16015a680 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe160151af0 0x7fe1601465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 --2- 172.21.15.12:0/715199455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe16006e300 0x7fe160158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 -- 172.21.15.12:0/715199455 >> 172.21.15.12:0/715199455 conn(0x7fe160066360 msgr2=0x7fe1600c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 -- 172.21.15.12:0/715199455 shutdown_connections 2023-11-28T21:53:07.117 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:07.110+0000 7fe167db8640 1 -- 172.21.15.12:0/715199455 wait complete. 2023-11-28T21:53:07.136 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 74, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 74, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6330": { "addr": "172.21.15.12:6839/754378087", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 754378087, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 754378087, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6330, "incarnation": 63, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 16 }, "gid_6513": { "addr": "172.21.15.12:6838/2286739028", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2286739028, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 2286739028, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6513, "incarnation": 73, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:resolve", "state_seq": 27 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:53:05.093208+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6513, "mds_1": 6330 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:53:09.137 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:09.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.290+0000 7f2f7f809640 1 Processor -- start 2023-11-28T21:53:09.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.290+0000 7f2f7f809640 1 -- start start 2023-11-28T21:53:09.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.290+0000 7f2f7f809640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f78151af0 0x7f2f780c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:09.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.290+0000 7f2f7f809640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f78151ed0 0x7f2f780c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:09.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7f809640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f78157b70 con 0x7f2f7806e300 2023-11-28T21:53:09.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7f809640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f780707f0 con 0x7f2f78151af0 2023-11-28T21:53:09.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7f809640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780c7b90 con 0x7f2f78151ed0 2023-11-28T21:53:09.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7d5a5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f78151af0 0x7f2f780c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:09.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7d5a5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f78151af0 0x7f2f780c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47862/0 (socket says 172.21.15.12:47862) 2023-11-28T21:53:09.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7d5a5640 1 -- 172.21.15.12:0/3696601828 learned_addr learned my addr 172.21.15.12:0/3696601828 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:09.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7cda4640 1 --2- 172.21.15.12:0/3696601828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f78151ed0 0x7f2f780c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:09.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7d5a5640 1 -- 172.21.15.12:0/3696601828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f78151ed0 msgr2=0x7f2f780c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:09.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7d5a5640 1 --2- 172.21.15.12:0/3696601828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f78151ed0 0x7f2f780c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:09.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7d5a5640 1 -- 172.21.15.12:0/3696601828 >> v1:172.21.15.144:6789/0 conn(0x7f2f7806e300 legacy=0x7f2f78072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:09.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7d5a5640 1 -- 172.21.15.12:0/3696601828 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f78157e50 con 0x7f2f78151af0 2023-11-28T21:53:09.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f7d5a5640 1 --2- 172.21.15.12:0/3696601828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 0x7f2f780c7650 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009870 tx=0x7f2f7401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620715fc764dfba3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:09.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f6ffff640 1 -- 172.21.15.12:0/3696601828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740ab600 con 0x7f2f78151af0 2023-11-28T21:53:09.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f6ffff640 1 -- 172.21.15.12:0/3696601828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f740a7050 con 0x7f2f78151af0 2023-11-28T21:53:09.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.294+0000 7f2f6ffff640 1 -- 172.21.15.12:0/3696601828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740ab980 con 0x7f2f78151af0 2023-11-28T21:53:09.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 -- 172.21.15.12:0/3696601828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 msgr2=0x7f2f780c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:09.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 --2- 172.21.15.12:0/3696601828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 0x7f2f780c7650 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009870 tx=0x7f2f7401c500 comp rx=0 tx=0).stop 2023-11-28T21:53:09.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 -- 172.21.15.12:0/3696601828 shutdown_connections 2023-11-28T21:53:09.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 --2- 172.21.15.12:0/3696601828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f78151ed0 0x7f2f780c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:09.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 --2- 172.21.15.12:0/3696601828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 0x7f2f780c7650 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:09.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 -- 172.21.15.12:0/3696601828 >> 172.21.15.12:0/3696601828 conn(0x7f2f78066360 msgr2=0x7f2f78066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:09.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 -- 172.21.15.12:0/3696601828 shutdown_connections 2023-11-28T21:53:09.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 -- 172.21.15.12:0/3696601828 wait complete. 2023-11-28T21:53:09.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 Processor -- start 2023-11-28T21:53:09.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 -- start start 2023-11-28T21:53:09.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7806e300 0x7f2f781f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:09.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 0x7f2f7814ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f780c9320 0x7f2f780c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f781f3a00 con 0x7f2f78151af0 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f781f7410 con 0x7f2f780c9320 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7f809640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7814f390 con 0x7f2f7806e300 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7cda4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 0x7f2f7814ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7cda4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 0x7f2f7814ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47872/0 (socket says 172.21.15.12:47872) 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7cda4640 1 -- 172.21.15.12:0/4259429768 learned_addr learned my addr 172.21.15.12:0/4259429768 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7d5a5640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7806e300 0x7f2f781f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.298+0000 7f2f7dda6640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f780c9320 0x7f2f780c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f7cda4640 1 -- 172.21.15.12:0/4259429768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7806e300 msgr2=0x7f2f781f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:09.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f7cda4640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7806e300 0x7f2f781f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:09.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f7cda4640 1 -- 172.21.15.12:0/4259429768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f780c9320 msgr2=0x7f2f780c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:09.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f7cda4640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f780c9320 0x7f2f780c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:09.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f7cda4640 1 -- 172.21.15.12:0/4259429768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f740094d0 con 0x7f2f78151af0 2023-11-28T21:53:09.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f7cda4640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 0x7f2f7814ee50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800d600 tx=0x7f2f6800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:09.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f6dffb640 1 -- 172.21.15.12:0/4259429768 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68004240 con 0x7f2f78151af0 2023-11-28T21:53:09.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f6dffb640 1 -- 172.21.15.12:0/4259429768 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f680043a0 con 0x7f2f78151af0 2023-11-28T21:53:09.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f7f809640 1 -- 172.21.15.12:0/4259429768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f780c4920 con 0x7f2f78151af0 2023-11-28T21:53:09.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f6dffb640 1 -- 172.21.15.12:0/4259429768 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68003840 con 0x7f2f78151af0 2023-11-28T21:53:09.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f7f809640 1 -- 172.21.15.12:0/4259429768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7814f8e0 con 0x7f2f78151af0 2023-11-28T21:53:09.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f7f809640 1 -- 172.21.15.12:0/4259429768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f781f3b40 con 0x7f2f78151af0 2023-11-28T21:53:09.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f6dffb640 1 -- 172.21.15.12:0/4259429768 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2f680039a0 con 0x7f2f78151af0 2023-11-28T21:53:09.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f6dffb640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f5c066470 0x7f2f5c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:09.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f6dffb640 1 -- 172.21.15.12:0/4259429768 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f2f68084730 con 0x7f2f78151af0 2023-11-28T21:53:09.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.302+0000 7f2f7d5a5640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f5c066470 0x7f2f5c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:09.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.306+0000 7f2f7d5a5640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f5c066470 0x7f2f5c068930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2f740ad000 tx=0x7f2f74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:09.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.310+0000 7f2f6dffb640 1 -- 172.21.15.12:0/4259429768 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2f781f3b40 con 0x7f2f78151af0 2023-11-28T21:53:09.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.502+0000 7f2f7f809640 1 -- 172.21.15.12:0/4259429768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f781f3b40 con 0x7f2f78151af0 2023-11-28T21:53:09.509 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.502+0000 7f2f6dffb640 1 -- 172.21.15.12:0/4259429768 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5726 (secure 0 0 0) 0x7f2f781f3b40 con 0x7f2f78151af0 2023-11-28T21:53:09.510 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:09.510 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:53:05.093208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:resolve","state_seq":27,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:09.510 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 74 2023-11-28T21:53:09.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 -- 172.21.15.12:0/4259429768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f5c066470 msgr2=0x7f2f5c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:09.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f5c066470 0x7f2f5c068930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2f740ad000 tx=0x7f2f74008040 comp rx=0 tx=0).stop 2023-11-28T21:53:09.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 -- 172.21.15.12:0/4259429768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 msgr2=0x7f2f7814ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:09.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 0x7f2f7814ee50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800d600 tx=0x7f2f6800dac0 comp rx=0 tx=0).stop 2023-11-28T21:53:09.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 -- 172.21.15.12:0/4259429768 shutdown_connections 2023-11-28T21:53:09.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f5c066470 0x7f2f5c068930 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:09.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f780c9320 0x7f2f780c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:09.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f78151af0 0x7f2f7814ee50 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:09.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 --2- 172.21.15.12:0/4259429768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7806e300 0x7f2f781f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:09.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 -- 172.21.15.12:0/4259429768 >> 172.21.15.12:0/4259429768 conn(0x7f2f78066360 msgr2=0x7f2f7806d410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:09.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 -- 172.21.15.12:0/4259429768 shutdown_connections 2023-11-28T21:53:09.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:09.506+0000 7f2f7f809640 1 -- 172.21.15.12:0/4259429768 wait complete. 2023-11-28T21:53:09.531 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:53:11.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb534bf5640 1 Processor -- start 2023-11-28T21:53:11.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb534bf5640 1 -- start start 2023-11-28T21:53:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb534bf5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb530150c20 0x7fb530156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:11.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb534bf5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb53006e300 0x7fb5300c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb534bf5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb53006cf40 0x7fb5300c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb534bf5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb530067410 con 0x7fb53006e300 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb534bf5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53006e090 con 0x7fb53006cf40 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb534bf5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb530157f30 con 0x7fb530150c20 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb527fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb53006e300 0x7fb5300c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb527fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb53006e300 0x7fb5300c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47888/0 (socket says 172.21.15.12:47888) 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb527fff640 1 -- 172.21.15.12:0/4161371960 learned_addr learned my addr 172.21.15.12:0/4161371960 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb52e59c640 1 --2- 172.21.15.12:0/4161371960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb530150c20 0x7fb530156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb52ed9d640 1 --2- 172.21.15.12:0/4161371960 >> v2:172.21.15.144:3300/0 conn(0x7fb53006cf40 0x7fb5300c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb527fff640 1 -- 172.21.15.12:0/4161371960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb530150c20 msgr2=0x7fb530156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:11.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb527fff640 1 --2- 172.21.15.12:0/4161371960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb530150c20 0x7fb530156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb527fff640 1 -- 172.21.15.12:0/4161371960 >> v2:172.21.15.144:3300/0 conn(0x7fb53006cf40 msgr2=0x7fb5300c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb527fff640 1 --2- 172.21.15.12:0/4161371960 >> v2:172.21.15.144:3300/0 conn(0x7fb53006cf40 0x7fb5300c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb527fff640 1 -- 172.21.15.12:0/4161371960 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb530151000 con 0x7fb53006e300 2023-11-28T21:53:11.691 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.682+0000 7fb52ed9d640 1 --2- 172.21.15.12:0/4161371960 >> v2:172.21.15.144:3300/0 conn(0x7fb53006cf40 0x7fb5300c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:53:11.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb527fff640 1 --2- 172.21.15.12:0/4161371960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb53006e300 0x7fb5300c6dd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb5300c7d70 tx=0x7fb51801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e37a5e7b5fd144e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:11.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb52dd9b640 1 -- 172.21.15.12:0/4161371960 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180ac630 con 0x7fb53006e300 2023-11-28T21:53:11.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb52dd9b640 1 -- 172.21.15.12:0/4161371960 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5180ac790 con 0x7fb53006e300 2023-11-28T21:53:11.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb52dd9b640 1 -- 172.21.15.12:0/4161371960 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180acb20 con 0x7fb53006e300 2023-11-28T21:53:11.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb534bf5640 1 -- 172.21.15.12:0/4161371960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb53006e300 msgr2=0x7fb5300c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:11.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb534bf5640 1 --2- 172.21.15.12:0/4161371960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb53006e300 0x7fb5300c6dd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb5300c7d70 tx=0x7fb51801c500 comp rx=0 tx=0).stop 2023-11-28T21:53:11.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb534bf5640 1 -- 172.21.15.12:0/4161371960 shutdown_connections 2023-11-28T21:53:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb534bf5640 1 --2- 172.21.15.12:0/4161371960 >> v2:172.21.15.144:3300/0 conn(0x7fb53006cf40 0x7fb5300c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb534bf5640 1 --2- 172.21.15.12:0/4161371960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb53006e300 0x7fb5300c6dd0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb534bf5640 1 --2- 172.21.15.12:0/4161371960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb530150c20 0x7fb530156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb534bf5640 1 -- 172.21.15.12:0/4161371960 >> 172.21.15.12:0/4161371960 conn(0x7fb530066360 msgr2=0x7fb530066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb534bf5640 1 -- 172.21.15.12:0/4161371960 shutdown_connections 2023-11-28T21:53:11.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.686+0000 7fb534bf5640 1 -- 172.21.15.12:0/4161371960 wait complete. 2023-11-28T21:53:11.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb534bf5640 1 Processor -- start 2023-11-28T21:53:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb534bf5640 1 -- start start 2023-11-28T21:53:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb534bf5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb53006cf40 0x7fb530145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb534bf5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53006e300 0x7fb530145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb534bf5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb530150c20 0x7fb530071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb534bf5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb530071660 con 0x7fb530150c20 2023-11-28T21:53:11.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb534bf5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5300cb000 con 0x7fb53006cf40 2023-11-28T21:53:11.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb534bf5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb530070d70 con 0x7fb53006e300 2023-11-28T21:53:11.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb52ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb530150c20 0x7fb530071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:11.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb52ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb530150c20 0x7fb530071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47894/0 (socket says 172.21.15.12:47894) 2023-11-28T21:53:11.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb52ed9d640 1 -- 172.21.15.12:0/2786399511 learned_addr learned my addr 172.21.15.12:0/2786399511 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:11.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb527fff640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53006e300 0x7fb530145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:11.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb52e59c640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb53006cf40 0x7fb530145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:11.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb52ed9d640 1 -- 172.21.15.12:0/2786399511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53006e300 msgr2=0x7fb530145590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:11.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb52ed9d640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53006e300 0x7fb530145590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb52ed9d640 1 -- 172.21.15.12:0/2786399511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb53006cf40 msgr2=0x7fb530145050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:11.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb52ed9d640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb53006cf40 0x7fb530145050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb52ed9d640 1 -- 172.21.15.12:0/2786399511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180094d0 con 0x7fb530150c20 2023-11-28T21:53:11.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb52ed9d640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb530150c20 0x7fb530071af0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb528007e10 tx=0x7fb52800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:11.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb526ffd640 1 -- 172.21.15.12:0/2786399511 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528003ba0 con 0x7fb530150c20 2023-11-28T21:53:11.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb526ffd640 1 -- 172.21.15.12:0/2786399511 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52800f040 con 0x7fb530150c20 2023-11-28T21:53:11.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb534bf5640 1 -- 172.21.15.12:0/2786399511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5300c3fd0 con 0x7fb530150c20 2023-11-28T21:53:11.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb526ffd640 1 -- 172.21.15.12:0/2786399511 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800de80 con 0x7fb530150c20 2023-11-28T21:53:11.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.690+0000 7fb534bf5640 1 -- 172.21.15.12:0/2786399511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5300712f0 con 0x7fb530150c20 2023-11-28T21:53:11.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.694+0000 7fb534bf5640 1 -- 172.21.15.12:0/2786399511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53006fbc0 con 0x7fb530150c20 2023-11-28T21:53:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.694+0000 7fb526ffd640 1 -- 172.21.15.12:0/2786399511 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb528004030 con 0x7fb530150c20 2023-11-28T21:53:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.694+0000 7fb526ffd640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb510066470 0x7fb510068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.694+0000 7fb526ffd640 1 -- 172.21.15.12:0/2786399511 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fb528084860 con 0x7fb530150c20 2023-11-28T21:53:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.694+0000 7fb52e59c640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb510066470 0x7fb510068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:11.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.694+0000 7fb52e59c640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb510066470 0x7fb510068930 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5200079c0 tx=0x7fb520008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:11.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.698+0000 7fb526ffd640 1 -- 172.21.15.12:0/2786399511 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb528051630 con 0x7fb530150c20 2023-11-28T21:53:11.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.890+0000 7fb534bf5640 1 -- 172.21.15.12:0/2786399511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5300c27d0 con 0x7fb530150c20 2023-11-28T21:53:11.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.890+0000 7fb526ffd640 1 -- 172.21.15.12:0/2786399511 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5728 (secure 0 0 0) 0x7fb528054c50 con 0x7fb530150c20 2023-11-28T21:53:11.896 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:11.896 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:53:10.759858+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:reconnect","state_seq":29,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:11.897 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 75 2023-11-28T21:53:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 -- 172.21.15.12:0/2786399511 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb510066470 msgr2=0x7fb510068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb510066470 0x7fb510068930 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb5200079c0 tx=0x7fb520008040 comp rx=0 tx=0).stop 2023-11-28T21:53:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 -- 172.21.15.12:0/2786399511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb530150c20 msgr2=0x7fb530071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:11.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb530150c20 0x7fb530071af0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb528007e10 tx=0x7fb52800d4a0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 -- 172.21.15.12:0/2786399511 shutdown_connections 2023-11-28T21:53:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb510066470 0x7fb510068930 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb530150c20 0x7fb530071af0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53006e300 0x7fb530145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 --2- 172.21.15.12:0/2786399511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb53006cf40 0x7fb530145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 -- 172.21.15.12:0/2786399511 >> 172.21.15.12:0/2786399511 conn(0x7fb530066360 msgr2=0x7fb530155290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 -- 172.21.15.12:0/2786399511 shutdown_connections 2023-11-28T21:53:11.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:11.894+0000 7fb534bf5640 1 -- 172.21.15.12:0/2786399511 wait complete. 2023-11-28T21:53:11.917 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:53:13.918 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.066+0000 7f9a796bb640 1 Processor -- start 2023-11-28T21:53:14.074 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a796bb640 1 -- start start 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a796bb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9a74151af0 0x7f9a740c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a796bb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a74151ed0 0x7f9a740c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a796bb640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a74157b70 con 0x7f9a7406e300 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a796bb640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a740707f0 con 0x7f9a74151af0 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a796bb640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a740c7b90 con 0x7f9a74151ed0 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a72ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9a74151af0 0x7f9a740c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a72ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9a74151af0 0x7f9a740c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47896/0 (socket says 172.21.15.12:47896) 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a72ffd640 1 -- 172.21.15.12:0/1072064898 learned_addr learned my addr 172.21.15.12:0/1072064898 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a6bfff640 1 --2- 172.21.15.12:0/1072064898 >> v2:172.21.15.144:3300/0 conn(0x7f9a74151ed0 0x7f9a740c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a72ffd640 1 -- 172.21.15.12:0/1072064898 >> v2:172.21.15.144:3300/0 conn(0x7f9a74151ed0 msgr2=0x7f9a740c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:14.076 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a72ffd640 1 --2- 172.21.15.12:0/1072064898 >> v2:172.21.15.144:3300/0 conn(0x7f9a74151ed0 0x7f9a740c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a72ffd640 1 -- 172.21.15.12:0/1072064898 >> v1:172.21.15.144:6789/0 conn(0x7f9a7406e300 legacy=0x7f9a74072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:14.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.070+0000 7f9a72ffd640 1 -- 172.21.15.12:0/1072064898 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a74157e50 con 0x7f9a74151af0 2023-11-28T21:53:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a72ffd640 1 --2- 172.21.15.12:0/1072064898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a74151af0 0x7f9a740c7650 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9a740c8e60 tx=0x7f9a6401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a2aea2ea6595a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a727fc640 1 -- 172.21.15.12:0/1072064898 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a640ab600 con 0x7f9a74151af0 2023-11-28T21:53:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a727fc640 1 -- 172.21.15.12:0/1072064898 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a640a7050 con 0x7f9a74151af0 2023-11-28T21:53:14.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a727fc640 1 -- 172.21.15.12:0/1072064898 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a640ab980 con 0x7f9a74151af0 2023-11-28T21:53:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a796bb640 1 -- 172.21.15.12:0/1072064898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a74151af0 msgr2=0x7f9a740c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a796bb640 1 --2- 172.21.15.12:0/1072064898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a74151af0 0x7f9a740c7650 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9a740c8e60 tx=0x7f9a6401c4c0 comp rx=0 tx=0).stop 2023-11-28T21:53:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a796bb640 1 -- 172.21.15.12:0/1072064898 shutdown_connections 2023-11-28T21:53:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a796bb640 1 --2- 172.21.15.12:0/1072064898 >> v2:172.21.15.144:3300/0 conn(0x7f9a74151ed0 0x7f9a740c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a796bb640 1 --2- 172.21.15.12:0/1072064898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a74151af0 0x7f9a740c7650 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:14.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a796bb640 1 -- 172.21.15.12:0/1072064898 >> 172.21.15.12:0/1072064898 conn(0x7f9a74066360 msgr2=0x7f9a74066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a796bb640 1 -- 172.21.15.12:0/1072064898 shutdown_connections 2023-11-28T21:53:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a796bb640 1 -- 172.21.15.12:0/1072064898 wait complete. 2023-11-28T21:53:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.074+0000 7f9a796bb640 1 Processor -- start 2023-11-28T21:53:14.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a796bb640 1 -- start start 2023-11-28T21:53:14.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a796bb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a7406e300 0x7f9a74156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a796bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a74151af0 0x7f9a741f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a796bb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a741453b0 0x7f9a7414cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a796bb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741f3c30 con 0x7f9a741453b0 2023-11-28T21:53:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a796bb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741570d0 con 0x7f9a7406e300 2023-11-28T21:53:14.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a796bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741573b0 con 0x7f9a74151af0 2023-11-28T21:53:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a741453b0 0x7f9a7414cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a741453b0 0x7f9a7414cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47904/0 (socket says 172.21.15.12:47904) 2023-11-28T21:53:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a737fe640 1 -- 172.21.15.12:0/715532107 learned_addr learned my addr 172.21.15.12:0/715532107 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a72ffd640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a7406e300 0x7f9a74156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a6bfff640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a74151af0 0x7f9a741f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:14.086 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a737fe640 1 -- 172.21.15.12:0/715532107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a74151af0 msgr2=0x7f9a741f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a737fe640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a74151af0 0x7f9a741f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a737fe640 1 -- 172.21.15.12:0/715532107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a7406e300 msgr2=0x7f9a74156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a737fe640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a7406e300 0x7f9a74156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a737fe640 1 -- 172.21.15.12:0/715532107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a640094d0 con 0x7f9a741453b0 2023-11-28T21:53:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a737fe640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a741453b0 0x7f9a7414cbe0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00c780 tx=0x7f9a6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:14.087 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a6b7fe640 1 -- 172.21.15.12:0/715532107 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00a9f0 con 0x7f9a741453b0 2023-11-28T21:53:14.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a6b7fe640 1 -- 172.21.15.12:0/715532107 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c00ce90 con 0x7f9a741453b0 2023-11-28T21:53:14.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a6b7fe640 1 -- 172.21.15.12:0/715532107 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00add0 con 0x7f9a741453b0 2023-11-28T21:53:14.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a796bb640 1 -- 172.21.15.12:0/715532107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a74157690 con 0x7f9a741453b0 2023-11-28T21:53:14.088 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a796bb640 1 -- 172.21.15.12:0/715532107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7414c6c0 con 0x7f9a741453b0 2023-11-28T21:53:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.078+0000 7f9a796bb640 1 -- 172.21.15.12:0/715532107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7414c240 con 0x7f9a741453b0 2023-11-28T21:53:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.082+0000 7f9a6b7fe640 1 -- 172.21.15.12:0/715532107 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9a6c012420 con 0x7f9a741453b0 2023-11-28T21:53:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.082+0000 7f9a6b7fe640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a540664a0 0x7f9a54068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:14.090 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.082+0000 7f9a6b7fe640 1 -- 172.21.15.12:0/715532107 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f9a6c084340 con 0x7f9a741453b0 2023-11-28T21:53:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.082+0000 7f9a72ffd640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a540664a0 0x7f9a54068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.082+0000 7f9a72ffd640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a540664a0 0x7f9a54068960 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9a640ad000 tx=0x7f9a64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:14.091 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.086+0000 7f9a6b7fe640 1 -- 172.21.15.12:0/715532107 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a7414c240 con 0x7f9a741453b0 2023-11-28T21:53:14.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.274+0000 7f9a796bb640 1 -- 172.21.15.12:0/715532107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a7414c240 con 0x7f9a741453b0 2023-11-28T21:53:14.283 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a6b7fe640 1 -- 172.21.15.12:0/715532107 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5728 (secure 0 0 0) 0x7f9a7414c240 con 0x7f9a741453b0 2023-11-28T21:53:14.283 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:14.283 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:53:10.759858+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:reconnect","state_seq":29,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 75 2023-11-28T21:53:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 -- 172.21.15.12:0/715532107 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a540664a0 msgr2=0x7f9a54068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:14.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a540664a0 0x7f9a54068960 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9a640ad000 tx=0x7f9a64008040 comp rx=0 tx=0).stop 2023-11-28T21:53:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 -- 172.21.15.12:0/715532107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a741453b0 msgr2=0x7f9a7414cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a741453b0 0x7f9a7414cbe0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00c780 tx=0x7f9a6c00cc40 comp rx=0 tx=0).stop 2023-11-28T21:53:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 -- 172.21.15.12:0/715532107 shutdown_connections 2023-11-28T21:53:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9a540664a0 0x7f9a54068960 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a741453b0 0x7f9a7414cbe0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:14.286 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a74151af0 0x7f9a741f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 --2- 172.21.15.12:0/715532107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a7406e300 0x7f9a74156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 -- 172.21.15.12:0/715532107 >> 172.21.15.12:0/715532107 conn(0x7f9a74066360 msgr2=0x7f9a740c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 -- 172.21.15.12:0/715532107 shutdown_connections 2023-11-28T21:53:14.287 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:14.278+0000 7f9a796bb640 1 -- 172.21.15.12:0/715532107 wait complete. 2023-11-28T21:53:14.305 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:53:16.306 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:16.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae62c8640 1 Processor -- start 2023-11-28T21:53:16.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae62c8640 1 -- start start 2023-11-28T21:53:16.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae62c8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6ae0153fb0 0x7f6ae00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:16.464 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae62c8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 0x7f6ae00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:16.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae62c8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae015d390 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae62c8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae00b2b60 con 0x7f6ae0153fb0 2023-11-28T21:53:16.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae62c8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae015e1b0 con 0x7f6ae015a980 2023-11-28T21:53:16.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae5ac7640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6ae00b3ad0 0x7f6ae00b3ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58040/0 (socket says 172.21.15.12:58040) 2023-11-28T21:53:16.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae5ac7640 1 -- 172.21.15.12:0/2578177888 learned_addr learned my addr 172.21.15.12:0/2578177888 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:53:16.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae4ac5640 1 --2- 172.21.15.12:0/2578177888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 0x7f6ae00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:16.466 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae52c6640 1 --2- 172.21.15.12:0/2578177888 >> v2:172.21.15.144:3300/0 conn(0x7f6ae0153fb0 0x7f6ae00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:16.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ad7fff640 1 -- 172.21.15.12:0/2578177888 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025771665 0 0) 0x7f6ae015d390 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ad7fff640 1 -- 172.21.15.12:0/2578177888 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6acc003530 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae4ac5640 1 -- 172.21.15.12:0/2578177888 >> v2:172.21.15.144:3300/0 conn(0x7f6ae0153fb0 msgr2=0x7f6ae00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:16.467 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae4ac5640 1 --2- 172.21.15.12:0/2578177888 >> v2:172.21.15.144:3300/0 conn(0x7f6ae0153fb0 0x7f6ae00ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:16.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae4ac5640 1 -- 172.21.15.12:0/2578177888 >> v1:172.21.15.12:6789/0 conn(0x7f6ae00b3ad0 legacy=0x7f6ae00b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:16.468 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.458+0000 7f6ae4ac5640 1 -- 172.21.15.12:0/2578177888 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae015d630 con 0x7f6ae015a980 2023-11-28T21:53:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ae4ac5640 1 --2- 172.21.15.12:0/2578177888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 0x7f6ae00caa30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00097c0 tx=0x7f6ad001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b20548b729bf6db1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:16.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ad7fff640 1 -- 172.21.15.12:0/2578177888 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00ab650 con 0x7f6ae015a980 2023-11-28T21:53:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ad7fff640 1 -- 172.21.15.12:0/2578177888 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad00ab7b0 con 0x7f6ae015a980 2023-11-28T21:53:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ad7fff640 1 -- 172.21.15.12:0/2578177888 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00abb20 con 0x7f6ae015a980 2023-11-28T21:53:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ae62c8640 1 -- 172.21.15.12:0/2578177888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 msgr2=0x7f6ae00caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ae62c8640 1 --2- 172.21.15.12:0/2578177888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 0x7f6ae00caa30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00097c0 tx=0x7f6ad001c450 comp rx=0 tx=0).stop 2023-11-28T21:53:16.470 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ae62c8640 1 -- 172.21.15.12:0/2578177888 shutdown_connections 2023-11-28T21:53:16.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ae62c8640 1 --2- 172.21.15.12:0/2578177888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 0x7f6ae00caa30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:16.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ae62c8640 1 --2- 172.21.15.12:0/2578177888 >> v2:172.21.15.144:3300/0 conn(0x7f6ae0153fb0 0x7f6ae00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:16.471 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ae62c8640 1 -- 172.21.15.12:0/2578177888 >> 172.21.15.12:0/2578177888 conn(0x7f6ae0071ce0 msgr2=0x7f6ae00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:16.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ae62c8640 1 -- 172.21.15.12:0/2578177888 shutdown_connections 2023-11-28T21:53:16.472 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.462+0000 7f6ae62c8640 1 -- 172.21.15.12:0/2578177888 wait complete. 2023-11-28T21:53:16.473 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae62c8640 1 Processor -- start 2023-11-28T21:53:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae62c8640 1 -- start start 2023-11-28T21:53:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae62c8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae00b3ad0 0x7f6ae014f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae62c8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae0153fb0 0x7f6ae01f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae62c8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 0x7f6ae0156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae62c8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae0150440 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae62c8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae0155fa0 con 0x7f6ae0153fb0 2023-11-28T21:53:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae62c8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae0150180 con 0x7f6ae015a980 2023-11-28T21:53:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae52c6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae00b3ad0 0x7f6ae014f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae52c6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae00b3ad0 0x7f6ae014f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50056/0 (socket says 172.21.15.12:50056) 2023-11-28T21:53:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae52c6640 1 -- 172.21.15.12:0/439629332 learned_addr learned my addr 172.21.15.12:0/439629332 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:16.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae4ac5640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae0153fb0 0x7f6ae01f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae5ac7640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 0x7f6ae0156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae52c6640 1 -- 172.21.15.12:0/439629332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 msgr2=0x7f6ae0156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae52c6640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 0x7f6ae0156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae52c6640 1 -- 172.21.15.12:0/439629332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae0153fb0 msgr2=0x7f6ae01f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:16.476 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae52c6640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae0153fb0 0x7f6ae01f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae52c6640 1 -- 172.21.15.12:0/439629332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad0009420 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae5ac7640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 0x7f6ae0156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:53:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae52c6640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae00b3ad0 0x7f6ae014f110 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00e930 tx=0x7f6adc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ad5ffb640 1 -- 172.21.15.12:0/439629332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc009690 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ad5ffb640 1 -- 172.21.15.12:0/439629332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc00ca00 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ad5ffb640 1 -- 172.21.15.12:0/439629332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc00cd70 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae62c8640 1 -- 172.21.15.12:0/439629332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae0155a00 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.466+0000 7f6ae62c8640 1 -- 172.21.15.12:0/439629332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae0155cc0 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.470+0000 7f6ae62c8640 1 -- 172.21.15.12:0/439629332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae0074f60 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.470+0000 7f6ad5ffb640 1 -- 172.21.15.12:0/439629332 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6adc013470 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.470+0000 7f6ad5ffb640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6ac4066470 0x7f6ac4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:16.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.470+0000 7f6ad5ffb640 1 -- 172.21.15.12:0/439629332 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f6adc0846d0 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.470+0000 7f6ae4ac5640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6ac4066470 0x7f6ac4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:16.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.470+0000 7f6ae4ac5640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6ac4066470 0x7f6ac4068930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00af000 tx=0x7f6ad00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:16.480 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.474+0000 7f6ad5ffb640 1 -- 172.21.15.12:0/439629332 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6adc0515f0 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.512 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:53:16.516 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:53:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.666+0000 7f6ae62c8640 1 -- 172.21.15.12:0/439629332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ae014e040 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.666+0000 7f6ad5ffb640 1 -- 172.21.15.12:0/439629332 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5728 (secure 0 0 0) 0x7f6adc054c10 con 0x7f6ae00b3ad0 2023-11-28T21:53:16.672 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:16.673 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:53:10.759858+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:reconnect","state_seq":29,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:16.673 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 75 2023-11-28T21:53:16.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 -- 172.21.15.12:0/439629332 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6ac4066470 msgr2=0x7f6ac4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6ac4066470 0x7f6ac4068930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00af000 tx=0x7f6ad00a7000 comp rx=0 tx=0).stop 2023-11-28T21:53:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 -- 172.21.15.12:0/439629332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae00b3ad0 msgr2=0x7f6ae014f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae00b3ad0 0x7f6ae014f110 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00e930 tx=0x7f6adc00edf0 comp rx=0 tx=0).stop 2023-11-28T21:53:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 -- 172.21.15.12:0/439629332 shutdown_connections 2023-11-28T21:53:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6ac4066470 0x7f6ac4068930 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ae015a980 0x7f6ae0156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ae0153fb0 0x7f6ae01f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:16.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 --2- 172.21.15.12:0/439629332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ae00b3ad0 0x7f6ae014f110 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 -- 172.21.15.12:0/439629332 >> 172.21.15.12:0/439629332 conn(0x7f6ae0071ce0 msgr2=0x7f6ae00ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 -- 172.21.15.12:0/439629332 shutdown_connections 2023-11-28T21:53:16.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:16.670+0000 7f6ae62c8640 1 -- 172.21.15.12:0/439629332 wait complete. 2023-11-28T21:53:16.694 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:53:18.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:18.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96c2edd640 1 Processor -- start 2023-11-28T21:53:18.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96c2edd640 1 -- start start 2023-11-28T21:53:18.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96c2edd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f96bc151ed0 0x7f96bc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:18.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96c2edd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc157b70 con 0x7f96bc151af0 2023-11-28T21:53:18.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96c2edd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc157e50 con 0x7f96bc06e300 2023-11-28T21:53:18.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96c2edd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96bc0707f0 con 0x7f96bc151ed0 2023-11-28T21:53:18.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96c0c79640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f96bc151af0 0x7f96bc0c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58050/0 (socket says 172.21.15.12:58050) 2023-11-28T21:53:18.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96c0c79640 1 -- 172.21.15.12:0/4073085114 learned_addr learned my addr 172.21.15.12:0/4073085114 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:53:18.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96bbfff640 1 --2- 172.21.15.12:0/4073085114 >> v2:172.21.15.144:3300/0 conn(0x7f96bc151ed0 0x7f96bc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:18.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678608991 0 0) 0x7f96bc157b70 con 0x7f96bc151af0 2023-11-28T21:53:18.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.846+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96ac003530 con 0x7f96bc151af0 2023-11-28T21:53:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1313596193 0 0) 0x7f96ac003530 con 0x7f96bc151af0 2023-11-28T21:53:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 >> v2:172.21.15.144:3300/0 conn(0x7f96bc151ed0 msgr2=0x7f96bc0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 --2- 172.21.15.12:0/4073085114 >> v2:172.21.15.144:3300/0 conn(0x7f96bc151ed0 0x7f96bc0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 >> v1:172.21.15.144:6789/0 conn(0x7f96bc06e300 legacy=0x7f96bc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc158130 con 0x7f96bc151af0 2023-11-28T21:53:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f96b0001680 con 0x7f96bc151af0 2023-11-28T21:53:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 >> v1:172.21.15.12:6789/0 conn(0x7f96bc151af0 legacy=0x7f96bc0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac0039f0 0x7f96ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96ac0047d0 0x7f96ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:18.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96ac005190 0x7f96ac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0707f0 con 0x7f96ac0039f0 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac00c0c0 con 0x7f96ac0047d0 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ac00c280 con 0x7f96ac005190 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96c0c79640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac0039f0 0x7f96ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bbfff640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96ac0047d0 0x7f96ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96c147a640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96ac005190 0x7f96ac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96c0c79640 1 -- 172.21.15.12:0/4073085114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96ac005190 msgr2=0x7f96ac00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96c0c79640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96ac005190 0x7f96ac00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96c0c79640 1 -- 172.21.15.12:0/4073085114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96ac0047d0 msgr2=0x7f96ac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96c0c79640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96ac0047d0 0x7f96ac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96c0c79640 1 -- 172.21.15.12:0/4073085114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac003530 con 0x7f96ac0039f0 2023-11-28T21:53:18.857 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96c147a640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96ac005190 0x7f96ac00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:53:18.858 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96c147a640 1 -- 172.21.15.12:0/4073085114 reap_dead start 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96c0c79640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac0039f0 0x7f96ac003e10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c240 tx=0x7f96b001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c41d0fd1904f06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0005ae0 con 0x7f96ac0039f0 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b0005c40 con 0x7f96ac0039f0 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.850+0000 7f96bb7fe640 1 -- 172.21.15.12:0/4073085114 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b00072b0 con 0x7f96ac0039f0 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 -- 172.21.15.12:0/4073085114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac0039f0 msgr2=0x7f96ac003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac0039f0 0x7f96ac003e10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f96b000c240 tx=0x7f96b001cc40 comp rx=0 tx=0).stop 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 -- 172.21.15.12:0/4073085114 shutdown_connections 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 --2- 172.21.15.12:0/4073085114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96ac0039f0 0x7f96ac003e10 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 -- 172.21.15.12:0/4073085114 >> 172.21.15.12:0/4073085114 conn(0x7f96bc066360 msgr2=0x7f96bc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 -- 172.21.15.12:0/4073085114 shutdown_connections 2023-11-28T21:53:18.859 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 -- 172.21.15.12:0/4073085114 wait complete. 2023-11-28T21:53:18.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 Processor -- start 2023-11-28T21:53:18.861 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 -- start start 2023-11-28T21:53:18.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96ac0039f0 0x7f96bc145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:18.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96bc144b20 0x7f96bc1f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:18.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96bc14f4c0 0x7f96bc15b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:18.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc158b90 con 0x7f96bc14f4c0 2023-11-28T21:53:18.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc158de0 con 0x7f96bc144b20 2023-11-28T21:53:18.862 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c2edd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc1590c0 con 0x7f96ac0039f0 2023-11-28T21:53:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c147a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96bc14f4c0 0x7f96bc15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c147a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96bc14f4c0 0x7f96bc15b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50074/0 (socket says 172.21.15.12:50074) 2023-11-28T21:53:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c147a640 1 -- 172.21.15.12:0/3489548577 learned_addr learned my addr 172.21.15.12:0/3489548577 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96bbfff640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96bc144b20 0x7f96bc1f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c0c79640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96ac0039f0 0x7f96bc145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c147a640 1 -- 172.21.15.12:0/3489548577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96ac0039f0 msgr2=0x7f96bc145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.854+0000 7f96c147a640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96ac0039f0 0x7f96bc145000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96c147a640 1 -- 172.21.15.12:0/3489548577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96bc144b20 msgr2=0x7f96bc1f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:18.866 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96c147a640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96bc144b20 0x7f96bc1f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96c147a640 1 -- 172.21.15.12:0/3489548577 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc1593a0 con 0x7f96bc14f4c0 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96bbfff640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96bc144b20 0x7f96bc1f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96c147a640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96bc14f4c0 0x7f96bc15b310 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f96bc071130 tx=0x7f96b400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96b97fa640 1 -- 172.21.15.12:0/3489548577 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4003b80 con 0x7f96bc14f4c0 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96b97fa640 1 -- 172.21.15.12:0/3489548577 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b4003ce0 con 0x7f96bc14f4c0 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96b97fa640 1 -- 172.21.15.12:0/3489548577 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b40052b0 con 0x7f96bc14f4c0 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96c2edd640 1 -- 172.21.15.12:0/3489548577 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc159680 con 0x7f96bc14f4c0 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96c2edd640 1 -- 172.21.15.12:0/3489548577 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc14f950 con 0x7f96bc14f4c0 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96c2edd640 1 -- 172.21.15.12:0/3489548577 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc074f60 con 0x7f96bc14f4c0 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96b97fa640 1 -- 172.21.15.12:0/3489548577 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f96b4005410 con 0x7f96bc14f4c0 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96b97fa640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f96a00664a0 0x7f96a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:18.867 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96b97fa640 1 -- 172.21.15.12:0/3489548577 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f96b4083e00 con 0x7f96bc14f4c0 2023-11-28T21:53:18.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.858+0000 7f96c0c79640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f96a00664a0 0x7f96a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:18.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.862+0000 7f96c0c79640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f96a00664a0 0x7f96a0068960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f96b00ab000 tx=0x7f96b00b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:18.869 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:18.866+0000 7f96b97fa640 1 -- 172.21.15.12:0/3489548577 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f96b4050c50 con 0x7f96bc14f4c0 2023-11-28T21:53:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 -- 172.21.15.12:0/3489548577 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96bc159b30 con 0x7f96bc14f4c0 2023-11-28T21:53:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96b97fa640 1 -- 172.21.15.12:0/3489548577 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5725 (secure 0 0 0) 0x7f96b4018070 con 0x7f96bc14f4c0 2023-11-28T21:53:19.064 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:19.065 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:53:16.751064+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:rejoin","state_seq":31,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 76 2023-11-28T21:53:19.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 -- 172.21.15.12:0/3489548577 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f96a00664a0 msgr2=0x7f96a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:19.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f96a00664a0 0x7f96a0068960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f96b00ab000 tx=0x7f96b00b1040 comp rx=0 tx=0).stop 2023-11-28T21:53:19.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 -- 172.21.15.12:0/3489548577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96bc14f4c0 msgr2=0x7f96bc15b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:19.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96bc14f4c0 0x7f96bc15b310 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f96bc071130 tx=0x7f96b400dc00 comp rx=0 tx=0).stop 2023-11-28T21:53:19.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 -- 172.21.15.12:0/3489548577 shutdown_connections 2023-11-28T21:53:19.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f96a00664a0 0x7f96a0068960 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:19.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96bc14f4c0 0x7f96bc15b310 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:19.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96bc144b20 0x7f96bc1f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:19.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 --2- 172.21.15.12:0/3489548577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96ac0039f0 0x7f96bc145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:19.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 -- 172.21.15.12:0/3489548577 >> 172.21.15.12:0/3489548577 conn(0x7f96bc066360 msgr2=0x7f96bc06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:19.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 -- 172.21.15.12:0/3489548577 shutdown_connections 2023-11-28T21:53:19.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:19.058+0000 7f96c2edd640 1 -- 172.21.15.12:0/3489548577 wait complete. 2023-11-28T21:53:19.085 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 76, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 76, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6330": { "addr": "172.21.15.12:6839/754378087", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6836", "nonce": 754378087, "type": "v2" }, { "addr": "172.21.15.12:6839", "nonce": 754378087, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6330, "incarnation": 63, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 16 }, "gid_6513": { "addr": "172.21.15.12:6838/2286739028", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2286739028, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 2286739028, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6513, "incarnation": 73, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 51, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:53:16.751064+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6513, "mds_1": 6330 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:53:21.087 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:21.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.234+0000 7f1f4aefd640 1 Processor -- start 2023-11-28T21:53:21.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.234+0000 7f1f4aefd640 1 -- start start 2023-11-28T21:53:21.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f4aefd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f44150c20 0x7f1f44156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:21.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f4aefd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406e300 0x7f1f440c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:21.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f4aefd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f44067410 con 0x7f1f4406cf40 2023-11-28T21:53:21.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f4aefd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4406e090 con 0x7f1f44150c20 2023-11-28T21:53:21.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f4aefd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f44157f30 con 0x7f1f4406e300 2023-11-28T21:53:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406e300 0x7f1f440c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f48c99640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f44150c20 0x7f1f44156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f48c99640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f44150c20 0x7f1f44156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:34928/0 (socket says 172.21.15.12:34928) 2023-11-28T21:53:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406e300 0x7f1f440c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:44558/0 (socket says 172.21.15.12:44558) 2023-11-28T21:53:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f48c99640 1 -- 172.21.15.12:0/1390318747 learned_addr learned my addr 172.21.15.12:0/1390318747 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f48c99640 1 -- 172.21.15.12:0/1390318747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406e300 msgr2=0x7f1f440c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:21.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f48c99640 1 --2- 172.21.15.12:0/1390318747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406e300 0x7f1f440c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:21.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f48c99640 1 -- 172.21.15.12:0/1390318747 >> v1:172.21.15.144:6789/0 conn(0x7f1f4406cf40 legacy=0x7f1f44158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:21.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f48c99640 1 -- 172.21.15.12:0/1390318747 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f44151000 con 0x7f1f44150c20 2023-11-28T21:53:21.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.238+0000 7f1f43fff640 1 --2- 172.21.15.12:0/1390318747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406e300 0x7f1f440c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:53:21.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f48c99640 1 --2- 172.21.15.12:0/1390318747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f44150c20 0x7f1f44156910 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009870 tx=0x7f1f3801e750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81e369a56ace4978 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:21.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f437fe640 1 -- 172.21.15.12:0/1390318747 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f380ab630 con 0x7f1f44150c20 2023-11-28T21:53:21.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f437fe640 1 -- 172.21.15.12:0/1390318747 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f380ab790 con 0x7f1f44150c20 2023-11-28T21:53:21.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f437fe640 1 -- 172.21.15.12:0/1390318747 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f380abb20 con 0x7f1f44150c20 2023-11-28T21:53:21.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1390318747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f44150c20 msgr2=0x7f1f44156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:21.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f4aefd640 1 --2- 172.21.15.12:0/1390318747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f44150c20 0x7f1f44156910 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009870 tx=0x7f1f3801e750 comp rx=0 tx=0).stop 2023-11-28T21:53:21.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1390318747 shutdown_connections 2023-11-28T21:53:21.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f4aefd640 1 --2- 172.21.15.12:0/1390318747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406e300 0x7f1f440c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:21.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f4aefd640 1 --2- 172.21.15.12:0/1390318747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f44150c20 0x7f1f44156910 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:21.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1390318747 >> 172.21.15.12:0/1390318747 conn(0x7f1f44066360 msgr2=0x7f1f44066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:21.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1390318747 shutdown_connections 2023-11-28T21:53:21.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.242+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1390318747 wait complete. 2023-11-28T21:53:21.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4aefd640 1 Processor -- start 2023-11-28T21:53:21.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4aefd640 1 -- start start 2023-11-28T21:53:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4aefd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406cf40 0x7f1f440c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4aefd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f4406e300 0x7f1f4413ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4aefd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f44150c20 0x7f1f44160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4aefd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4413e4a0 con 0x7f1f44150c20 2023-11-28T21:53:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4aefd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440c9880 con 0x7f1f4406e300 2023-11-28T21:53:21.251 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4aefd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440c90c0 con 0x7f1f4406cf40 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4949a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f44150c20 0x7f1f44160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f48c99640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406cf40 0x7f1f440c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4949a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f44150c20 0x7f1f44160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50078/0 (socket says 172.21.15.12:50078) 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4949a640 1 -- 172.21.15.12:0/1007323046 learned_addr learned my addr 172.21.15.12:0/1007323046 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f43fff640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f4406e300 0x7f1f4413ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4949a640 1 -- 172.21.15.12:0/1007323046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406cf40 msgr2=0x7f1f440c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4949a640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406cf40 0x7f1f440c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4949a640 1 -- 172.21.15.12:0/1007323046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f4406e300 msgr2=0x7f1f4413ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4949a640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f4406e300 0x7f1f4413ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4949a640 1 -- 172.21.15.12:0/1007323046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30009420 con 0x7f1f44150c20 2023-11-28T21:53:21.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4949a640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f44150c20 0x7f1f44160680 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00c7a0 tx=0x7f1f3c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f417fa640 1 -- 172.21.15.12:0/1007323046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c003b80 con 0x7f1f44150c20 2023-11-28T21:53:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f417fa640 1 -- 172.21.15.12:0/1007323046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c003ce0 con 0x7f1f44150c20 2023-11-28T21:53:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1007323046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f380094d0 con 0x7f1f44150c20 2023-11-28T21:53:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f417fa640 1 -- 172.21.15.12:0/1007323046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c00aa30 con 0x7f1f44150c20 2023-11-28T21:53:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.246+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1007323046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4415f400 con 0x7f1f44150c20 2023-11-28T21:53:21.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.250+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1007323046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f44074f60 con 0x7f1f44150c20 2023-11-28T21:53:21.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.250+0000 7f1f417fa640 1 -- 172.21.15.12:0/1007323046 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1f3c00ab90 con 0x7f1f44150c20 2023-11-28T21:53:21.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.250+0000 7f1f417fa640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1f28066470 0x7f1f28068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:21.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.250+0000 7f1f417fa640 1 -- 172.21.15.12:0/1007323046 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f1f3c084300 con 0x7f1f44150c20 2023-11-28T21:53:21.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.250+0000 7f1f48c99640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1f28066470 0x7f1f28068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:21.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.250+0000 7f1f48c99640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1f28066470 0x7f1f28068930 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1f380b0000 tx=0x7f1f380b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:21.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.254+0000 7f1f417fa640 1 -- 172.21.15.12:0/1007323046 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1f3c051040 con 0x7f1f44150c20 2023-11-28T21:53:21.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1007323046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f440c2c30 con 0x7f1f44150c20 2023-11-28T21:53:21.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f417fa640 1 -- 172.21.15.12:0/1007323046 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5725 (secure 0 0 0) 0x7f1f3c054660 con 0x7f1f44150c20 2023-11-28T21:53:21.451 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:21.451 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:53:16.751064+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:rejoin","state_seq":31,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:21.451 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 76 2023-11-28T21:53:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1007323046 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1f28066470 msgr2=0x7f1f28068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1f28066470 0x7f1f28068930 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1f380b0000 tx=0x7f1f380b5040 comp rx=0 tx=0).stop 2023-11-28T21:53:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1007323046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f44150c20 msgr2=0x7f1f44160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f44150c20 0x7f1f44160680 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00c7a0 tx=0x7f1f3c00cc60 comp rx=0 tx=0).stop 2023-11-28T21:53:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1007323046 shutdown_connections 2023-11-28T21:53:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1f28066470 0x7f1f28068930 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f44150c20 0x7f1f44160680 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f4406e300 0x7f1f4413ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:21.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 --2- 172.21.15.12:0/1007323046 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f4406cf40 0x7f1f440c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:21.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1007323046 >> 172.21.15.12:0/1007323046 conn(0x7f1f44066360 msgr2=0x7f1f4406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:21.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1007323046 shutdown_connections 2023-11-28T21:53:21.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:21.446+0000 7f1f4aefd640 1 -- 172.21.15.12:0/1007323046 wait complete. 2023-11-28T21:53:21.472 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:53:23.473 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:23.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.622+0000 7f42eaf62640 1 Processor -- start 2023-11-28T21:53:23.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.622+0000 7f42eaf62640 1 -- start start 2023-11-28T21:53:23.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42eaf62640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e4153fb0 0x7f42e40ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:23.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42eaf62640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f42e415a980 0x7f42e40caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:23.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42eaf62640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e415d390 con 0x7f42e40b3ad0 2023-11-28T21:53:23.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42eaf62640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e40b2b60 con 0x7f42e415a980 2023-11-28T21:53:23.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42eaf62640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e415e1b0 con 0x7f42e4153fb0 2023-11-28T21:53:23.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42e9f60640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e4153fb0 0x7f42e40ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:23.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42ea761640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f42e40b3ad0 0x7f42e40b3ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58064/0 (socket says 172.21.15.12:58064) 2023-11-28T21:53:23.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42ea761640 1 -- 172.21.15.12:0/1102255106 learned_addr learned my addr 172.21.15.12:0/1102255106 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:53:23.632 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42e975f640 1 --2- 172.21.15.12:0/1102255106 >> v2:172.21.15.144:3300/0 conn(0x7f42e415a980 0x7f42e40caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:23.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42e8f5e640 1 -- 172.21.15.12:0/1102255106 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368536265 0 0) 0x7f42e415d390 con 0x7f42e40b3ad0 2023-11-28T21:53:23.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42e8f5e640 1 -- 172.21.15.12:0/1102255106 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d0003530 con 0x7f42e40b3ad0 2023-11-28T21:53:23.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42e9f60640 1 -- 172.21.15.12:0/1102255106 >> v2:172.21.15.144:3300/0 conn(0x7f42e415a980 msgr2=0x7f42e40caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:23.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42e9f60640 1 --2- 172.21.15.12:0/1102255106 >> v2:172.21.15.144:3300/0 conn(0x7f42e415a980 0x7f42e40caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:23.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42e9f60640 1 -- 172.21.15.12:0/1102255106 >> v1:172.21.15.12:6789/0 conn(0x7f42e40b3ad0 legacy=0x7f42e40b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:23.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.626+0000 7f42e9f60640 1 -- 172.21.15.12:0/1102255106 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e415d630 con 0x7f42e4153fb0 2023-11-28T21:53:23.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42e9f60640 1 --2- 172.21.15.12:0/1102255106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e4153fb0 0x7f42e40ca4f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f42e00097c0 tx=0x7f42e001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8e71efa37abeec4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:23.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42e8f5e640 1 -- 172.21.15.12:0/1102255106 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e00ab650 con 0x7f42e4153fb0 2023-11-28T21:53:23.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42e8f5e640 1 -- 172.21.15.12:0/1102255106 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e00ab7b0 con 0x7f42e4153fb0 2023-11-28T21:53:23.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42e8f5e640 1 -- 172.21.15.12:0/1102255106 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e00abb20 con 0x7f42e4153fb0 2023-11-28T21:53:23.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42eaf62640 1 -- 172.21.15.12:0/1102255106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e4153fb0 msgr2=0x7f42e40ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:23.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42eaf62640 1 --2- 172.21.15.12:0/1102255106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e4153fb0 0x7f42e40ca4f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f42e00097c0 tx=0x7f42e001c450 comp rx=0 tx=0).stop 2023-11-28T21:53:23.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42eaf62640 1 -- 172.21.15.12:0/1102255106 shutdown_connections 2023-11-28T21:53:23.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42eaf62640 1 --2- 172.21.15.12:0/1102255106 >> v2:172.21.15.144:3300/0 conn(0x7f42e415a980 0x7f42e40caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:23.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42eaf62640 1 --2- 172.21.15.12:0/1102255106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e4153fb0 0x7f42e40ca4f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:23.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42eaf62640 1 -- 172.21.15.12:0/1102255106 >> 172.21.15.12:0/1102255106 conn(0x7f42e4071ce0 msgr2=0x7f42e40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:23.636 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42eaf62640 1 -- 172.21.15.12:0/1102255106 shutdown_connections 2023-11-28T21:53:23.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42eaf62640 1 -- 172.21.15.12:0/1102255106 wait complete. 2023-11-28T21:53:23.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.630+0000 7f42eaf62640 1 Processor -- start 2023-11-28T21:53:23.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42eaf62640 1 -- start start 2023-11-28T21:53:23.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42eaf62640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42e40b3ad0 0x7f42e415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:23.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42eaf62640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e4153fb0 0x7f42e415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:23.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42eaf62640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e415a980 0x7f42e4142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:23.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42eaf62640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e415b960 con 0x7f42e40b3ad0 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42eaf62640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e4149f50 con 0x7f42e4153fb0 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42eaf62640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e414a230 con 0x7f42e415a980 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42e9f60640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42e40b3ad0 0x7f42e415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42e9f60640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42e40b3ad0 0x7f42e415bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50086/0 (socket says 172.21.15.12:50086) 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42e9f60640 1 -- 172.21.15.12:0/4051524554 learned_addr learned my addr 172.21.15.12:0/4051524554 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42e975f640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e4153fb0 0x7f42e415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42ea761640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e415a980 0x7f42e4142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42e9f60640 1 -- 172.21.15.12:0/4051524554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e415a980 msgr2=0x7f42e4142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42e9f60640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e415a980 0x7f42e4142560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42e9f60640 1 -- 172.21.15.12:0/4051524554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e4153fb0 msgr2=0x7f42e415c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42e9f60640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e4153fb0 0x7f42e415c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:23.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42e9f60640 1 -- 172.21.15.12:0/4051524554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e0009420 con 0x7f42e40b3ad0 2023-11-28T21:53:23.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42e9f60640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42e40b3ad0 0x7f42e415bd80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f42e001a080 tx=0x7f42e00191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:23.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42da7fc640 1 -- 172.21.15.12:0/4051524554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e00ab920 con 0x7f42e40b3ad0 2023-11-28T21:53:23.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42da7fc640 1 -- 172.21.15.12:0/4051524554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e00aba80 con 0x7f42e40b3ad0 2023-11-28T21:53:23.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42da7fc640 1 -- 172.21.15.12:0/4051524554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e00abe10 con 0x7f42e40b3ad0 2023-11-28T21:53:23.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42eaf62640 1 -- 172.21.15.12:0/4051524554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e4149320 con 0x7f42e40b3ad0 2023-11-28T21:53:23.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42eaf62640 1 -- 172.21.15.12:0/4051524554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e41498e0 con 0x7f42e40b3ad0 2023-11-28T21:53:23.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.634+0000 7f42eaf62640 1 -- 172.21.15.12:0/4051524554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e4074f60 con 0x7f42e40b3ad0 2023-11-28T21:53:23.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.638+0000 7f42da7fc640 1 -- 172.21.15.12:0/4051524554 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f42e0003620 con 0x7f42e40b3ad0 2023-11-28T21:53:23.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.638+0000 7f42da7fc640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42c8066470 0x7f42c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:23.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.638+0000 7f42da7fc640 1 -- 172.21.15.12:0/4051524554 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f42e011ec50 con 0x7f42e40b3ad0 2023-11-28T21:53:23.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.638+0000 7f42e975f640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42c8066470 0x7f42c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:23.643 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.638+0000 7f42e975f640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42c8066470 0x7f42c8068930 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f42e40727a0 tx=0x7f42d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:23.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.642+0000 7f42da7fc640 1 -- 172.21.15.12:0/4051524554 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f42e00eba10 con 0x7f42e40b3ad0 2023-11-28T21:53:23.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 -- 172.21.15.12:0/4051524554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42e40b4f00 con 0x7f42e40b3ad0 2023-11-28T21:53:23.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42da7fc640 1 -- 172.21.15.12:0/4051524554 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5725 (secure 0 0 0) 0x7f42e00ef030 con 0x7f42e40b3ad0 2023-11-28T21:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:23.840 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:53:21.833525+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:23.840 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 77 2023-11-28T21:53:23.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 -- 172.21.15.12:0/4051524554 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42c8066470 msgr2=0x7f42c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:23.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42c8066470 0x7f42c8068930 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f42e40727a0 tx=0x7f42d4008040 comp rx=0 tx=0).stop 2023-11-28T21:53:23.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 -- 172.21.15.12:0/4051524554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42e40b3ad0 msgr2=0x7f42e415bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42e40b3ad0 0x7f42e415bd80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f42e001a080 tx=0x7f42e00191f0 comp rx=0 tx=0).stop 2023-11-28T21:53:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 -- 172.21.15.12:0/4051524554 shutdown_connections 2023-11-28T21:53:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42c8066470 0x7f42c8068930 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42e415a980 0x7f42e4142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e4153fb0 0x7f42e415c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 --2- 172.21.15.12:0/4051524554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42e40b3ad0 0x7f42e415bd80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 -- 172.21.15.12:0/4051524554 >> 172.21.15.12:0/4051524554 conn(0x7f42e4071ce0 msgr2=0x7f42e40ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 -- 172.21.15.12:0/4051524554 shutdown_connections 2023-11-28T21:53:23.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:23.834+0000 7f42eaf62640 1 -- 172.21.15.12:0/4051524554 wait complete. 2023-11-28T21:53:23.860 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=5057 2023-11-28T21:53:23.861 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before reviving mds.f 2023-11-28T21:53:26.862 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2023-11-28T21:53:26.862 INFO:tasks.ceph.mds.f:Restarting daemon 2023-11-28T21:53:26.862 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-11-28T21:53:26.865 INFO:tasks.ceph.mds.f:Started 2023-11-28T21:53:26.865 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:26.945 INFO:tasks.ceph.mds.f.smithi144.stdout:starting mds.f at 2023-11-28T21:53:27.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc91aed6640 1 Processor -- start 2023-11-28T21:53:27.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc91aed6640 1 -- start start 2023-11-28T21:53:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc91aed6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9140c27c0 0x7fc914156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc91aed6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc91406d7d0 0x7fc91406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc91aed6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9140c6ce0 con 0x7fc9141500d0 2023-11-28T21:53:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc91aed6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc914157030 con 0x7fc91406d7d0 2023-11-28T21:53:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc91aed6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9140c7170 con 0x7fc9140c27c0 2023-11-28T21:53:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc919473640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc91406d7d0 0x7fc91406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc919473640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc91406d7d0 0x7fc91406dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41612/0 (socket says 172.21.15.12:41612) 2023-11-28T21:53:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc919473640 1 -- 172.21.15.12:0/351887755 learned_addr learned my addr 172.21.15.12:0/351887755 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.014+0000 7fc913fff640 1 --2- 172.21.15.12:0/351887755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9140c27c0 0x7fc914156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:27.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.018+0000 7fc919473640 1 -- 172.21.15.12:0/351887755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9140c27c0 msgr2=0x7fc914156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:27.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.018+0000 7fc919473640 1 --2- 172.21.15.12:0/351887755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9140c27c0 0x7fc914156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.018+0000 7fc919473640 1 -- 172.21.15.12:0/351887755 >> v1:172.21.15.144:6789/0 conn(0x7fc9141500d0 legacy=0x7fc9140c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:27.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.018+0000 7fc919473640 1 -- 172.21.15.12:0/351887755 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9140c2c00 con 0x7fc91406d7d0 2023-11-28T21:53:27.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.018+0000 7fc919473640 1 --2- 172.21.15.12:0/351887755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc91406d7d0 0x7fc91406dc20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc90c00d180 tx=0x7fc90c01ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf5d4e621034587 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:27.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.018+0000 7fc9137fe640 1 -- 172.21.15.12:0/351887755 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c00b2d0 con 0x7fc91406d7d0 2023-11-28T21:53:27.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.018+0000 7fc9137fe640 1 -- 172.21.15.12:0/351887755 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c00b430 con 0x7fc91406d7d0 2023-11-28T21:53:27.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.018+0000 7fc9137fe640 1 -- 172.21.15.12:0/351887755 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0a7c30 con 0x7fc91406d7d0 2023-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.018+0000 7fc91aed6640 1 -- 172.21.15.12:0/351887755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc91406d7d0 msgr2=0x7fc91406dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.018+0000 7fc91aed6640 1 --2- 172.21.15.12:0/351887755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc91406d7d0 0x7fc91406dc20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc90c00d180 tx=0x7fc90c01ebc0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 -- 172.21.15.12:0/351887755 shutdown_connections 2023-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 --2- 172.21.15.12:0/351887755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc91406d7d0 0x7fc91406dc20 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 --2- 172.21.15.12:0/351887755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9140c27c0 0x7fc914156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 -- 172.21.15.12:0/351887755 >> 172.21.15.12:0/351887755 conn(0x7fc914066360 msgr2=0x7fc914066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 -- 172.21.15.12:0/351887755 shutdown_connections 2023-11-28T21:53:27.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 -- 172.21.15.12:0/351887755 wait complete. 2023-11-28T21:53:27.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 Processor -- start 2023-11-28T21:53:27.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 -- start start 2023-11-28T21:53:27.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91406d7d0 0x7fc91415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:27.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9140c27c0 0x7fc914157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:27.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9141500d0 0x7fc91415f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:27.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141f6b30 con 0x7fc9140c27c0 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc914156760 con 0x7fc91406d7d0 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc91aed6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc914156a40 con 0x7fc9141500d0 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc913fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9140c27c0 0x7fc914157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc913fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9140c27c0 0x7fc914157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41614/0 (socket says 172.21.15.12:41614) 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc913fff640 1 -- 172.21.15.12:0/2012640356 learned_addr learned my addr 172.21.15.12:0/2012640356 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc919473640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9141500d0 0x7fc91415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc918c72640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91406d7d0 0x7fc91415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc913fff640 1 -- 172.21.15.12:0/2012640356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9141500d0 msgr2=0x7fc91415f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc913fff640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9141500d0 0x7fc91415f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc913fff640 1 -- 172.21.15.12:0/2012640356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91406d7d0 msgr2=0x7fc91415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc913fff640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91406d7d0 0x7fc91415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.022+0000 7fc913fff640 1 -- 172.21.15.12:0/2012640356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c00bd40 con 0x7fc9140c27c0 2023-11-28T21:53:27.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc913fff640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9140c27c0 0x7fc914157490 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e930 tx=0x7fc90000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc9117fa640 1 -- 172.21.15.12:0/2012640356 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900009690 con 0x7fc9140c27c0 2023-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc9117fa640 1 -- 172.21.15.12:0/2012640356 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90000ca00 con 0x7fc9140c27c0 2023-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc9117fa640 1 -- 172.21.15.12:0/2012640356 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90000cd70 con 0x7fc9140c27c0 2023-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc91aed6640 1 -- 172.21.15.12:0/2012640356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc914156d20 con 0x7fc9140c27c0 2023-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc91aed6640 1 -- 172.21.15.12:0/2012640356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91415ee40 con 0x7fc9140c27c0 2023-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc91aed6640 1 -- 172.21.15.12:0/2012640356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc914074f60 con 0x7fc9140c27c0 2023-11-28T21:53:27.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc9117fa640 1 -- 172.21.15.12:0/2012640356 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc900013470 con 0x7fc9140c27c0 2023-11-28T21:53:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc9117fa640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc8f80664a0 0x7fc8f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc9117fa640 1 -- 172.21.15.12:0/2012640356 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fc900084860 con 0x7fc9140c27c0 2023-11-28T21:53:27.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.026+0000 7fc918c72640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc8f80664a0 0x7fc8f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:27.034 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.030+0000 7fc918c72640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc8f80664a0 0x7fc8f8068960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc908007fd0 tx=0x7fc908009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:27.037 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.030+0000 7fc9117fa640 1 -- 172.21.15.12:0/2012640356 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc900051660 con 0x7fc9140c27c0 2023-11-28T21:53:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 -- 172.21.15.12:0/2012640356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc91414e200 con 0x7fc9140c27c0 2023-11-28T21:53:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc9117fa640 1 -- 172.21.15.12:0/2012640356 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5725 (secure 0 0 0) 0x7fc900054c80 con 0x7fc9140c27c0 2023-11-28T21:53:27.228 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:27.228 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:53:21.833525+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 77 2023-11-28T21:53:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 -- 172.21.15.12:0/2012640356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc8f80664a0 msgr2=0x7fc8f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc8f80664a0 0x7fc8f8068960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc908007fd0 tx=0x7fc908009040 comp rx=0 tx=0).stop 2023-11-28T21:53:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 -- 172.21.15.12:0/2012640356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9140c27c0 msgr2=0x7fc914157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9140c27c0 0x7fc914157490 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e930 tx=0x7fc90000edf0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 -- 172.21.15.12:0/2012640356 shutdown_connections 2023-11-28T21:53:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc8f80664a0 0x7fc8f8068960 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9141500d0 0x7fc91415f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9140c27c0 0x7fc914157490 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 --2- 172.21.15.12:0/2012640356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc91406d7d0 0x7fc91415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 -- 172.21.15.12:0/2012640356 >> 172.21.15.12:0/2012640356 conn(0x7fc914066360 msgr2=0x7fc9140c9330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 -- 172.21.15.12:0/2012640356 shutdown_connections 2023-11-28T21:53:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:27.222+0000 7fc91aed6640 1 -- 172.21.15.12:0/2012640356 wait complete. 2023-11-28T21:53:27.249 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2023-11-28T21:53:29.250 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:53:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26ac5c4640 1 Processor -- start 2023-11-28T21:53:29.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26ac5c4640 1 -- start start 2023-11-28T21:53:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26ac5c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a4150c20 0x7f26a4156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26ac5c4640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f26a406e300 0x7f26a40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26ac5c4640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f26a406cf40 0x7f26a40c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:29.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26ac5c4640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a4067410 con 0x7f26a406e300 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26ac5c4640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a406e090 con 0x7f26a406cf40 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26ac5c4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a4157f30 con 0x7f26a4150c20 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26a9b5f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f26a406e300 0x7f26a40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26a9b5f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f26a406e300 0x7f26a40c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41630/0 (socket says 172.21.15.12:41630) 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26a9b5f640 1 -- 172.21.15.12:0/1970043472 learned_addr learned my addr 172.21.15.12:0/1970043472 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26aab61640 1 --2- 172.21.15.12:0/1970043472 >> v2:172.21.15.144:3300/0 conn(0x7f26a406cf40 0x7f26a40c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26aa360640 1 --2- 172.21.15.12:0/1970043472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a4150c20 0x7f26a4156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26a9b5f640 1 -- 172.21.15.12:0/1970043472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a4150c20 msgr2=0x7f26a4156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26a9b5f640 1 --2- 172.21.15.12:0/1970043472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a4150c20 0x7f26a4156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26a9b5f640 1 -- 172.21.15.12:0/1970043472 >> v2:172.21.15.144:3300/0 conn(0x7f26a406cf40 msgr2=0x7f26a40c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:53:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26a9b5f640 1 --2- 172.21.15.12:0/1970043472 >> v2:172.21.15.144:3300/0 conn(0x7f26a406cf40 0x7f26a40c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26a9b5f640 1 -- 172.21.15.12:0/1970043472 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a4151000 con 0x7f26a406e300 2023-11-28T21:53:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.394+0000 7f26aab61640 1 --2- 172.21.15.12:0/1970043472 >> v2:172.21.15.144:3300/0 conn(0x7f26a406cf40 0x7f26a40c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:53:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26a9b5f640 1 --2- 172.21.15.12:0/1970043472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406e300 0x7f26a40c6dd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f26a40c7d70 tx=0x7f269401e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fe9ff7f1788c1d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26a935e640 1 -- 172.21.15.12:0/1970043472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940aa610 con 0x7f26a406e300 2023-11-28T21:53:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26a935e640 1 -- 172.21.15.12:0/1970043472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26940aa770 con 0x7f26a406e300 2023-11-28T21:53:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26a935e640 1 -- 172.21.15.12:0/1970043472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940aab00 con 0x7f26a406e300 2023-11-28T21:53:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26ac5c4640 1 -- 172.21.15.12:0/1970043472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406e300 msgr2=0x7f26a40c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26ac5c4640 1 --2- 172.21.15.12:0/1970043472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406e300 0x7f26a40c6dd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f26a40c7d70 tx=0x7f269401e710 comp rx=0 tx=0).stop 2023-11-28T21:53:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26ac5c4640 1 -- 172.21.15.12:0/1970043472 shutdown_connections 2023-11-28T21:53:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26ac5c4640 1 --2- 172.21.15.12:0/1970043472 >> v2:172.21.15.144:3300/0 conn(0x7f26a406cf40 0x7f26a40c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26ac5c4640 1 --2- 172.21.15.12:0/1970043472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a406e300 0x7f26a40c6dd0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26ac5c4640 1 --2- 172.21.15.12:0/1970043472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a4150c20 0x7f26a4156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26ac5c4640 1 -- 172.21.15.12:0/1970043472 >> 172.21.15.12:0/1970043472 conn(0x7f26a4066360 msgr2=0x7f26a4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26ac5c4640 1 -- 172.21.15.12:0/1970043472 shutdown_connections 2023-11-28T21:53:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.398+0000 7f26ac5c4640 1 -- 172.21.15.12:0/1970043472 wait complete. 2023-11-28T21:53:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26ac5c4640 1 Processor -- start 2023-11-28T21:53:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26ac5c4640 1 -- start start 2023-11-28T21:53:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26ac5c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a406cf40 0x7f26a41f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26ac5c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406e300 0x7f26a4071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26ac5c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a4150c20 0x7f26a40c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26ac5c4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41f7110 con 0x7f26a4150c20 2023-11-28T21:53:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26ac5c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a41f35d0 con 0x7f26a406cf40 2023-11-28T21:53:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26ac5c4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a4072a40 con 0x7f26a406e300 2023-11-28T21:53:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26aab61640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a4150c20 0x7f26a40c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26aab61640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a4150c20 0x7f26a40c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41646/0 (socket says 172.21.15.12:41646) 2023-11-28T21:53:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26aab61640 1 -- 172.21.15.12:0/641836780 learned_addr learned my addr 172.21.15.12:0/641836780 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:53:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26aa360640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a406cf40 0x7f26a41f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26a9b5f640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406e300 0x7f26a4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26aab61640 1 -- 172.21.15.12:0/641836780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406e300 msgr2=0x7f26a4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26aab61640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406e300 0x7f26a4071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26aab61640 1 -- 172.21.15.12:0/641836780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a406cf40 msgr2=0x7f26a41f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26aab61640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a406cf40 0x7f26a41f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26aab61640 1 -- 172.21.15.12:0/641836780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940094d0 con 0x7f26a4150c20 2023-11-28T21:53:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26aab61640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a4150c20 0x7f26a40c5e90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f269c007e10 tx=0x7f269c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f269affd640 1 -- 172.21.15.12:0/641836780 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c003ba0 con 0x7f26a4150c20 2023-11-28T21:53:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f269affd640 1 -- 172.21.15.12:0/641836780 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c00f040 con 0x7f26a4150c20 2023-11-28T21:53:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f269affd640 1 -- 172.21.15.12:0/641836780 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00de80 con 0x7f26a4150c20 2023-11-28T21:53:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.402+0000 7f26ac5c4640 1 -- 172.21.15.12:0/641836780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a4072d00 con 0x7f26a4150c20 2023-11-28T21:53:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.406+0000 7f26ac5c4640 1 -- 172.21.15.12:0/641836780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a41f3260 con 0x7f26a4150c20 2023-11-28T21:53:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.406+0000 7f26ac5c4640 1 -- 172.21.15.12:0/641836780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a4074f60 con 0x7f26a4150c20 2023-11-28T21:53:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.406+0000 7f269affd640 1 -- 172.21.15.12:0/641836780 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f269c004030 con 0x7f26a4150c20 2023-11-28T21:53:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.406+0000 7f269affd640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f26880664a0 0x7f2688068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:53:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.406+0000 7f269affd640 1 -- 172.21.15.12:0/641836780 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f269c084830 con 0x7f26a4150c20 2023-11-28T21:53:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.406+0000 7f26aa360640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f26880664a0 0x7f2688068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:53:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.410+0000 7f26aa360640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f26880664a0 0x7f2688068960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f26a000b930 tx=0x7f26a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:53:29.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.410+0000 7f269affd640 1 -- 172.21.15.12:0/641836780 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f269c0515e0 con 0x7f26a4150c20 2023-11-28T21:53:29.610 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 -- 172.21.15.12:0/641836780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26a41f3710 con 0x7f26a4150c20 2023-11-28T21:53:29.611 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f269affd640 1 -- 172.21.15.12:0/641836780 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6519 (secure 0 0 0) 0x7f269c054c00 con 0x7f26a4150c20 2023-11-28T21:53:29.611 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:53:29.611 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:53:21.833525+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:53:29.612 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 78 2023-11-28T21:53:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 -- 172.21.15.12:0/641836780 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f26880664a0 msgr2=0x7f2688068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f26880664a0 0x7f2688068960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f26a000b930 tx=0x7f26a0008040 comp rx=0 tx=0).stop 2023-11-28T21:53:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 -- 172.21.15.12:0/641836780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a4150c20 msgr2=0x7f26a40c5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:53:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a4150c20 0x7f26a40c5e90 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f269c007e10 tx=0x7f269c00d4a0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 -- 172.21.15.12:0/641836780 shutdown_connections 2023-11-28T21:53:29.613 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f26880664a0 0x7f2688068960 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f26a4150c20 0x7f26a40c5e90 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26a406e300 0x7f26a4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 --2- 172.21.15.12:0/641836780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26a406cf40 0x7f26a41f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:53:29.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 -- 172.21.15.12:0/641836780 >> 172.21.15.12:0/641836780 conn(0x7f26a4066360 msgr2=0x7f26a4154a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:53:29.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 -- 172.21.15.12:0/641836780 shutdown_connections 2023-11-28T21:53:29.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:53:29.606+0000 7f26ac5c4640 1 -- 172.21.15.12:0/641836780 wait complete. 2023-11-28T21:53:29.639 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2023-11-28T21:53:29.640 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 116 secs before thrashing 2023-11-28T21:53:32.611 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-11-28T21:53:32.606+0000 7f959cbc2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T21:53:32.611 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-11-28T21:53:32.606+0000 7f959cbc2640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T21:53:46.600 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:53:46.606 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:54:16.663 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:54:16.667 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:54:46.794 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:54:46.798 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:55:16.874 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:55:16.878 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:55:25.641 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:25.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca38565640 1 Processor -- start 2023-11-28T21:55:25.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca38565640 1 -- start start 2023-11-28T21:55:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca38565640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fca3006b130 0x7fca300c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca38565640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fca300cb020 0x7fca30159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:25.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca38565640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3015a5e0 con 0x7fca300cac40 2023-11-28T21:55:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca38565640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3006b550 con 0x7fca300cb020 2023-11-28T21:55:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca38565640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3015a8c0 con 0x7fca3006b130 2023-11-28T21:55:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca36301640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fca300cac40 0x7fca300ca030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60828/0 (socket says 172.21.15.12:60828) 2023-11-28T21:55:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca36301640 1 -- 172.21.15.12:0/408152621 learned_addr learned my addr 172.21.15.12:0/408152621 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:55:25.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca36b02640 1 --2- 172.21.15.12:0/408152621 >> v2:172.21.15.12:3300/0 conn(0x7fca300cb020 0x7fca30159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca35b00640 1 --2- 172.21.15.12:0/408152621 >> v2:172.21.15.144:3300/0 conn(0x7fca3006b130 0x7fca300c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca36b02640 1 -- 172.21.15.12:0/408152621 >> v2:172.21.15.144:3300/0 conn(0x7fca3006b130 msgr2=0x7fca300c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:25.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca36b02640 1 --2- 172.21.15.12:0/408152621 >> v2:172.21.15.144:3300/0 conn(0x7fca3006b130 0x7fca300c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca36b02640 1 -- 172.21.15.12:0/408152621 >> v1:172.21.15.12:6789/0 conn(0x7fca300cac40 legacy=0x7fca300ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca36b02640 1 -- 172.21.15.12:0/408152621 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3006a650 con 0x7fca300cb020 2023-11-28T21:55:25.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.781+0000 7fca352ff640 1 -- 172.21.15.12:0/408152621 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804770555 0 0) 0x7fca3015a5e0 con 0x7fca300cac40 2023-11-28T21:55:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca36b02640 1 --2- 172.21.15.12:0/408152621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca300cb020 0x7fca30159d50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fca300c9cf0 tx=0x7fca2801f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f9bec3b4d295db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca352ff640 1 -- 172.21.15.12:0/408152621 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280b0440 con 0x7fca300cb020 2023-11-28T21:55:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca352ff640 1 -- 172.21.15.12:0/408152621 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca280b05a0 con 0x7fca300cb020 2023-11-28T21:55:25.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca352ff640 1 -- 172.21.15.12:0/408152621 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280b0930 con 0x7fca300cb020 2023-11-28T21:55:25.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca38565640 1 -- 172.21.15.12:0/408152621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca300cb020 msgr2=0x7fca30159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:25.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca38565640 1 --2- 172.21.15.12:0/408152621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca300cb020 0x7fca30159d50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fca300c9cf0 tx=0x7fca2801f0f0 comp rx=0 tx=0).stop 2023-11-28T21:55:25.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca38565640 1 -- 172.21.15.12:0/408152621 shutdown_connections 2023-11-28T21:55:25.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca38565640 1 --2- 172.21.15.12:0/408152621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca300cb020 0x7fca30159d50 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:25.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca38565640 1 --2- 172.21.15.12:0/408152621 >> v2:172.21.15.144:3300/0 conn(0x7fca3006b130 0x7fca300c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:25.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca38565640 1 -- 172.21.15.12:0/408152621 >> 172.21.15.12:0/408152621 conn(0x7fca300720e0 msgr2=0x7fca300724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:25.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca38565640 1 -- 172.21.15.12:0/408152621 shutdown_connections 2023-11-28T21:55:25.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.785+0000 7fca38565640 1 -- 172.21.15.12:0/408152621 wait complete. 2023-11-28T21:55:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca38565640 1 Processor -- start 2023-11-28T21:55:25.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca38565640 1 -- start start 2023-11-28T21:55:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca38565640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca3006b130 0x7fca3014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca38565640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca300cac40 0x7fca3015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca38565640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca300cb020 0x7fca3015b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca38565640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3014fb50 con 0x7fca3006b130 2023-11-28T21:55:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca38565640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3015b420 con 0x7fca300cac40 2023-11-28T21:55:25.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca38565640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3014f020 con 0x7fca300cb020 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36301640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca3006b130 0x7fca3014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36301640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca3006b130 0x7fca3014eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52642/0 (socket says 172.21.15.12:52642) 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36301640 1 -- 172.21.15.12:0/10809898 learned_addr learned my addr 172.21.15.12:0/10809898 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca35b00640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca300cac40 0x7fca3015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36b02640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca300cb020 0x7fca3015b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36301640 1 -- 172.21.15.12:0/10809898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca300cb020 msgr2=0x7fca3015b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36301640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca300cb020 0x7fca3015b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36301640 1 -- 172.21.15.12:0/10809898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca300cac40 msgr2=0x7fca3015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36301640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca300cac40 0x7fca3015db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36301640 1 -- 172.21.15.12:0/10809898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca28009a70 con 0x7fca3006b130 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36b02640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca300cb020 0x7fca3015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:55:25.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca36301640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca3006b130 0x7fca3014eae0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c520 tx=0x7fca2c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca26ffd640 1 -- 172.21.15.12:0/10809898 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0074e0 con 0x7fca3006b130 2023-11-28T21:55:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca26ffd640 1 -- 172.21.15.12:0/10809898 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c007640 con 0x7fca3006b130 2023-11-28T21:55:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca26ffd640 1 -- 172.21.15.12:0/10809898 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0049f0 con 0x7fca3006b130 2023-11-28T21:55:25.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca38565640 1 -- 172.21.15.12:0/10809898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3014f2b0 con 0x7fca3006b130 2023-11-28T21:55:25.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.789+0000 7fca38565640 1 -- 172.21.15.12:0/10809898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3014f870 con 0x7fca3006b130 2023-11-28T21:55:25.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.793+0000 7fca38565640 1 -- 172.21.15.12:0/10809898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca30067c70 con 0x7fca3006b130 2023-11-28T21:55:25.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.793+0000 7fca26ffd640 1 -- 172.21.15.12:0/10809898 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fca2c004b50 con 0x7fca3006b130 2023-11-28T21:55:25.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.793+0000 7fca26ffd640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fca140664a0 0x7fca14068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.793+0000 7fca26ffd640 1 -- 172.21.15.12:0/10809898 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7fca2c083770 con 0x7fca3006b130 2023-11-28T21:55:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.793+0000 7fca35b00640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fca140664a0 0x7fca14068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:25.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.793+0000 7fca35b00640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fca140664a0 0x7fca14068960 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fca300b42d0 tx=0x7fca20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:25.804 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:25.797+0000 7fca26ffd640 1 -- 172.21.15.12:0/10809898 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fca2c050550 con 0x7fca3006b130 2023-11-28T21:55:26.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.253+0000 7fca38565640 1 -- 172.21.15.12:0/10809898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca30074f60 con 0x7fca3006b130 2023-11-28T21:55:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:26.281 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:22.698388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:26.282 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-11-28T21:55:26.282 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.269+0000 7fca26ffd640 1 -- 172.21.15.12:0/10809898 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6521 (secure 0 0 0) 0x7fca2c018020 con 0x7fca3006b130 2023-11-28T21:55:26.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.277+0000 7fca38565640 1 -- 172.21.15.12:0/10809898 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fca140664a0 msgr2=0x7fca14068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:26.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.277+0000 7fca38565640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fca140664a0 0x7fca14068960 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fca300b42d0 tx=0x7fca20008040 comp rx=0 tx=0).stop 2023-11-28T21:55:26.284 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.277+0000 7fca38565640 1 -- 172.21.15.12:0/10809898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca3006b130 msgr2=0x7fca3014eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:26.285 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.277+0000 7fca38565640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca3006b130 0x7fca3014eae0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fca2c00c520 tx=0x7fca2c00c9e0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.289+0000 7fca38565640 1 -- 172.21.15.12:0/10809898 shutdown_connections 2023-11-28T21:55:26.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.289+0000 7fca38565640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fca140664a0 0x7fca14068960 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.289+0000 7fca38565640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca300cb020 0x7fca3015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.289+0000 7fca38565640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca300cac40 0x7fca3015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.289+0000 7fca38565640 1 --2- 172.21.15.12:0/10809898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fca3006b130 0x7fca3014eae0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.289+0000 7fca38565640 1 -- 172.21.15.12:0/10809898 >> 172.21.15.12:0/10809898 conn(0x7fca300720e0 msgr2=0x7fca300b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:26.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.297+0000 7fca38565640 1 -- 172.21.15.12:0/10809898 shutdown_connections 2023-11-28T21:55:26.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.301+0000 7fca38565640 1 -- 172.21.15.12:0/10809898 wait complete. 2023-11-28T21:55:26.383 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=1) 2023-11-28T21:55:26.383 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-11-28T21:55:26.383 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:55:26.466 INFO:tasks.ceph.mds.e:Stopped 2023-11-28T21:55:26.466 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:26.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8c6cfe640 1 Processor -- start 2023-11-28T21:55:26.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8c6cfe640 1 -- start start 2023-11-28T21:55:26.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8c6cfe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff8c015a910 0x7ff8c0070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:26.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8c6cfe640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c006f050 con 0x7ff8c0143070 2023-11-28T21:55:26.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8c6cfe640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c006f580 con 0x7ff8c00710e0 2023-11-28T21:55:26.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8c6cfe640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c015e330 con 0x7ff8c015a910 2023-11-28T21:55:26.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8c4a9a640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff8c0143070 0x7ff8c015b340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60844/0 (socket says 172.21.15.12:60844) 2023-11-28T21:55:26.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8c4a9a640 1 -- 172.21.15.12:0/1772661664 learned_addr learned my addr 172.21.15.12:0/1772661664 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:55:26.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bffff640 1 --2- 172.21.15.12:0/1772661664 >> v2:172.21.15.144:3300/0 conn(0x7ff8c015a910 0x7ff8c0070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:26.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39851520 0 0) 0x7ff8c006f050 con 0x7ff8c0143070 2023-11-28T21:55:26.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b0003530 con 0x7ff8c0143070 2023-11-28T21:55:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655750585 0 0) 0x7ff8c006f580 con 0x7ff8c00710e0 2023-11-28T21:55:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c006f050 con 0x7ff8c00710e0 2023-11-28T21:55:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 479577496 0 0) 0x7ff8b0003530 con 0x7ff8c0143070 2023-11-28T21:55:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 >> v2:172.21.15.144:3300/0 conn(0x7ff8c015a910 msgr2=0x7ff8c0070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 --2- 172.21.15.12:0/1772661664 >> v2:172.21.15.144:3300/0 conn(0x7ff8c015a910 0x7ff8c0070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 >> v1:172.21.15.144:6789/0 conn(0x7ff8c00710e0 legacy=0x7ff8c00714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c006f2a0 con 0x7ff8c0143070 2023-11-28T21:55:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7ff8b4001680 con 0x7ff8c0143070 2023-11-28T21:55:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 >> v1:172.21.15.12:6789/0 conn(0x7ff8c0143070 legacy=0x7ff8c015b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:26.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b0003b80 0x7ff8b0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b00047d0 0x7ff8b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b0005190 0x7ff8b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c015e330 con 0x7ff8b00047d0 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b000c090 con 0x7ff8b0003b80 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b000c250 con 0x7ff8b0005190 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bffff640 1 --2- 172.21.15.12:0/1772661664 >> v2:172.21.15.144:3300/0 conn(0x7ff8c015a910 0x7ff8c0070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8c4a9a640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b0003b80 0x7ff8b0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8bffff640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b00047d0 0x7ff8b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.757+0000 7ff8c529b640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b0005190 0x7ff8b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8bffff640 1 -- 172.21.15.12:0/1772661664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b0005190 msgr2=0x7ff8b000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8bffff640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b0005190 0x7ff8b000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8bffff640 1 -- 172.21.15.12:0/1772661664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b0003b80 msgr2=0x7ff8b0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8bffff640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b0003b80 0x7ff8b0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8bffff640 1 -- 172.21.15.12:0/1772661664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b0003530 con 0x7ff8b00047d0 2023-11-28T21:55:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8c4a9a640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b0003b80 0x7ff8b0003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:55:26.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8c529b640 1 -- 172.21.15.12:0/1772661664 reap_dead start 2023-11-28T21:55:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8bffff640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b00047d0 0x7ff8b0004c50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009c90 tx=0x7ff8ac01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e004be81318fda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac004020 con 0x7ff8b00047d0 2023-11-28T21:55:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac004180 con 0x7ff8b00047d0 2023-11-28T21:55:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8bf7fe640 1 -- 172.21.15.12:0/1772661664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0a5780 con 0x7ff8b00047d0 2023-11-28T21:55:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/1772661664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b00047d0 msgr2=0x7ff8b0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.761+0000 7ff8c6cfe640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b00047d0 0x7ff8b0004c50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009c90 tx=0x7ff8ac01cbd0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/1772661664 shutdown_connections 2023-11-28T21:55:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 --2- 172.21.15.12:0/1772661664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b00047d0 0x7ff8b0004c50 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/1772661664 >> 172.21.15.12:0/1772661664 conn(0x7ff8c006b6c0 msgr2=0x7ff8c006bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:26.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/1772661664 shutdown_connections 2023-11-28T21:55:26.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/1772661664 wait complete. 2023-11-28T21:55:26.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 Processor -- start 2023-11-28T21:55:26.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 -- start start 2023-11-28T21:55:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00047d0 0x7ff8c013f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c01572c0 0x7ff8c0154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8c0155440 0x7ff8c00c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c013f820 con 0x7ff8c01572c0 2023-11-28T21:55:26.773 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00c6d90 con 0x7ff8b00047d0 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c6cfe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0154a70 con 0x7ff8c0155440 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c01572c0 0x7ff8c0154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c01572c0 0x7ff8c0154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52654/0 (socket says 172.21.15.12:52654) 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8bffff640 1 -- 172.21.15.12:0/3004620659 learned_addr learned my addr 172.21.15.12:0/3004620659 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c4a9a640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00047d0 0x7ff8c013f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c529b640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8c0155440 0x7ff8c00c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8bffff640 1 -- 172.21.15.12:0/3004620659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8c0155440 msgr2=0x7ff8c00c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8bffff640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8c0155440 0x7ff8c00c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8bffff640 1 -- 172.21.15.12:0/3004620659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00047d0 msgr2=0x7ff8c013f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8bffff640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00047d0 0x7ff8c013f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8bffff640 1 -- 172.21.15.12:0/3004620659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c01558d0 con 0x7ff8c01572c0 2023-11-28T21:55:26.774 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.765+0000 7ff8c529b640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8c0155440 0x7ff8c00c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:55:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8bffff640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c01572c0 0x7ff8c0154f00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0af040 tx=0x7ff8b0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8bd7fa640 1 -- 172.21.15.12:0/3004620659 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0bb070 con 0x7ff8c01572c0 2023-11-28T21:55:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8bd7fa640 1 -- 172.21.15.12:0/3004620659 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac0a5780 con 0x7ff8c01572c0 2023-11-28T21:55:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8bd7fa640 1 -- 172.21.15.12:0/3004620659 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0a5af0 con 0x7ff8c01572c0 2023-11-28T21:55:26.775 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/3004620659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c01eff90 con 0x7ff8c01572c0 2023-11-28T21:55:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/3004620659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c01566d0 con 0x7ff8c01572c0 2023-11-28T21:55:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/3004620659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c0154c20 con 0x7ff8c01572c0 2023-11-28T21:55:26.776 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8bd7fa640 1 -- 172.21.15.12:0/3004620659 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff8ac0a5c50 con 0x7ff8c01572c0 2023-11-28T21:55:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8bd7fa640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff8a4066470 0x7ff8a4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8bd7fa640 1 -- 172.21.15.12:0/3004620659 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7ff8ac11e6c0 con 0x7ff8c01572c0 2023-11-28T21:55:26.777 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.769+0000 7ff8c4a9a640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff8a4066470 0x7ff8a4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:26.778 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.773+0000 7ff8c4a9a640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff8a4066470 0x7ff8a4068930 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4007fd0 tx=0x7ff8b4001e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:26.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.773+0000 7ff8bd7fa640 1 -- 172.21.15.12:0/3004620659 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff8c0154c20 con 0x7ff8c01572c0 2023-11-28T21:55:26.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/3004620659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8c0154c20 con 0x7ff8c01572c0 2023-11-28T21:55:26.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8bd7fa640 1 -- 172.21.15.12:0/3004620659 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6521 (secure 0 0 0) 0x7ff8c0154c20 con 0x7ff8c01572c0 2023-11-28T21:55:26.975 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:26.975 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:22.698388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:26.975 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-11-28T21:55:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/3004620659 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff8a4066470 msgr2=0x7ff8a4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff8a4066470 0x7ff8a4068930 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4007fd0 tx=0x7ff8b4001e30 comp rx=0 tx=0).stop 2023-11-28T21:55:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/3004620659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c01572c0 msgr2=0x7ff8c0154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c01572c0 0x7ff8c0154f00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0af040 tx=0x7ff8b0005800 comp rx=0 tx=0).stop 2023-11-28T21:55:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/3004620659 shutdown_connections 2023-11-28T21:55:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff8a4066470 0x7ff8a4068930 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8c0155440 0x7ff8c00c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8c01572c0 0x7ff8c0154f00 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 --2- 172.21.15.12:0/3004620659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00047d0 0x7ff8c013f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:26.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/3004620659 >> 172.21.15.12:0/3004620659 conn(0x7ff8c006b6c0 msgr2=0x7ff8c00ca2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:26.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/3004620659 shutdown_connections 2023-11-28T21:55:26.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:26.969+0000 7ff8c6cfe640 1 -- 172.21.15.12:0/3004620659 wait complete. 2023-11-28T21:55:26.996 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:55:28.997 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:29.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d57f01640 1 Processor -- start 2023-11-28T21:55:29.152 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d57f01640 1 -- start start 2023-11-28T21:55:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d57f01640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d50158420 0x7f5d5006d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d57f01640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5d50161600 0x7f5d50161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d57f01640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d500c3460 con 0x7f5d500c38f0 2023-11-28T21:55:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d57f01640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d50158800 con 0x7f5d50158420 2023-11-28T21:55:29.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d57f01640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d500c52c0 con 0x7f5d50161600 2023-11-28T21:55:29.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d55c9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d50158420 0x7f5d5006d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:29.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d55c9d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d50158420 0x7f5d5006d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52656/0 (socket says 172.21.15.12:52656) 2023-11-28T21:55:29.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d55c9d640 1 -- 172.21.15.12:0/172035329 learned_addr learned my addr 172.21.15.12:0/172035329 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:29.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d5549c640 1 --1- 172.21.15.12:0/172035329 >> v1:172.21.15.12:6789/0 conn(0x7f5d500c38f0 0x7f5d500c3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60858/0 (socket says 172.21.15.12:60858) 2023-11-28T21:55:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.145+0000 7f5d5649e640 1 --2- 172.21.15.12:0/172035329 >> v2:172.21.15.144:3300/0 conn(0x7f5d50161600 0x7f5d50161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d55c9d640 1 -- 172.21.15.12:0/172035329 >> v2:172.21.15.144:3300/0 conn(0x7f5d50161600 msgr2=0x7f5d50161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d55c9d640 1 --2- 172.21.15.12:0/172035329 >> v2:172.21.15.144:3300/0 conn(0x7f5d50161600 0x7f5d50161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d55c9d640 1 -- 172.21.15.12:0/172035329 >> v1:172.21.15.12:6789/0 conn(0x7f5d500c38f0 legacy=0x7f5d500c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d55c9d640 1 -- 172.21.15.12:0/172035329 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500c2b70 con 0x7f5d50158420 2023-11-28T21:55:29.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d5649e640 1 --2- 172.21.15.12:0/172035329 >> v2:172.21.15.144:3300/0 conn(0x7f5d50161600 0x7f5d50161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:55:29.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d55c9d640 1 --2- 172.21.15.12:0/172035329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 0x7f5d5006d950 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5d500c5510 tx=0x7f5d4c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e6c25d782ba2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d54c9b640 1 -- 172.21.15.12:0/172035329 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0ab600 con 0x7f5d50158420 2023-11-28T21:55:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d54c9b640 1 -- 172.21.15.12:0/172035329 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c0a7050 con 0x7f5d50158420 2023-11-28T21:55:29.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d54c9b640 1 -- 172.21.15.12:0/172035329 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0ab980 con 0x7f5d50158420 2023-11-28T21:55:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d57f01640 1 -- 172.21.15.12:0/172035329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 msgr2=0x7f5d5006d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d57f01640 1 --2- 172.21.15.12:0/172035329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 0x7f5d5006d950 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5d500c5510 tx=0x7f5d4c01c500 comp rx=0 tx=0).stop 2023-11-28T21:55:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d57f01640 1 -- 172.21.15.12:0/172035329 shutdown_connections 2023-11-28T21:55:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d57f01640 1 --2- 172.21.15.12:0/172035329 >> v2:172.21.15.144:3300/0 conn(0x7f5d50161600 0x7f5d50161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d57f01640 1 --2- 172.21.15.12:0/172035329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 0x7f5d5006d950 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.149+0000 7f5d57f01640 1 -- 172.21.15.12:0/172035329 >> 172.21.15.12:0/172035329 conn(0x7f5d50066360 msgr2=0x7f5d50066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d57f01640 1 -- 172.21.15.12:0/172035329 shutdown_connections 2023-11-28T21:55:29.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d57f01640 1 -- 172.21.15.12:0/172035329 wait complete. 2023-11-28T21:55:29.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d57f01640 1 Processor -- start 2023-11-28T21:55:29.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d57f01640 1 -- start start 2023-11-28T21:55:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d57f01640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d500c38f0 0x7f5d50159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d57f01640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 0x7f5d50155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d57f01640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d50161600 0x7f5d50159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d57f01640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50156950 con 0x7f5d50158420 2023-11-28T21:55:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d57f01640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50156600 con 0x7f5d500c38f0 2023-11-28T21:55:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d57f01640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5015b010 con 0x7f5d50161600 2023-11-28T21:55:29.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d5549c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 0x7f5d50155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d5549c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 0x7f5d50155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52664/0 (socket says 172.21.15.12:52664) 2023-11-28T21:55:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d5549c640 1 -- 172.21.15.12:0/3001163482 learned_addr learned my addr 172.21.15.12:0/3001163482 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d5649e640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d50161600 0x7f5d50159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d55c9d640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d500c38f0 0x7f5d50159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d5549c640 1 -- 172.21.15.12:0/3001163482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d50161600 msgr2=0x7f5d50159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d5549c640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d50161600 0x7f5d50159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d5549c640 1 -- 172.21.15.12:0/3001163482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d500c38f0 msgr2=0x7f5d50159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d5549c640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d500c38f0 0x7f5d50159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d5549c640 1 -- 172.21.15.12:0/3001163482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0094d0 con 0x7f5d50158420 2023-11-28T21:55:29.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d5549c640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 0x7f5d50155ab0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c490 tx=0x7f5d4000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d467fc640 1 -- 172.21.15.12:0/3001163482 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40007480 con 0x7f5d50158420 2023-11-28T21:55:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d467fc640 1 -- 172.21.15.12:0/3001163482 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d400075e0 con 0x7f5d50158420 2023-11-28T21:55:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.153+0000 7f5d467fc640 1 -- 172.21.15.12:0/3001163482 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40004a10 con 0x7f5d50158420 2023-11-28T21:55:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.157+0000 7f5d57f01640 1 -- 172.21.15.12:0/3001163482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5015b2f0 con 0x7f5d50158420 2023-11-28T21:55:29.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.157+0000 7f5d57f01640 1 -- 172.21.15.12:0/3001163482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5015b8b0 con 0x7f5d50158420 2023-11-28T21:55:29.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.157+0000 7f5d57f01640 1 -- 172.21.15.12:0/3001163482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d50156b00 con 0x7f5d50158420 2023-11-28T21:55:29.164 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.157+0000 7f5d467fc640 1 -- 172.21.15.12:0/3001163482 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5d40004b70 con 0x7f5d50158420 2023-11-28T21:55:29.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.157+0000 7f5d467fc640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5d34066470 0x7f5d34068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:29.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.157+0000 7f5d467fc640 1 -- 172.21.15.12:0/3001163482 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f5d40083700 con 0x7f5d50158420 2023-11-28T21:55:29.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.157+0000 7f5d55c9d640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5d34066470 0x7f5d34068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:29.165 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.161+0000 7f5d55c9d640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5d34066470 0x7f5d34068930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0ad000 tx=0x7f5d4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:29.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.161+0000 7f5d467fc640 1 -- 172.21.15.12:0/3001163482 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5d400504b0 con 0x7f5d50158420 2023-11-28T21:55:29.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.353+0000 7f5d57f01640 1 -- 172.21.15.12:0/3001163482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d50156770 con 0x7f5d50158420 2023-11-28T21:55:29.360 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.353+0000 7f5d467fc640 1 -- 172.21.15.12:0/3001163482 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6521 (secure 0 0 0) 0x7f5d40053ad0 con 0x7f5d50158420 2023-11-28T21:55:29.361 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:29.361 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:22.698388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:29.361 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-11-28T21:55:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 -- 172.21.15.12:0/3001163482 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5d34066470 msgr2=0x7f5d34068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5d34066470 0x7f5d34068930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0ad000 tx=0x7f5d4c008040 comp rx=0 tx=0).stop 2023-11-28T21:55:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 -- 172.21.15.12:0/3001163482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 msgr2=0x7f5d50155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 0x7f5d50155ab0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c490 tx=0x7f5d4000c950 comp rx=0 tx=0).stop 2023-11-28T21:55:29.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 -- 172.21.15.12:0/3001163482 shutdown_connections 2023-11-28T21:55:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5d34066470 0x7f5d34068930 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d50161600 0x7f5d50159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d50158420 0x7f5d50155ab0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 --2- 172.21.15.12:0/3001163482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d500c38f0 0x7f5d50159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 -- 172.21.15.12:0/3001163482 >> 172.21.15.12:0/3001163482 conn(0x7f5d50066360 msgr2=0x7f5d500c3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 -- 172.21.15.12:0/3001163482 shutdown_connections 2023-11-28T21:55:29.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:29.357+0000 7f5d57f01640 1 -- 172.21.15.12:0/3001163482 wait complete. 2023-11-28T21:55:29.382 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:55:31.383 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:31.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.517+0000 7f9a0aa59640 1 Processor -- start 2023-11-28T21:55:31.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a0aa59640 1 -- start start 2023-11-28T21:55:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a0aa59640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9a04153fb0 0x7f9a040ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a0aa59640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 0x7f9a040caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a0aa59640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0415d390 con 0x7f9a0415a980 2023-11-28T21:55:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a0aa59640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040b2b60 con 0x7f9a04153fb0 2023-11-28T21:55:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a0aa59640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0415e1b0 con 0x7f9a040b41e0 2023-11-28T21:55:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a09a57640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9a04153fb0 0x7f9a040ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a09a57640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9a04153fb0 0x7f9a040ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52684/0 (socket says 172.21.15.12:52684) 2023-11-28T21:55:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a09a57640 1 -- 172.21.15.12:0/246407874 learned_addr learned my addr 172.21.15.12:0/246407874 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a0a258640 1 --2- 172.21.15.12:0/246407874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 0x7f9a040caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a09a57640 1 -- 172.21.15.12:0/246407874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 msgr2=0x7f9a040caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:31.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a09a57640 1 --2- 172.21.15.12:0/246407874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 0x7f9a040caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:31.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a09a57640 1 -- 172.21.15.12:0/246407874 >> v1:172.21.15.144:6789/0 conn(0x7f9a0415a980 legacy=0x7f9a040b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:31.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.521+0000 7f9a09a57640 1 -- 172.21.15.12:0/246407874 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0415d630 con 0x7f9a04153fb0 2023-11-28T21:55:31.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a09a57640 1 --2- 172.21.15.12:0/246407874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a04153fb0 0x7f9a040ca4f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009870 tx=0x7f9a0001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784534632dd1d2e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:31.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a08a55640 1 -- 172.21.15.12:0/246407874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a000aa670 con 0x7f9a04153fb0 2023-11-28T21:55:31.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a08a55640 1 -- 172.21.15.12:0/246407874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a000aa7d0 con 0x7f9a04153fb0 2023-11-28T21:55:31.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a08a55640 1 -- 172.21.15.12:0/246407874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a000aab60 con 0x7f9a04153fb0 2023-11-28T21:55:31.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a0aa59640 1 -- 172.21.15.12:0/246407874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a04153fb0 msgr2=0x7f9a040ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:31.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a0aa59640 1 --2- 172.21.15.12:0/246407874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a04153fb0 0x7f9a040ca4f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009870 tx=0x7f9a0001c510 comp rx=0 tx=0).stop 2023-11-28T21:55:31.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a0aa59640 1 -- 172.21.15.12:0/246407874 shutdown_connections 2023-11-28T21:55:31.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a0aa59640 1 --2- 172.21.15.12:0/246407874 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 0x7f9a040caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:31.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a0aa59640 1 --2- 172.21.15.12:0/246407874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a04153fb0 0x7f9a040ca4f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:31.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a0aa59640 1 -- 172.21.15.12:0/246407874 >> 172.21.15.12:0/246407874 conn(0x7f9a04071ce0 msgr2=0x7f9a040720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:31.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a0aa59640 1 -- 172.21.15.12:0/246407874 shutdown_connections 2023-11-28T21:55:31.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a0aa59640 1 -- 172.21.15.12:0/246407874 wait complete. 2023-11-28T21:55:31.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a0aa59640 1 Processor -- start 2023-11-28T21:55:31.533 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.525+0000 7f9a0aa59640 1 -- start start 2023-11-28T21:55:31.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0aa59640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 0x7f9a0415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:31.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0aa59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a04153fb0 0x7f9a041f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:31.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0aa59640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a0415a980 0x7f9a041f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:31.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0aa59640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0415bcc0 con 0x7f9a0415a980 2023-11-28T21:55:31.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0aa59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041575a0 con 0x7f9a04153fb0 2023-11-28T21:55:31.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0aa59640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040c55d0 con 0x7f9a040b41e0 2023-11-28T21:55:31.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0a258640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a0415a980 0x7f9a041f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0a258640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a0415a980 0x7f9a041f6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52698/0 (socket says 172.21.15.12:52698) 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0a258640 1 -- 172.21.15.12:0/3248457173 learned_addr learned my addr 172.21.15.12:0/3248457173 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a09a57640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 0x7f9a0415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a09256640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a04153fb0 0x7f9a041f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0a258640 1 -- 172.21.15.12:0/3248457173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 msgr2=0x7f9a0415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0a258640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 0x7f9a0415b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0a258640 1 -- 172.21.15.12:0/3248457173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a04153fb0 msgr2=0x7f9a041f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0a258640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a04153fb0 0x7f9a041f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0a258640 1 -- 172.21.15.12:0/3248457173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000094d0 con 0x7f9a0415a980 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a09a57640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 0x7f9a0415b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:55:31.536 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0a258640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a0415a980 0x7f9a041f6b30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f99fc007e40 tx=0x7f99fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:31.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f99fa7fc640 1 -- 172.21.15.12:0/3248457173 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc014620 con 0x7f9a0415a980 2023-11-28T21:55:31.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f99fa7fc640 1 -- 172.21.15.12:0/3248457173 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc00ee90 con 0x7f9a0415a980 2023-11-28T21:55:31.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0aa59640 1 -- 172.21.15.12:0/3248457173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a040c58b0 con 0x7f9a0415a980 2023-11-28T21:55:31.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f99fa7fc640 1 -- 172.21.15.12:0/3248457173 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0149e0 con 0x7f9a0415a980 2023-11-28T21:55:31.537 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0aa59640 1 -- 172.21.15.12:0/3248457173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a040c5e70 con 0x7f9a0415a980 2023-11-28T21:55:31.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f9a0aa59640 1 -- 172.21.15.12:0/3248457173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a04074f60 con 0x7f9a0415a980 2023-11-28T21:55:31.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.529+0000 7f99fa7fc640 1 -- 172.21.15.12:0/3248457173 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f99fc014b40 con 0x7f9a0415a980 2023-11-28T21:55:31.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.533+0000 7f99fa7fc640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99e80664a0 0x7f99e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:31.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.533+0000 7f99fa7fc640 1 -- 172.21.15.12:0/3248457173 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f99fc084f10 con 0x7f9a0415a980 2023-11-28T21:55:31.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.533+0000 7f9a09a57640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99e80664a0 0x7f99e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:31.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.533+0000 7f9a09a57640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99e80664a0 0x7f99e8068960 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9a000ae000 tx=0x7f9a000a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:31.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.537+0000 7f99fa7fc640 1 -- 172.21.15.12:0/3248457173 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f99fc051c30 con 0x7f9a0415a980 2023-11-28T21:55:31.733 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.725+0000 7f9a0aa59640 1 -- 172.21.15.12:0/3248457173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a041f6350 con 0x7f9a0415a980 2023-11-28T21:55:31.734 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f99fa7fc640 1 -- 172.21.15.12:0/3248457173 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6521 (secure 0 0 0) 0x7f99fc055250 con 0x7f9a0415a980 2023-11-28T21:55:31.734 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:31.734 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:22.698388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:31.735 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-11-28T21:55:31.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 -- 172.21.15.12:0/3248457173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99e80664a0 msgr2=0x7f99e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:31.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99e80664a0 0x7f99e8068960 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9a000ae000 tx=0x7f9a000a7000 comp rx=0 tx=0).stop 2023-11-28T21:55:31.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 -- 172.21.15.12:0/3248457173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a0415a980 msgr2=0x7f9a041f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:31.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a0415a980 0x7f9a041f6b30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f99fc007e40 tx=0x7f99fc00e4b0 comp rx=0 tx=0).stop 2023-11-28T21:55:31.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 -- 172.21.15.12:0/3248457173 shutdown_connections 2023-11-28T21:55:31.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f99e80664a0 0x7f99e8068960 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:31.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9a0415a980 0x7f9a041f6b30 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:31.736 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a04153fb0 0x7f9a041f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:31.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 --2- 172.21.15.12:0/3248457173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a040b41e0 0x7f9a0415b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:31.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 -- 172.21.15.12:0/3248457173 >> 172.21.15.12:0/3248457173 conn(0x7f9a04071ce0 msgr2=0x7f9a040cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:31.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 -- 172.21.15.12:0/3248457173 shutdown_connections 2023-11-28T21:55:31.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:31.729+0000 7f9a0aa59640 1 -- 172.21.15.12:0/3248457173 wait complete. 2023-11-28T21:55:31.755 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:55:33.755 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.905+0000 7f2ad5f3e640 1 Processor -- start 2023-11-28T21:55:33.912 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.905+0000 7f2ad5f3e640 1 -- start start 2023-11-28T21:55:33.913 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.905+0000 7f2ad5f3e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2ad006e300 0x7f2ad0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.905+0000 7f2ad5f3e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad0067410 con 0x7f2ad006cf40 2023-11-28T21:55:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.905+0000 7f2ad5f3e640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad006e090 con 0x7f2ad0150c20 2023-11-28T21:55:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.905+0000 7f2ad5f3e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad0157f30 con 0x7f2ad006e300 2023-11-28T21:55:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2acffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2ad006cf40 0x7f2ad0158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60862/0 (socket says 172.21.15.12:60862) 2023-11-28T21:55:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2acffff640 1 -- 172.21.15.12:0/1848086117 learned_addr learned my addr 172.21.15.12:0/1848086117 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:55:33.914 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2aceffd640 1 --2- 172.21.15.12:0/1848086117 >> v2:172.21.15.144:3300/0 conn(0x7f2ad006e300 0x7f2ad0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208468915 0 0) 0x7f2ad0067410 con 0x7f2ad006cf40 2023-11-28T21:55:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ab8005740 con 0x7f2ad006cf40 2023-11-28T21:55:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3375735364 0 0) 0x7f2ab8005740 con 0x7f2ad006cf40 2023-11-28T21:55:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 >> v2:172.21.15.144:3300/0 conn(0x7f2ad006e300 msgr2=0x7f2ad0156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 --2- 172.21.15.12:0/1848086117 >> v2:172.21.15.144:3300/0 conn(0x7f2ad006e300 0x7f2ad0156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 >> v1:172.21.15.144:6789/0 conn(0x7f2ad0150c20 legacy=0x7f2ad00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad0151000 con 0x7f2ad006cf40 2023-11-28T21:55:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f2ac8001a00 con 0x7f2ad006cf40 2023-11-28T21:55:33.916 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 >> v1:172.21.15.12:6789/0 conn(0x7f2ad006cf40 legacy=0x7f2ad0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ab8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ab80069e0 0x7f2ab8006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ab80073a0 0x7f2ab800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0157f30 con 0x7f2ab8005c00 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab800e220 con 0x7f2ab80069e0 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab800e3e0 con 0x7f2ab80073a0 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2acf7fe640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ab8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2acffff640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ab80073a0 0x7f2ab800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2aceffd640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ab80069e0 0x7f2ab8006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2acf7fe640 1 -- 172.21.15.12:0/1848086117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ab80073a0 msgr2=0x7f2ab800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2acf7fe640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ab80073a0 0x7f2ab800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:33.917 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2acf7fe640 1 -- 172.21.15.12:0/1848086117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ab80069e0 msgr2=0x7f2ab8006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:33.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2acf7fe640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ab80069e0 0x7f2ab8006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:33.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2acf7fe640 1 -- 172.21.15.12:0/1848086117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab8005740 con 0x7f2ab8005c00 2023-11-28T21:55:33.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.909+0000 7f2acffff640 1 -- 172.21.15.12:0/1848086117 reap_dead start 2023-11-28T21:55:33.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2acf7fe640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ab8005fe0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0001860 tx=0x7f2ac001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95af4f8a7a63429e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:33.918 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00ab600 con 0x7f2ab8005c00 2023-11-28T21:55:33.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac00a7050 con 0x7f2ab8005c00 2023-11-28T21:55:33.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ace7fc640 1 -- 172.21.15.12:0/1848086117 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00ab9e0 con 0x7f2ab8005c00 2023-11-28T21:55:33.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1848086117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 msgr2=0x7f2ab8005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:33.919 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ad5f3e640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ab8005fe0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0001860 tx=0x7f2ac001c500 comp rx=0 tx=0).stop 2023-11-28T21:55:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1848086117 shutdown_connections 2023-11-28T21:55:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ad5f3e640 1 --2- 172.21.15.12:0/1848086117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ab8005fe0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1848086117 >> 172.21.15.12:0/1848086117 conn(0x7f2ad0066360 msgr2=0x7f2ad0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1848086117 shutdown_connections 2023-11-28T21:55:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1848086117 wait complete. 2023-11-28T21:55:33.920 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ad5f3e640 1 Processor -- start 2023-11-28T21:55:33.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.913+0000 7f2ad5f3e640 1 -- start start 2023-11-28T21:55:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ad5f3e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ad01f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:33.923 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ad5f3e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ad00c4510 0x7f2ad01f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ad5f3e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0151850 0x7f2ad00c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ad5f3e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00c4080 con 0x7f2ab8005c00 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ad5f3e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad01f41f0 con 0x7f2ad0151850 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ad5f3e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00c3190 con 0x7f2ad00c4510 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2acf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ad01f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2acf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ad01f2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52716/0 (socket says 172.21.15.12:52716) 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2acf7fe640 1 -- 172.21.15.12:0/1172253434 learned_addr learned my addr 172.21.15.12:0/1172253434 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2aceffd640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ad00c4510 0x7f2ad01f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2acffff640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0151850 0x7f2ad00c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2acf7fe640 1 -- 172.21.15.12:0/1172253434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ad00c4510 msgr2=0x7f2ad01f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2acf7fe640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ad00c4510 0x7f2ad01f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:33.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2acf7fe640 1 -- 172.21.15.12:0/1172253434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0151850 msgr2=0x7f2ad00c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2acf7fe640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0151850 0x7f2ad00c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2acf7fe640 1 -- 172.21.15.12:0/1172253434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad00c3470 con 0x7f2ab8005c00 2023-11-28T21:55:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2acf7fe640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ad01f2840 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2ac001c330 tx=0x7f2ab8007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ab7fff640 1 -- 172.21.15.12:0/1172253434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00038c0 con 0x7f2ab8005c00 2023-11-28T21:55:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ab7fff640 1 -- 172.21.15.12:0/1172253434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac0003a20 con 0x7f2ab8005c00 2023-11-28T21:55:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ab7fff640 1 -- 172.21.15.12:0/1172253434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0003d90 con 0x7f2ab8005c00 2023-11-28T21:55:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1172253434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad00c3750 con 0x7f2ab8005c00 2023-11-28T21:55:33.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1172253434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad00c3d10 con 0x7f2ab8005c00 2023-11-28T21:55:33.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1172253434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad0074f60 con 0x7f2ab8005c00 2023-11-28T21:55:33.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.917+0000 7f2ab7fff640 1 -- 172.21.15.12:0/1172253434 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2ac00b2040 con 0x7f2ab8005c00 2023-11-28T21:55:33.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.921+0000 7f2ab7fff640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2abc0664f0 0x7f2abc0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:33.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.921+0000 7f2ab7fff640 1 -- 172.21.15.12:0/1172253434 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f2ac011e290 con 0x7f2ab8005c00 2023-11-28T21:55:33.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.921+0000 7f2aceffd640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2abc0664f0 0x7f2abc0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:33.927 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.921+0000 7f2aceffd640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2abc0664f0 0x7f2abc0689b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2ad01f0470 tx=0x7f2ac4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:33.931 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:33.925+0000 7f2ab7fff640 1 -- 172.21.15.12:0/1172253434 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2ac00eb030 con 0x7f2ab8005c00 2023-11-28T21:55:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.113+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1172253434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ad006bae0 con 0x7f2ab8005c00 2023-11-28T21:55:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.113+0000 7f2ab7fff640 1 -- 172.21.15.12:0/1172253434 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6521 (secure 0 0 0) 0x7f2ac00ee650 con 0x7f2ab8005c00 2023-11-28T21:55:34.120 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:34.120 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:22.698388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-11-28T21:55:34.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1172253434 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2abc0664f0 msgr2=0x7f2abc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:34.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2abc0664f0 0x7f2abc0689b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2ad01f0470 tx=0x7f2ac4008040 comp rx=0 tx=0).stop 2023-11-28T21:55:34.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1172253434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 msgr2=0x7f2ad01f2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:34.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ad01f2840 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2ac001c330 tx=0x7f2ab8007a10 comp rx=0 tx=0).stop 2023-11-28T21:55:34.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1172253434 shutdown_connections 2023-11-28T21:55:34.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2abc0664f0 0x7f2abc0689b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:34.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ad0151850 0x7f2ad00c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:34.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ad00c4510 0x7f2ad01f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:34.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 --2- 172.21.15.12:0/1172253434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ab8005c00 0x7f2ad01f2840 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:34.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1172253434 >> 172.21.15.12:0/1172253434 conn(0x7f2ad0066360 msgr2=0x7f2ad00ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:34.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1172253434 shutdown_connections 2023-11-28T21:55:34.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:34.117+0000 7f2ad5f3e640 1 -- 172.21.15.12:0/1172253434 wait complete. 2023-11-28T21:55:34.142 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:55:36.143 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:36.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.289+0000 7f03fb7f8640 1 Processor -- start 2023-11-28T21:55:36.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.289+0000 7f03fb7f8640 1 -- start start 2023-11-28T21:55:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03fb7f8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f03f40c4000 0x7f03f406d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03fb7f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4161600 0x7f03f4161a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03fb7f8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f40c3460 con 0x7f03f4158420 2023-11-28T21:55:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03fb7f8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f4158800 con 0x7f03f40c4000 2023-11-28T21:55:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03fb7f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f40c52c0 con 0x7f03f4161600 2023-11-28T21:55:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03f9594640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f03f4158420 0x7f03f40c38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50298/0 (socket says 172.21.15.12:50298) 2023-11-28T21:55:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03f9594640 1 -- 172.21.15.12:0/1715247421 learned_addr learned my addr 172.21.15.12:0/1715247421 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:55:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03f8d93640 1 --2- 172.21.15.12:0/1715247421 >> v2:172.21.15.12:3300/0 conn(0x7f03f40c4000 0x7f03f406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03f9d95640 1 --2- 172.21.15.12:0/1715247421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4161600 0x7f03f4161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:36.300 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03ebfff640 1 -- 172.21.15.12:0/1715247421 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13722296 0 0) 0x7f03f40c3460 con 0x7f03f4158420 2023-11-28T21:55:36.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03ebfff640 1 -- 172.21.15.12:0/1715247421 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e0003530 con 0x7f03f4158420 2023-11-28T21:55:36.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03f8d93640 1 -- 172.21.15.12:0/1715247421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4161600 msgr2=0x7f03f4161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:36.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03f8d93640 1 --2- 172.21.15.12:0/1715247421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4161600 0x7f03f4161a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:36.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03f8d93640 1 -- 172.21.15.12:0/1715247421 >> v1:172.21.15.12:6789/0 conn(0x7f03f4158420 legacy=0x7f03f40c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:36.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03f8d93640 1 -- 172.21.15.12:0/1715247421 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f40c2b70 con 0x7f03f40c4000 2023-11-28T21:55:36.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.293+0000 7f03f9d95640 1 --2- 172.21.15.12:0/1715247421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4161600 0x7f03f4161a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:55:36.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03f8d93640 1 --2- 172.21.15.12:0/1715247421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f40c4000 0x7f03f406d950 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f03e40097c0 tx=0x7f03e401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d76d6bb943e0f49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03ebfff640 1 -- 172.21.15.12:0/1715247421 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40ab600 con 0x7f03f40c4000 2023-11-28T21:55:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03ebfff640 1 -- 172.21.15.12:0/1715247421 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e40a7050 con 0x7f03f40c4000 2023-11-28T21:55:36.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03ebfff640 1 -- 172.21.15.12:0/1715247421 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40ab980 con 0x7f03f40c4000 2023-11-28T21:55:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1715247421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f40c4000 msgr2=0x7f03f406d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03fb7f8640 1 --2- 172.21.15.12:0/1715247421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f40c4000 0x7f03f406d950 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f03e40097c0 tx=0x7f03e401c480 comp rx=0 tx=0).stop 2023-11-28T21:55:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1715247421 shutdown_connections 2023-11-28T21:55:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03fb7f8640 1 --2- 172.21.15.12:0/1715247421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4161600 0x7f03f4161a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03fb7f8640 1 --2- 172.21.15.12:0/1715247421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f40c4000 0x7f03f406d950 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1715247421 >> 172.21.15.12:0/1715247421 conn(0x7f03f4066360 msgr2=0x7f03f4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1715247421 shutdown_connections 2023-11-28T21:55:36.304 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1715247421 wait complete. 2023-11-28T21:55:36.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03fb7f8640 1 Processor -- start 2023-11-28T21:55:36.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.297+0000 7f03fb7f8640 1 -- start start 2023-11-28T21:55:36.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03fb7f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f40c4000 0x7f03f41f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:36.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03fb7f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4158420 0x7f03f41f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:36.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03fb7f8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4161600 0x7f03f4156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:36.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03fb7f8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f41f8950 con 0x7f03f4161600 2023-11-28T21:55:36.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03fb7f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f4159ec0 con 0x7f03f40c4000 2023-11-28T21:55:36.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03fb7f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f415a1a0 con 0x7f03f4158420 2023-11-28T21:55:36.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f9d95640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4161600 0x7f03f4156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:36.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f9d95640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4161600 0x7f03f4156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43318/0 (socket says 172.21.15.12:43318) 2023-11-28T21:55:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f9d95640 1 -- 172.21.15.12:0/1835722645 learned_addr learned my addr 172.21.15.12:0/1835722645 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f8d93640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4158420 0x7f03f41f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f9594640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f40c4000 0x7f03f41f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f9d95640 1 -- 172.21.15.12:0/1835722645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4158420 msgr2=0x7f03f41f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f9d95640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4158420 0x7f03f41f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f9d95640 1 -- 172.21.15.12:0/1835722645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f40c4000 msgr2=0x7f03f41f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f9d95640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f40c4000 0x7f03f41f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f9d95640 1 -- 172.21.15.12:0/1835722645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e4009420 con 0x7f03f4161600 2023-11-28T21:55:36.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03f9d95640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4161600 0x7f03f4156540 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f03ec007df0 tx=0x7f03ec00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:36.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03e9ffb640 1 -- 172.21.15.12:0/1835722645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0039a0 con 0x7f03f4161600 2023-11-28T21:55:36.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03e9ffb640 1 -- 172.21.15.12:0/1835722645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ec00f040 con 0x7f03f4161600 2023-11-28T21:55:36.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03e9ffb640 1 -- 172.21.15.12:0/1835722645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec003d40 con 0x7f03f4161600 2023-11-28T21:55:36.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1835722645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f415a480 con 0x7f03f4161600 2023-11-28T21:55:36.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1835722645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f415aa40 con 0x7f03f4161600 2023-11-28T21:55:36.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.301+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1835722645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f4074f60 con 0x7f03f4161600 2023-11-28T21:55:36.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.305+0000 7f03e9ffb640 1 -- 172.21.15.12:0/1835722645 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f03ec004030 con 0x7f03f4161600 2023-11-28T21:55:36.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.305+0000 7f03e9ffb640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f03d80664a0 0x7f03d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:36.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.305+0000 7f03e9ffb640 1 -- 172.21.15.12:0/1835722645 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 7437+0+0 (secure 0 0 0) 0x7f03ec084880 con 0x7f03f4161600 2023-11-28T21:55:36.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.305+0000 7f03f9594640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f03d80664a0 0x7f03d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:36.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.305+0000 7f03f9594640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f03d80664a0 0x7f03d8068960 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f03f00011f0 tx=0x7f03f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:36.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.309+0000 7f03e9ffb640 1 -- 172.21.15.12:0/1835722645 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03ec0515c0 con 0x7f03f4161600 2023-11-28T21:55:36.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.497+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1835722645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03f406e4f0 con 0x7f03f4161600 2023-11-28T21:55:36.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.497+0000 7f03e9ffb640 1 -- 172.21.15.12:0/1835722645 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+6521 (secure 0 0 0) 0x7f03ec054be0 con 0x7f03f4161600 2023-11-28T21:55:36.505 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:36.505 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:22.698388+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":51,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6330},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6330":{"gid":6330,"name":"e","rank":1,"incarnation":63,"state":"up:active","state_seq":16,"addr":"172.21.15.12:6839/754378087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":754378087},{"type":"v1","addr":"172.21.15.12:6839","nonce":754378087}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:36.505 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 80 2023-11-28T21:55:36.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1835722645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f03d80664a0 msgr2=0x7f03d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:36.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f03d80664a0 0x7f03d8068960 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f03f00011f0 tx=0x7f03f0009040 comp rx=0 tx=0).stop 2023-11-28T21:55:36.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1835722645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4161600 msgr2=0x7f03f4156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:36.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4161600 0x7f03f4156540 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f03ec007df0 tx=0x7f03ec00d4a0 comp rx=0 tx=0).stop 2023-11-28T21:55:36.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1835722645 shutdown_connections 2023-11-28T21:55:36.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f03d80664a0 0x7f03d8068960 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:36.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03f4161600 0x7f03f4156540 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:36.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03f4158420 0x7f03f41f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:36.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 --2- 172.21.15.12:0/1835722645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f40c4000 0x7f03f41f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:36.507 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1835722645 >> 172.21.15.12:0/1835722645 conn(0x7f03f4066360 msgr2=0x7f03f40c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:36.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1835722645 shutdown_connections 2023-11-28T21:55:36.508 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:36.501+0000 7f03fb7f8640 1 -- 172.21.15.12:0/1835722645 wait complete. 2023-11-28T21:55:36.526 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-11-28T21:55:38.527 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:38.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.677+0000 7faf0349f640 1 Processor -- start 2023-11-28T21:55:38.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.677+0000 7faf0349f640 1 -- start start 2023-11-28T21:55:38.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.677+0000 7faf0349f640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faefc153fb0 0x7faefc0ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:38.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.677+0000 7faf0349f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faefc15a980 0x7faefc0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:38.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.677+0000 7faf0349f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc0b3ad0 0x7faefc0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:38.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.677+0000 7faf0349f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faefc0b2b60 con 0x7faefc15a980 2023-11-28T21:55:38.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.677+0000 7faf0349f640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faefc15e1b0 con 0x7faefc153fb0 2023-11-28T21:55:38.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.677+0000 7faf0349f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc0b44a0 con 0x7faefc0b3ad0 2023-11-28T21:55:38.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf01c9c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faefc15a980 0x7faefc0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:38.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf01c9c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7faefc15a980 0x7faefc0caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43320/0 (socket says 172.21.15.12:43320) 2023-11-28T21:55:38.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf01c9c640 1 -- 172.21.15.12:0/3445241016 learned_addr learned my addr 172.21.15.12:0/3445241016 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:38.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf02c9e640 1 --2- 172.21.15.12:0/3445241016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc0b3ad0 0x7faefc0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:38.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf0249d640 1 --2- 172.21.15.12:0/3445241016 >> v2:172.21.15.144:3300/0 conn(0x7faefc153fb0 0x7faefc0ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:38.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf01c9c640 1 -- 172.21.15.12:0/3445241016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc0b3ad0 msgr2=0x7faefc0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:38.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf01c9c640 1 --2- 172.21.15.12:0/3445241016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc0b3ad0 0x7faefc0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf01c9c640 1 -- 172.21.15.12:0/3445241016 >> v2:172.21.15.144:3300/0 conn(0x7faefc153fb0 msgr2=0x7faefc0ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:38.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf01c9c640 1 --2- 172.21.15.12:0/3445241016 >> v2:172.21.15.144:3300/0 conn(0x7faefc153fb0 0x7faefc0ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf01c9c640 1 -- 172.21.15.12:0/3445241016 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc15d390 con 0x7faefc15a980 2023-11-28T21:55:38.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf01c9c640 1 --2- 172.21.15.12:0/3445241016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faefc15a980 0x7faefc0caa30 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faefc0d04c0 tx=0x7faef001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19413a6c2dc0c46c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:38.692 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf0149b640 1 -- 172.21.15.12:0/3445241016 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00a5820 con 0x7faefc15a980 2023-11-28T21:55:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf0149b640 1 -- 172.21.15.12:0/3445241016 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef00a5980 con 0x7faefc15a980 2023-11-28T21:55:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.681+0000 7faf0149b640 1 -- 172.21.15.12:0/3445241016 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00a5d10 con 0x7faefc15a980 2023-11-28T21:55:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 -- 172.21.15.12:0/3445241016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faefc15a980 msgr2=0x7faefc0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 --2- 172.21.15.12:0/3445241016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faefc15a980 0x7faefc0caa30 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7faefc0d04c0 tx=0x7faef001c9a0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.693 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 -- 172.21.15.12:0/3445241016 shutdown_connections 2023-11-28T21:55:38.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 --2- 172.21.15.12:0/3445241016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc0b3ad0 0x7faefc0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 --2- 172.21.15.12:0/3445241016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faefc15a980 0x7faefc0caa30 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 --2- 172.21.15.12:0/3445241016 >> v2:172.21.15.144:3300/0 conn(0x7faefc153fb0 0x7faefc0ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 -- 172.21.15.12:0/3445241016 >> 172.21.15.12:0/3445241016 conn(0x7faefc071ce0 msgr2=0x7faefc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:38.694 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 -- 172.21.15.12:0/3445241016 shutdown_connections 2023-11-28T21:55:38.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 -- 172.21.15.12:0/3445241016 wait complete. 2023-11-28T21:55:38.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 Processor -- start 2023-11-28T21:55:38.695 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 -- start start 2023-11-28T21:55:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0b3ad0 0x7faefc0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc153fb0 0x7faefc157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faefc15a980 0x7faefc152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc0c4a10 con 0x7faefc15a980 2023-11-28T21:55:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc1f5b90 con 0x7faefc0b3ad0 2023-11-28T21:55:38.697 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0349f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc1f5e70 con 0x7faefc153fb0 2023-11-28T21:55:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf01c9c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc153fb0 0x7faefc157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf01c9c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc153fb0 0x7faefc157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.12:47022/0 (socket says 172.21.15.12:47022) 2023-11-28T21:55:38.698 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf01c9c640 1 -- 172.21.15.12:0/3532581334 learned_addr learned my addr 172.21.15.12:0/3532581334 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:38.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf0249d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0b3ad0 0x7faefc0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:38.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.685+0000 7faf02c9e640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faefc15a980 0x7faefc152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:38.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faf01c9c640 1 -- 172.21.15.12:0/3532581334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0b3ad0 msgr2=0x7faefc0c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:38.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faf01c9c640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0b3ad0 0x7faefc0c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faf01c9c640 1 -- 172.21.15.12:0/3532581334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faefc15a980 msgr2=0x7faefc152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:38.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faf01c9c640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faefc15a980 0x7faefc152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faf01c9c640 1 -- 172.21.15.12:0/3532581334 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef0009b30 con 0x7faefc153fb0 2023-11-28T21:55:38.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faf01c9c640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc153fb0 0x7faefc157cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faef00af040 tx=0x7faef0019740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:38.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faeeeffd640 1 -- 172.21.15.12:0/3532581334 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00bd070 con 0x7faefc153fb0 2023-11-28T21:55:38.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faeeeffd640 1 -- 172.21.15.12:0/3532581334 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef00a5af0 con 0x7faefc153fb0 2023-11-28T21:55:38.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faf0349f640 1 -- 172.21.15.12:0/3532581334 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc1f6150 con 0x7faefc153fb0 2023-11-28T21:55:38.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faeeeffd640 1 -- 172.21.15.12:0/3532581334 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00b8470 con 0x7faefc153fb0 2023-11-28T21:55:38.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faf0349f640 1 -- 172.21.15.12:0/3532581334 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc1f6710 con 0x7faefc153fb0 2023-11-28T21:55:38.700 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faf0349f640 1 -- 172.21.15.12:0/3532581334 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc0c6600 con 0x7faefc153fb0 2023-11-28T21:55:38.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faeeeffd640 1 -- 172.21.15.12:0/3532581334 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faef00b85d0 con 0x7faefc153fb0 2023-11-28T21:55:38.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.689+0000 7faeeeffd640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faee0066470 0x7faee0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:38.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.693+0000 7faeeeffd640 1 -- 172.21.15.12:0/3532581334 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7faef011ede0 con 0x7faefc153fb0 2023-11-28T21:55:38.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.693+0000 7faf0249d640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faee0066470 0x7faee0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:38.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.693+0000 7faf0249d640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faee0066470 0x7faee0068930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faef4009b60 tx=0x7faef4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:38.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.697+0000 7faeeeffd640 1 -- 172.21.15.12:0/3532581334 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faef00ebc70 con 0x7faefc153fb0 2023-11-28T21:55:38.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.885+0000 7faf0349f640 1 -- 172.21.15.12:0/3532581334 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faefc074f60 con 0x7faefc153fb0 2023-11-28T21:55:38.892 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.885+0000 7faeeeffd640 1 -- 172.21.15.12:0/3532581334 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5731 (secure 0 0 0) 0x7faef00ef290 con 0x7faefc153fb0 2023-11-28T21:55:38.893 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:38.893 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:38.673816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:38.893 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 81 2023-11-28T21:55:38.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 -- 172.21.15.12:0/3532581334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faee0066470 msgr2=0x7faee0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:38.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faee0066470 0x7faee0068930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faef4009b60 tx=0x7faef4009190 comp rx=0 tx=0).stop 2023-11-28T21:55:38.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 -- 172.21.15.12:0/3532581334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc153fb0 msgr2=0x7faefc157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:38.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc153fb0 0x7faefc157cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faef00af040 tx=0x7faef0019740 comp rx=0 tx=0).stop 2023-11-28T21:55:38.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 -- 172.21.15.12:0/3532581334 shutdown_connections 2023-11-28T21:55:38.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7faee0066470 0x7faee0068930 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faefc15a980 0x7faefc152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faefc153fb0 0x7faefc157cd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.895 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 --2- 172.21.15.12:0/3532581334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0b3ad0 0x7faefc0c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:38.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 -- 172.21.15.12:0/3532581334 >> 172.21.15.12:0/3532581334 conn(0x7faefc071ce0 msgr2=0x7faefc149fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:38.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 -- 172.21.15.12:0/3532581334 shutdown_connections 2023-11-28T21:55:38.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:38.889+0000 7faf0349f640 1 -- 172.21.15.12:0/3532581334 wait complete. 2023-11-28T21:55:38.914 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-11-28T21:55:38.914 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-11-28T21:55:38.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:39.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049b03640 1 Processor -- start 2023-11-28T21:55:39.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049b03640 1 -- start start 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049b03640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6044153fb0 0x7f60440ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049b03640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f604415a980 0x7f60440caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049b03640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f60440b3ad0 0x7f60440b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049b03640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60440b2b60 con 0x7f60440b3ad0 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049b03640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604415e1b0 con 0x7f6044153fb0 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049b03640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604415d390 con 0x7f604415a980 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049302640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f60440b3ad0 0x7f60440b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049302640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f60440b3ad0 0x7f60440b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43340/0 (socket says 172.21.15.12:43340) 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049302640 1 -- 172.21.15.12:0/3645028896 learned_addr learned my addr 172.21.15.12:0/3645028896 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6043fff640 1 --2- 172.21.15.12:0/3645028896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f604415a980 0x7f60440caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:39.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6048b01640 1 --2- 172.21.15.12:0/3645028896 >> v2:172.21.15.144:3300/0 conn(0x7f6044153fb0 0x7f60440ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049302640 1 -- 172.21.15.12:0/3645028896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f604415a980 msgr2=0x7f60440caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049302640 1 --2- 172.21.15.12:0/3645028896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f604415a980 0x7f60440caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049302640 1 -- 172.21.15.12:0/3645028896 >> v2:172.21.15.144:3300/0 conn(0x7f6044153fb0 msgr2=0x7f60440ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049302640 1 --2- 172.21.15.12:0/3645028896 >> v2:172.21.15.144:3300/0 conn(0x7f6044153fb0 0x7f60440ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.053+0000 7f6049302640 1 -- 172.21.15.12:0/3645028896 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604415d630 con 0x7f60440b3ad0 2023-11-28T21:55:39.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049302640 1 --2- 172.21.15.12:0/3645028896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60440b3ad0 0x7f60440b3ed0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f60440d04c0 tx=0x7f603c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8382280c9e204b8d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:39.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f60437fe640 1 -- 172.21.15.12:0/3645028896 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0a5820 con 0x7f60440b3ad0 2023-11-28T21:55:39.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f60437fe640 1 -- 172.21.15.12:0/3645028896 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c0a5980 con 0x7f60440b3ad0 2023-11-28T21:55:39.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f60437fe640 1 -- 172.21.15.12:0/3645028896 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0a5d10 con 0x7f60440b3ad0 2023-11-28T21:55:39.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 -- 172.21.15.12:0/3645028896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60440b3ad0 msgr2=0x7f60440b3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:39.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 --2- 172.21.15.12:0/3645028896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60440b3ad0 0x7f60440b3ed0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f60440d04c0 tx=0x7f603c01c9a0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 -- 172.21.15.12:0/3645028896 shutdown_connections 2023-11-28T21:55:39.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 --2- 172.21.15.12:0/3645028896 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60440b3ad0 0x7f60440b3ed0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 --2- 172.21.15.12:0/3645028896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f604415a980 0x7f60440caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 --2- 172.21.15.12:0/3645028896 >> v2:172.21.15.144:3300/0 conn(0x7f6044153fb0 0x7f60440ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 -- 172.21.15.12:0/3645028896 >> 172.21.15.12:0/3645028896 conn(0x7f6044071ce0 msgr2=0x7f60440720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:39.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 -- 172.21.15.12:0/3645028896 shutdown_connections 2023-11-28T21:55:39.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 -- 172.21.15.12:0/3645028896 wait complete. 2023-11-28T21:55:39.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 Processor -- start 2023-11-28T21:55:39.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.057+0000 7f6049b03640 1 -- start start 2023-11-28T21:55:39.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6049b03640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60440b3ad0 0x7f6044148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:39.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6049b03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6044153fb0 0x7f60441f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:39.067 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6049b03640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604415a980 0x7f60441f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6049b03640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6044152450 con 0x7f6044153fb0 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6049b03640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60441f6ba0 con 0x7f604415a980 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6049b03640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60441f2c90 con 0x7f60440b3ad0 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6043fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6044153fb0 0x7f60441f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6043fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6044153fb0 0x7f60441f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43346/0 (socket says 172.21.15.12:43346) 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6043fff640 1 -- 172.21.15.12:0/2373979382 learned_addr learned my addr 172.21.15.12:0/2373979382 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6048b01640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60440b3ad0 0x7f6044148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6049302640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604415a980 0x7f60441f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6043fff640 1 -- 172.21.15.12:0/2373979382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60440b3ad0 msgr2=0x7f6044148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6043fff640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60440b3ad0 0x7f6044148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6043fff640 1 -- 172.21.15.12:0/2373979382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604415a980 msgr2=0x7f60441f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:39.068 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6043fff640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604415a980 0x7f60441f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6043fff640 1 -- 172.21.15.12:0/2373979382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c009b30 con 0x7f6044153fb0 2023-11-28T21:55:39.069 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6043fff640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6044153fb0 0x7f60441f3170 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f603000e870 tx=0x7f603000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:39.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f60417fa640 1 -- 172.21.15.12:0/2373979382 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030009510 con 0x7f6044153fb0 2023-11-28T21:55:39.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f60417fa640 1 -- 172.21.15.12:0/2373979382 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603000eec0 con 0x7f6044153fb0 2023-11-28T21:55:39.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f60417fa640 1 -- 172.21.15.12:0/2373979382 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000cb80 con 0x7f6044153fb0 2023-11-28T21:55:39.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6049b03640 1 -- 172.21.15.12:0/2373979382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60441f6e10 con 0x7f6044153fb0 2023-11-28T21:55:39.070 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6049b03640 1 -- 172.21.15.12:0/2373979382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6044152980 con 0x7f6044153fb0 2023-11-28T21:55:39.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.061+0000 7f6049b03640 1 -- 172.21.15.12:0/2373979382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60440c6600 con 0x7f6044153fb0 2023-11-28T21:55:39.071 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.065+0000 7f60417fa640 1 -- 172.21.15.12:0/2373979382 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f603000cce0 con 0x7f6044153fb0 2023-11-28T21:55:39.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.065+0000 7f60417fa640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6028066470 0x7f6028068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:39.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.065+0000 7f60417fa640 1 -- 172.21.15.12:0/2373979382 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f6030084620 con 0x7f6044153fb0 2023-11-28T21:55:39.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.065+0000 7f6048b01640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6028066470 0x7f6028068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:39.072 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.065+0000 7f6048b01640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6028066470 0x7f6028068930 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6038009ae0 tx=0x7f6038009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:39.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.069+0000 7f60417fa640 1 -- 172.21.15.12:0/2373979382 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6030051420 con 0x7f6044153fb0 2023-11-28T21:55:39.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 -- 172.21.15.12:0/2373979382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6044074f60 con 0x7f6044153fb0 2023-11-28T21:55:39.262 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f60417fa640 1 -- 172.21.15.12:0/2373979382 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f6030054a40 con 0x7f6044153fb0 2023-11-28T21:55:39.263 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:39.263 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:38.673816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:39.263 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 81 2023-11-28T21:55:39.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 -- 172.21.15.12:0/2373979382 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6028066470 msgr2=0x7f6028068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:39.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6028066470 0x7f6028068930 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6038009ae0 tx=0x7f6038009190 comp rx=0 tx=0).stop 2023-11-28T21:55:39.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 -- 172.21.15.12:0/2373979382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6044153fb0 msgr2=0x7f60441f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:39.265 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6044153fb0 0x7f60441f3170 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f603000e870 tx=0x7f603000ec70 comp rx=0 tx=0).stop 2023-11-28T21:55:39.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 -- 172.21.15.12:0/2373979382 shutdown_connections 2023-11-28T21:55:39.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6028066470 0x7f6028068930 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f604415a980 0x7f60441f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6044153fb0 0x7f60441f3170 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 --2- 172.21.15.12:0/2373979382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60440b3ad0 0x7f6044148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:39.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 -- 172.21.15.12:0/2373979382 >> 172.21.15.12:0/2373979382 conn(0x7f6044071ce0 msgr2=0x7f6044159d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:39.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 -- 172.21.15.12:0/2373979382 shutdown_connections 2023-11-28T21:55:39.267 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:39.257+0000 7f6049b03640 1 -- 172.21.15.12:0/2373979382 wait complete. 2023-11-28T21:55:39.288 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 81, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 81, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6502": { "addr": "172.21.15.144:6833/3873278110", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 3873278110, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 3873278110, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6502, "incarnation": 81, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:replay", "state_seq": 1 }, "gid_6513": { "addr": "172.21.15.12:6838/2286739028", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2286739028, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 2286739028, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 6513, "incarnation": 73, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 33 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:55:38.673816+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6513, "mds_1": 6502 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:55:41.289 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe97e11d640 1 Processor -- start 2023-11-28T21:55:41.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe97e11d640 1 -- start start 2023-11-28T21:55:41.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe97e11d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe97806e1b0 0x7fe978066e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:41.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe97e11d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe978159a10 0x7fe978159e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:41.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe97e11d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe97815a3c0 con 0x7fe97814cae0 2023-11-28T21:55:41.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe97e11d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9780673c0 con 0x7fe97806e1b0 2023-11-28T21:55:41.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe97e11d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe978156140 con 0x7fe978159a10 2023-11-28T21:55:41.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe9777fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe97814cae0 0x7fe97806d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50306/0 (socket says 172.21.15.12:50306) 2023-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe9777fe640 1 -- 172.21.15.12:0/3099905762 learned_addr learned my addr 172.21.15.12:0/3099905762 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe976ffd640 1 --2- 172.21.15.12:0/3099905762 >> v2:172.21.15.12:3300/0 conn(0x7fe97806e1b0 0x7fe978066e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe977fff640 1 --2- 172.21.15.12:0/3099905762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe978159a10 0x7fe978159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe9767fc640 1 -- 172.21.15.12:0/3099905762 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460305307 0 0) 0x7fe97815a3c0 con 0x7fe97814cae0 2023-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe9767fc640 1 -- 172.21.15.12:0/3099905762 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe960003530 con 0x7fe97814cae0 2023-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe976ffd640 1 -- 172.21.15.12:0/3099905762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe978159a10 msgr2=0x7fe978159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe976ffd640 1 --2- 172.21.15.12:0/3099905762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe978159a10 0x7fe978159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe976ffd640 1 -- 172.21.15.12:0/3099905762 >> v1:172.21.15.12:6789/0 conn(0x7fe97814cae0 legacy=0x7fe97806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe976ffd640 1 -- 172.21.15.12:0/3099905762 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97806fb80 con 0x7fe97806e1b0 2023-11-28T21:55:41.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.441+0000 7fe977fff640 1 --2- 172.21.15.12:0/3099905762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe978159a10 0x7fe978159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:55:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe976ffd640 1 --2- 172.21.15.12:0/3099905762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe97806e1b0 0x7fe978066e80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0097c0 tx=0x7fe96c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2005d5e43e7ff3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe9767fc640 1 -- 172.21.15.12:0/3099905762 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c0a8970 con 0x7fe97806e1b0 2023-11-28T21:55:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe9767fc640 1 -- 172.21.15.12:0/3099905762 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c0a8ad0 con 0x7fe97806e1b0 2023-11-28T21:55:41.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe9767fc640 1 -- 172.21.15.12:0/3099905762 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c0a8e60 con 0x7fe97806e1b0 2023-11-28T21:55:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe97e11d640 1 -- 172.21.15.12:0/3099905762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe97806e1b0 msgr2=0x7fe978066e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe97e11d640 1 --2- 172.21.15.12:0/3099905762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe97806e1b0 0x7fe978066e80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0097c0 tx=0x7fe96c01ec70 comp rx=0 tx=0).stop 2023-11-28T21:55:41.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe97e11d640 1 -- 172.21.15.12:0/3099905762 shutdown_connections 2023-11-28T21:55:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe97e11d640 1 --2- 172.21.15.12:0/3099905762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe978159a10 0x7fe978159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe97e11d640 1 --2- 172.21.15.12:0/3099905762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe97806e1b0 0x7fe978066e80 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe97e11d640 1 -- 172.21.15.12:0/3099905762 >> 172.21.15.12:0/3099905762 conn(0x7fe978066360 msgr2=0x7fe978066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe97e11d640 1 -- 172.21.15.12:0/3099905762 shutdown_connections 2023-11-28T21:55:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.445+0000 7fe97e11d640 1 -- 172.21.15.12:0/3099905762 wait complete. 2023-11-28T21:55:41.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 Processor -- start 2023-11-28T21:55:41.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 -- start start 2023-11-28T21:55:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe97806e1b0 0x7fe978158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe97814cae0 0x7fe978140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe978159a10 0x7fe978141200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780ca2c0 con 0x7fe978159a10 2023-11-28T21:55:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780cb050 con 0x7fe97806e1b0 2023-11-28T21:55:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780c9ce0 con 0x7fe97814cae0 2023-11-28T21:55:41.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe977fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe978159a10 0x7fe978141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe977fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe978159a10 0x7fe978141200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43372/0 (socket says 172.21.15.12:43372) 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe977fff640 1 -- 172.21.15.12:0/144518733 learned_addr learned my addr 172.21.15.12:0/144518733 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe976ffd640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe97814cae0 0x7fe978140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe9777fe640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe97806e1b0 0x7fe978158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe977fff640 1 -- 172.21.15.12:0/144518733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe97814cae0 msgr2=0x7fe978140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe977fff640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe97814cae0 0x7fe978140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe977fff640 1 -- 172.21.15.12:0/144518733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe97806e1b0 msgr2=0x7fe978158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe977fff640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe97806e1b0 0x7fe978158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe977fff640 1 -- 172.21.15.12:0/144518733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c009420 con 0x7fe978159a10 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe977fff640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe978159a10 0x7fe978141200 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe970007f20 tx=0x7fe97000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe95ffff640 1 -- 172.21.15.12:0/144518733 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97000dee0 con 0x7fe978159a10 2023-11-28T21:55:41.457 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe95ffff640 1 -- 172.21.15.12:0/144518733 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97000f040 con 0x7fe978159a10 2023-11-28T21:55:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 -- 172.21.15.12:0/144518733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe978148080 con 0x7fe978159a10 2023-11-28T21:55:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe95ffff640 1 -- 172.21.15.12:0/144518733 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970003de0 con 0x7fe978159a10 2023-11-28T21:55:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 -- 172.21.15.12:0/144518733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe978148360 con 0x7fe978159a10 2023-11-28T21:55:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.449+0000 7fe97e11d640 1 -- 172.21.15.12:0/144518733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe978074f60 con 0x7fe978159a10 2023-11-28T21:55:41.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.453+0000 7fe95ffff640 1 -- 172.21.15.12:0/144518733 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe970004030 con 0x7fe978159a10 2023-11-28T21:55:41.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.453+0000 7fe95ffff640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe964066470 0x7fe964068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:41.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.453+0000 7fe95ffff640 1 -- 172.21.15.12:0/144518733 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fe970084940 con 0x7fe978159a10 2023-11-28T21:55:41.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.453+0000 7fe9777fe640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe964066470 0x7fe964068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:41.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.453+0000 7fe9777fe640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe964066470 0x7fe964068930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe9680011f0 tx=0x7fe968009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:41.463 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.457+0000 7fe95ffff640 1 -- 172.21.15.12:0/144518733 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe970051690 con 0x7fe978159a10 2023-11-28T21:55:41.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 -- 172.21.15.12:0/144518733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe978153d90 con 0x7fe978159a10 2023-11-28T21:55:41.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe95ffff640 1 -- 172.21.15.12:0/144518733 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5731 (secure 0 0 0) 0x7fe970054cb0 con 0x7fe978159a10 2023-11-28T21:55:41.655 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:41.655 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:38.673816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:41.655 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 81 2023-11-28T21:55:41.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 -- 172.21.15.12:0/144518733 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe964066470 msgr2=0x7fe964068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:41.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe964066470 0x7fe964068930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe9680011f0 tx=0x7fe968009040 comp rx=0 tx=0).stop 2023-11-28T21:55:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 -- 172.21.15.12:0/144518733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe978159a10 msgr2=0x7fe978141200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe978159a10 0x7fe978141200 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe970007f20 tx=0x7fe97000d560 comp rx=0 tx=0).stop 2023-11-28T21:55:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 -- 172.21.15.12:0/144518733 shutdown_connections 2023-11-28T21:55:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fe964066470 0x7fe964068930 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe978159a10 0x7fe978141200 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe97814cae0 0x7fe978140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 --2- 172.21.15.12:0/144518733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe97806e1b0 0x7fe978158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 -- 172.21.15.12:0/144518733 >> 172.21.15.12:0/144518733 conn(0x7fe978066360 msgr2=0x7fe9780c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 -- 172.21.15.12:0/144518733 shutdown_connections 2023-11-28T21:55:41.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:41.649+0000 7fe97e11d640 1 -- 172.21.15.12:0/144518733 wait complete. 2023-11-28T21:55:41.676 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:55:43.677 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:43.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf3f0f640 1 Processor -- start 2023-11-28T21:55:43.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf3f0f640 1 -- start start 2023-11-28T21:55:43.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf3f0f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 0x7f2cec06d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:43.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf3f0f640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2cec161600 0x7f2cec161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:43.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf3f0f640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec0c3460 con 0x7f2cec158420 2023-11-28T21:55:43.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf3f0f640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec158800 con 0x7f2cec161600 2023-11-28T21:55:43.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf3f0f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0c52c0 con 0x7f2cec0c4000 2023-11-28T21:55:43.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf1cab640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2cec158420 0x7f2cec0c38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:50318/0 (socket says 172.21.15.12:50318) 2023-11-28T21:55:43.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf1cab640 1 -- 172.21.15.12:0/3409861968 learned_addr learned my addr 172.21.15.12:0/3409861968 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:55:43.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf24ac640 1 --2- 172.21.15.12:0/3409861968 >> v2:172.21.15.12:3300/0 conn(0x7f2cec161600 0x7f2cec161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:43.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf14aa640 1 --2- 172.21.15.12:0/3409861968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 0x7f2cec06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:43.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf0ca9640 1 -- 172.21.15.12:0/3409861968 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832321767 0 0) 0x7f2cec0c3460 con 0x7f2cec158420 2023-11-28T21:55:43.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf24ac640 1 -- 172.21.15.12:0/3409861968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 msgr2=0x7f2cec06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:43.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf24ac640 1 --2- 172.21.15.12:0/3409861968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 0x7f2cec06d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:43.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf24ac640 1 -- 172.21.15.12:0/3409861968 >> v1:172.21.15.12:6789/0 conn(0x7f2cec158420 legacy=0x7f2cec0c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:43.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.837+0000 7f2cf24ac640 1 -- 172.21.15.12:0/3409861968 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0c2b70 con 0x7f2cec161600 2023-11-28T21:55:43.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf24ac640 1 --2- 172.21.15.12:0/3409861968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec161600 0x7f2cec161a10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009d90 tx=0x7f2ce401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b67a04a12fbe9ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:43.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf0ca9640 1 -- 172.21.15.12:0/3409861968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce40a5810 con 0x7f2cec161600 2023-11-28T21:55:43.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf0ca9640 1 -- 172.21.15.12:0/3409861968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce40a5970 con 0x7f2cec161600 2023-11-28T21:55:43.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf0ca9640 1 -- 172.21.15.12:0/3409861968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce40a5d00 con 0x7f2cec161600 2023-11-28T21:55:43.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/3409861968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec161600 msgr2=0x7f2cec161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:43.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf3f0f640 1 --2- 172.21.15.12:0/3409861968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec161600 0x7f2cec161a10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009d90 tx=0x7f2ce401c950 comp rx=0 tx=0).stop 2023-11-28T21:55:43.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/3409861968 shutdown_connections 2023-11-28T21:55:43.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf3f0f640 1 --2- 172.21.15.12:0/3409861968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec161600 0x7f2cec161a10 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:43.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf3f0f640 1 --2- 172.21.15.12:0/3409861968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 0x7f2cec06d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:43.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/3409861968 >> 172.21.15.12:0/3409861968 conn(0x7f2cec066360 msgr2=0x7f2cec066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:43.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/3409861968 shutdown_connections 2023-11-28T21:55:43.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.841+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/3409861968 wait complete. 2023-11-28T21:55:43.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 Processor -- start 2023-11-28T21:55:43.850 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 -- start start 2023-11-28T21:55:43.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 0x7f2cec155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:43.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec158420 0x7f2cec15adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:43.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec161600 0x7f2cec15f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec1558d0 con 0x7f2cec158420 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec15a6e0 con 0x7f2cec161600 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec1611d0 con 0x7f2cec0c4000 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf14aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec158420 0x7f2cec15adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf14aa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec158420 0x7f2cec15adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43384/0 (socket says 172.21.15.12:43384) 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf14aa640 1 -- 172.21.15.12:0/1110493047 learned_addr learned my addr 172.21.15.12:0/1110493047 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf1cab640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 0x7f2cec155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf24ac640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec161600 0x7f2cec15f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf14aa640 1 -- 172.21.15.12:0/1110493047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 msgr2=0x7f2cec155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf14aa640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 0x7f2cec155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:43.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf14aa640 1 -- 172.21.15.12:0/1110493047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec161600 msgr2=0x7f2cec15f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:43.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf14aa640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec161600 0x7f2cec15f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:43.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf14aa640 1 -- 172.21.15.12:0/1110493047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4009a70 con 0x7f2cec158420 2023-11-28T21:55:43.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf1cab640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 0x7f2cec155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:55:43.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf14aa640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec158420 0x7f2cec15adb0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00e930 tx=0x7f2cdc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:43.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2ce27fc640 1 -- 172.21.15.12:0/1110493047 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc009690 con 0x7f2cec158420 2023-11-28T21:55:43.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2ce27fc640 1 -- 172.21.15.12:0/1110493047 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc00ca00 con 0x7f2cec158420 2023-11-28T21:55:43.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2ce27fc640 1 -- 172.21.15.12:0/1110493047 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc00cd70 con 0x7f2cec158420 2023-11-28T21:55:43.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/1110493047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec158940 con 0x7f2cec158420 2023-11-28T21:55:43.854 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/1110493047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec158f00 con 0x7f2cec158420 2023-11-28T21:55:43.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.845+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/1110493047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec15f1d0 con 0x7f2cec158420 2023-11-28T21:55:43.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.849+0000 7f2ce27fc640 1 -- 172.21.15.12:0/1110493047 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2cdc013470 con 0x7f2cec158420 2023-11-28T21:55:43.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.849+0000 7f2ce27fc640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2cd0066470 0x7f2cd0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:43.855 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.849+0000 7f2ce27fc640 1 -- 172.21.15.12:0/1110493047 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f2cdc084c80 con 0x7f2cec158420 2023-11-28T21:55:43.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.849+0000 7f2cf1cab640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2cd0066470 0x7f2cd0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:43.856 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.849+0000 7f2cf1cab640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2cd0066470 0x7f2cd0068930 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8007fd0 tx=0x7f2ce8007c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:43.860 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:43.853+0000 7f2ce27fc640 1 -- 172.21.15.12:0/1110493047 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2cec15f1d0 con 0x7f2cec158420 2023-11-28T21:55:44.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.041+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/1110493047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cec15f1d0 con 0x7f2cec158420 2023-11-28T21:55:44.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.041+0000 7f2ce27fc640 1 -- 172.21.15.12:0/1110493047 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5731 (secure 0 0 0) 0x7f2cec15f1d0 con 0x7f2cec158420 2023-11-28T21:55:44.050 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:44.050 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:38.673816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:44.050 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 81 2023-11-28T21:55:44.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/1110493047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2cd0066470 msgr2=0x7f2cd0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:44.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2cd0066470 0x7f2cd0068930 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8007fd0 tx=0x7f2ce8007c20 comp rx=0 tx=0).stop 2023-11-28T21:55:44.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/1110493047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec158420 msgr2=0x7f2cec15adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:44.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec158420 0x7f2cec15adb0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00e930 tx=0x7f2cdc00edf0 comp rx=0 tx=0).stop 2023-11-28T21:55:44.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/1110493047 shutdown_connections 2023-11-28T21:55:44.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2cd0066470 0x7f2cd0068930 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:44.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cec161600 0x7f2cec15f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:44.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cec158420 0x7f2cec15adb0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:44.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 --2- 172.21.15.12:0/1110493047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cec0c4000 0x7f2cec155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:44.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/1110493047 >> 172.21.15.12:0/1110493047 conn(0x7f2cec066360 msgr2=0x7f2cec06d410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:44.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/1110493047 shutdown_connections 2023-11-28T21:55:44.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:44.045+0000 7f2cf3f0f640 1 -- 172.21.15.12:0/1110493047 wait complete. 2023-11-28T21:55:44.071 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:55:46.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f22957f5640 1 Processor -- start 2023-11-28T21:55:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f22957f5640 1 -- start start 2023-11-28T21:55:46.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f22957f5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2290150c20 0x7f2290156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:46.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f22957f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229006e300 0x7f22900c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:46.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f22957f5640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2290067410 con 0x7f229006cf40 2023-11-28T21:55:46.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f22957f5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f229006e090 con 0x7f2290150c20 2023-11-28T21:55:46.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f22957f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2290157f30 con 0x7f229006e300 2023-11-28T21:55:46.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f228effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2290150c20 0x7f2290156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:46.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f228effd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2290150c20 0x7f2290156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41350/0 (socket says 172.21.15.12:41350) 2023-11-28T21:55:46.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f228effd640 1 -- 172.21.15.12:0/1297443921 learned_addr learned my addr 172.21.15.12:0/1297443921 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:46.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f228f7fe640 1 --1- 172.21.15.12:0/1297443921 >> v1:172.21.15.12:6789/0 conn(0x7f229006cf40 0x7f2290158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49458/0 (socket says 172.21.15.12:49458) 2023-11-28T21:55:46.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.221+0000 7f2287fff640 1 --2- 172.21.15.12:0/1297443921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229006e300 0x7f22900c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.225+0000 7f228effd640 1 -- 172.21.15.12:0/1297443921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229006e300 msgr2=0x7f22900c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.225+0000 7f228effd640 1 --2- 172.21.15.12:0/1297443921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229006e300 0x7f22900c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.225+0000 7f228effd640 1 -- 172.21.15.12:0/1297443921 >> v1:172.21.15.12:6789/0 conn(0x7f229006cf40 legacy=0x7f2290158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.225+0000 7f228effd640 1 -- 172.21.15.12:0/1297443921 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2290151000 con 0x7f2290150c20 2023-11-28T21:55:46.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.225+0000 7f228effd640 1 --2- 172.21.15.12:0/1297443921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 0x7f2290156910 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2280009870 tx=0x7f228001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26def0075176dec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:46.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.225+0000 7f228e7fc640 1 -- 172.21.15.12:0/1297443921 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22800ab650 con 0x7f2290150c20 2023-11-28T21:55:46.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.225+0000 7f228e7fc640 1 -- 172.21.15.12:0/1297443921 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22800ab7b0 con 0x7f2290150c20 2023-11-28T21:55:46.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.225+0000 7f228e7fc640 1 -- 172.21.15.12:0/1297443921 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22800abb40 con 0x7f2290150c20 2023-11-28T21:55:46.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.225+0000 7f22957f5640 1 -- 172.21.15.12:0/1297443921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 msgr2=0x7f2290156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:46.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.225+0000 7f22957f5640 1 --2- 172.21.15.12:0/1297443921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 0x7f2290156910 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2280009870 tx=0x7f228001c540 comp rx=0 tx=0).stop 2023-11-28T21:55:46.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 -- 172.21.15.12:0/1297443921 shutdown_connections 2023-11-28T21:55:46.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 --2- 172.21.15.12:0/1297443921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229006e300 0x7f22900c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:46.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 --2- 172.21.15.12:0/1297443921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 0x7f2290156910 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:46.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 -- 172.21.15.12:0/1297443921 >> 172.21.15.12:0/1297443921 conn(0x7f2290066360 msgr2=0x7f2290066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:46.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 -- 172.21.15.12:0/1297443921 shutdown_connections 2023-11-28T21:55:46.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 -- 172.21.15.12:0/1297443921 wait complete. 2023-11-28T21:55:46.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 Processor -- start 2023-11-28T21:55:46.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 -- start start 2023-11-28T21:55:46.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229006cf40 0x7f22901f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:46.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f229006e300 0x7f22900c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:46.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 0x7f2290151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:46.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22900c3ef0 con 0x7f2290150c20 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22901f73d0 con 0x7f229006e300 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f22957f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2290154400 con 0x7f229006cf40 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f228f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 0x7f2290151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f228f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 0x7f2290151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41352/0 (socket says 172.21.15.12:41352) 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f228f7fe640 1 -- 172.21.15.12:0/2960773785 learned_addr learned my addr 172.21.15.12:0/2960773785 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f228effd640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229006cf40 0x7f22901f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f2287fff640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f229006e300 0x7f22900c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f228f7fe640 1 -- 172.21.15.12:0/2960773785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229006cf40 msgr2=0x7f22901f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f228f7fe640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229006cf40 0x7f22901f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f228f7fe640 1 -- 172.21.15.12:0/2960773785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f229006e300 msgr2=0x7f22900c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:46.238 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f228f7fe640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f229006e300 0x7f22900c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:46.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f228f7fe640 1 -- 172.21.15.12:0/2960773785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22800094d0 con 0x7f2290150c20 2023-11-28T21:55:46.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.229+0000 7f228f7fe640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 0x7f2290151fc0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f228800c7a0 tx=0x7f228800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:46.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.233+0000 7f22877fe640 1 -- 172.21.15.12:0/2960773785 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800a640 con 0x7f2290150c20 2023-11-28T21:55:46.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.233+0000 7f22877fe640 1 -- 172.21.15.12:0/2960773785 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228800a7a0 con 0x7f2290150c20 2023-11-28T21:55:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.233+0000 7f22877fe640 1 -- 172.21.15.12:0/2960773785 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800ab10 con 0x7f2290150c20 2023-11-28T21:55:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.233+0000 7f22957f5640 1 -- 172.21.15.12:0/2960773785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22900c2e50 con 0x7f2290150c20 2023-11-28T21:55:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.233+0000 7f22957f5640 1 -- 172.21.15.12:0/2960773785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22900c3410 con 0x7f2290150c20 2023-11-28T21:55:46.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.233+0000 7f22957f5640 1 -- 172.21.15.12:0/2960773785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2290074f60 con 0x7f2290150c20 2023-11-28T21:55:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.233+0000 7f22877fe640 1 -- 172.21.15.12:0/2960773785 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f228800ac70 con 0x7f2290150c20 2023-11-28T21:55:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.233+0000 7f22877fe640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2270066470 0x7f2270068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.233+0000 7f22877fe640 1 -- 172.21.15.12:0/2960773785 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f2288083e10 con 0x7f2290150c20 2023-11-28T21:55:46.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.233+0000 7f228effd640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2270066470 0x7f2270068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:46.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.237+0000 7f228effd640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2270066470 0x7f2270068930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f22800af000 tx=0x7f22800b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:46.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.237+0000 7f22877fe640 1 -- 172.21.15.12:0/2960773785 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2288087030 con 0x7f2290150c20 2023-11-28T21:55:46.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.429+0000 7f22957f5640 1 -- 172.21.15.12:0/2960773785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f229014e6c0 con 0x7f2290150c20 2023-11-28T21:55:46.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22877fe640 1 -- 172.21.15.12:0/2960773785 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5733 (secure 0 0 0) 0x7f2288017020 con 0x7f2290150c20 2023-11-28T21:55:46.438 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:46.438 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:44.615182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:resolve","state_seq":36,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:46.439 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 82 2023-11-28T21:55:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 -- 172.21.15.12:0/2960773785 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2270066470 msgr2=0x7f2270068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:46.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2270066470 0x7f2270068930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f22800af000 tx=0x7f22800b6040 comp rx=0 tx=0).stop 2023-11-28T21:55:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 -- 172.21.15.12:0/2960773785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 msgr2=0x7f2290151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 0x7f2290151fc0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f228800c7a0 tx=0x7f228800cc60 comp rx=0 tx=0).stop 2023-11-28T21:55:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 -- 172.21.15.12:0/2960773785 shutdown_connections 2023-11-28T21:55:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2270066470 0x7f2270068930 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2290150c20 0x7f2290151fc0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:46.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f229006e300 0x7f22900c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 --2- 172.21.15.12:0/2960773785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f229006cf40 0x7f22901f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 -- 172.21.15.12:0/2960773785 >> 172.21.15.12:0/2960773785 conn(0x7f2290066360 msgr2=0x7f22900c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 -- 172.21.15.12:0/2960773785 shutdown_connections 2023-11-28T21:55:46.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:46.433+0000 7f22957f5640 1 -- 172.21.15.12:0/2960773785 wait complete. 2023-11-28T21:55:46.459 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:55:46.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:55:46.934 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:55:48.460 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.609+0000 7f45bb4e3640 1 Processor -- start 2023-11-28T21:55:48.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.609+0000 7f45bb4e3640 1 -- start start 2023-11-28T21:55:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45bb4e3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f45b406e300 0x7f45b4156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45bb4e3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b4067410 con 0x7f45b406cf40 2023-11-28T21:55:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45bb4e3640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b406e090 con 0x7f45b4150c20 2023-11-28T21:55:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45bb4e3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b4157f30 con 0x7f45b406e300 2023-11-28T21:55:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45b8a7e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f45b406e300 0x7f45b4156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45b8a7e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f45b406e300 0x7f45b4156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41360/0 (socket says 172.21.15.12:41360) 2023-11-28T21:55:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45b8a7e640 1 -- 172.21.15.12:0/804688820 learned_addr learned my addr 172.21.15.12:0/804688820 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45b9a80640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f45b406cf40 0x7f45b4158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49468/0 (socket says 172.21.15.12:49468) 2023-11-28T21:55:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45b8a7e640 1 -- 172.21.15.12:0/804688820 >> v1:172.21.15.144:6789/0 conn(0x7f45b4150c20 legacy=0x7f45b40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45b8a7e640 1 -- 172.21.15.12:0/804688820 >> v1:172.21.15.12:6789/0 conn(0x7f45b406cf40 legacy=0x7f45b4158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.613+0000 7f45b8a7e640 1 -- 172.21.15.12:0/804688820 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4151000 con 0x7f45b406e300 2023-11-28T21:55:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45b8a7e640 1 --2- 172.21.15.12:0/804688820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b406e300 0x7f45b4156910 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f45a400bb10 tx=0x7f45a401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d35c0de933c4d7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45abfff640 1 -- 172.21.15.12:0/804688820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a40a7af0 con 0x7f45b406e300 2023-11-28T21:55:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45abfff640 1 -- 172.21.15.12:0/804688820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a40a7c50 con 0x7f45b406e300 2023-11-28T21:55:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45abfff640 1 -- 172.21.15.12:0/804688820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a40af440 con 0x7f45b406e300 2023-11-28T21:55:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45bb4e3640 1 -- 172.21.15.12:0/804688820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b406e300 msgr2=0x7f45b4156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45bb4e3640 1 --2- 172.21.15.12:0/804688820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b406e300 0x7f45b4156910 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f45a400bb10 tx=0x7f45a401ec10 comp rx=0 tx=0).stop 2023-11-28T21:55:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45bb4e3640 1 -- 172.21.15.12:0/804688820 shutdown_connections 2023-11-28T21:55:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45bb4e3640 1 --2- 172.21.15.12:0/804688820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b406e300 0x7f45b4156910 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45bb4e3640 1 -- 172.21.15.12:0/804688820 >> 172.21.15.12:0/804688820 conn(0x7f45b4066360 msgr2=0x7f45b4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45bb4e3640 1 -- 172.21.15.12:0/804688820 shutdown_connections 2023-11-28T21:55:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45bb4e3640 1 -- 172.21.15.12:0/804688820 wait complete. 2023-11-28T21:55:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45bb4e3640 1 Processor -- start 2023-11-28T21:55:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.617+0000 7f45bb4e3640 1 -- start start 2023-11-28T21:55:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45bb4e3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b406cf40 0x7f45b41f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45bb4e3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b406e300 0x7f45b40c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45bb4e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b4150c20 0x7f45b415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45bb4e3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b40c49a0 con 0x7f45b4150c20 2023-11-28T21:55:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45bb4e3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b41f3e10 con 0x7f45b406e300 2023-11-28T21:55:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45bb4e3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b41505c0 con 0x7f45b406cf40 2023-11-28T21:55:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b9a80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b4150c20 0x7f45b415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b9a80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b4150c20 0x7f45b415b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41362/0 (socket says 172.21.15.12:41362) 2023-11-28T21:55:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b9a80640 1 -- 172.21.15.12:0/3273552319 learned_addr learned my addr 172.21.15.12:0/3273552319 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b927f640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b406cf40 0x7f45b41f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b8a7e640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b406e300 0x7f45b40c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b9a80640 1 -- 172.21.15.12:0/3273552319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b406cf40 msgr2=0x7f45b41f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b9a80640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b406cf40 0x7f45b41f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b9a80640 1 -- 172.21.15.12:0/3273552319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b406e300 msgr2=0x7f45b40c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b9a80640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b406e300 0x7f45b40c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b9a80640 1 -- 172.21.15.12:0/3273552319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a400b770 con 0x7f45b4150c20 2023-11-28T21:55:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45b9a80640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b4150c20 0x7f45b415b090 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00c7a0 tx=0x7f45ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:48.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45a9ffb640 1 -- 172.21.15.12:0/3273552319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00a640 con 0x7f45b4150c20 2023-11-28T21:55:48.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45a9ffb640 1 -- 172.21.15.12:0/3273552319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac00a7a0 con 0x7f45b4150c20 2023-11-28T21:55:48.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45a9ffb640 1 -- 172.21.15.12:0/3273552319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00ab10 con 0x7f45b4150c20 2023-11-28T21:55:48.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45bb4e3640 1 -- 172.21.15.12:0/3273552319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b40c5290 con 0x7f45b4150c20 2023-11-28T21:55:48.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45bb4e3640 1 -- 172.21.15.12:0/3273552319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b41502e0 con 0x7f45b4150c20 2023-11-28T21:55:48.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45bb4e3640 1 -- 172.21.15.12:0/3273552319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b415adb0 con 0x7f45b4150c20 2023-11-28T21:55:48.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.621+0000 7f45a9ffb640 1 -- 172.21.15.12:0/3273552319 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f45ac00ac70 con 0x7f45b4150c20 2023-11-28T21:55:48.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.625+0000 7f45a9ffb640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4598066470 0x7f4598068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:48.630 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.625+0000 7f45a9ffb640 1 -- 172.21.15.12:0/3273552319 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f45ac083d60 con 0x7f45b4150c20 2023-11-28T21:55:48.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.625+0000 7f45b927f640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4598066470 0x7f4598068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:48.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.625+0000 7f45b927f640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4598066470 0x7f4598068930 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f45b0007fd0 tx=0x7f45b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:48.634 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.629+0000 7f45a9ffb640 1 -- 172.21.15.12:0/3273552319 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f45b415adb0 con 0x7f45b4150c20 2023-11-28T21:55:48.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.817+0000 7f45bb4e3640 1 -- 172.21.15.12:0/3273552319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45b415adb0 con 0x7f45b4150c20 2023-11-28T21:55:48.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45a9ffb640 1 -- 172.21.15.12:0/3273552319 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5733 (secure 0 0 0) 0x7f45b415adb0 con 0x7f45b4150c20 2023-11-28T21:55:48.827 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:48.827 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:44.615182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:resolve","state_seq":36,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:48.827 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 82 2023-11-28T21:55:48.828 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 -- 172.21.15.12:0/3273552319 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4598066470 msgr2=0x7f4598068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4598066470 0x7f4598068930 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f45b0007fd0 tx=0x7f45b0009040 comp rx=0 tx=0).stop 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 -- 172.21.15.12:0/3273552319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b4150c20 msgr2=0x7f45b415b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b4150c20 0x7f45b415b090 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00c7a0 tx=0x7f45ac00cc60 comp rx=0 tx=0).stop 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 -- 172.21.15.12:0/3273552319 shutdown_connections 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4598066470 0x7f4598068930 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f45b4150c20 0x7f45b415b090 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b406e300 0x7f45b40c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 --2- 172.21.15.12:0/3273552319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b406cf40 0x7f45b41f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 -- 172.21.15.12:0/3273552319 >> 172.21.15.12:0/3273552319 conn(0x7f45b4066360 msgr2=0x7f45b40c9740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 -- 172.21.15.12:0/3273552319 shutdown_connections 2023-11-28T21:55:48.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:48.821+0000 7f45bb4e3640 1 -- 172.21.15.12:0/3273552319 wait complete. 2023-11-28T21:55:48.848 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:55:50.849 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.001+0000 7f1a19f72640 1 Processor -- start 2023-11-28T21:55:51.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.001+0000 7f1a19f72640 1 -- start start 2023-11-28T21:55:51.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.001+0000 7f1a19f72640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140cac40 0x7f1a1406b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:51.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.001+0000 7f1a19f72640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a140c4da0 con 0x7f1a140c4980 2023-11-28T21:55:51.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.001+0000 7f1a19f72640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a140cb4b0 con 0x7f1a140cb020 2023-11-28T21:55:51.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.001+0000 7f1a19f72640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1415a5e0 con 0x7f1a140cac40 2023-11-28T21:55:51.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.005+0000 7f1a12ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f1a140c4980 0x7f1a140ca030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49478/0 (socket says 172.21.15.12:49478) 2023-11-28T21:55:51.010 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.005+0000 7f1a12ffd640 1 -- 172.21.15.12:0/790576353 learned_addr learned my addr 172.21.15.12:0/790576353 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:55:51.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.005+0000 7f1a137fe640 1 --2- 172.21.15.12:0/790576353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140cac40 0x7f1a1406b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:51.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.005+0000 7f1a127fc640 1 -- 172.21.15.12:0/790576353 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030290424 0 0) 0x7f1a140c4da0 con 0x7f1a140c4980 2023-11-28T21:55:51.011 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.005+0000 7f1a127fc640 1 -- 172.21.15.12:0/790576353 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19fc003530 con 0x7f1a140c4980 2023-11-28T21:55:51.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.005+0000 7f1a127fc640 1 -- 172.21.15.12:0/790576353 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403614947 0 0) 0x7f1a140cb4b0 con 0x7f1a140cb020 2023-11-28T21:55:51.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.005+0000 7f1a137fe640 1 -- 172.21.15.12:0/790576353 >> v1:172.21.15.144:6789/0 conn(0x7f1a140cb020 legacy=0x7f1a14159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:51.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.005+0000 7f1a137fe640 1 -- 172.21.15.12:0/790576353 >> v1:172.21.15.12:6789/0 conn(0x7f1a140c4980 legacy=0x7f1a140ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:51.012 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.005+0000 7f1a137fe640 1 -- 172.21.15.12:0/790576353 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1415a8c0 con 0x7f1a140cac40 2023-11-28T21:55:51.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a137fe640 1 --2- 172.21.15.12:0/790576353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140cac40 0x7f1a1406b130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a040097f0 tx=0x7f1a0401ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cdeafad8c455263 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:51.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a127fc640 1 -- 172.21.15.12:0/790576353 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040a89b0 con 0x7f1a140cac40 2023-11-28T21:55:51.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a127fc640 1 -- 172.21.15.12:0/790576353 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a040a8b10 con 0x7f1a140cac40 2023-11-28T21:55:51.014 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a127fc640 1 -- 172.21.15.12:0/790576353 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040a8e80 con 0x7f1a140cac40 2023-11-28T21:55:51.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a19f72640 1 -- 172.21.15.12:0/790576353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140cac40 msgr2=0x7f1a1406b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:51.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a19f72640 1 --2- 172.21.15.12:0/790576353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140cac40 0x7f1a1406b130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a040097f0 tx=0x7f1a0401ecb0 comp rx=0 tx=0).stop 2023-11-28T21:55:51.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a19f72640 1 -- 172.21.15.12:0/790576353 shutdown_connections 2023-11-28T21:55:51.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a19f72640 1 --2- 172.21.15.12:0/790576353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140cac40 0x7f1a1406b130 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:51.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a19f72640 1 -- 172.21.15.12:0/790576353 >> 172.21.15.12:0/790576353 conn(0x7f1a140720e0 msgr2=0x7f1a140724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:51.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a19f72640 1 -- 172.21.15.12:0/790576353 shutdown_connections 2023-11-28T21:55:51.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.009+0000 7f1a19f72640 1 -- 172.21.15.12:0/790576353 wait complete. 2023-11-28T21:55:51.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a19f72640 1 Processor -- start 2023-11-28T21:55:51.018 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a19f72640 1 -- start start 2023-11-28T21:55:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a19f72640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140c4980 0x7f1a14154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a19f72640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a140cac40 0x7f1a140c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:51.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a19f72640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a140cb020 0x7f1a14159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a19f72640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140c7e00 con 0x7f1a140cb020 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a19f72640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a14154740 con 0x7f1a140cac40 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a19f72640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a141f0580 con 0x7f1a140c4980 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a13fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a140cb020 0x7f1a14159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a137fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140c4980 0x7f1a14154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a13fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a140cb020 0x7f1a14159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41376/0 (socket says 172.21.15.12:41376) 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a13fff640 1 -- 172.21.15.12:0/799478317 learned_addr learned my addr 172.21.15.12:0/799478317 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a12ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a140cac40 0x7f1a140c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a13fff640 1 -- 172.21.15.12:0/799478317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140c4980 msgr2=0x7f1a14154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a13fff640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140c4980 0x7f1a14154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a13fff640 1 -- 172.21.15.12:0/799478317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a140cac40 msgr2=0x7f1a140c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:51.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a13fff640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a140cac40 0x7f1a140c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:51.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a13fff640 1 -- 172.21.15.12:0/799478317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a04009420 con 0x7f1a140cb020 2023-11-28T21:55:51.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a13fff640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a140cb020 0x7f1a14159c50 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00c780 tx=0x7f1a0c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:51.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f19fbfff640 1 -- 172.21.15.12:0/799478317 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c003db0 con 0x7f1a140cb020 2023-11-28T21:55:51.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f19fbfff640 1 -- 172.21.15.12:0/799478317 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c00ce90 con 0x7f1a140cb020 2023-11-28T21:55:51.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f19fbfff640 1 -- 172.21.15.12:0/799478317 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c0052b0 con 0x7f1a140cb020 2023-11-28T21:55:51.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a19f72640 1 -- 172.21.15.12:0/799478317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a140c9310 con 0x7f1a140cb020 2023-11-28T21:55:51.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.013+0000 7f1a19f72640 1 -- 172.21.15.12:0/799478317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a140c7a90 con 0x7f1a140cb020 2023-11-28T21:55:51.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.017+0000 7f1a19f72640 1 -- 172.21.15.12:0/799478317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a140c7c50 con 0x7f1a140cb020 2023-11-28T21:55:51.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.017+0000 7f19fbfff640 1 -- 172.21.15.12:0/799478317 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1a0c005410 con 0x7f1a140cb020 2023-11-28T21:55:51.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.017+0000 7f19fbfff640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1a00066470 0x7f1a00068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:51.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.017+0000 7f19fbfff640 1 -- 172.21.15.12:0/799478317 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f1a0c084330 con 0x7f1a140cb020 2023-11-28T21:55:51.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.017+0000 7f1a137fe640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1a00066470 0x7f1a00068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:51.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.021+0000 7f1a137fe640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1a00066470 0x7f1a00068930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1a040b2000 tx=0x7f1a040aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:51.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.025+0000 7f19fbfff640 1 -- 172.21.15.12:0/799478317 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1a0c0510a0 con 0x7f1a140cb020 2023-11-28T21:55:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.229+0000 7f1a19f72640 1 -- 172.21.15.12:0/799478317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a14067c70 con 0x7f1a140cb020 2023-11-28T21:55:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.229+0000 7f19fbfff640 1 -- 172.21.15.12:0/799478317 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f1a0c0546c0 con 0x7f1a140cb020 2023-11-28T21:55:51.237 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:51.238 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:49.812029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:reconnect","state_seq":38,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 83 2023-11-28T21:55:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 -- 172.21.15.12:0/799478317 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1a00066470 msgr2=0x7f1a00068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1a00066470 0x7f1a00068930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1a040b2000 tx=0x7f1a040aa070 comp rx=0 tx=0).stop 2023-11-28T21:55:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 -- 172.21.15.12:0/799478317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a140cb020 msgr2=0x7f1a14159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a140cb020 0x7f1a14159c50 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00c780 tx=0x7f1a0c00cc40 comp rx=0 tx=0).stop 2023-11-28T21:55:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 -- 172.21.15.12:0/799478317 shutdown_connections 2023-11-28T21:55:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1a00066470 0x7f1a00068930 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a140cb020 0x7f1a14159c50 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a140cac40 0x7f1a140c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 --2- 172.21.15.12:0/799478317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a140c4980 0x7f1a14154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 -- 172.21.15.12:0/799478317 >> 172.21.15.12:0/799478317 conn(0x7f1a140720e0 msgr2=0x7f1a141578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 -- 172.21.15.12:0/799478317 shutdown_connections 2023-11-28T21:55:51.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:51.233+0000 7f1a19f72640 1 -- 172.21.15.12:0/799478317 wait complete. 2023-11-28T21:55:51.264 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 83, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 83, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6502": { "addr": "172.21.15.144:6833/3873278110", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 3873278110, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 3873278110, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6502, "incarnation": 81, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:reconnect", "state_seq": 38 }, "gid_6513": { "addr": "172.21.15.12:6838/2286739028", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2286739028, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 2286739028, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 6513, "incarnation": 73, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 33 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:55:49.812029+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6513, "mds_1": 6502 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:55:53.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:53.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.413+0000 7f6b5758d640 1 Processor -- start 2023-11-28T21:55:53.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.413+0000 7f6b5758d640 1 -- start start 2023-11-28T21:55:53.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b5758d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5006d950 0x7f6b50161d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:53.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b5758d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b500c3460 con 0x7f6b50158420 2023-11-28T21:55:53.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b5758d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b500c2b70 con 0x7f6b500c4000 2023-11-28T21:55:53.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b5758d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50158800 con 0x7f6b5006d950 2023-11-28T21:55:53.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55329640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f6b50158420 0x7f6b500c38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:49482/0 (socket says 172.21.15.12:49482) 2023-11-28T21:55:53.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55329640 1 -- 172.21.15.12:0/280532216 learned_addr learned my addr 172.21.15.12:0/280532216 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:55:53.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55b2a640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5006d950 0x7f6b50161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312904173 0 0) 0x7f6b500c3460 con 0x7f6b50158420 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b3c003530 con 0x7f6b50158420 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1014962402 0 0) 0x7f6b3c003530 con 0x7f6b50158420 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5006d950 msgr2=0x7f6b50161d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5006d950 0x7f6b50161d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 >> v1:172.21.15.144:6789/0 conn(0x7f6b500c4000 legacy=0x7f6b50161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5006e090 con 0x7f6b50158420 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f6b4c001680 con 0x7f6b50158420 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 >> v1:172.21.15.12:6789/0 conn(0x7f6b50158420 legacy=0x7f6b500c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b3c003a20 0x7f6b3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:53.425 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b3c0047d0 0x7f6b3c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5006d950 0x7f6b3c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50158800 con 0x7f6b3c003a20 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c00ba40 con 0x7f6b3c0047d0 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c00bbd0 con 0x7f6b5006d950 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55329640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b3c003a20 0x7f6b3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b54b28640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b3c0047d0 0x7f6b3c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55b2a640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5006d950 0x7f6b3c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55329640 1 -- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5006d950 msgr2=0x7f6b3c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55329640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5006d950 0x7f6b3c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55329640 1 -- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b3c0047d0 msgr2=0x7f6b3c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55329640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b3c0047d0 0x7f6b3c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:53.426 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55329640 1 -- 172.21.15.12:0/280532216 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c003530 con 0x7f6b3c003a20 2023-11-28T21:55:53.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.417+0000 7f6b55b2a640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5006d950 0x7f6b3c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:55:53.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b55329640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b3c003a20 0x7f6b3c003e00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00c210 tx=0x7f6b4c01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b45a82ce744265 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:53.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c005ae0 con 0x7f6b3c003a20 2023-11-28T21:55:53.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c005c60 con 0x7f6b3c003a20 2023-11-28T21:55:53.427 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b47fff640 1 -- 172.21.15.12:0/280532216 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c007070 con 0x7f6b3c003a20 2023-11-28T21:55:53.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b5758d640 1 -- 172.21.15.12:0/280532216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b3c003a20 msgr2=0x7f6b3c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:53.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b5758d640 1 --2- 172.21.15.12:0/280532216 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b3c003a20 0x7f6b3c003e00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00c210 tx=0x7f6b4c01cc00 comp rx=0 tx=0).stop 2023-11-28T21:55:53.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b55b2a640 1 -- 172.21.15.12:0/280532216 reap_dead start 2023-11-28T21:55:53.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b5758d640 1 -- 172.21.15.12:0/280532216 shutdown_connections 2023-11-28T21:55:53.428 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b5758d640 1 -- 172.21.15.12:0/280532216 >> 172.21.15.12:0/280532216 conn(0x7f6b50066360 msgr2=0x7f6b50066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:53.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b5758d640 1 -- 172.21.15.12:0/280532216 shutdown_connections 2023-11-28T21:55:53.429 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.421+0000 7f6b5758d640 1 -- 172.21.15.12:0/280532216 wait complete. 2023-11-28T21:55:53.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b5758d640 1 Processor -- start 2023-11-28T21:55:53.430 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b5758d640 1 -- start start 2023-11-28T21:55:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b5758d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b3c003a20 0x7f6b501f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b5758d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b501521c0 0x7f6b5006cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b5758d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5015b640 0x7f6b50153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b5758d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b501f6ef0 con 0x7f6b501521c0 2023-11-28T21:55:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b5758d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b501fa660 con 0x7f6b3c003a20 2023-11-28T21:55:53.431 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b5758d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b501fa050 con 0x7f6b5015b640 2023-11-28T21:55:53.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b54b28640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b501521c0 0x7f6b5006cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:53.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b54b28640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b501521c0 0x7f6b5006cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41388/0 (socket says 172.21.15.12:41388) 2023-11-28T21:55:53.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b54b28640 1 -- 172.21.15.12:0/4178049721 learned_addr learned my addr 172.21.15.12:0/4178049721 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:53.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b55329640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b3c003a20 0x7f6b501f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:53.432 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b55b2a640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5015b640 0x7f6b50153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:53.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b54b28640 1 -- 172.21.15.12:0/4178049721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5015b640 msgr2=0x7f6b50153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:53.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b54b28640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5015b640 0x7f6b50153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:53.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b54b28640 1 -- 172.21.15.12:0/4178049721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b3c003a20 msgr2=0x7f6b501f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:53.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b54b28640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b3c003a20 0x7f6b501f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:53.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b54b28640 1 -- 172.21.15.12:0/4178049721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5015bfb0 con 0x7f6b501521c0 2023-11-28T21:55:53.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b54b28640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b501521c0 0x7f6b5006cf20 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000f7c0 tx=0x7f6b4000fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:53.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b45ffb640 1 -- 172.21.15.12:0/4178049721 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400050d0 con 0x7f6b501521c0 2023-11-28T21:55:53.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b45ffb640 1 -- 172.21.15.12:0/4178049721 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b40005230 con 0x7f6b501521c0 2023-11-28T21:55:53.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b45ffb640 1 -- 172.21.15.12:0/4178049721 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4000d730 con 0x7f6b501521c0 2023-11-28T21:55:53.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b5758d640 1 -- 172.21.15.12:0/4178049721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5015c290 con 0x7f6b501521c0 2023-11-28T21:55:53.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.425+0000 7f6b5758d640 1 -- 172.21.15.12:0/4178049721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5015d3f0 con 0x7f6b501521c0 2023-11-28T21:55:53.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.429+0000 7f6b5758d640 1 -- 172.21.15.12:0/4178049721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b501fa810 con 0x7f6b501521c0 2023-11-28T21:55:53.435 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.429+0000 7f6b45ffb640 1 -- 172.21.15.12:0/4178049721 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6b4000d890 con 0x7f6b501521c0 2023-11-28T21:55:53.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.429+0000 7f6b45ffb640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6b34066470 0x7f6b34068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:53.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.429+0000 7f6b45ffb640 1 -- 172.21.15.12:0/4178049721 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f6b400843f0 con 0x7f6b501521c0 2023-11-28T21:55:53.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.429+0000 7f6b55329640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6b34066470 0x7f6b34068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:53.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.429+0000 7f6b55329640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6b34066470 0x7f6b34068930 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0ab000 tx=0x7f6b4c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:53.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.433+0000 7f6b45ffb640 1 -- 172.21.15.12:0/4178049721 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b501fa810 con 0x7f6b501521c0 2023-11-28T21:55:53.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 -- 172.21.15.12:0/4178049721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b501fa7d0 con 0x7f6b501521c0 2023-11-28T21:55:53.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b45ffb640 1 -- 172.21.15.12:0/4178049721 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f6b40051200 con 0x7f6b501521c0 2023-11-28T21:55:53.635 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:53.636 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:49.812029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:reconnect","state_seq":38,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:53.636 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 83 2023-11-28T21:55:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 -- 172.21.15.12:0/4178049721 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6b34066470 msgr2=0x7f6b34068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6b34066470 0x7f6b34068930 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0ab000 tx=0x7f6b4c0a7070 comp rx=0 tx=0).stop 2023-11-28T21:55:53.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 -- 172.21.15.12:0/4178049721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b501521c0 msgr2=0x7f6b5006cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:53.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b501521c0 0x7f6b5006cf20 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000f7c0 tx=0x7f6b4000fc80 comp rx=0 tx=0).stop 2023-11-28T21:55:53.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 -- 172.21.15.12:0/4178049721 shutdown_connections 2023-11-28T21:55:53.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f6b34066470 0x7f6b34068930 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:53.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b5015b640 0x7f6b50153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:53.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b501521c0 0x7f6b5006cf20 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:53.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 --2- 172.21.15.12:0/4178049721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b3c003a20 0x7f6b501f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:53.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 -- 172.21.15.12:0/4178049721 >> 172.21.15.12:0/4178049721 conn(0x7f6b50066360 msgr2=0x7f6b50070570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:53.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 -- 172.21.15.12:0/4178049721 shutdown_connections 2023-11-28T21:55:53.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:53.629+0000 7f6b5758d640 1 -- 172.21.15.12:0/4178049721 wait complete. 2023-11-28T21:55:53.657 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:55:55.657 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:55.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.805+0000 7f05d1dd8640 1 Processor -- start 2023-11-28T21:55:55.813 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.805+0000 7f05d1dd8640 1 -- start start 2023-11-28T21:55:55.814 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05d1dd8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 0x7f05cc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:55.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05d1dd8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05cc157b70 con 0x7f05cc151ed0 2023-11-28T21:55:55.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05d1dd8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05cc157e50 con 0x7f05cc06e300 2023-11-28T21:55:55.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05d1dd8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0707f0 con 0x7f05cc151af0 2023-11-28T21:55:55.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05cb7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 0x7f05cc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:55.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05c3fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f05cc151ed0 0x7f05cc0c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59250/0 (socket says 172.21.15.12:59250) 2023-11-28T21:55:55.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05c3fff640 1 -- 172.21.15.12:0/1535798204 learned_addr learned my addr 172.21.15.12:0/1535798204 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:55:55.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05cbfff640 1 --1- 172.21.15.12:0/1535798204 >> v1:172.21.15.144:6789/0 conn(0x7f05cc06e300 0x7f05cc072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:40174/0 (socket says 172.21.15.12:40174) 2023-11-28T21:55:55.815 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05cb7fe640 1 -- 172.21.15.12:0/1535798204 >> v1:172.21.15.144:6789/0 conn(0x7f05cc06e300 legacy=0x7f05cc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:55.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05cb7fe640 1 -- 172.21.15.12:0/1535798204 >> v1:172.21.15.12:6789/0 conn(0x7f05cc151ed0 legacy=0x7f05cc0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:55.816 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.809+0000 7f05cb7fe640 1 -- 172.21.15.12:0/1535798204 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc158130 con 0x7f05cc151af0 2023-11-28T21:55:55.818 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.813+0000 7f05cb7fe640 1 --2- 172.21.15.12:0/1535798204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 0x7f05cc0c7650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0097f0 tx=0x7f05bc01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e064114f91cc67c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:55.824 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.817+0000 7f05caffd640 1 -- 172.21.15.12:0/1535798204 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc0ab600 con 0x7f05cc151af0 2023-11-28T21:55:55.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.817+0000 7f05caffd640 1 -- 172.21.15.12:0/1535798204 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc0a7050 con 0x7f05cc151af0 2023-11-28T21:55:55.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.817+0000 7f05caffd640 1 -- 172.21.15.12:0/1535798204 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc0ab980 con 0x7f05cc151af0 2023-11-28T21:55:55.825 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.817+0000 7f05d1dd8640 1 -- 172.21.15.12:0/1535798204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 msgr2=0x7f05cc0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:55.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.817+0000 7f05d1dd8640 1 --2- 172.21.15.12:0/1535798204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 0x7f05cc0c7650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0097f0 tx=0x7f05bc01c4c0 comp rx=0 tx=0).stop 2023-11-28T21:55:55.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.817+0000 7f05d1dd8640 1 -- 172.21.15.12:0/1535798204 shutdown_connections 2023-11-28T21:55:55.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.817+0000 7f05d1dd8640 1 --2- 172.21.15.12:0/1535798204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 0x7f05cc0c7650 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:55.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.817+0000 7f05d1dd8640 1 -- 172.21.15.12:0/1535798204 >> 172.21.15.12:0/1535798204 conn(0x7f05cc066360 msgr2=0x7f05cc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:55.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.817+0000 7f05d1dd8640 1 -- 172.21.15.12:0/1535798204 shutdown_connections 2023-11-28T21:55:55.826 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05d1dd8640 1 -- 172.21.15.12:0/1535798204 wait complete. 2023-11-28T21:55:55.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05d1dd8640 1 Processor -- start 2023-11-28T21:55:55.827 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05d1dd8640 1 -- start start 2023-11-28T21:55:55.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05d1dd8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05cc06e300 0x7f05cc13f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:55.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05d1dd8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 0x7f05cc13aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:55.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05d1dd8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05cc13faf0 0x7f05cc13b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:55.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05d1dd8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc140c80 con 0x7f05cc13faf0 2023-11-28T21:55:55.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05d1dd8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0ca400 con 0x7f05cc06e300 2023-11-28T21:55:55.829 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05d1dd8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc141490 con 0x7f05cc151af0 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05cc13faf0 0x7f05cc13b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05cc13faf0 0x7f05cc13b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46208/0 (socket says 172.21.15.12:46208) 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05cbfff640 1 -- 172.21.15.12:0/2108714613 learned_addr learned my addr 172.21.15.12:0/2108714613 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05c3fff640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 0x7f05cc13aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05cb7fe640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05cc06e300 0x7f05cc13f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05cbfff640 1 -- 172.21.15.12:0/2108714613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 msgr2=0x7f05cc13aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05cbfff640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 0x7f05cc13aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05cbfff640 1 -- 172.21.15.12:0/2108714613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05cc06e300 msgr2=0x7f05cc13f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05cbfff640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05cc06e300 0x7f05cc13f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05cbfff640 1 -- 172.21.15.12:0/2108714613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc009420 con 0x7f05cc13faf0 2023-11-28T21:55:55.830 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05cbfff640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05cc13faf0 0x7f05cc13b020 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f05c400c7a0 tx=0x7f05c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:55.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.821+0000 7f05c8ff9640 1 -- 172.21.15.12:0/2108714613 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c400a640 con 0x7f05cc13faf0 2023-11-28T21:55:55.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.825+0000 7f05c8ff9640 1 -- 172.21.15.12:0/2108714613 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c400a7a0 con 0x7f05cc13faf0 2023-11-28T21:55:55.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.825+0000 7f05d1dd8640 1 -- 172.21.15.12:0/2108714613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc0c2180 con 0x7f05cc13faf0 2023-11-28T21:55:55.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.825+0000 7f05c8ff9640 1 -- 172.21.15.12:0/2108714613 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c400ab10 con 0x7f05cc13faf0 2023-11-28T21:55:55.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.825+0000 7f05d1dd8640 1 -- 172.21.15.12:0/2108714613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc0c2440 con 0x7f05cc13faf0 2023-11-28T21:55:55.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.825+0000 7f05d1dd8640 1 -- 172.21.15.12:0/2108714613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc074f60 con 0x7f05cc13faf0 2023-11-28T21:55:55.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.825+0000 7f05c8ff9640 1 -- 172.21.15.12:0/2108714613 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f05c400ac70 con 0x7f05cc13faf0 2023-11-28T21:55:55.832 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.825+0000 7f05c8ff9640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f05ac066470 0x7f05ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:55.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.825+0000 7f05c8ff9640 1 -- 172.21.15.12:0/2108714613 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f05c4083d90 con 0x7f05cc13faf0 2023-11-28T21:55:55.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.825+0000 7f05cb7fe640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f05ac066470 0x7f05ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:55.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.825+0000 7f05cb7fe640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f05ac066470 0x7f05ac068930 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0ad000 tx=0x7f05bc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:55.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:55.829+0000 7f05c8ff9640 1 -- 172.21.15.12:0/2108714613 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f05c4050a80 con 0x7f05cc13faf0 2023-11-28T21:55:56.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.021+0000 7f05d1dd8640 1 -- 172.21.15.12:0/2108714613 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05cc072c10 con 0x7f05cc13faf0 2023-11-28T21:55:56.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.021+0000 7f05c8ff9640 1 -- 172.21.15.12:0/2108714613 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5735 (secure 0 0 0) 0x7f05c40540a0 con 0x7f05cc13faf0 2023-11-28T21:55:56.029 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:56.029 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:49.812029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:reconnect","state_seq":38,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:56.029 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 83 2023-11-28T21:55:56.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 -- 172.21.15.12:0/2108714613 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f05ac066470 msgr2=0x7f05ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:56.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f05ac066470 0x7f05ac068930 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0ad000 tx=0x7f05bc0a6000 comp rx=0 tx=0).stop 2023-11-28T21:55:56.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 -- 172.21.15.12:0/2108714613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05cc13faf0 msgr2=0x7f05cc13b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:56.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05cc13faf0 0x7f05cc13b020 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f05c400c7a0 tx=0x7f05c400cc60 comp rx=0 tx=0).stop 2023-11-28T21:55:56.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 -- 172.21.15.12:0/2108714613 shutdown_connections 2023-11-28T21:55:56.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f05ac066470 0x7f05ac068930 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:56.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05cc13faf0 0x7f05cc13b020 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:56.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05cc151af0 0x7f05cc13aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:56.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 --2- 172.21.15.12:0/2108714613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05cc06e300 0x7f05cc13f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:56.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 -- 172.21.15.12:0/2108714613 >> 172.21.15.12:0/2108714613 conn(0x7f05cc066360 msgr2=0x7f05cc14f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:56.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 -- 172.21.15.12:0/2108714613 shutdown_connections 2023-11-28T21:55:56.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:56.025+0000 7f05d1dd8640 1 -- 172.21.15.12:0/2108714613 wait complete. 2023-11-28T21:55:56.051 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:55:58.052 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:55:58.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.193+0000 7fd1ae52a640 1 Processor -- start 2023-11-28T21:55:58.201 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.193+0000 7fd1ae52a640 1 -- start start 2023-11-28T21:55:58.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1ae52a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd1a8151ed0 0x7fd1a80c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:58.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1ae52a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a806e300 0x7fd1a8072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:58.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1ae52a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a8157b70 con 0x7fd1a8151af0 2023-11-28T21:55:58.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1ae52a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a80707f0 con 0x7fd1a8151ed0 2023-11-28T21:55:58.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1ae52a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80c7b90 con 0x7fd1a806e300 2023-11-28T21:55:58.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1a77fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd1a8151ed0 0x7fd1a80c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:58.203 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1acac7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a806e300 0x7fd1a8072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1a77fe640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd1a8151ed0 0x7fd1a80c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:47924/0 (socket says 172.21.15.12:47924) 2023-11-28T21:55:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1a77fe640 1 -- 172.21.15.12:0/4263835381 learned_addr learned my addr 172.21.15.12:0/4263835381 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1a7fff640 1 --1- 172.21.15.12:0/4263835381 >> v1:172.21.15.144:6789/0 conn(0x7fd1a8151af0 0x7fd1a80c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:40178/0 (socket says 172.21.15.12:40178) 2023-11-28T21:55:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1a77fe640 1 -- 172.21.15.12:0/4263835381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a806e300 msgr2=0x7fd1a8072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:55:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1a77fe640 1 --2- 172.21.15.12:0/4263835381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a806e300 0x7fd1a8072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1a77fe640 1 -- 172.21.15.12:0/4263835381 >> v1:172.21.15.144:6789/0 conn(0x7fd1a8151af0 legacy=0x7fd1a80c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:58.204 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.197+0000 7fd1a77fe640 1 -- 172.21.15.12:0/4263835381 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8157e50 con 0x7fd1a8151ed0 2023-11-28T21:55:58.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1a77fe640 1 --2- 172.21.15.12:0/4263835381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a8151ed0 0x7fd1a80c7650 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009870 tx=0x7fd19c01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5268ad20e2db49ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:58.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1a6ffd640 1 -- 172.21.15.12:0/4263835381 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0ab600 con 0x7fd1a8151ed0 2023-11-28T21:55:58.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1a6ffd640 1 -- 172.21.15.12:0/4263835381 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c0a7050 con 0x7fd1a8151ed0 2023-11-28T21:55:58.206 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1a6ffd640 1 -- 172.21.15.12:0/4263835381 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0ab980 con 0x7fd1a8151ed0 2023-11-28T21:55:58.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1ae52a640 1 -- 172.21.15.12:0/4263835381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a8151ed0 msgr2=0x7fd1a80c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:58.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1ae52a640 1 --2- 172.21.15.12:0/4263835381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a8151ed0 0x7fd1a80c7650 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009870 tx=0x7fd19c01c540 comp rx=0 tx=0).stop 2023-11-28T21:55:58.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1ae52a640 1 -- 172.21.15.12:0/4263835381 shutdown_connections 2023-11-28T21:55:58.207 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1ae52a640 1 --2- 172.21.15.12:0/4263835381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a806e300 0x7fd1a8072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:58.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1ae52a640 1 --2- 172.21.15.12:0/4263835381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a8151ed0 0x7fd1a80c7650 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:58.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1ae52a640 1 -- 172.21.15.12:0/4263835381 >> 172.21.15.12:0/4263835381 conn(0x7fd1a8066360 msgr2=0x7fd1a8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:58.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1ae52a640 1 -- 172.21.15.12:0/4263835381 shutdown_connections 2023-11-28T21:55:58.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1ae52a640 1 -- 172.21.15.12:0/4263835381 wait complete. 2023-11-28T21:55:58.208 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1ae52a640 1 Processor -- start 2023-11-28T21:55:58.209 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.201+0000 7fd1ae52a640 1 -- start start 2023-11-28T21:55:58.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1ae52a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a806e300 0x7fd1a8150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:58.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1ae52a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a8151af0 0x7fd1a81f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:58.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1ae52a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a8158b90 0x7fd1a81f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:58.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1ae52a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a81f7020 con 0x7fd1a806e300 2023-11-28T21:55:58.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1ae52a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a81f2b20 con 0x7fd1a8151af0 2023-11-28T21:55:58.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1ae52a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a81f61c0 con 0x7fd1a8158b90 2023-11-28T21:55:58.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a806e300 0x7fd1a8150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:58.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a806e300 0x7fd1a8150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46220/0 (socket says 172.21.15.12:46220) 2023-11-28T21:55:58.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a7fff640 1 -- 172.21.15.12:0/1674657993 learned_addr learned my addr 172.21.15.12:0/1674657993 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:55:58.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1acac7640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a8158b90 0x7fd1a81f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:58.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a77fe640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a8151af0 0x7fd1a81f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:58.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a7fff640 1 -- 172.21.15.12:0/1674657993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a8158b90 msgr2=0x7fd1a81f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:58.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a7fff640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a8158b90 0x7fd1a81f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:58.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a7fff640 1 -- 172.21.15.12:0/1674657993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a8151af0 msgr2=0x7fd1a81f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:58.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a7fff640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a8151af0 0x7fd1a81f2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:58.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a7fff640 1 -- 172.21.15.12:0/1674657993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c0094d0 con 0x7fd1a806e300 2023-11-28T21:55:58.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a7fff640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a806e300 0x7fd1a8150cc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c420 tx=0x7fd19800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:58.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a4ff9640 1 -- 172.21.15.12:0/1674657993 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1980074e0 con 0x7fd1a806e300 2023-11-28T21:55:58.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a4ff9640 1 -- 172.21.15.12:0/1674657993 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd198007640 con 0x7fd1a806e300 2023-11-28T21:55:58.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a4ff9640 1 -- 172.21.15.12:0/1674657993 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198004a30 con 0x7fd1a806e300 2023-11-28T21:55:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1ae52a640 1 -- 172.21.15.12:0/1674657993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a81f64a0 con 0x7fd1a806e300 2023-11-28T21:55:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1ae52a640 1 -- 172.21.15.12:0/1674657993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a81f6a60 con 0x7fd1a806e300 2023-11-28T21:55:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1ae52a640 1 -- 172.21.15.12:0/1674657993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a81f2c90 con 0x7fd1a806e300 2023-11-28T21:55:58.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.205+0000 7fd1a4ff9640 1 -- 172.21.15.12:0/1674657993 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd198004b90 con 0x7fd1a806e300 2023-11-28T21:55:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.209+0000 7fd1a4ff9640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd1940664a0 0x7fd194068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:55:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.209+0000 7fd1a4ff9640 1 -- 172.21.15.12:0/1674657993 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fd198083e90 con 0x7fd1a806e300 2023-11-28T21:55:58.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.209+0000 7fd1a77fe640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd1940664a0 0x7fd194068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:55:58.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.209+0000 7fd1a77fe640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd1940664a0 0x7fd194068960 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80c7d20 tx=0x7fd19c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:55:58.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.213+0000 7fd1a4ff9640 1 -- 172.21.15.12:0/1674657993 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd1a81f2c90 con 0x7fd1a806e300 2023-11-28T21:55:58.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 -- 172.21.15.12:0/1674657993 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1a81f2c90 con 0x7fd1a806e300 2023-11-28T21:55:58.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1a4ff9640 1 -- 172.21.15.12:0/1674657993 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5732 (secure 0 0 0) 0x7fd1a81f2c90 con 0x7fd1a806e300 2023-11-28T21:55:58.411 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:55:58.411 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:55.823170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:rejoin","state_seq":40,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:55:58.411 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 84 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 -- 172.21.15.12:0/1674657993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd1940664a0 msgr2=0x7fd194068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd1940664a0 0x7fd194068960 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80c7d20 tx=0x7fd19c0a6000 comp rx=0 tx=0).stop 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 -- 172.21.15.12:0/1674657993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a806e300 msgr2=0x7fd1a8150cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a806e300 0x7fd1a8150cc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c420 tx=0x7fd19800c8e0 comp rx=0 tx=0).stop 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 -- 172.21.15.12:0/1674657993 shutdown_connections 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd1940664a0 0x7fd194068960 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1a8158b90 0x7fd1a81f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1a8151af0 0x7fd1a81f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 --2- 172.21.15.12:0/1674657993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1a806e300 0x7fd1a8150cc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 -- 172.21.15.12:0/1674657993 >> 172.21.15.12:0/1674657993 conn(0x7fd1a8066360 msgr2=0x7fd1a80c49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:55:58.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 -- 172.21.15.12:0/1674657993 shutdown_connections 2023-11-28T21:55:58.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:55:58.405+0000 7fd1ae52a640 1 -- 172.21.15.12:0/1674657993 wait complete. 2023-11-28T21:55:58.432 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:56:00.434 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:56:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.581+0000 7fc4624d6640 1 Processor -- start 2023-11-28T21:56:00.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.581+0000 7fc4624d6640 1 -- start start 2023-11-28T21:56:00.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc4624d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc45c151af0 0x7fc45c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc4624d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc45c06e300 0x7fc45c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc4624d6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45c157b70 con 0x7fc45c151ed0 2023-11-28T21:56:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc4624d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c0707f0 con 0x7fc45c06e300 2023-11-28T21:56:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc4624d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0c7b90 con 0x7fc45c151af0 2023-11-28T21:56:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc460a73640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc45c06e300 0x7fc45c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc460a73640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc45c06e300 0x7fc45c072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46230/0 (socket says 172.21.15.12:46230) 2023-11-28T21:56:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc460a73640 1 -- 172.21.15.12:0/1296264276 learned_addr learned my addr 172.21.15.12:0/1296264276 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:00.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc45bfff640 1 --2- 172.21.15.12:0/1296264276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc45c151af0 0x7fc45c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:00.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc460a73640 1 -- 172.21.15.12:0/1296264276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc45c151af0 msgr2=0x7fc45c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:00.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc460a73640 1 --2- 172.21.15.12:0/1296264276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc45c151af0 0x7fc45c0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:00.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc460a73640 1 -- 172.21.15.12:0/1296264276 >> v1:172.21.15.144:6789/0 conn(0x7fc45c151ed0 legacy=0x7fc45c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:00.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.585+0000 7fc460a73640 1 -- 172.21.15.12:0/1296264276 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c157e50 con 0x7fc45c06e300 2023-11-28T21:56:00.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc460a73640 1 --2- 172.21.15.12:0/1296264276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 0x7fc45c072070 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc454009e50 tx=0x7fc45401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fbcae7919d78b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:00.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc45affd640 1 -- 172.21.15.12:0/1296264276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540a56e0 con 0x7fc45c06e300 2023-11-28T21:56:00.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc45affd640 1 -- 172.21.15.12:0/1296264276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4540a5840 con 0x7fc45c06e300 2023-11-28T21:56:00.594 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc45affd640 1 -- 172.21.15.12:0/1296264276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540a5bd0 con 0x7fc45c06e300 2023-11-28T21:56:00.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc4624d6640 1 -- 172.21.15.12:0/1296264276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 msgr2=0x7fc45c072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:00.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc4624d6640 1 --2- 172.21.15.12:0/1296264276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 0x7fc45c072070 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc454009e50 tx=0x7fc45401ca10 comp rx=0 tx=0).stop 2023-11-28T21:56:00.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc4624d6640 1 -- 172.21.15.12:0/1296264276 shutdown_connections 2023-11-28T21:56:00.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc4624d6640 1 --2- 172.21.15.12:0/1296264276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 0x7fc45c072070 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:00.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc4624d6640 1 --2- 172.21.15.12:0/1296264276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc45c151af0 0x7fc45c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:00.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc4624d6640 1 -- 172.21.15.12:0/1296264276 >> 172.21.15.12:0/1296264276 conn(0x7fc45c066360 msgr2=0x7fc45c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:00.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc4624d6640 1 -- 172.21.15.12:0/1296264276 shutdown_connections 2023-11-28T21:56:00.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc4624d6640 1 -- 172.21.15.12:0/1296264276 wait complete. 2023-11-28T21:56:00.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc4624d6640 1 Processor -- start 2023-11-28T21:56:00.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.589+0000 7fc4624d6640 1 -- start start 2023-11-28T21:56:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc4624d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 0x7fc45c138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc4624d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc45c151af0 0x7fc45c0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc4624d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc45c0c9600 0x7fc45c147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc4624d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0cac00 con 0x7fc45c06e300 2023-11-28T21:56:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc4624d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c139c50 con 0x7fc45c151af0 2023-11-28T21:56:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc4624d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c13a0e0 con 0x7fc45c0c9600 2023-11-28T21:56:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc45bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 0x7fc45c138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc45bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 0x7fc45c138100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46236/0 (socket says 172.21.15.12:46236) 2023-11-28T21:56:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc45bfff640 1 -- 172.21.15.12:0/1347877190 learned_addr learned my addr 172.21.15.12:0/1347877190 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:00.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc460a73640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc45c0c9600 0x7fc45c147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:00.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc45b7fe640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc45c151af0 0x7fc45c0c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:00.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc45bfff640 1 -- 172.21.15.12:0/1347877190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc45c0c9600 msgr2=0x7fc45c147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:00.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc45bfff640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc45c0c9600 0x7fc45c147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:00.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc45bfff640 1 -- 172.21.15.12:0/1347877190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc45c151af0 msgr2=0x7fc45c0c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:00.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc45bfff640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc45c151af0 0x7fc45c0c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:00.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc45bfff640 1 -- 172.21.15.12:0/1347877190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc454009b30 con 0x7fc45c06e300 2023-11-28T21:56:00.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc45bfff640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 0x7fc45c138100 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00e960 tx=0x7fc44c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:00.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc458ff9640 1 -- 172.21.15.12:0/1347877190 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c009690 con 0x7fc45c06e300 2023-11-28T21:56:00.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc458ff9640 1 -- 172.21.15.12:0/1347877190 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44c00ca00 con 0x7fc45c06e300 2023-11-28T21:56:00.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc458ff9640 1 -- 172.21.15.12:0/1347877190 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c00cd70 con 0x7fc45c06e300 2023-11-28T21:56:00.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc4624d6640 1 -- 172.21.15.12:0/1347877190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c13a720 con 0x7fc45c06e300 2023-11-28T21:56:00.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc4624d6640 1 -- 172.21.15.12:0/1347877190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45c139040 con 0x7fc45c06e300 2023-11-28T21:56:00.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc4624d6640 1 -- 172.21.15.12:0/1347877190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45c06a080 con 0x7fc45c06e300 2023-11-28T21:56:00.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.593+0000 7fc458ff9640 1 -- 172.21.15.12:0/1347877190 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc44c013470 con 0x7fc45c06e300 2023-11-28T21:56:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.597+0000 7fc458ff9640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc4480664a0 0x7fc448068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.597+0000 7fc458ff9640 1 -- 172.21.15.12:0/1347877190 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fc44c084790 con 0x7fc45c06e300 2023-11-28T21:56:00.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.597+0000 7fc45b7fe640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc4480664a0 0x7fc448068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:00.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.597+0000 7fc45b7fe640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc4480664a0 0x7fc448068960 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc45c071e00 tx=0x7fc450009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:00.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.601+0000 7fc458ff9640 1 -- 172.21.15.12:0/1347877190 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc44c051500 con 0x7fc45c06e300 2023-11-28T21:56:00.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.789+0000 7fc4624d6640 1 -- 172.21.15.12:0/1347877190 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc45c074f60 con 0x7fc45c06e300 2023-11-28T21:56:00.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.789+0000 7fc458ff9640 1 -- 172.21.15.12:0/1347877190 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5732 (secure 0 0 0) 0x7fc44c054b20 con 0x7fc45c06e300 2023-11-28T21:56:00.796 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:00.797 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:55:55.823170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:rejoin","state_seq":40,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:56:00.797 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 84 2023-11-28T21:56:00.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 -- 172.21.15.12:0/1347877190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc4480664a0 msgr2=0x7fc448068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:00.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc4480664a0 0x7fc448068960 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc45c071e00 tx=0x7fc450009040 comp rx=0 tx=0).stop 2023-11-28T21:56:00.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 -- 172.21.15.12:0/1347877190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 msgr2=0x7fc45c138100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:00.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 0x7fc45c138100 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00e960 tx=0x7fc44c00ee20 comp rx=0 tx=0).stop 2023-11-28T21:56:00.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 -- 172.21.15.12:0/1347877190 shutdown_connections 2023-11-28T21:56:00.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc4480664a0 0x7fc448068960 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:00.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc45c0c9600 0x7fc45c147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:00.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc45c151af0 0x7fc45c0c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:00.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 --2- 172.21.15.12:0/1347877190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc45c06e300 0x7fc45c138100 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 -- 172.21.15.12:0/1347877190 >> 172.21.15.12:0/1347877190 conn(0x7fc45c066360 msgr2=0x7fc45c073a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 -- 172.21.15.12:0/1347877190 shutdown_connections 2023-11-28T21:56:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:00.793+0000 7fc4624d6640 1 -- 172.21.15.12:0/1347877190 wait complete. 2023-11-28T21:56:00.818 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:56:02.819 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:56:02.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.965+0000 7f20e3654640 1 Processor -- start 2023-11-28T21:56:02.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e3654640 1 -- start start 2023-11-28T21:56:02.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e3654640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc158b20 0x7f20dc156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:02.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e3654640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20dc067410 con 0x7f20dc06e300 2023-11-28T21:56:02.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e3654640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20dc06e090 con 0x7f20dc150c20 2023-11-28T21:56:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e3654640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc157f30 con 0x7f20dc158b20 2023-11-28T21:56:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e0bef640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f20dc06e300 0x7f20dc158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59252/0 (socket says 172.21.15.12:59252) 2023-11-28T21:56:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e0bef640 1 -- 172.21.15.12:0/1934383539 learned_addr learned my addr 172.21.15.12:0/1934383539 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:02.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e1bf1640 1 --2- 172.21.15.12:0/1934383539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc158b20 0x7f20dc156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20d3fff640 1 -- 172.21.15.12:0/1934383539 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2555421952 0 0) 0x7f20dc067410 con 0x7f20dc06e300 2023-11-28T21:56:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20d3fff640 1 -- 172.21.15.12:0/1934383539 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c8003530 con 0x7f20dc06e300 2023-11-28T21:56:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20d3fff640 1 -- 172.21.15.12:0/1934383539 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663959147 0 0) 0x7f20dc06e090 con 0x7f20dc150c20 2023-11-28T21:56:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20d3fff640 1 -- 172.21.15.12:0/1934383539 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20dc067410 con 0x7f20dc150c20 2023-11-28T21:56:02.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e1bf1640 1 -- 172.21.15.12:0/1934383539 >> v1:172.21.15.144:6789/0 conn(0x7f20dc150c20 legacy=0x7f20dc0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e1bf1640 1 -- 172.21.15.12:0/1934383539 >> v1:172.21.15.12:6789/0 conn(0x7f20dc06e300 legacy=0x7f20dc158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20d3fff640 1 -- 172.21.15.12:0/1934383539 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 478189766 0 0) 0x7f20c8003530 con 0x7f20dc06e300 2023-11-28T21:56:02.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.969+0000 7f20e1bf1640 1 -- 172.21.15.12:0/1934383539 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc151000 con 0x7f20dc158b20 2023-11-28T21:56:02.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20e1bf1640 1 --2- 172.21.15.12:0/1934383539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc158b20 0x7f20dc156910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009dc0 tx=0x7f20d401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d4fdefe583310c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:02.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20d3fff640 1 -- 172.21.15.12:0/1934383539 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc001680 con 0x7f20dc158b20 2023-11-28T21:56:02.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20d3fff640 1 -- 172.21.15.12:0/1934383539 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d40a55e0 con 0x7f20dc158b20 2023-11-28T21:56:02.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20d3fff640 1 -- 172.21.15.12:0/1934383539 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40a5950 con 0x7f20dc158b20 2023-11-28T21:56:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20e3654640 1 -- 172.21.15.12:0/1934383539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc158b20 msgr2=0x7f20dc156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20e3654640 1 --2- 172.21.15.12:0/1934383539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc158b20 0x7f20dc156910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009dc0 tx=0x7f20d401c890 comp rx=0 tx=0).stop 2023-11-28T21:56:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20e3654640 1 -- 172.21.15.12:0/1934383539 shutdown_connections 2023-11-28T21:56:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20e3654640 1 --2- 172.21.15.12:0/1934383539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc158b20 0x7f20dc156910 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20e3654640 1 -- 172.21.15.12:0/1934383539 >> 172.21.15.12:0/1934383539 conn(0x7f20dc066360 msgr2=0x7f20dc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20e3654640 1 -- 172.21.15.12:0/1934383539 shutdown_connections 2023-11-28T21:56:02.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.973+0000 7f20e3654640 1 -- 172.21.15.12:0/1934383539 wait complete. 2023-11-28T21:56:02.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e3654640 1 Processor -- start 2023-11-28T21:56:02.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e3654640 1 -- start start 2023-11-28T21:56:02.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e3654640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc06e300 0x7f20dc0c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:02.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e3654640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc150c20 0x7f20dc1f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:02.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e3654640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20dc158b20 0x7f20dc1f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:02.984 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e3654640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc0c3eb0 con 0x7f20dc158b20 2023-11-28T21:56:02.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e3654640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc0c4880 con 0x7f20dc150c20 2023-11-28T21:56:02.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e3654640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc1f6d80 con 0x7f20dc06e300 2023-11-28T21:56:02.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e1bf1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20dc158b20 0x7f20dc1f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:02.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e1bf1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20dc158b20 0x7f20dc1f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46240/0 (socket says 172.21.15.12:46240) 2023-11-28T21:56:02.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e1bf1640 1 -- 172.21.15.12:0/2078866516 learned_addr learned my addr 172.21.15.12:0/2078866516 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:02.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e13f0640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc06e300 0x7f20dc0c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:02.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e0bef640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc150c20 0x7f20dc1f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:02.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e1bf1640 1 -- 172.21.15.12:0/2078866516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc06e300 msgr2=0x7f20dc0c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:02.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e1bf1640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc06e300 0x7f20dc0c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:02.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e1bf1640 1 -- 172.21.15.12:0/2078866516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc150c20 msgr2=0x7f20dc1f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:02.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e1bf1640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc150c20 0x7f20dc1f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:02.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e1bf1640 1 -- 172.21.15.12:0/2078866516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4009a70 con 0x7f20dc158b20 2023-11-28T21:56:02.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e0bef640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc150c20 0x7f20dc1f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:02.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e1bf1640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20dc158b20 0x7f20dc1f6670 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f20d401af40 tx=0x7f20d4019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:02.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20d1ffb640 1 -- 172.21.15.12:0/2078866516 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40bc070 con 0x7f20dc158b20 2023-11-28T21:56:02.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20d1ffb640 1 -- 172.21.15.12:0/2078866516 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d40a5960 con 0x7f20dc158b20 2023-11-28T21:56:02.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20d1ffb640 1 -- 172.21.15.12:0/2078866516 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40a5cd0 con 0x7f20dc158b20 2023-11-28T21:56:02.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e3654640 1 -- 172.21.15.12:0/2078866516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc14cec0 con 0x7f20dc158b20 2023-11-28T21:56:02.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.977+0000 7f20e3654640 1 -- 172.21.15.12:0/2078866516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc0c3860 con 0x7f20dc158b20 2023-11-28T21:56:02.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.981+0000 7f20e3654640 1 -- 172.21.15.12:0/2078866516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20dc074f60 con 0x7f20dc158b20 2023-11-28T21:56:02.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.981+0000 7f20d1ffb640 1 -- 172.21.15.12:0/2078866516 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f20d40b4070 con 0x7f20dc158b20 2023-11-28T21:56:02.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.981+0000 7f20d1ffb640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c0066470 0x7f20c0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:02.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.981+0000 7f20d1ffb640 1 -- 172.21.15.12:0/2078866516 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f20d411e960 con 0x7f20dc158b20 2023-11-28T21:56:02.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.981+0000 7f20e13f0640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c0066470 0x7f20c0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:02.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.981+0000 7f20e13f0640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c0066470 0x7f20c0068930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f20d80017c0 tx=0x7f20d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:02.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:02.985+0000 7f20d1ffb640 1 -- 172.21.15.12:0/2078866516 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f20d40eb620 con 0x7f20dc158b20 2023-11-28T21:56:03.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.173+0000 7f20e3654640 1 -- 172.21.15.12:0/2078866516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20dc0c3ff0 con 0x7f20dc158b20 2023-11-28T21:56:03.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.173+0000 7f20d1ffb640 1 -- 172.21.15.12:0/2078866516 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f20d40f5030 con 0x7f20dc158b20 2023-11-28T21:56:03.181 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:03.182 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:56:02.009527+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:clientreplay","state_seq":42,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:56:03.182 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 85 2023-11-28T21:56:03.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 -- 172.21.15.12:0/2078866516 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c0066470 msgr2=0x7f20c0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:03.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c0066470 0x7f20c0068930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f20d80017c0 tx=0x7f20d8009040 comp rx=0 tx=0).stop 2023-11-28T21:56:03.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 -- 172.21.15.12:0/2078866516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20dc158b20 msgr2=0x7f20dc1f6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:03.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20dc158b20 0x7f20dc1f6670 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f20d401af40 tx=0x7f20d4019630 comp rx=0 tx=0).stop 2023-11-28T21:56:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 -- 172.21.15.12:0/2078866516 shutdown_connections 2023-11-28T21:56:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f20c0066470 0x7f20c0068930 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f20dc158b20 0x7f20dc1f6670 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20dc150c20 0x7f20dc1f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 --2- 172.21.15.12:0/2078866516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20dc06e300 0x7f20dc0c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 -- 172.21.15.12:0/2078866516 >> 172.21.15.12:0/2078866516 conn(0x7f20dc066360 msgr2=0x7f20dc0c9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 -- 172.21.15.12:0/2078866516 shutdown_connections 2023-11-28T21:56:03.184 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:03.177+0000 7f20e3654640 1 -- 172.21.15.12:0/2078866516 wait complete. 2023-11-28T21:56:03.205 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 85, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-28T21:34:36.378416+0000", "damaged": [], "data_pools": [ 3, 4 ], "enabled": true, "epoch": 85, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_6502": { "addr": "172.21.15.144:6833/3873278110", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6832", "nonce": 3873278110, "type": "v2" }, { "addr": "172.21.15.144:6833", "nonce": 3873278110, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6502, "incarnation": 81, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:clientreplay", "state_seq": 42 }, "gid_6513": { "addr": "172.21.15.12:6838/2286739028", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6834", "nonce": 2286739028, "type": "v2" }, { "addr": "172.21.15.12:6838", "nonce": 2286739028, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 6513, "incarnation": 73, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 33 } }, "last_failure": 0, "last_failure_osd_epoch": 52, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2023-11-28T21:56:02.009527+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 6513, "mds_1": 6502 } } } ], "standbys": [ { "addr": "172.21.15.144:6836/3101383243", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6834", "nonce": 3101383243, "type": "v2" }, { "addr": "172.21.15.144:6836", "nonce": 3101383243, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4238, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.144:6837/4030769266", "addrs": { "addrvec": [ { "addr": "172.21.15.144:6835", "nonce": 4030769266, "type": "v2" }, { "addr": "172.21.15.144:6837", "nonce": 4030769266, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4241, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.12:6837/1717483656", "addrs": { "addrvec": [ { "addr": "172.21.15.12:6835", "nonce": 1717483656, "type": "v2" }, { "addr": "172.21.15.12:6837", "nonce": 1717483656, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-28T21:56:05.206 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:56:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e25f2a640 1 Processor -- start 2023-11-28T21:56:05.362 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e25f2a640 1 -- start start 2023-11-28T21:56:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e25f2a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7e2006e300 0x7f7e200c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e25f2a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e20157b70 con 0x7f7e20151ed0 2023-11-28T21:56:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e25f2a640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e20157e50 con 0x7f7e20151af0 2023-11-28T21:56:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e25f2a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e200707f0 con 0x7f7e2006e300 2023-11-28T21:56:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e16ffd640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f7e20151ed0 0x7f7e20072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45056/0 (socket says 172.21.15.12:45056) 2023-11-28T21:56:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e16ffd640 1 -- 172.21.15.12:0/2791777995 learned_addr learned my addr 172.21.15.12:0/2791777995 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e1ffff640 1 --2- 172.21.15.12:0/2791777995 >> v2:172.21.15.144:3300/0 conn(0x7f7e2006e300 0x7f7e200c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e1effd640 1 -- 172.21.15.12:0/2791777995 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113562938 0 0) 0x7f7e20157b70 con 0x7f7e20151ed0 2023-11-28T21:56:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e1effd640 1 -- 172.21.15.12:0/2791777995 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e0c003530 con 0x7f7e20151ed0 2023-11-28T21:56:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e1effd640 1 -- 172.21.15.12:0/2791777995 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842092338 0 0) 0x7f7e20157e50 con 0x7f7e20151af0 2023-11-28T21:56:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e1effd640 1 -- 172.21.15.12:0/2791777995 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e20157b70 con 0x7f7e20151af0 2023-11-28T21:56:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e1effd640 1 -- 172.21.15.12:0/2791777995 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1356516182 0 0) 0x7f7e0c003530 con 0x7f7e20151ed0 2023-11-28T21:56:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e1ffff640 1 -- 172.21.15.12:0/2791777995 >> v1:172.21.15.144:6789/0 conn(0x7f7e20151af0 legacy=0x7f7e200c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e1ffff640 1 -- 172.21.15.12:0/2791777995 >> v1:172.21.15.12:6789/0 conn(0x7f7e20151ed0 legacy=0x7f7e20072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:05.365 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.357+0000 7f7e1ffff640 1 -- 172.21.15.12:0/2791777995 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20158130 con 0x7f7e2006e300 2023-11-28T21:56:05.367 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e1ffff640 1 --2- 172.21.15.12:0/2791777995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e2006e300 0x7f7e200c7650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009d90 tx=0x7f7e1801c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfa808ac23063d45 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:05.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e1effd640 1 -- 172.21.15.12:0/2791777995 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10001680 con 0x7f7e2006e300 2023-11-28T21:56:05.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e1effd640 1 -- 172.21.15.12:0/2791777995 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e180a64c0 con 0x7f7e2006e300 2023-11-28T21:56:05.368 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e1effd640 1 -- 172.21.15.12:0/2791777995 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e180a6830 con 0x7f7e2006e300 2023-11-28T21:56:05.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e25f2a640 1 -- 172.21.15.12:0/2791777995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e2006e300 msgr2=0x7f7e200c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:05.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e25f2a640 1 --2- 172.21.15.12:0/2791777995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e2006e300 0x7f7e200c7650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009d90 tx=0x7f7e1801c870 comp rx=0 tx=0).stop 2023-11-28T21:56:05.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e25f2a640 1 -- 172.21.15.12:0/2791777995 shutdown_connections 2023-11-28T21:56:05.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e25f2a640 1 --2- 172.21.15.12:0/2791777995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e2006e300 0x7f7e200c7650 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:05.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e25f2a640 1 -- 172.21.15.12:0/2791777995 >> 172.21.15.12:0/2791777995 conn(0x7f7e20066360 msgr2=0x7f7e20066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:05.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e25f2a640 1 -- 172.21.15.12:0/2791777995 shutdown_connections 2023-11-28T21:56:05.369 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.361+0000 7f7e25f2a640 1 -- 172.21.15.12:0/2791777995 wait complete. 2023-11-28T21:56:05.370 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e25f2a640 1 Processor -- start 2023-11-28T21:56:05.371 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e25f2a640 1 -- start start 2023-11-28T21:56:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e25f2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2006e300 0x7f7e20154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e25f2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e20151af0 0x7f7e20156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e25f2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e20155560 0x7f7e20156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e25f2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20154320 con 0x7f7e2006e300 2023-11-28T21:56:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e25f2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20156620 con 0x7f7e20155560 2023-11-28T21:56:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e25f2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e20152440 con 0x7f7e20151af0 2023-11-28T21:56:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2006e300 0x7f7e20154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2006e300 0x7f7e20154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49664/0 (socket says 172.21.15.12:49664) 2023-11-28T21:56:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1f7fe640 1 -- 172.21.15.12:0/1874165284 learned_addr learned my addr 172.21.15.12:0/1874165284 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:05.373 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e16ffd640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e20151af0 0x7f7e20156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1ffff640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e20155560 0x7f7e20156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1f7fe640 1 -- 172.21.15.12:0/1874165284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e20151af0 msgr2=0x7f7e20156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1f7fe640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e20151af0 0x7f7e20156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1f7fe640 1 -- 172.21.15.12:0/1874165284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e20155560 msgr2=0x7f7e20156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1f7fe640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e20155560 0x7f7e20156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1f7fe640 1 -- 172.21.15.12:0/1874165284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e18009a70 con 0x7f7e2006e300 2023-11-28T21:56:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1f7fe640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2006e300 0x7f7e20154bf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800c490 tx=0x7f7e0800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:05.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1cff9640 1 -- 172.21.15.12:0/1874165284 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e080052d0 con 0x7f7e2006e300 2023-11-28T21:56:05.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1cff9640 1 -- 172.21.15.12:0/1874165284 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e08005430 con 0x7f7e2006e300 2023-11-28T21:56:05.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e1cff9640 1 -- 172.21.15.12:0/1874165284 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08003840 con 0x7f7e2006e300 2023-11-28T21:56:05.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e25f2a640 1 -- 172.21.15.12:0/1874165284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e20143800 con 0x7f7e2006e300 2023-11-28T21:56:05.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.365+0000 7f7e25f2a640 1 -- 172.21.15.12:0/1874165284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e201529e0 con 0x7f7e2006e300 2023-11-28T21:56:05.375 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.369+0000 7f7e25f2a640 1 -- 172.21.15.12:0/1874165284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e20074f60 con 0x7f7e2006e300 2023-11-28T21:56:05.376 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.369+0000 7f7e1cff9640 1 -- 172.21.15.12:0/1874165284 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7e080039a0 con 0x7f7e2006e300 2023-11-28T21:56:05.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.369+0000 7f7e1cff9640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7e000664a0 0x7f7e00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:05.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.369+0000 7f7e1cff9640 1 -- 172.21.15.12:0/1874165284 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f7e08083ca0 con 0x7f7e2006e300 2023-11-28T21:56:05.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.369+0000 7f7e16ffd640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7e000664a0 0x7f7e00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:05.377 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.373+0000 7f7e16ffd640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7e000664a0 0x7f7e00068960 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7e20163210 tx=0x7f7e10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:05.382 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.377+0000 7f7e1cff9640 1 -- 172.21.15.12:0/1874165284 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7e08050970 con 0x7f7e2006e300 2023-11-28T21:56:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 -- 172.21.15.12:0/1874165284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e20072c10 con 0x7f7e2006e300 2023-11-28T21:56:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e1cff9640 1 -- 172.21.15.12:0/1874165284 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5738 (secure 0 0 0) 0x7f7e08017090 con 0x7f7e2006e300 2023-11-28T21:56:05.571 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:05.572 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:56:02.009527+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:clientreplay","state_seq":42,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:56:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 85 2023-11-28T21:56:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 -- 172.21.15.12:0/1874165284 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7e000664a0 msgr2=0x7f7e00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7e000664a0 0x7f7e00068960 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7e20163210 tx=0x7f7e10009040 comp rx=0 tx=0).stop 2023-11-28T21:56:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 -- 172.21.15.12:0/1874165284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2006e300 msgr2=0x7f7e20154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2006e300 0x7f7e20154bf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f7e0800c490 tx=0x7f7e0800c950 comp rx=0 tx=0).stop 2023-11-28T21:56:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 -- 172.21.15.12:0/1874165284 shutdown_connections 2023-11-28T21:56:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f7e000664a0 0x7f7e00068960 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e20155560 0x7f7e20156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e20151af0 0x7f7e20156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 --2- 172.21.15.12:0/1874165284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2006e300 0x7f7e20154bf0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 -- 172.21.15.12:0/1874165284 >> 172.21.15.12:0/1874165284 conn(0x7f7e20066360 msgr2=0x7f7e200c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 -- 172.21.15.12:0/1874165284 shutdown_connections 2023-11-28T21:56:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:05.565+0000 7f7e25f2a640 1 -- 172.21.15.12:0/1874165284 wait complete. 2023-11-28T21:56:05.593 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:56:07.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:56:07.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.741+0000 7fb800180640 1 Processor -- start 2023-11-28T21:56:07.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.741+0000 7fb800180640 1 -- start start 2023-11-28T21:56:07.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb800180640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f806d950 0x7fb7f8161d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb800180640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80c3460 con 0x7fb7f8158420 2023-11-28T21:56:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb800180640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80c2b70 con 0x7fb7f80c4000 2023-11-28T21:56:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb800180640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f8158800 con 0x7fb7f806d950 2023-11-28T21:56:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fdf1c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb7f8158420 0x7fb7f80c38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45058/0 (socket says 172.21.15.12:45058) 2023-11-28T21:56:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fdf1c640 1 -- 172.21.15.12:0/476676348 learned_addr learned my addr 172.21.15.12:0/476676348 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:07.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fe71d640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f806d950 0x7fb7f8161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208162711 0 0) 0x7fb7f80c3460 con 0x7fb7f8158420 2023-11-28T21:56:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e4003530 con 0x7fb7f8158420 2023-11-28T21:56:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3039425202 0 0) 0x7fb7e4003530 con 0x7fb7f8158420 2023-11-28T21:56:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f806d950 msgr2=0x7fb7f8161d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f806d950 0x7fb7f8161d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 >> v1:172.21.15.144:6789/0 conn(0x7fb7f80c4000 legacy=0x7fb7f8161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f806e090 con 0x7fb7f8158420 2023-11-28T21:56:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fb7f4001680 con 0x7fb7f8158420 2023-11-28T21:56:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 >> v1:172.21.15.12:6789/0 conn(0x7fb7f8158420 legacy=0x7fb7f80c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:07.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e4003a20 0x7fb7e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f806d950 0x7fb7e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7e4004d10 0x7fb7e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f8158800 con 0x7fb7e4003a20 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e400ba40 con 0x7fb7e4004d10 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e400bbb0 con 0x7fb7f806d950 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fdf1c640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e4003a20 0x7fb7e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fd71b640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f806d950 0x7fb7e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fe71d640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7e4004d10 0x7fb7e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fdf1c640 1 -- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f806d950 msgr2=0x7fb7e40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fdf1c640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f806d950 0x7fb7e40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fdf1c640 1 -- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7e4004d10 msgr2=0x7fb7e4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:07.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fdf1c640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7e4004d10 0x7fb7e4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:07.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.745+0000 7fb7fdf1c640 1 -- 172.21.15.12:0/476676348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e4003530 con 0x7fb7e4003a20 2023-11-28T21:56:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb7fdf1c640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e4003a20 0x7fb7e4003e00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c210 tx=0x7fb7f401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806f56be1997f1b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4005ae0 con 0x7fb7e4003a20 2023-11-28T21:56:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f4005c60 con 0x7fb7e4003a20 2023-11-28T21:56:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb7fcf1a640 1 -- 172.21.15.12:0/476676348 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4007070 con 0x7fb7e4003a20 2023-11-28T21:56:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb800180640 1 -- 172.21.15.12:0/476676348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e4003a20 msgr2=0x7fb7e4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb800180640 1 --2- 172.21.15.12:0/476676348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7e4003a20 0x7fb7e4003e00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c210 tx=0x7fb7f401cc60 comp rx=0 tx=0).stop 2023-11-28T21:56:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb7fe71d640 1 -- 172.21.15.12:0/476676348 reap_dead start 2023-11-28T21:56:07.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb800180640 1 -- 172.21.15.12:0/476676348 shutdown_connections 2023-11-28T21:56:07.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb800180640 1 -- 172.21.15.12:0/476676348 >> 172.21.15.12:0/476676348 conn(0x7fb7f8066360 msgr2=0x7fb7f8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:07.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb800180640 1 -- 172.21.15.12:0/476676348 shutdown_connections 2023-11-28T21:56:07.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb800180640 1 -- 172.21.15.12:0/476676348 wait complete. 2023-11-28T21:56:07.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.749+0000 7fb800180640 1 Processor -- start 2023-11-28T21:56:07.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb800180640 1 -- start start 2023-11-28T21:56:07.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb800180640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7e4003a20 0x7fb7f81f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:07.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb800180640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f81521c0 0x7fb7f806cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:07.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb800180640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f815b640 0x7fb7f8153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:07.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb800180640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81f6ef0 con 0x7fb7f815b640 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb800180640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81fa660 con 0x7fb7e4003a20 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb800180640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81fa050 con 0x7fb7f81521c0 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fe71d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f815b640 0x7fb7f8153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fe71d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f815b640 0x7fb7f8153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49686/0 (socket says 172.21.15.12:49686) 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fdf1c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7e4003a20 0x7fb7f81f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fe71d640 1 -- 172.21.15.12:0/384854745 learned_addr learned my addr 172.21.15.12:0/384854745 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fd71b640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f81521c0 0x7fb7f806cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fe71d640 1 -- 172.21.15.12:0/384854745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f81521c0 msgr2=0x7fb7f806cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fe71d640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f81521c0 0x7fb7f806cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fe71d640 1 -- 172.21.15.12:0/384854745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7e4003a20 msgr2=0x7fb7f81f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fe71d640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7e4003a20 0x7fb7f81f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:07.760 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fe71d640 1 -- 172.21.15.12:0/384854745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f815bfb0 con 0x7fb7f815b640 2023-11-28T21:56:07.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7fe71d640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f815b640 0x7fb7f8153200 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb7f806ba30 tx=0x7fb7f00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:07.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7ee7fc640 1 -- 172.21.15.12:0/384854745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000e070 con 0x7fb7f815b640 2023-11-28T21:56:07.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7ee7fc640 1 -- 172.21.15.12:0/384854745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f0011e10 con 0x7fb7f815b640 2023-11-28T21:56:07.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb800180640 1 -- 172.21.15.12:0/384854745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f815c290 con 0x7fb7f815b640 2023-11-28T21:56:07.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb7ee7fc640 1 -- 172.21.15.12:0/384854745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f00039a0 con 0x7fb7f815b640 2023-11-28T21:56:07.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb800180640 1 -- 172.21.15.12:0/384854745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f815d3f0 con 0x7fb7f815b640 2023-11-28T21:56:07.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.753+0000 7fb800180640 1 -- 172.21.15.12:0/384854745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f81fa810 con 0x7fb7f815b640 2023-11-28T21:56:07.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.757+0000 7fb7ee7fc640 1 -- 172.21.15.12:0/384854745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb7f0003b00 con 0x7fb7f815b640 2023-11-28T21:56:07.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.757+0000 7fb7ee7fc640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7dc066470 0x7fb7dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:07.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.757+0000 7fb7ee7fc640 1 -- 172.21.15.12:0/384854745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fb7f0084a90 con 0x7fb7f815b640 2023-11-28T21:56:07.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.757+0000 7fb7fdf1c640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7dc066470 0x7fb7dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:07.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.757+0000 7fb7fdf1c640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7dc066470 0x7fb7dc068930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40ab000 tx=0x7fb7f40a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:07.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.761+0000 7fb7ee7fc640 1 -- 172.21.15.12:0/384854745 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb7f81fa810 con 0x7fb7f815b640 2023-11-28T21:56:07.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.969+0000 7fb800180640 1 -- 172.21.15.12:0/384854745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7f81fa7d0 con 0x7fb7f815b640 2023-11-28T21:56:07.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb7ee7fc640 1 -- 172.21.15.12:0/384854745 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5738 (secure 0 0 0) 0x7fb7f0051800 con 0x7fb7f815b640 2023-11-28T21:56:07.979 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:07.979 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:56:02.009527+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:clientreplay","state_seq":42,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:56:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 85 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 -- 172.21.15.12:0/384854745 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7dc066470 msgr2=0x7fb7dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7dc066470 0x7fb7dc068930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40ab000 tx=0x7fb7f40a7070 comp rx=0 tx=0).stop 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 -- 172.21.15.12:0/384854745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f815b640 msgr2=0x7fb7f8153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f815b640 0x7fb7f8153200 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb7f806ba30 tx=0x7fb7f00115a0 comp rx=0 tx=0).stop 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 -- 172.21.15.12:0/384854745 shutdown_connections 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb7dc066470 0x7fb7dc068930 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb7f815b640 0x7fb7f8153200 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7f81521c0 0x7fb7f806cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 --2- 172.21.15.12:0/384854745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7e4003a20 0x7fb7f81f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 -- 172.21.15.12:0/384854745 >> 172.21.15.12:0/384854745 conn(0x7fb7f8066360 msgr2=0x7fb7f8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 -- 172.21.15.12:0/384854745 shutdown_connections 2023-11-28T21:56:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:07.973+0000 7fb800180640 1 -- 172.21.15.12:0/384854745 wait complete. 2023-11-28T21:56:08.000 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-11-28T21:56:10.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:56:13.480 INFO:tasks.ceph.mds.a.smithi012.stderr:2023-11-28T21:56:12.613+0000 7f959cbc2640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-28T21:56:13.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.625+0000 7efd7eabb640 1 Processor -- start 2023-11-28T21:56:13.633 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.625+0000 7efd7eabb640 1 -- start start 2023-11-28T21:56:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7eabb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd78151ed0 0x7efd780c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7eabb640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd78157b70 con 0x7efd78151af0 2023-11-28T21:56:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7eabb640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd78157e50 con 0x7efd7806e300 2023-11-28T21:56:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7eabb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd780707f0 con 0x7efd78151ed0 2023-11-28T21:56:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7c857640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efd78151af0 0x7efd780c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:45062/0 (socket says 172.21.15.12:45062) 2023-11-28T21:56:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7c857640 1 -- 172.21.15.12:0/2249317743 learned_addr learned my addr 172.21.15.12:0/2249317743 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:13.635 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd77fff640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd78151ed0 0x7efd780c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:13.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753536631 0 0) 0x7efd78157b70 con 0x7efd78151af0 2023-11-28T21:56:13.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd68003530 con 0x7efd78151af0 2023-11-28T21:56:13.637 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1780529617 0 0) 0x7efd68003530 con 0x7efd78151af0 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd78151ed0 msgr2=0x7efd780c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd78151ed0 0x7efd780c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 >> v1:172.21.15.144:6789/0 conn(0x7efd7806e300 legacy=0x7efd78072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd78158130 con 0x7efd78151af0 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7efd6c001680 con 0x7efd78151af0 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 >> v1:172.21.15.12:6789/0 conn(0x7efd78151af0 legacy=0x7efd780c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd68003a20 0x7efd68003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680047d0 0x7efd68004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd68005190 0x7efd6800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd780707f0 con 0x7efd68003a20 2023-11-28T21:56:13.638 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6800bed0 con 0x7efd680047d0 2023-11-28T21:56:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6800c010 con 0x7efd68005190 2023-11-28T21:56:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7c857640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd68003a20 0x7efd68003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd77fff640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680047d0 0x7efd68004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7d058640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd68005190 0x7efd6800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7c857640 1 -- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd68005190 msgr2=0x7efd6800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7c857640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd68005190 0x7efd6800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7c857640 1 -- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680047d0 msgr2=0x7efd68004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7c857640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680047d0 0x7efd68004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd7c857640 1 -- 172.21.15.12:0/2249317743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68003530 con 0x7efd68003a20 2023-11-28T21:56:13.639 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.629+0000 7efd77fff640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd680047d0 0x7efd68004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd7c857640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd68003a20 0x7efd68003e00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00c210 tx=0x7efd6c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8817295932b574 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:13.640 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c005ae0 con 0x7efd68003a20 2023-11-28T21:56:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c005c60 con 0x7efd68003a20 2023-11-28T21:56:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd777fe640 1 -- 172.21.15.12:0/2249317743 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c007070 con 0x7efd68003a20 2023-11-28T21:56:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd7eabb640 1 -- 172.21.15.12:0/2249317743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd68003a20 msgr2=0x7efd68003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd7eabb640 1 --2- 172.21.15.12:0/2249317743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd68003a20 0x7efd68003e00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00c210 tx=0x7efd6c01cc60 comp rx=0 tx=0).stop 2023-11-28T21:56:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd7d058640 1 -- 172.21.15.12:0/2249317743 reap_dead start 2023-11-28T21:56:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd7eabb640 1 -- 172.21.15.12:0/2249317743 shutdown_connections 2023-11-28T21:56:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd7eabb640 1 -- 172.21.15.12:0/2249317743 >> 172.21.15.12:0/2249317743 conn(0x7efd78066360 msgr2=0x7efd78066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd7eabb640 1 -- 172.21.15.12:0/2249317743 shutdown_connections 2023-11-28T21:56:13.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.633+0000 7efd7eabb640 1 -- 172.21.15.12:0/2249317743 wait complete. 2023-11-28T21:56:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7eabb640 1 Processor -- start 2023-11-28T21:56:13.642 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7eabb640 1 -- start start 2023-11-28T21:56:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7eabb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd68003a20 0x7efd78161170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7eabb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd780c9e90 0x7efd780c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7eabb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd78162120 0x7efd78140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7eabb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd78160580 con 0x7efd78162120 2023-11-28T21:56:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7eabb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd780c9a00 con 0x7efd780c9e90 2023-11-28T21:56:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7eabb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7814b2e0 con 0x7efd68003a20 2023-11-28T21:56:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7d058640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd78162120 0x7efd78140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7d058640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd78162120 0x7efd78140a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49698/0 (socket says 172.21.15.12:49698) 2023-11-28T21:56:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7d058640 1 -- 172.21.15.12:0/1351374290 learned_addr learned my addr 172.21.15.12:0/1351374290 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:13.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7c857640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd68003a20 0x7efd78161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd77fff640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd780c9e90 0x7efd780c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7d058640 1 -- 172.21.15.12:0/1351374290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd68003a20 msgr2=0x7efd78161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7d058640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd68003a20 0x7efd78161170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7d058640 1 -- 172.21.15.12:0/1351374290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd780c9e90 msgr2=0x7efd780c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7d058640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd780c9e90 0x7efd780c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7d058640 1 -- 172.21.15.12:0/1351374290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7814b8a0 con 0x7efd78162120 2023-11-28T21:56:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7d058640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd78162120 0x7efd78140a50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7efd78071410 tx=0x7efd7000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:13.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd757fa640 1 -- 172.21.15.12:0/1351374290 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70003980 con 0x7efd78162120 2023-11-28T21:56:13.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd757fa640 1 -- 172.21.15.12:0/1351374290 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd70003b00 con 0x7efd78162120 2023-11-28T21:56:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd757fa640 1 -- 172.21.15.12:0/1351374290 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd700050d0 con 0x7efd78162120 2023-11-28T21:56:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7eabb640 1 -- 172.21.15.12:0/1351374290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd780c9690 con 0x7efd78162120 2023-11-28T21:56:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.637+0000 7efd7eabb640 1 -- 172.21.15.12:0/1351374290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd780c46b0 con 0x7efd78162120 2023-11-28T21:56:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.641+0000 7efd7eabb640 1 -- 172.21.15.12:0/1351374290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd780c9850 con 0x7efd78162120 2023-11-28T21:56:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.641+0000 7efd757fa640 1 -- 172.21.15.12:0/1351374290 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7efd70005230 con 0x7efd78162120 2023-11-28T21:56:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.641+0000 7efd757fa640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 0x7efd5c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.641+0000 7efd757fa640 1 -- 172.21.15.12:0/1351374290 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7efd70084250 con 0x7efd78162120 2023-11-28T21:56:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.641+0000 7efd7c857640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 0x7efd5c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.641+0000 7efd7c857640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 0x7efd5c068960 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0ab000 tx=0x7efd6c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.645+0000 7efd757fa640 1 -- 172.21.15.12:0/1351374290 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efd70051050 con 0x7efd78162120 2023-11-28T21:56:13.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.837+0000 7efd7eabb640 1 -- 172.21.15.12:0/1351374290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd780c9b70 con 0x7efd78162120 2023-11-28T21:56:13.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.837+0000 7efd757fa640 1 -- 172.21.15.12:0/1351374290 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5732 (secure 0 0 0) 0x7efd70054670 con 0x7efd78162120 2023-11-28T21:56:13.844 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:13.845 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:56:08.485531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:active","state_seq":44,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:56:13.845 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 86 2023-11-28T21:56:13.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 -- 172.21.15.12:0/1351374290 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 msgr2=0x7efd5c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 0x7efd5c068960 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0ab000 tx=0x7efd6c0a7070 comp rx=0 tx=0).stop 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 -- 172.21.15.12:0/1351374290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd78162120 msgr2=0x7efd78140a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd78162120 0x7efd78140a50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7efd78071410 tx=0x7efd7000eb90 comp rx=0 tx=0).stop 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 -- 172.21.15.12:0/1351374290 shutdown_connections 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7efd5c0664a0 0x7efd5c068960 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd78162120 0x7efd78140a50 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd780c9e90 0x7efd780c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 --2- 172.21.15.12:0/1351374290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd68003a20 0x7efd78161170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 -- 172.21.15.12:0/1351374290 >> 172.21.15.12:0/1351374290 conn(0x7efd78066360 msgr2=0x7efd7814f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 -- 172.21.15.12:0/1351374290 shutdown_connections 2023-11-28T21:56:13.847 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:13.841+0000 7efd7eabb640 1 -- 172.21.15.12:0/1351374290 wait complete. 2023-11-28T21:56:13.867 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=1, replacing gid=6330 2023-11-28T21:56:13.867 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.e 2023-11-28T21:56:17.059 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:56:17.063 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:56:18.868 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-11-28T21:56:18.869 INFO:tasks.ceph.mds.e:Restarting daemon 2023-11-28T21:56:18.869 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-11-28T21:56:18.873 INFO:tasks.ceph.mds.e:Started 2023-11-28T21:56:18.873 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:56:18.962 INFO:tasks.ceph.mds.e.smithi012.stdout:starting mds.e at 2023-11-28T21:56:19.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416c0e3640 1 Processor -- start 2023-11-28T21:56:19.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416c0e3640 1 -- start start 2023-11-28T21:56:19.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416c0e3640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f41640c4000 0x7f416406d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:19.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416c0e3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4164161600 0x7f4164161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416c0e3640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41640c3460 con 0x7f4164158420 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416c0e3640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4164158800 con 0x7f4164161600 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416c0e3640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41640c52c0 con 0x7f41640c4000 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416a680640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4164161600 0x7f4164161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416a680640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4164161600 0x7f4164161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49248/0 (socket says 172.21.15.12:49248) 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416a680640 1 -- 172.21.15.12:0/2849428845 learned_addr learned my addr 172.21.15.12:0/2849428845 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.017+0000 7f416967e640 1 --2- 172.21.15.12:0/2849428845 >> v2:172.21.15.144:3300/0 conn(0x7f41640c4000 0x7f416406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416a680640 1 -- 172.21.15.12:0/2849428845 >> v2:172.21.15.144:3300/0 conn(0x7f41640c4000 msgr2=0x7f416406d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416a680640 1 --2- 172.21.15.12:0/2849428845 >> v2:172.21.15.144:3300/0 conn(0x7f41640c4000 0x7f416406d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416a680640 1 -- 172.21.15.12:0/2849428845 >> v1:172.21.15.144:6789/0 conn(0x7f4164158420 legacy=0x7f41640c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:19.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416a680640 1 -- 172.21.15.12:0/2849428845 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41640c2b70 con 0x7f4164161600 2023-11-28T21:56:19.023 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416967e640 1 --2- 172.21.15.12:0/2849428845 >> v2:172.21.15.144:3300/0 conn(0x7f41640c4000 0x7f416406d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:19.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416a680640 1 --2- 172.21.15.12:0/2849428845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164161600 0x7f4164161a10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f416406b930 tx=0x7f415c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5393f1ed5d91d37b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:19.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f4168e7d640 1 -- 172.21.15.12:0/2849428845 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c0a5690 con 0x7f4164161600 2023-11-28T21:56:19.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f4168e7d640 1 -- 172.21.15.12:0/2849428845 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c0a57f0 con 0x7f4164161600 2023-11-28T21:56:19.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f4168e7d640 1 -- 172.21.15.12:0/2849428845 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c0a5b80 con 0x7f4164161600 2023-11-28T21:56:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416c0e3640 1 -- 172.21.15.12:0/2849428845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164161600 msgr2=0x7f4164161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416c0e3640 1 --2- 172.21.15.12:0/2849428845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164161600 0x7f4164161a10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f416406b930 tx=0x7f415c01c9d0 comp rx=0 tx=0).stop 2023-11-28T21:56:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416c0e3640 1 -- 172.21.15.12:0/2849428845 shutdown_connections 2023-11-28T21:56:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416c0e3640 1 --2- 172.21.15.12:0/2849428845 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164161600 0x7f4164161a10 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416c0e3640 1 --2- 172.21.15.12:0/2849428845 >> v2:172.21.15.144:3300/0 conn(0x7f41640c4000 0x7f416406d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.021+0000 7f416c0e3640 1 -- 172.21.15.12:0/2849428845 >> 172.21.15.12:0/2849428845 conn(0x7f4164066360 msgr2=0x7f4164066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416c0e3640 1 -- 172.21.15.12:0/2849428845 shutdown_connections 2023-11-28T21:56:19.026 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416c0e3640 1 -- 172.21.15.12:0/2849428845 wait complete. 2023-11-28T21:56:19.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416c0e3640 1 Processor -- start 2023-11-28T21:56:19.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416c0e3640 1 -- start start 2023-11-28T21:56:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416c0e3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41640c4000 0x7f416415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416c0e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164158420 0x7f41641efec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416c0e3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4164161600 0x7f4164156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416c0e3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416415f650 con 0x7f4164158420 2023-11-28T21:56:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416c0e3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41641f0f70 con 0x7f41640c4000 2023-11-28T21:56:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416c0e3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41641f0400 con 0x7f4164161600 2023-11-28T21:56:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416967e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164158420 0x7f41641efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416967e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164158420 0x7f41641efec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49258/0 (socket says 172.21.15.12:49258) 2023-11-28T21:56:19.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416967e640 1 -- 172.21.15.12:0/1021397140 learned_addr learned my addr 172.21.15.12:0/1021397140 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f4169e7f640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41640c4000 0x7f416415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416a680640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4164161600 0x7f4164156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416967e640 1 -- 172.21.15.12:0/1021397140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4164161600 msgr2=0x7f4164156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416967e640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4164161600 0x7f4164156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416967e640 1 -- 172.21.15.12:0/1021397140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41640c4000 msgr2=0x7f416415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416967e640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41640c4000 0x7f416415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416967e640 1 -- 172.21.15.12:0/1021397140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c009b30 con 0x7f4164158420 2023-11-28T21:56:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.025+0000 7f416967e640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164158420 0x7f41641efec0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f415400e840 tx=0x7f415400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f415a7fc640 1 -- 172.21.15.12:0/1021397140 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415400c8d0 con 0x7f4164158420 2023-11-28T21:56:19.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f415a7fc640 1 -- 172.21.15.12:0/1021397140 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415400ca30 con 0x7f4164158420 2023-11-28T21:56:19.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f415a7fc640 1 -- 172.21.15.12:0/1021397140 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415400cdc0 con 0x7f4164158420 2023-11-28T21:56:19.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f416c0e3640 1 -- 172.21.15.12:0/1021397140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41641518c0 con 0x7f4164158420 2023-11-28T21:56:19.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f416c0e3640 1 -- 172.21.15.12:0/1021397140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416415fa10 con 0x7f4164158420 2023-11-28T21:56:19.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f416c0e3640 1 -- 172.21.15.12:0/1021397140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4164074f60 con 0x7f4164158420 2023-11-28T21:56:19.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f415a7fc640 1 -- 172.21.15.12:0/1021397140 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4154013410 con 0x7f4164158420 2023-11-28T21:56:19.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f415a7fc640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4148066470 0x7f4148068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:19.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f415a7fc640 1 -- 172.21.15.12:0/1021397140 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f4154084380 con 0x7f4164158420 2023-11-28T21:56:19.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f4169e7f640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4148066470 0x7f4148068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:19.033 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.029+0000 7f4169e7f640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4148066470 0x7f4148068930 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4160007fd0 tx=0x7f4160009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:19.036 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.033+0000 7f415a7fc640 1 -- 172.21.15.12:0/1021397140 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4154051100 con 0x7f4164158420 2023-11-28T21:56:19.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.265+0000 7f416c0e3640 1 -- 172.21.15.12:0/1021397140 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41640c79f0 con 0x7f4164158420 2023-11-28T21:56:19.270 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.265+0000 7f415a7fc640 1 -- 172.21.15.12:0/1021397140 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5732 (secure 0 0 0) 0x7f4154054720 con 0x7f4164158420 2023-11-28T21:56:19.270 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:19.270 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:56:08.485531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:active","state_seq":44,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:56:19.270 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 86 2023-11-28T21:56:19.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 -- 172.21.15.12:0/1021397140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4148066470 msgr2=0x7f4148068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:19.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4148066470 0x7f4148068930 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4160007fd0 tx=0x7f4160009040 comp rx=0 tx=0).stop 2023-11-28T21:56:19.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 -- 172.21.15.12:0/1021397140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164158420 msgr2=0x7f41641efec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:19.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164158420 0x7f41641efec0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f415400e840 tx=0x7f415400ed00 comp rx=0 tx=0).stop 2023-11-28T21:56:19.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 -- 172.21.15.12:0/1021397140 shutdown_connections 2023-11-28T21:56:19.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4148066470 0x7f4148068930 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:19.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4164161600 0x7f4164156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:19.272 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4164158420 0x7f41641efec0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:19.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 --2- 172.21.15.12:0/1021397140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41640c4000 0x7f416415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:19.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 -- 172.21.15.12:0/1021397140 >> 172.21.15.12:0/1021397140 conn(0x7f4164066360 msgr2=0x7f41640c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:19.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 -- 172.21.15.12:0/1021397140 shutdown_connections 2023-11-28T21:56:19.273 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:19.269+0000 7f416c0e3640 1 -- 172.21.15.12:0/1021397140 wait complete. 2023-11-28T21:56:19.290 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-11-28T21:56:21.291 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:56:21.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.433+0000 7f7a03901640 1 Processor -- start 2023-11-28T21:56:21.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.433+0000 7f7a03901640 1 -- start start 2023-11-28T21:56:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a03901640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79fc151ed0 0x7f79fc0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a03901640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f79fc06e300 0x7f79fc072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a03901640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc157b70 con 0x7f79fc151af0 2023-11-28T21:56:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a03901640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0707f0 con 0x7f79fc151ed0 2023-11-28T21:56:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a03901640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc0c7b90 con 0x7f79fc06e300 2023-11-28T21:56:21.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a00e9c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79fc151ed0 0x7f79fc0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a00e9c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f79fc151ed0 0x7f79fc0c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49264/0 (socket says 172.21.15.12:49264) 2023-11-28T21:56:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a00e9c640 1 -- 172.21.15.12:0/2098166688 learned_addr learned my addr 172.21.15.12:0/2098166688 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a01e9e640 1 --2- 172.21.15.12:0/2098166688 >> v2:172.21.15.144:3300/0 conn(0x7f79fc06e300 0x7f79fc072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a00e9c640 1 -- 172.21.15.12:0/2098166688 >> v2:172.21.15.144:3300/0 conn(0x7f79fc06e300 msgr2=0x7f79fc072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:21.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a00e9c640 1 --2- 172.21.15.12:0/2098166688 >> v2:172.21.15.144:3300/0 conn(0x7f79fc06e300 0x7f79fc072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a00e9c640 1 -- 172.21.15.12:0/2098166688 >> v1:172.21.15.12:6789/0 conn(0x7f79fc151af0 legacy=0x7f79fc0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f79f3fff640 1 -- 172.21.15.12:0/2098166688 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441122906 0 0) 0x7f79fc157b70 con 0x7f79fc151af0 2023-11-28T21:56:21.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.437+0000 7f7a00e9c640 1 -- 172.21.15.12:0/2098166688 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc157e50 con 0x7f79fc151ed0 2023-11-28T21:56:21.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a00e9c640 1 --2- 172.21.15.12:0/2098166688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151ed0 0x7f79fc0c7650 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0716f0 tx=0x7f79ec01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90273ed440814656 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:21.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f79f3fff640 1 -- 172.21.15.12:0/2098166688 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0ab600 con 0x7f79fc151ed0 2023-11-28T21:56:21.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f79f3fff640 1 -- 172.21.15.12:0/2098166688 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec0a7050 con 0x7f79fc151ed0 2023-11-28T21:56:21.443 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f79f3fff640 1 -- 172.21.15.12:0/2098166688 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0ab980 con 0x7f79fc151ed0 2023-11-28T21:56:21.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a03901640 1 -- 172.21.15.12:0/2098166688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151ed0 msgr2=0x7f79fc0c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:21.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a03901640 1 --2- 172.21.15.12:0/2098166688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151ed0 0x7f79fc0c7650 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0716f0 tx=0x7f79ec01c500 comp rx=0 tx=0).stop 2023-11-28T21:56:21.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a03901640 1 -- 172.21.15.12:0/2098166688 shutdown_connections 2023-11-28T21:56:21.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a03901640 1 --2- 172.21.15.12:0/2098166688 >> v2:172.21.15.144:3300/0 conn(0x7f79fc06e300 0x7f79fc072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:21.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a03901640 1 --2- 172.21.15.12:0/2098166688 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151ed0 0x7f79fc0c7650 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:21.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a03901640 1 -- 172.21.15.12:0/2098166688 >> 172.21.15.12:0/2098166688 conn(0x7f79fc066360 msgr2=0x7f79fc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:21.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a03901640 1 -- 172.21.15.12:0/2098166688 shutdown_connections 2023-11-28T21:56:21.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a03901640 1 -- 172.21.15.12:0/2098166688 wait complete. 2023-11-28T21:56:21.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a03901640 1 Processor -- start 2023-11-28T21:56:21.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.441+0000 7f7a03901640 1 -- start start 2023-11-28T21:56:21.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a03901640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79fc06e300 0x7f79fc14aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:21.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a03901640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151af0 0x7f79fc141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:21.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a03901640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79fc141770 0x7f79fc14d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:21.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a03901640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc141c20 con 0x7f79fc151af0 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a03901640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc149600 con 0x7f79fc06e300 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a03901640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc14afd0 con 0x7f79fc141770 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a00e9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151af0 0x7f79fc141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a00e9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151af0 0x7f79fc141230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49274/0 (socket says 172.21.15.12:49274) 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a00e9c640 1 -- 172.21.15.12:0/2181603946 learned_addr learned my addr 172.21.15.12:0/2181603946 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a0169d640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79fc06e300 0x7f79fc14aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a01e9e640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79fc141770 0x7f79fc14d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a00e9c640 1 -- 172.21.15.12:0/2181603946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79fc141770 msgr2=0x7f79fc14d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a00e9c640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79fc141770 0x7f79fc14d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a00e9c640 1 -- 172.21.15.12:0/2181603946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79fc06e300 msgr2=0x7f79fc14aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a00e9c640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79fc06e300 0x7f79fc14aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:21.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a00e9c640 1 -- 172.21.15.12:0/2181603946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0094d0 con 0x7f79fc151af0 2023-11-28T21:56:21.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a00e9c640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151af0 0x7f79fc141230 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0ae040 tx=0x7f79ec0192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:21.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f79f1ffb640 1 -- 172.21.15.12:0/2181603946 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0b4640 con 0x7f79fc151af0 2023-11-28T21:56:21.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f79f1ffb640 1 -- 172.21.15.12:0/2181603946 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec0b47c0 con 0x7f79fc151af0 2023-11-28T21:56:21.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f79f1ffb640 1 -- 172.21.15.12:0/2181603946 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0b4b70 con 0x7f79fc151af0 2023-11-28T21:56:21.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a03901640 1 -- 172.21.15.12:0/2181603946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc14b530 con 0x7f79fc151af0 2023-11-28T21:56:21.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a03901640 1 -- 172.21.15.12:0/2181603946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc149890 con 0x7f79fc151af0 2023-11-28T21:56:21.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f7a03901640 1 -- 172.21.15.12:0/2181603946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc074f60 con 0x7f79fc151af0 2023-11-28T21:56:21.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.445+0000 7f79f1ffb640 1 -- 172.21.15.12:0/2181603946 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f79ec0b2070 con 0x7f79fc151af0 2023-11-28T21:56:21.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.449+0000 7f79f1ffb640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79e0066470 0x7f79e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:21.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.449+0000 7f79f1ffb640 1 -- 172.21.15.12:0/2181603946 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f79ec11e4e0 con 0x7f79fc151af0 2023-11-28T21:56:21.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.449+0000 7f7a0169d640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79e0066470 0x7f79e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:21.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.449+0000 7f7a0169d640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79e0066470 0x7f79e0068930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000f70 tx=0x7f79f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:21.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.453+0000 7f79f1ffb640 1 -- 172.21.15.12:0/2181603946 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79ec0eb230 con 0x7f79fc151af0 2023-11-28T21:56:21.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.641+0000 7f7a03901640 1 -- 172.21.15.12:0/2181603946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79fc162130 con 0x7f79fc151af0 2023-11-28T21:56:21.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.641+0000 7f79f1ffb640 1 -- 172.21.15.12:0/2181603946 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+6518 (secure 0 0 0) 0x7f79ec0ee850 con 0x7f79fc151af0 2023-11-28T21:56:21.645 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:21.646 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6813,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/35262902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":35262902},{"type":"v1","addr":"172.21.15.12:6839","nonce":35262902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:56:18.921322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:active","state_seq":44,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:56:21.646 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 87 2023-11-28T21:56:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 -- 172.21.15.12:0/2181603946 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79e0066470 msgr2=0x7f79e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:21.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79e0066470 0x7f79e0068930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000f70 tx=0x7f79f8009040 comp rx=0 tx=0).stop 2023-11-28T21:56:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 -- 172.21.15.12:0/2181603946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151af0 msgr2=0x7f79fc141230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151af0 0x7f79fc141230 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0ae040 tx=0x7f79ec0192a0 comp rx=0 tx=0).stop 2023-11-28T21:56:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 -- 172.21.15.12:0/2181603946 shutdown_connections 2023-11-28T21:56:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f79e0066470 0x7f79e0068930 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79fc141770 0x7f79fc14d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f79fc151af0 0x7f79fc141230 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 --2- 172.21.15.12:0/2181603946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79fc06e300 0x7f79fc14aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 -- 172.21.15.12:0/2181603946 >> 172.21.15.12:0/2181603946 conn(0x7f79fc066360 msgr2=0x7f79fc0c6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 -- 172.21.15.12:0/2181603946 shutdown_connections 2023-11-28T21:56:21.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:21.645+0000 7f7a03901640 1 -- 172.21.15.12:0/2181603946 wait complete. 2023-11-28T21:56:21.666 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-11-28T21:56:21.666 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 108 secs before thrashing 2023-11-28T21:56:30.748 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 2% (280/12117) Updating files: 2% (353/12117) Updating files: 3% (364/12117) Updating files: 3% (452/12117) Updating files: 4% (485/12117) Updating files: 5% (606/12117) Updating files: 5% (683/12117) Updating files: 5% (685/12117) Updating files: 6% (728/12117) Updating files: 6% (730/12117) Updating files: 7% (849/12117) Updating files: 8% (970/12117) Updating files: 8% (986/12117) Updating files: 8% (1032/12117) Updating files: 9% (1091/12117) Updating files: 9% (1159/12117) Updating files: 10% (1212/12117) Updating files: 10% (1267/12117) Updating files: 11% (1333/12117) Updating files: 12% (1455/12117) Updating files: 12% (1509/12117) Updating files: 13% (1576/12117) Updating files: 13% (1586/12117) Updating files: 13% (1590/12117) Updating files: 14% (1697/12117) Updating files: 14% (1706/12117) Updating files: 15% (1818/12117) Updating files: 15% (1820/12117) Updating files: 15% (1865/12117) Updating files: 16% (1939/12117) Updating files: 16% (1946/12117) Updating files: 17% (2060/12117) Updating files: 17% (2070/12117) Updating files: 18% (2182/12117) Updating files: 18% (2221/12117) Updating files: 19% (2303/12117) Updating files: 20% (2424/12117) Updating files: 21% (2545/12117) Updating files: 21% (2555/12117) Updating files: 21% (2568/12117) Updating files: 21% (2569/12117) Updating files: 22% (2666/12117) Updating files: 22% (2731/12117) Updating files: 23% (2787/12117) Updating files: 23% (2819/12117) Updating files: 24% (2909/12117) Updating files: 24% (2924/12117) Updating files: 25% (3030/12117) Updating files: 26% (3151/12117) Updating files: 26% (3160/12117) Updating files: 26% (3198/12117) Updating files: 26% (3202/12117) Updating files: 27% (3272/12117) Updating files: 27% (3287/12117) Updating files: 28% (3393/12117) Updating files: 28% (3462/12117) Updating files: 29% (3514/12117) Updating files: 29% (3587/12117) Updating files: 30% (3636/12117) Updating files: 31% (3757/12117) Updating files: 31% (3806/12117) Updating files: 32% (3878/12117) Updating files: 32% (3935/12117) Updating files: 33% (3999/12117) Updating files: 33% (4044/12117) Updating files: 33% (4046/12117) Updating files: 33% (4083/12117) Updating files: 34% (4120/12117) Updating files: 35% (4241/12117) Updating files: 35% (4285/12117) Updating files: 35% (4287/12117) Updating files: 36% (4363/12117) Updating files: 36% (4375/12117) Updating files: 36% (4460/12117) Updating files: 37% (4484/12117) Updating files: 38% (4605/12117) Updating files: 38% (4685/12117) Updating files: 39% (4726/12117) Updating files: 40% (4847/12117) Updating files: 40% (4890/12117) Updating files: 40% (4901/12117) Updating files: 41% (4968/12117) Updating files: 41% (5007/12117) Updating files: 42% (5090/12117) Updating files: 43% (5211/12117) Updating files: 43% (5246/12117) Updating files: 44% (5332/12117) Updating files: 44% (5442/12117) Updating files: 45% (5453/12117) Updating files: 45% (5505/12117) Updating files: 45% (5506/12117) Updating files: 46% (5574/12117) Updating files: 46% (5622/12117) Updating files: 47% (5695/12117) Updating files: 47% (5706/12117) Updating files: 47% (5771/12117) Updating files: 48% (5817/12117) Updating files: 49% (5938/12117) Updating files: 49% (6005/12117) Updating files: 50% (6059/12117) Updating files: 50% (6127/12117) Updating files: 51% (6180/12117) Updating files: 52% (6301/12117) Updating files: 52% (6351/12117) Updating files: 53% (6423/12117) Updating files: 53% (6455/12117) Updating files: 54% (6544/12117) Updating files: 54% (6637/12117) Updating files: 55% (6665/12117) Updating files: 55% (6762/12117) Updating files: 56% (6786/12117) Updating files: 57% (6907/12117) Updating files: 58% (7028/12117) Updating files: 59% (7150/12117) Updating files: 59% (7228/12117) Updating files: 60% (7271/12117) Updating files: 61% (7392/12117) Updating files: 62% (7513/12117) Updating files: 62% (7612/12117) Updating files: 63% (7634/12117) Updating files: 63% (7667/12117) Updating files: 63% (7731/12117) Updating files: 64% (7755/12117) Updating files: 64% (7845/12117) Updating files: 65% (7877/12117) Updating files: 65% (7923/12117) Updating files: 65% (7928/12117) Updating files: 65% (7938/12117) Updating files: 66% (7998/12117) Updating files: 66% (8036/12117) Updating files: 67% (8119/12117) Updating files: 67% (8228/12117) Updating files: 68% (8240/12117) Updating files: 68% (8274/12117) Updating files: 68% (8278/12117) Updating files: 69% (8361/12117) Updating files: 70% (8482/12117) Updating files: 70% (8561/12117) Updating files: 70% (8591/12117) Updating files: 71% (8604/12117) Updating files: 72% (8725/12117) Updating files: 72% (8810/12117) Updating files: 73% (8846/12117) Updating files: 73% (8963/12117) Updating files: 74% (8967/12117) Updating files: 75% (9088/12117) Updating files: 75% (9093/12117) Updating files: 76% (9209/12117) Updating files: 76% (9274/12117) Updating files: 77% (9331/12117) Updating files: 77% (9346/12117) Updating files: 78% (9452/12117) Updating files: 78% (9454/12117) Updating files: 78% (9567/12117) Updating files: 79% (9573/12117) Updating files: 80% (9694/12117) Updating files: 80% (9718/12117) Updating files: 81% (9815/12117) Updating files: 81% (9860/12117) Updating files: 81% (9906/12117) Updating files: 81% (9930/12117) Updating files: 82% (9936/12117) Updating files: 82% (9987/12117) Updating files: 83% (10058/12117) Updating files: 83% (10085/12117) Updating files: 84% (10179/12117) Updating files: 85% (10300/12117) Updating files: 85% (10370/12117) Updating files: 85% (10371/12117) Updating files: 86% (10421/12117) Updating files: 86% (10530/12117) Updating files: 87% (10542/12117) Updating files: 88% (10663/12117) Updating files: 88% (10729/12117) Updating files: 88% (10769/12117) Updating files: 89% (10785/12117) Updating files: 90% (10906/12117) Updating files: 90% (10961/12117) Updating files: 91% (11027/12117) Updating files: 91% (11062/12117) Updating files: 92% (11148/12117) Updating files: 92% (11175/12117) Updating files: 93% (11269/12117) Updating files: 94% (11390/12117) Updating files: 95% (11512/12117) Updating files: 95% (11515/12117) Updating files: 95% (11600/12117) Updating files: 96% (11633/12117) Updating files: 96% (11649/12117) Updating files: 97% (11754/12117) Updating files: 98% (11875/12117) Updating files: 98% (11878/12117) Updating files: 99% (11996/12117) Updating files: 99% (12016/12117) Updating files: 100% (12117/12117) Updating files: 100% (12117/12117), done. 2023-11-28T21:56:30.789 DEBUG:teuthology.orchestra.run:got remote process result: 128 2023-11-28T21:56:30.790 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2023-11-28T21:56:30.790 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-11-28T21:56:31.663 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_43dd94c190073cf16cff1e711dd7f5a7b1cbc883/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_43dd94c190073cf16cff1e711dd7f5a7b1cbc883/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_43dd94c190073cf16cff1e711dd7f5a7b1cbc883/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi012 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-11-28T21:56:31.731 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f7eaf334d44b4e81a85f4d0780286075 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_43dd94c190073cf16cff1e711dd7f5a7b1cbc883/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_43dd94c190073cf16cff1e711dd7f5a7b1cbc883/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_43dd94c190073cf16cff1e711dd7f5a7b1cbc883/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi012 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-11-28T21:56:31.734 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2023-11-28T21:56:31.743 INFO:tasks.mds_thrash:joining mds_thrasher 2023-11-28T21:56:31.745 INFO:tasks.mds_thrash.fs.[cephfs]:stat['max_mds'] = 2 2023-11-28T21:56:31.745 INFO:tasks.mds_thrash.fs.[cephfs]:stat['deactivate'] = 0 2023-11-28T21:56:31.745 INFO:tasks.mds_thrash.fs.[cephfs]:stat['kill'] = 11 2023-11-28T21:56:31.746 INFO:tasks.mds_thrash:done joining 2023-11-28T21:56:31.746 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-28T21:56:31.794 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-28T21:56:31.794 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:56:31.794 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T21:56:31.807 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2023-11-28T21:56:31.807 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T21:56:31.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:56:32.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.040+0000 7faba9fd8640 1 Processor -- start 2023-11-28T21:56:32.044 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.040+0000 7faba9fd8640 1 -- start start 2023-11-28T21:56:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.040+0000 7faba9fd8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faba4151ba0 0x7faba41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.040+0000 7faba9fd8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba415be30 con 0x7faba415b9e0 2023-11-28T21:56:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.040+0000 7faba9fd8640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba406da10 con 0x7faba4066fa0 2023-11-28T21:56:32.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.040+0000 7faba9fd8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba40c24f0 con 0x7faba4151ba0 2023-11-28T21:56:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.040+0000 7faba3fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7faba415b9e0 0x7faba406cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33794/0 (socket says 172.21.15.12:33794) 2023-11-28T21:56:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.040+0000 7faba3fff640 1 -- 172.21.15.12:0/3228527640 learned_addr learned my addr 172.21.15.12:0/3228527640 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba37fe640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faba4151ba0 0x7faba41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625515610 0 0) 0x7faba415be30 con 0x7faba415b9e0 2023-11-28T21:56:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab8c003530 con 0x7faba415b9e0 2023-11-28T21:56:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538553951 0 0) 0x7faba406da10 con 0x7faba4066fa0 2023-11-28T21:56:32.048 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba415be30 con 0x7faba4066fa0 2023-11-28T21:56:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3238825144 0 0) 0x7fab8c003530 con 0x7faba415b9e0 2023-11-28T21:56:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faba4151ba0 msgr2=0x7faba41579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faba4151ba0 0x7faba41579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 >> v1:172.21.15.144:6789/0 conn(0x7faba4066fa0 legacy=0x7faba4071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba406dcd0 con 0x7faba415b9e0 2023-11-28T21:56:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fab9c001a00 con 0x7faba415b9e0 2023-11-28T21:56:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 >> v1:172.21.15.12:6789/0 conn(0x7faba415b9e0 legacy=0x7faba406cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faba4151ba0 0x7fab8c003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8c004110 0x7fab8c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab8c004d10 0x7fab8c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba40c24f0 con 0x7fab8c004d10 2023-11-28T21:56:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c00bad0 con 0x7fab8c004110 2023-11-28T21:56:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c00bc90 con 0x7faba4151ba0 2023-11-28T21:56:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba3fff640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab8c004d10 0x7fab8c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba37fe640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faba4151ba0 0x7fab8c003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba2ffd640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8c004110 0x7fab8c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba3fff640 1 -- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faba4151ba0 msgr2=0x7fab8c003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba3fff640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faba4151ba0 0x7fab8c003bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba3fff640 1 -- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8c004110 msgr2=0x7fab8c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba3fff640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8c004110 0x7fab8c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba3fff640 1 -- 172.21.15.12:0/3228527640 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c003530 con 0x7fab8c004d10 2023-11-28T21:56:32.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.044+0000 7faba37fe640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faba4151ba0 0x7fab8c003bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba3fff640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab8c004d10 0x7fab8c005190 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00a7c0 tx=0x7fab9c01fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb442f460623c16 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0aa630 con 0x7fab8c004d10 2023-11-28T21:56:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c0aa7b0 con 0x7fab8c004d10 2023-11-28T21:56:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba27fc640 1 -- 172.21.15.12:0/3228527640 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0aaba0 con 0x7fab8c004d10 2023-11-28T21:56:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba9fd8640 1 -- 172.21.15.12:0/3228527640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab8c004d10 msgr2=0x7fab8c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.053 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba9fd8640 1 --2- 172.21.15.12:0/3228527640 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab8c004d10 0x7fab8c005190 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00a7c0 tx=0x7fab9c01fa00 comp rx=0 tx=0).stop 2023-11-28T21:56:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba3fff640 1 -- 172.21.15.12:0/3228527640 reap_dead start 2023-11-28T21:56:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba9fd8640 1 -- 172.21.15.12:0/3228527640 shutdown_connections 2023-11-28T21:56:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba9fd8640 1 -- 172.21.15.12:0/3228527640 >> 172.21.15.12:0/3228527640 conn(0x7faba4066360 msgr2=0x7faba4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba9fd8640 1 -- 172.21.15.12:0/3228527640 shutdown_connections 2023-11-28T21:56:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba9fd8640 1 -- 172.21.15.12:0/3228527640 wait complete. 2023-11-28T21:56:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba9fd8640 1 Processor -- start 2023-11-28T21:56:32.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.048+0000 7faba9fd8640 1 -- start start 2023-11-28T21:56:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba9fd8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c004d10 0x7faba4159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba9fd8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faba41f0b70 0x7faba41ed600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba9fd8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faba41428f0 0x7faba40ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba9fd8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba4159fe0 con 0x7faba41428f0 2023-11-28T21:56:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba9fd8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba4159050 con 0x7faba41f0b70 2023-11-28T21:56:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba9fd8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba41ede20 con 0x7fab8c004d10 2023-11-28T21:56:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faba41428f0 0x7faba40ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faba41428f0 0x7faba40ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39942/0 (socket says 172.21.15.12:39942) 2023-11-28T21:56:32.057 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba3fff640 1 -- 172.21.15.12:0/796618260 learned_addr learned my addr 172.21.15.12:0/796618260 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba37fe640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c004d10 0x7faba4159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba2ffd640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faba41f0b70 0x7faba41ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba3fff640 1 -- 172.21.15.12:0/796618260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c004d10 msgr2=0x7faba4159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba3fff640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c004d10 0x7faba4159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba3fff640 1 -- 172.21.15.12:0/796618260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faba41f0b70 msgr2=0x7faba41ed600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba3fff640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faba41f0b70 0x7faba41ed600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba3fff640 1 -- 172.21.15.12:0/796618260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba41ee100 con 0x7faba41428f0 2023-11-28T21:56:32.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba3fff640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faba41428f0 0x7faba40ca300 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0ad040 tx=0x7fab8c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7fab8bfff640 1 -- 172.21.15.12:0/796618260 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0b9070 con 0x7faba41428f0 2023-11-28T21:56:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7fab8bfff640 1 -- 172.21.15.12:0/796618260 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c0aa430 con 0x7faba41428f0 2023-11-28T21:56:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba9fd8640 1 -- 172.21.15.12:0/796618260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba41ee3e0 con 0x7faba41428f0 2023-11-28T21:56:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7fab8bfff640 1 -- 172.21.15.12:0/796618260 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c004b70 con 0x7faba41428f0 2023-11-28T21:56:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba9fd8640 1 -- 172.21.15.12:0/796618260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba4159580 con 0x7faba41428f0 2023-11-28T21:56:32.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7faba9fd8640 1 -- 172.21.15.12:0/796618260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba41ee570 con 0x7faba41428f0 2023-11-28T21:56:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.052+0000 7fab8bfff640 1 -- 172.21.15.12:0/796618260 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fab9c004cd0 con 0x7faba41428f0 2023-11-28T21:56:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.056+0000 7fab8bfff640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fab90066470 0x7fab90068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.056+0000 7fab8bfff640 1 -- 172.21.15.12:0/796618260 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fab9c121bf0 con 0x7faba41428f0 2023-11-28T21:56:32.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.056+0000 7faba37fe640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fab90066470 0x7fab90068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.056+0000 7faba37fe640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fab90066470 0x7fab90068930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fab94009ab0 tx=0x7fab94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:32.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.060+0000 7fab8bfff640 1 -- 172.21.15.12:0/796618260 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fab9c0ee820 con 0x7faba41428f0 2023-11-28T21:56:32.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.216+0000 7faba9fd8640 1 -- 172.21.15.12:0/796618260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faba4074f60 con 0x7faba41428f0 2023-11-28T21:56:32.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7fab8bfff640 1 -- 172.21.15.12:0/796618260 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+20091 (secure 0 0 0) 0x7fab9c0f1e40 con 0x7faba41428f0 2023-11-28T21:56:32.223 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:32.223 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":52,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:55:38.673845+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6839/754378087":"2023-11-29T21:55:38.673802+0000","172.21.15.12:6836/754378087":"2023-11-29T21:55:38.673802+0000","172.21.15.144:6832/3053317078":"2023-11-29T21:39:58.381227+0000","172.21.15.144:6833/3053317078":"2023-11-29T21:39:58.381227+0000","172.21.15.12:6839/2705421041":"2023-11-29T21:48:53.535998+0000","172.21.15.12:6834/3741155090":"2023-11-29T21:41:03.398724+0000","172.21.15.12:6838/3474424840":"2023-11-29T21:46:48.503169+0000","172.21.15.144:6832/126572048":"2023-11-29T21:52:53.611902+0000","172.21.15.12:6838/3741155090":"2023-11-29T21:41:03.398724+0000","172.21.15.12:6834/3474424840":"2023-11-29T21:46:48.503169+0000","172.21.15.12:6834/1551825785":"2023-11-29T21:50:38.562185+0000","172.21.15.12:6836/3009112102":"2023-11-29T21:42:38.424201+0000","172.21.15.12:6834/291731973":"2023-11-29T21:44:08.460474+0000","172.21.15.144:6833/126572048":"2023-11-29T21:52:53.611902+0000","172.21.15.12:6838/291731973":"2023-11-29T21:44:08.460474+0000","172.21.15.12:6836/3299030262":"2023-11-29T21:45:23.480028+0000","172.21.15.12:6839/3009112102":"2023-11-29T21:42:38.424201+0000","172.21.15.12:6839/3299030262":"2023-11-29T21:45:23.480028+0000","172.21.15.12:6838/1551825785":"2023-11-29T21:50:38.562185+0000","172.21.15.12:6836/2705421041":"2023-11-29T21:48:53.535998+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:56:32.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 -- 172.21.15.12:0/796618260 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fab90066470 msgr2=0x7fab90068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fab90066470 0x7fab90068930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fab94009ab0 tx=0x7fab94008040 comp rx=0 tx=0).stop 2023-11-28T21:56:32.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 -- 172.21.15.12:0/796618260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faba41428f0 msgr2=0x7faba40ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faba41428f0 0x7faba40ca300 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0ad040 tx=0x7fab8c005800 comp rx=0 tx=0).stop 2023-11-28T21:56:32.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 -- 172.21.15.12:0/796618260 shutdown_connections 2023-11-28T21:56:32.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fab90066470 0x7fab90068930 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faba41428f0 0x7faba40ca300 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faba41f0b70 0x7faba41ed600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 --2- 172.21.15.12:0/796618260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8c004d10 0x7faba4159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 -- 172.21.15.12:0/796618260 >> 172.21.15.12:0/796618260 conn(0x7faba4066360 msgr2=0x7faba41505e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:32.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 -- 172.21.15.12:0/796618260 shutdown_connections 2023-11-28T21:56:32.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.220+0000 7faba9fd8640 1 -- 172.21.15.12:0/796618260 wait complete. 2023-11-28T21:56:32.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-28T21:34:29.878190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T21:34:34.039032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T21:34:35.369379+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 505, 'snap_epoch': 41, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T21:34:39.373079+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 505, 'snap_epoch': 41, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T21:56:32.245 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T21:56:32.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78b011d640 1 Processor -- start 2023-11-28T21:56:32.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78b011d640 1 -- start start 2023-11-28T21:56:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78b011d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f78a8151af0 0x7f78a80c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78b011d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f78a806e300 0x7f78a8072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78b011d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a8157b70 con 0x7f78a8151ed0 2023-11-28T21:56:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78b011d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80707f0 con 0x7f78a8151af0 2023-11-28T21:56:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78b011d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80c7b90 con 0x7f78a806e300 2023-11-28T21:56:32.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78adeb9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f78a8151af0 0x7f78a80c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78adeb9640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f78a8151af0 0x7f78a80c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39958/0 (socket says 172.21.15.12:39958) 2023-11-28T21:56:32.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78adeb9640 1 -- 172.21.15.12:0/4009232697 learned_addr learned my addr 172.21.15.12:0/4009232697 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:32.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78ae6ba640 1 --2- 172.21.15.12:0/4009232697 >> v2:172.21.15.144:3300/0 conn(0x7f78a806e300 0x7f78a8072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78adeb9640 1 -- 172.21.15.12:0/4009232697 >> v2:172.21.15.144:3300/0 conn(0x7f78a806e300 msgr2=0x7f78a8072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:32.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78adeb9640 1 --2- 172.21.15.12:0/4009232697 >> v2:172.21.15.144:3300/0 conn(0x7f78a806e300 0x7f78a8072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78adeb9640 1 -- 172.21.15.12:0/4009232697 >> v1:172.21.15.144:6789/0 conn(0x7f78a8151ed0 legacy=0x7f78a80c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.388+0000 7f78adeb9640 1 -- 172.21.15.12:0/4009232697 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8157e50 con 0x7f78a8151af0 2023-11-28T21:56:32.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78adeb9640 1 --2- 172.21.15.12:0/4009232697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 0x7f78a80c7650 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f78a80716f0 tx=0x7f78a401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6499964eac514da0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78aceb7640 1 -- 172.21.15.12:0/4009232697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40ab600 con 0x7f78a8151af0 2023-11-28T21:56:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78aceb7640 1 -- 172.21.15.12:0/4009232697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a40a7050 con 0x7f78a8151af0 2023-11-28T21:56:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78aceb7640 1 -- 172.21.15.12:0/4009232697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40ab980 con 0x7f78a8151af0 2023-11-28T21:56:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78b011d640 1 -- 172.21.15.12:0/4009232697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 msgr2=0x7f78a80c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78b011d640 1 --2- 172.21.15.12:0/4009232697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 0x7f78a80c7650 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f78a80716f0 tx=0x7f78a401c500 comp rx=0 tx=0).stop 2023-11-28T21:56:32.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78b011d640 1 -- 172.21.15.12:0/4009232697 shutdown_connections 2023-11-28T21:56:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78b011d640 1 --2- 172.21.15.12:0/4009232697 >> v2:172.21.15.144:3300/0 conn(0x7f78a806e300 0x7f78a8072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78b011d640 1 --2- 172.21.15.12:0/4009232697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 0x7f78a80c7650 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78b011d640 1 -- 172.21.15.12:0/4009232697 >> 172.21.15.12:0/4009232697 conn(0x7f78a8066360 msgr2=0x7f78a8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78b011d640 1 -- 172.21.15.12:0/4009232697 shutdown_connections 2023-11-28T21:56:32.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.392+0000 7f78b011d640 1 -- 172.21.15.12:0/4009232697 wait complete. 2023-11-28T21:56:32.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78b011d640 1 Processor -- start 2023-11-28T21:56:32.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78b011d640 1 -- start start 2023-11-28T21:56:32.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78b011d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a806e300 0x7f78a81574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78b011d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 0x7f78a8137cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78b011d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a81389d0 0x7f78a81468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78b011d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a81f38c0 con 0x7f78a8151af0 2023-11-28T21:56:32.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78b011d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80c9400 con 0x7f78a806e300 2023-11-28T21:56:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78b011d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80c9160 con 0x7f78a81389d0 2023-11-28T21:56:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78ad6b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 0x7f78a8137cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78ad6b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 0x7f78a8137cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39964/0 (socket says 172.21.15.12:39964) 2023-11-28T21:56:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78ad6b8640 1 -- 172.21.15.12:0/3611770865 learned_addr learned my addr 172.21.15.12:0/3611770865 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78adeb9640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a806e300 0x7f78a81574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78ae6ba640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a81389d0 0x7f78a81468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78ad6b8640 1 -- 172.21.15.12:0/3611770865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a81389d0 msgr2=0x7f78a81468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78ad6b8640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a81389d0 0x7f78a81468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78ad6b8640 1 -- 172.21.15.12:0/3611770865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a806e300 msgr2=0x7f78a81574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78ad6b8640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a806e300 0x7f78a81574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78ad6b8640 1 -- 172.21.15.12:0/3611770865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a40094d0 con 0x7f78a8151af0 2023-11-28T21:56:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78adeb9640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a806e300 0x7f78a81574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78ad6b8640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 0x7f78a8137cf0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f789800c3f0 tx=0x7f789800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f789e7fc640 1 -- 172.21.15.12:0/3611770865 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980052d0 con 0x7f78a8151af0 2023-11-28T21:56:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f789e7fc640 1 -- 172.21.15.12:0/3611770865 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7898005430 con 0x7f78a8151af0 2023-11-28T21:56:32.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78b011d640 1 -- 172.21.15.12:0/3611770865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a81f3d50 con 0x7f78a8151af0 2023-11-28T21:56:32.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f789e7fc640 1 -- 172.21.15.12:0/3611770865 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898003840 con 0x7f78a8151af0 2023-11-28T21:56:32.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.396+0000 7f78b011d640 1 -- 172.21.15.12:0/3611770865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a81f3270 con 0x7f78a8151af0 2023-11-28T21:56:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.400+0000 7f78b011d640 1 -- 172.21.15.12:0/3611770865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a8074f60 con 0x7f78a8151af0 2023-11-28T21:56:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.400+0000 7f789e7fc640 1 -- 172.21.15.12:0/3611770865 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f78980039a0 con 0x7f78a8151af0 2023-11-28T21:56:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.400+0000 7f789e7fc640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f788c0664a0 0x7f788c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.400+0000 7f789e7fc640 1 -- 172.21.15.12:0/3611770865 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f7898083c30 con 0x7f78a8151af0 2023-11-28T21:56:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.400+0000 7f78adeb9640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f788c0664a0 0x7f788c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.400+0000 7f78adeb9640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f788c0664a0 0x7f788c068960 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f78a40ad000 tx=0x7f78a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:32.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.404+0000 7f789e7fc640 1 -- 172.21.15.12:0/3611770865 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7898050860 con 0x7f78a8151af0 2023-11-28T21:56:32.565 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.560+0000 7f78b011d640 1 -- 172.21.15.12:0/3611770865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78a8072c10 con 0x7f78a8151af0 2023-11-28T21:56:32.566 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f789e7fc640 1 -- 172.21.15.12:0/3611770865 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78980170a0 con 0x7f78a8151af0 2023-11-28T21:56:32.566 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2023-11-28T21:56:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 -- 172.21.15.12:0/3611770865 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f788c0664a0 msgr2=0x7f788c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f788c0664a0 0x7f788c068960 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f78a40ad000 tx=0x7f78a4008040 comp rx=0 tx=0).stop 2023-11-28T21:56:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 -- 172.21.15.12:0/3611770865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 msgr2=0x7f78a8137cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.568 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 0x7f78a8137cf0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f789800c3f0 tx=0x7f789800c8b0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 -- 172.21.15.12:0/3611770865 shutdown_connections 2023-11-28T21:56:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f788c0664a0 0x7f788c068960 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78a81389d0 0x7f78a81468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78a8151af0 0x7f78a8137cf0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 --2- 172.21.15.12:0/3611770865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78a806e300 0x7f78a81574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 -- 172.21.15.12:0/3611770865 >> 172.21.15.12:0/3611770865 conn(0x7f78a8066360 msgr2=0x7f78a8144470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 -- 172.21.15.12:0/3611770865 shutdown_connections 2023-11-28T21:56:32.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.564+0000 7f78b011d640 1 -- 172.21.15.12:0/3611770865 wait complete. 2023-11-28T21:56:32.588 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T21:56:32.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea50432640 1 Processor -- start 2023-11-28T21:56:32.735 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea50432640 1 -- start start 2023-11-28T21:56:32.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea50432640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea48150c20 0x7fea48156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea50432640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fea4806e300 0x7fea480c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea50432640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea48067410 con 0x7fea4806cf40 2023-11-28T21:56:32.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea50432640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4806e090 con 0x7fea4806e300 2023-11-28T21:56:32.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea50432640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea48157f30 con 0x7fea48150c20 2023-11-28T21:56:32.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea4d9cd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fea4806e300 0x7fea480c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea4d9cd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fea4806e300 0x7fea480c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39980/0 (socket says 172.21.15.12:39980) 2023-11-28T21:56:32.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea4d9cd640 1 -- 172.21.15.12:0/733563057 learned_addr learned my addr 172.21.15.12:0/733563057 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:32.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea4e1ce640 1 --2- 172.21.15.12:0/733563057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea48150c20 0x7fea48156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea4d9cd640 1 -- 172.21.15.12:0/733563057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea48150c20 msgr2=0x7fea48156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:32.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea4d9cd640 1 --2- 172.21.15.12:0/733563057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea48150c20 0x7fea48156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea4d9cd640 1 -- 172.21.15.12:0/733563057 >> v1:172.21.15.144:6789/0 conn(0x7fea4806cf40 legacy=0x7fea48158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.732+0000 7fea4d9cd640 1 -- 172.21.15.12:0/733563057 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea48151000 con 0x7fea4806e300 2023-11-28T21:56:32.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea4d9cd640 1 --2- 172.21.15.12:0/733563057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4806e300 0x7fea480c6dd0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fea3800bb10 tx=0x7fea3801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427f70ffefaab329 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:32.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea4d1cc640 1 -- 172.21.15.12:0/733563057 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea380a7af0 con 0x7fea4806e300 2023-11-28T21:56:32.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea4d1cc640 1 -- 172.21.15.12:0/733563057 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea380a7c50 con 0x7fea4806e300 2023-11-28T21:56:32.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea4d1cc640 1 -- 172.21.15.12:0/733563057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea380af440 con 0x7fea4806e300 2023-11-28T21:56:32.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea50432640 1 -- 172.21.15.12:0/733563057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4806e300 msgr2=0x7fea480c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea50432640 1 --2- 172.21.15.12:0/733563057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4806e300 0x7fea480c6dd0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fea3800bb10 tx=0x7fea3801ec10 comp rx=0 tx=0).stop 2023-11-28T21:56:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea50432640 1 -- 172.21.15.12:0/733563057 shutdown_connections 2023-11-28T21:56:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea50432640 1 --2- 172.21.15.12:0/733563057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea4806e300 0x7fea480c6dd0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea50432640 1 --2- 172.21.15.12:0/733563057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea48150c20 0x7fea48156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea50432640 1 -- 172.21.15.12:0/733563057 >> 172.21.15.12:0/733563057 conn(0x7fea48066360 msgr2=0x7fea48066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea50432640 1 -- 172.21.15.12:0/733563057 shutdown_connections 2023-11-28T21:56:32.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.736+0000 7fea50432640 1 -- 172.21.15.12:0/733563057 wait complete. 2023-11-28T21:56:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea50432640 1 Processor -- start 2023-11-28T21:56:32.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea50432640 1 -- start start 2023-11-28T21:56:32.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea50432640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4806cf40 0x7fea481603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea50432640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea4806e300 0x7fea4806a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea50432640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea48150c20 0x7fea481f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea50432640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea481f6aa0 con 0x7fea48150c20 2023-11-28T21:56:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea50432640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea481f7340 con 0x7fea4806e300 2023-11-28T21:56:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea50432640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4814cf50 con 0x7fea4806cf40 2023-11-28T21:56:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4e9cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea48150c20 0x7fea481f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4e9cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea48150c20 0x7fea481f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39986/0 (socket says 172.21.15.12:39986) 2023-11-28T21:56:32.746 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4e9cf640 1 -- 172.21.15.12:0/3272008920 learned_addr learned my addr 172.21.15.12:0/3272008920 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4d9cd640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea4806e300 0x7fea4806a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4e1ce640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4806cf40 0x7fea481603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4e9cf640 1 -- 172.21.15.12:0/3272008920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4806cf40 msgr2=0x7fea481603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4e9cf640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4806cf40 0x7fea481603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4e9cf640 1 -- 172.21.15.12:0/3272008920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea4806e300 msgr2=0x7fea4806a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4e9cf640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea4806e300 0x7fea4806a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4e9cf640 1 -- 172.21.15.12:0/3272008920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3800b770 con 0x7fea48150c20 2023-11-28T21:56:32.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea4e9cf640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea48150c20 0x7fea481f3530 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c770 tx=0x7fea4000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:32.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea3effd640 1 -- 172.21.15.12:0/3272008920 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40003b80 con 0x7fea48150c20 2023-11-28T21:56:32.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea3effd640 1 -- 172.21.15.12:0/3272008920 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea40003ce0 con 0x7fea48150c20 2023-11-28T21:56:32.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea3effd640 1 -- 172.21.15.12:0/3272008920 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000aa90 con 0x7fea48150c20 2023-11-28T21:56:32.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea50432640 1 -- 172.21.15.12:0/3272008920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4814d230 con 0x7fea48150c20 2023-11-28T21:56:32.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.740+0000 7fea50432640 1 -- 172.21.15.12:0/3272008920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea481f6fd0 con 0x7fea48150c20 2023-11-28T21:56:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.744+0000 7fea50432640 1 -- 172.21.15.12:0/3272008920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea481f74b0 con 0x7fea48150c20 2023-11-28T21:56:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.744+0000 7fea3effd640 1 -- 172.21.15.12:0/3272008920 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fea4000abf0 con 0x7fea48150c20 2023-11-28T21:56:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.744+0000 7fea3effd640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea2c0664a0 0x7fea2c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.744+0000 7fea3effd640 1 -- 172.21.15.12:0/3272008920 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fea40084380 con 0x7fea48150c20 2023-11-28T21:56:32.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.744+0000 7fea4e1ce640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea2c0664a0 0x7fea2c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:32.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.744+0000 7fea4e1ce640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea2c0664a0 0x7fea2c068960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fea440079c0 tx=0x7fea44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:32.752 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.748+0000 7fea3effd640 1 -- 172.21.15.12:0/3272008920 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fea40051060 con 0x7fea48150c20 2023-11-28T21:56:32.921 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.916+0000 7fea50432640 1 -- 172.21.15.12:0/3272008920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea48074f60 con 0x7fea48150c20 2023-11-28T21:56:32.922 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.916+0000 7fea3effd640 1 -- 172.21.15.12:0/3272008920 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea40054680 con 0x7fea48150c20 2023-11-28T21:56:32.922 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:56:32.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 -- 172.21.15.12:0/3272008920 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea2c0664a0 msgr2=0x7fea2c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea2c0664a0 0x7fea2c068960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fea440079c0 tx=0x7fea44008040 comp rx=0 tx=0).stop 2023-11-28T21:56:32.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 -- 172.21.15.12:0/3272008920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea48150c20 msgr2=0x7fea481f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:32.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea48150c20 0x7fea481f3530 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c770 tx=0x7fea4000cc30 comp rx=0 tx=0).stop 2023-11-28T21:56:32.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 -- 172.21.15.12:0/3272008920 shutdown_connections 2023-11-28T21:56:32.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fea2c0664a0 0x7fea2c068960 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.924 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea48150c20 0x7fea481f3530 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea4806e300 0x7fea4806a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 --2- 172.21.15.12:0/3272008920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4806cf40 0x7fea481603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:32.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 -- 172.21.15.12:0/3272008920 >> 172.21.15.12:0/3272008920 conn(0x7fea48066360 msgr2=0x7fea48071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:32.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 -- 172.21.15.12:0/3272008920 shutdown_connections 2023-11-28T21:56:32.925 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:32.920+0000 7fea50432640 1 -- 172.21.15.12:0/3272008920 wait complete. 2023-11-28T21:56:32.944 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T21:56:33.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.088+0000 7f0239993640 1 Processor -- start 2023-11-28T21:56:33.093 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.088+0000 7f0239993640 1 -- start start 2023-11-28T21:56:33.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0239993640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f023415a980 0x7f02340b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0239993640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340b4010 0x7f02340cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.096 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0239993640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02340b44a0 con 0x7f0234153fb0 2023-11-28T21:56:33.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0239993640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02340b2b60 con 0x7f023415a980 2023-11-28T21:56:33.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0239993640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023415e1b0 con 0x7f02340b4010 2023-11-28T21:56:33.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0233fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f023415a980 0x7f02340b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0233fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f023415a980 0x7f02340b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39990/0 (socket says 172.21.15.12:39990) 2023-11-28T21:56:33.097 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0233fff640 1 -- 172.21.15.12:0/23460564 learned_addr learned my addr 172.21.15.12:0/23460564 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:33.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0238991640 1 --1- 172.21.15.12:0/23460564 >> v1:172.21.15.12:6789/0 conn(0x7f0234153fb0 0x7f02340ca4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33808/0 (socket says 172.21.15.12:33808) 2023-11-28T21:56:33.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0239192640 1 --2- 172.21.15.12:0/23460564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340b4010 0x7f02340cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0233fff640 1 -- 172.21.15.12:0/23460564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340b4010 msgr2=0x7f02340cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:33.098 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0233fff640 1 --2- 172.21.15.12:0/23460564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340b4010 0x7f02340cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0233fff640 1 -- 172.21.15.12:0/23460564 >> v1:172.21.15.12:6789/0 conn(0x7f0234153fb0 legacy=0x7f02340ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0233fff640 1 -- 172.21.15.12:0/23460564 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023415d390 con 0x7f023415a980 2023-11-28T21:56:33.099 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f02337fe640 1 -- 172.21.15.12:0/23460564 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521607035 0 0) 0x7f02340b44a0 con 0x7f0234153fb0 2023-11-28T21:56:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.092+0000 7f0233fff640 1 --2- 172.21.15.12:0/23460564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 0x7f02340b3ad0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f02200098a0 tx=0x7f022001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee5b050379ff32e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:33.103 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f02337fe640 1 -- 172.21.15.12:0/23460564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02200aa650 con 0x7f023415a980 2023-11-28T21:56:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f02337fe640 1 -- 172.21.15.12:0/23460564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02200aa7b0 con 0x7f023415a980 2023-11-28T21:56:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f02337fe640 1 -- 172.21.15.12:0/23460564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02200aab40 con 0x7f023415a980 2023-11-28T21:56:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 -- 172.21.15.12:0/23460564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 msgr2=0x7f02340b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 --2- 172.21.15.12:0/23460564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 0x7f02340b3ad0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f02200098a0 tx=0x7f022001c4f0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.104 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 -- 172.21.15.12:0/23460564 shutdown_connections 2023-11-28T21:56:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 --2- 172.21.15.12:0/23460564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340b4010 0x7f02340cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 --2- 172.21.15.12:0/23460564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 0x7f02340b3ad0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 -- 172.21.15.12:0/23460564 >> 172.21.15.12:0/23460564 conn(0x7f0234071ce0 msgr2=0x7f02340720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 -- 172.21.15.12:0/23460564 shutdown_connections 2023-11-28T21:56:33.105 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 -- 172.21.15.12:0/23460564 wait complete. 2023-11-28T21:56:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 Processor -- start 2023-11-28T21:56:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 -- start start 2023-11-28T21:56:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340b4010 0x7f02340cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0234153fb0 0x7f0234144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 0x7f0234145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02340cd350 con 0x7f023415a980 2023-11-28T21:56:33.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02341f6000 con 0x7f0234153fb0 2023-11-28T21:56:33.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239993640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02340b1e10 con 0x7f02340b4010 2023-11-28T21:56:33.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239192640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 0x7f0234145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.107 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239192640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 0x7f0234145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39992/0 (socket says 172.21.15.12:39992) 2023-11-28T21:56:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0239192640 1 -- 172.21.15.12:0/567991987 learned_addr learned my addr 172.21.15.12:0/567991987 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.096+0000 7f0238991640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340b4010 0x7f02340cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f0233fff640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0234153fb0 0x7f0234144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f0239192640 1 -- 172.21.15.12:0/567991987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340b4010 msgr2=0x7f02340cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f0239192640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340b4010 0x7f02340cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f0239192640 1 -- 172.21.15.12:0/567991987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0234153fb0 msgr2=0x7f0234144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f0239192640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0234153fb0 0x7f0234144dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f0239192640 1 -- 172.21.15.12:0/567991987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02200094d0 con 0x7f023415a980 2023-11-28T21:56:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f0239192640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 0x7f0234145300 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f022c007e40 tx=0x7f022c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f02317fa640 1 -- 172.21.15.12:0/567991987 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c014620 con 0x7f023415a980 2023-11-28T21:56:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f02317fa640 1 -- 172.21.15.12:0/567991987 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022c00ee90 con 0x7f023415a980 2023-11-28T21:56:33.111 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f02317fa640 1 -- 172.21.15.12:0/567991987 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c0149e0 con 0x7f023415a980 2023-11-28T21:56:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f0239993640 1 -- 172.21.15.12:0/567991987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02341f62a0 con 0x7f023415a980 2023-11-28T21:56:33.112 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f0239993640 1 -- 172.21.15.12:0/567991987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02341f6860 con 0x7f023415a980 2023-11-28T21:56:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f0239993640 1 -- 172.21.15.12:0/567991987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02340c6600 con 0x7f023415a980 2023-11-28T21:56:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f02317fa640 1 -- 172.21.15.12:0/567991987 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f022c014b40 con 0x7f023415a980 2023-11-28T21:56:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.100+0000 7f02317fa640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02180664c0 0x7f0218068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.114 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.104+0000 7f02317fa640 1 -- 172.21.15.12:0/567991987 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f022c084e60 con 0x7f023415a980 2023-11-28T21:56:33.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.104+0000 7f0238991640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02180664c0 0x7f0218068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.104+0000 7f0238991640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02180664c0 0x7f0218068980 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0228007fd0 tx=0x7f0228007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:33.115 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.104+0000 7f02317fa640 1 -- 172.21.15.12:0/567991987 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f022c051c50 con 0x7f023415a980 2023-11-28T21:56:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.260+0000 7f0239993640 1 -- 172.21.15.12:0/567991987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02341f6a50 con 0x7f023415a980 2023-11-28T21:56:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f02317fa640 1 -- 172.21.15.12:0/567991987 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f022c055270 con 0x7f023415a980 2023-11-28T21:56:33.266 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2023-11-28T21:56:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 -- 172.21.15.12:0/567991987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02180664c0 msgr2=0x7f0218068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02180664c0 0x7f0218068980 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0228007fd0 tx=0x7f0228007d60 comp rx=0 tx=0).stop 2023-11-28T21:56:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 -- 172.21.15.12:0/567991987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 msgr2=0x7f0234145300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 0x7f0234145300 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f022c007e40 tx=0x7f022c00e4b0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 -- 172.21.15.12:0/567991987 shutdown_connections 2023-11-28T21:56:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f02180664c0 0x7f0218068980 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f023415a980 0x7f0234145300 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0234153fb0 0x7f0234144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 --2- 172.21.15.12:0/567991987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02340b4010 0x7f02340cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 -- 172.21.15.12:0/567991987 >> 172.21.15.12:0/567991987 conn(0x7f0234071ce0 msgr2=0x7f02341515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 -- 172.21.15.12:0/567991987 shutdown_connections 2023-11-28T21:56:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.264+0000 7f0239993640 1 -- 172.21.15.12:0/567991987 wait complete. 2023-11-28T21:56:33.287 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T21:56:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.428+0000 7fa8c54dc640 1 Processor -- start 2023-11-28T21:56:33.434 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.428+0000 7fa8c54dc640 1 -- start start 2023-11-28T21:56:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8c54dc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c006bc20 0x7fa8c0066360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8c54dc640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fa8c00caea0 0x7fa8c00cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8c54dc640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c0159b90 con 0x7fa8c0153a20 2023-11-28T21:56:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8c54dc640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c015a4d0 con 0x7fa8c00caea0 2023-11-28T21:56:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8c54dc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c006c260 con 0x7fa8c006bc20 2023-11-28T21:56:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8bffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fa8c0153a20 0x7fa8c00c4690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33824/0 (socket says 172.21.15.12:33824) 2023-11-28T21:56:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8bffff640 1 -- 172.21.15.12:0/117421123 learned_addr learned my addr 172.21.15.12:0/117421123 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8c4cdb640 1 --2- 172.21.15.12:0/117421123 >> v2:172.21.15.12:3300/0 conn(0x7fa8c00caea0 0x7fa8c00cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8bf7fe640 1 --2- 172.21.15.12:0/117421123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c006bc20 0x7fa8c0066360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8c4cdb640 1 -- 172.21.15.12:0/117421123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c006bc20 msgr2=0x7fa8c0066360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8c4cdb640 1 --2- 172.21.15.12:0/117421123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c006bc20 0x7fa8c0066360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8c4cdb640 1 -- 172.21.15.12:0/117421123 >> v1:172.21.15.12:6789/0 conn(0x7fa8c0153a20 legacy=0x7fa8c00c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.432+0000 7fa8c4cdb640 1 -- 172.21.15.12:0/117421123 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c006c540 con 0x7fa8c00caea0 2023-11-28T21:56:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8c4cdb640 1 --2- 172.21.15.12:0/117421123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c00caea0 0x7fa8c00cb330 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009d90 tx=0x7fa8b801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3491ae39bf26f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8beffd640 1 -- 172.21.15.12:0/117421123 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80a5900 con 0x7fa8c00caea0 2023-11-28T21:56:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8beffd640 1 -- 172.21.15.12:0/117421123 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b80a5a60 con 0x7fa8c00caea0 2023-11-28T21:56:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8beffd640 1 -- 172.21.15.12:0/117421123 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80a5df0 con 0x7fa8c00caea0 2023-11-28T21:56:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8c54dc640 1 -- 172.21.15.12:0/117421123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c00caea0 msgr2=0x7fa8c00cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8c54dc640 1 --2- 172.21.15.12:0/117421123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c00caea0 0x7fa8c00cb330 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009d90 tx=0x7fa8b801c950 comp rx=0 tx=0).stop 2023-11-28T21:56:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8c54dc640 1 -- 172.21.15.12:0/117421123 shutdown_connections 2023-11-28T21:56:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8c54dc640 1 --2- 172.21.15.12:0/117421123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c00caea0 0x7fa8c00cb330 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8c54dc640 1 --2- 172.21.15.12:0/117421123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c006bc20 0x7fa8c0066360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8c54dc640 1 -- 172.21.15.12:0/117421123 >> 172.21.15.12:0/117421123 conn(0x7fa8c00cf550 msgr2=0x7fa8c00cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8c54dc640 1 -- 172.21.15.12:0/117421123 shutdown_connections 2023-11-28T21:56:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8c54dc640 1 -- 172.21.15.12:0/117421123 wait complete. 2023-11-28T21:56:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.436+0000 7fa8c54dc640 1 Processor -- start 2023-11-28T21:56:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c54dc640 1 -- start start 2023-11-28T21:56:33.444 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c54dc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c006bc20 0x7fa8c01f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c54dc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c00caea0 0x7fa8c014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c54dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c0153a20 0x7fa8c0148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c54dc640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00b5630 con 0x7fa8c006bc20 2023-11-28T21:56:33.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c54dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c01f2b60 con 0x7fa8c0153a20 2023-11-28T21:56:33.445 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c54dc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00b3ad0 con 0x7fa8c00caea0 2023-11-28T21:56:33.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c006bc20 0x7fa8c01f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c006bc20 0x7fa8c01f2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40024/0 (socket says 172.21.15.12:40024) 2023-11-28T21:56:33.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bffff640 1 -- 172.21.15.12:0/1404558339 learned_addr learned my addr 172.21.15.12:0/1404558339 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:33.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bf7fe640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c00caea0 0x7fa8c014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.446 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c4cdb640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c0153a20 0x7fa8c0148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bffff640 1 -- 172.21.15.12:0/1404558339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c00caea0 msgr2=0x7fa8c014bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bffff640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c00caea0 0x7fa8c014bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bffff640 1 -- 172.21.15.12:0/1404558339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c0153a20 msgr2=0x7fa8c0148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.447 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bffff640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c0153a20 0x7fa8c0148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bffff640 1 -- 172.21.15.12:0/1404558339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b8009a70 con 0x7fa8c006bc20 2023-11-28T21:56:33.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bffff640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c006bc20 0x7fa8c01f2620 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000c500 tx=0x7fa8b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:33.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bcff9640 1 -- 172.21.15.12:0/1404558339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0005cf0 con 0x7fa8c006bc20 2023-11-28T21:56:33.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bcff9640 1 -- 172.21.15.12:0/1404558339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b0005e50 con 0x7fa8c006bc20 2023-11-28T21:56:33.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8bcff9640 1 -- 172.21.15.12:0/1404558339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b00072f0 con 0x7fa8c006bc20 2023-11-28T21:56:33.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c54dc640 1 -- 172.21.15.12:0/1404558339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c00b4590 con 0x7fa8c006bc20 2023-11-28T21:56:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c54dc640 1 -- 172.21.15.12:0/1404558339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c00b4b50 con 0x7fa8c006bc20 2023-11-28T21:56:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.440+0000 7fa8c54dc640 1 -- 172.21.15.12:0/1404558339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c0074f60 con 0x7fa8c006bc20 2023-11-28T21:56:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.444+0000 7fa8bcff9640 1 -- 172.21.15.12:0/1404558339 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa8b0007450 con 0x7fa8c006bc20 2023-11-28T21:56:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.444+0000 7fa8bcff9640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa8ac0664a0 0x7fa8ac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.444+0000 7fa8bcff9640 1 -- 172.21.15.12:0/1404558339 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fa8b0083a50 con 0x7fa8c006bc20 2023-11-28T21:56:33.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.444+0000 7fa8bf7fe640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa8ac0664a0 0x7fa8ac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.444+0000 7fa8bf7fe640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa8ac0664a0 0x7fa8ac068960 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa8c015e1f0 tx=0x7fa8b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:33.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.448+0000 7fa8bcff9640 1 -- 172.21.15.12:0/1404558339 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa8b0050850 con 0x7fa8c006bc20 2023-11-28T21:56:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 -- 172.21.15.12:0/1404558339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa8c00cac80 con 0x7fa8c006bc20 2023-11-28T21:56:33.615 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8bcff9640 1 -- 172.21.15.12:0/1404558339 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v52) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa8b00170a0 con 0x7fa8c006bc20 2023-11-28T21:56:33.615 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2023-11-28T21:56:33.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 -- 172.21.15.12:0/1404558339 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa8ac0664a0 msgr2=0x7fa8ac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa8ac0664a0 0x7fa8ac068960 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa8c015e1f0 tx=0x7fa8b4008040 comp rx=0 tx=0).stop 2023-11-28T21:56:33.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 -- 172.21.15.12:0/1404558339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c006bc20 msgr2=0x7fa8c01f2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c006bc20 0x7fa8c01f2620 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000c500 tx=0x7fa8b000c9c0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 -- 172.21.15.12:0/1404558339 shutdown_connections 2023-11-28T21:56:33.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fa8ac0664a0 0x7fa8ac068960 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c0153a20 0x7fa8c0148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c00caea0 0x7fa8c014bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 --2- 172.21.15.12:0/1404558339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8c006bc20 0x7fa8c01f2620 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 -- 172.21.15.12:0/1404558339 >> 172.21.15.12:0/1404558339 conn(0x7fa8c00cf550 msgr2=0x7fa8c0159040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:33.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 -- 172.21.15.12:0/1404558339 shutdown_connections 2023-11-28T21:56:33.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.612+0000 7fa8c54dc640 1 -- 172.21.15.12:0/1404558339 wait complete. 2023-11-28T21:56:33.637 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T21:56:33.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc48de79640 1 Processor -- start 2023-11-28T21:56:33.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc48de79640 1 -- start start 2023-11-28T21:56:33.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc48de79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48806b130 0x7fc4880c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc48de79640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc4880cb020 0x7fc488159d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc48de79640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4880cb4b0 con 0x7fc4880cac40 2023-11-28T21:56:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc48de79640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48806b550 con 0x7fc4880cb020 2023-11-28T21:56:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc48de79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48815a5e0 con 0x7fc48806b130 2023-11-28T21:56:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc4877fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc4880cac40 0x7fc4880ca030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33832/0 (socket says 172.21.15.12:33832) 2023-11-28T21:56:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc487fff640 1 --2- 172.21.15.12:0/1789702074 >> v2:172.21.15.12:3300/0 conn(0x7fc4880cb020 0x7fc488159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc4877fe640 1 -- 172.21.15.12:0/1789702074 learned_addr learned my addr 172.21.15.12:0/1789702074 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc486ffd640 1 --2- 172.21.15.12:0/1789702074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48806b130 0x7fc4880c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc487fff640 1 -- 172.21.15.12:0/1789702074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48806b130 msgr2=0x7fc4880c4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:33.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc487fff640 1 --2- 172.21.15.12:0/1789702074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48806b130 0x7fc4880c4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc487fff640 1 -- 172.21.15.12:0/1789702074 >> v1:172.21.15.12:6789/0 conn(0x7fc4880cac40 legacy=0x7fc4880ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc487fff640 1 -- 172.21.15.12:0/1789702074 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48815a8c0 con 0x7fc4880cb020 2023-11-28T21:56:33.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.780+0000 7fc4867fc640 1 -- 172.21.15.12:0/1789702074 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567924099 0 0) 0x7fc4880cb4b0 con 0x7fc4880cac40 2023-11-28T21:56:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc487fff640 1 --2- 172.21.15.12:0/1789702074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4880cb020 0x7fc488159d50 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc480009d90 tx=0x7fc48001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c75869d5dedd45 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc4867fc640 1 -- 172.21.15.12:0/1789702074 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800af600 con 0x7fc4880cb020 2023-11-28T21:56:33.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc4867fc640 1 -- 172.21.15.12:0/1789702074 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4800ab050 con 0x7fc4880cb020 2023-11-28T21:56:33.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc4867fc640 1 -- 172.21.15.12:0/1789702074 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800af980 con 0x7fc4880cb020 2023-11-28T21:56:33.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc48de79640 1 -- 172.21.15.12:0/1789702074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4880cb020 msgr2=0x7fc488159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc48de79640 1 --2- 172.21.15.12:0/1789702074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4880cb020 0x7fc488159d50 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc480009d90 tx=0x7fc48001f130 comp rx=0 tx=0).stop 2023-11-28T21:56:33.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc48de79640 1 -- 172.21.15.12:0/1789702074 shutdown_connections 2023-11-28T21:56:33.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc48de79640 1 --2- 172.21.15.12:0/1789702074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc4880cb020 0x7fc488159d50 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc48de79640 1 --2- 172.21.15.12:0/1789702074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc48806b130 0x7fc4880c4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc48de79640 1 -- 172.21.15.12:0/1789702074 >> 172.21.15.12:0/1789702074 conn(0x7fc4880720e0 msgr2=0x7fc4880724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:33.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc48de79640 1 -- 172.21.15.12:0/1789702074 shutdown_connections 2023-11-28T21:56:33.791 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.784+0000 7fc48de79640 1 -- 172.21.15.12:0/1789702074 wait complete. 2023-11-28T21:56:33.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc48de79640 1 Processor -- start 2023-11-28T21:56:33.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc48de79640 1 -- start start 2023-11-28T21:56:33.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc48de79640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc48806b130 0x7fc488159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc48de79640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4880cac40 0x7fc4881528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc48de79640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4880cb020 0x7fc4881e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc48de79640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc488154a00 con 0x7fc48806b130 2023-11-28T21:56:33.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc48de79640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881597e0 con 0x7fc4880cac40 2023-11-28T21:56:33.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc48de79640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc488159d10 con 0x7fc4880cb020 2023-11-28T21:56:33.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc48806b130 0x7fc488159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc48806b130 0x7fc488159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40038/0 (socket says 172.21.15.12:40038) 2023-11-28T21:56:33.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc4877fe640 1 -- 172.21.15.12:0/2032014864 learned_addr learned my addr 172.21.15.12:0/2032014864 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:33.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc487fff640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4880cb020 0x7fc4881e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc486ffd640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4880cac40 0x7fc4881528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc4877fe640 1 -- 172.21.15.12:0/2032014864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4880cb020 msgr2=0x7fc4881e53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc4877fe640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4880cb020 0x7fc4881e53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc4877fe640 1 -- 172.21.15.12:0/2032014864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4880cac40 msgr2=0x7fc4881528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc4877fe640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4880cac40 0x7fc4881528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc4877fe640 1 -- 172.21.15.12:0/2032014864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc480009a70 con 0x7fc48806b130 2023-11-28T21:56:33.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc4877fe640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc48806b130 0x7fc488159040 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc4780011f0 tx=0x7fc47800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:33.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc46ffff640 1 -- 172.21.15.12:0/2032014864 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4780074e0 con 0x7fc48806b130 2023-11-28T21:56:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc46ffff640 1 -- 172.21.15.12:0/2032014864 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc478007640 con 0x7fc48806b130 2023-11-28T21:56:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc46ffff640 1 -- 172.21.15.12:0/2032014864 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478004a10 con 0x7fc48806b130 2023-11-28T21:56:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc48de79640 1 -- 172.21.15.12:0/2032014864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4880b3f80 con 0x7fc48806b130 2023-11-28T21:56:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.788+0000 7fc48de79640 1 -- 172.21.15.12:0/2032014864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc488159a30 con 0x7fc48806b130 2023-11-28T21:56:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.792+0000 7fc48de79640 1 -- 172.21.15.12:0/2032014864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc488067c70 con 0x7fc48806b130 2023-11-28T21:56:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.792+0000 7fc46ffff640 1 -- 172.21.15.12:0/2032014864 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc478004b70 con 0x7fc48806b130 2023-11-28T21:56:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.792+0000 7fc46ffff640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc474066470 0x7fc474068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.792+0000 7fc46ffff640 1 -- 172.21.15.12:0/2032014864 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fc478083900 con 0x7fc48806b130 2023-11-28T21:56:33.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.792+0000 7fc486ffd640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc474066470 0x7fc474068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:33.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.792+0000 7fc486ffd640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc474066470 0x7fc474068930 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc488158fd0 tx=0x7fc47c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:33.801 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.796+0000 7fc46ffff640 1 -- 172.21.15.12:0/2032014864 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc4780505c0 con 0x7fc48806b130 2023-11-28T21:56:33.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.968+0000 7fc48de79640 1 -- 172.21.15.12:0/2032014864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc488074f60 con 0x7fc48806b130 2023-11-28T21:56:33.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.968+0000 7fc46ffff640 1 -- 172.21.15.12:0/2032014864 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 20 entries v52) v1 ==== 81+0+1218 (secure 0 0 0) 0x7fc478018020 con 0x7fc48806b130 2023-11-28T21:56:33.973 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6839/754378087 2023-11-29T21:55:38.673802+0000 2023-11-28T21:56:33.974 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6836/754378087 2023-11-29T21:55:38.673802+0000 2023-11-28T21:56:33.974 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.144:6832/3053317078 2023-11-29T21:39:58.381227+0000 2023-11-28T21:56:33.974 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.144:6833/3053317078 2023-11-29T21:39:58.381227+0000 2023-11-28T21:56:33.974 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6839/2705421041 2023-11-29T21:48:53.535998+0000 2023-11-28T21:56:33.974 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6834/3741155090 2023-11-29T21:41:03.398724+0000 2023-11-28T21:56:33.974 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6838/3474424840 2023-11-29T21:46:48.503169+0000 2023-11-28T21:56:33.974 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.144:6832/126572048 2023-11-29T21:52:53.611902+0000 2023-11-28T21:56:33.975 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6838/3741155090 2023-11-29T21:41:03.398724+0000 2023-11-28T21:56:33.975 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6834/3474424840 2023-11-29T21:46:48.503169+0000 2023-11-28T21:56:33.975 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6834/1551825785 2023-11-29T21:50:38.562185+0000 2023-11-28T21:56:33.975 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6836/3009112102 2023-11-29T21:42:38.424201+0000 2023-11-28T21:56:33.975 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6834/291731973 2023-11-29T21:44:08.460474+0000 2023-11-28T21:56:33.975 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.144:6833/126572048 2023-11-29T21:52:53.611902+0000 2023-11-28T21:56:33.975 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6838/291731973 2023-11-29T21:44:08.460474+0000 2023-11-28T21:56:33.975 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6836/3299030262 2023-11-29T21:45:23.480028+0000 2023-11-28T21:56:33.976 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6839/3009112102 2023-11-29T21:42:38.424201+0000 2023-11-28T21:56:33.976 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6839/3299030262 2023-11-29T21:45:23.480028+0000 2023-11-28T21:56:33.976 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6838/1551825785 2023-11-29T21:50:38.562185+0000 2023-11-28T21:56:33.976 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6836/2705421041 2023-11-29T21:48:53.535998+0000 2023-11-28T21:56:33.977 INFO:teuthology.orchestra.run.smithi012.stderr:listed 20 entries 2023-11-28T21:56:33.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 -- 172.21.15.12:0/2032014864 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc474066470 msgr2=0x7fc474068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc474066470 0x7fc474068930 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc488158fd0 tx=0x7fc47c008040 comp rx=0 tx=0).stop 2023-11-28T21:56:33.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 -- 172.21.15.12:0/2032014864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc48806b130 msgr2=0x7fc488159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc48806b130 0x7fc488159040 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc4780011f0 tx=0x7fc47800c9b0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 -- 172.21.15.12:0/2032014864 shutdown_connections 2023-11-28T21:56:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fc474066470 0x7fc474068930 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4880cb020 0x7fc4881e53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4880cac40 0x7fc4881528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 --2- 172.21.15.12:0/2032014864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc48806b130 0x7fc488159040 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 -- 172.21.15.12:0/2032014864 >> 172.21.15.12:0/2032014864 conn(0x7fc4880720e0 msgr2=0x7fc4880c4650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:33.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 -- 172.21.15.12:0/2032014864 shutdown_connections 2023-11-28T21:56:33.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:33.972+0000 7fc48de79640 1 -- 172.21.15.12:0/2032014864 wait complete. 2023-11-28T21:56:33.995 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T21:56:33.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:56:33.995 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T21:56:35.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:56:35.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:56:35.967 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T21:56:35.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:56:35.978 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2023-11-28T21:56:36.031 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T21:56:36.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:56:36.032 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T21:56:36.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T21:56:36.100 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2023-11-28T21:56:36.294 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T21:56:36.304 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T21:56:36.305 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:56:36.448 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.444+0000 7ff55b0bb640 1 Processor -- start 2023-11-28T21:56:36.449 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.444+0000 7ff55b0bb640 1 -- start start 2023-11-28T21:56:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.444+0000 7ff55b0bb640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff554151ed0 0x7ff5540c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.444+0000 7ff55b0bb640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff55406e300 0x7ff554072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.444+0000 7ff55b0bb640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff554157b70 con 0x7ff554151af0 2023-11-28T21:56:36.450 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.444+0000 7ff55b0bb640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540707f0 con 0x7ff554151ed0 2023-11-28T21:56:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.444+0000 7ff55b0bb640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540c7b90 con 0x7ff55406e300 2023-11-28T21:56:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.444+0000 7ff553fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff554151ed0 0x7ff5540c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff553fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff554151ed0 0x7ff5540c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48636/0 (socket says 172.21.15.12:48636) 2023-11-28T21:56:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff553fff640 1 -- 172.21.15.12:0/49662963 learned_addr learned my addr 172.21.15.12:0/49662963 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff559658640 1 --2- 172.21.15.12:0/49662963 >> v2:172.21.15.144:3300/0 conn(0x7ff55406e300 0x7ff554072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff553fff640 1 -- 172.21.15.12:0/49662963 >> v2:172.21.15.144:3300/0 conn(0x7ff55406e300 msgr2=0x7ff554072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff553fff640 1 --2- 172.21.15.12:0/49662963 >> v2:172.21.15.144:3300/0 conn(0x7ff55406e300 0x7ff554072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff553fff640 1 -- 172.21.15.12:0/49662963 >> v1:172.21.15.144:6789/0 conn(0x7ff554151af0 legacy=0x7ff5540c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:36.451 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff553fff640 1 -- 172.21.15.12:0/49662963 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff554157e50 con 0x7ff554151ed0 2023-11-28T21:56:36.452 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff559658640 1 --2- 172.21.15.12:0/49662963 >> v2:172.21.15.144:3300/0 conn(0x7ff55406e300 0x7ff554072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:36.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff553fff640 1 --2- 172.21.15.12:0/49662963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554151ed0 0x7ff5540c7650 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff5540716f0 tx=0x7ff54001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c834d88c6a19dbf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:36.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff5537fe640 1 -- 172.21.15.12:0/49662963 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400ab600 con 0x7ff554151ed0 2023-11-28T21:56:36.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff5537fe640 1 -- 172.21.15.12:0/49662963 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5400a7050 con 0x7ff554151ed0 2023-11-28T21:56:36.453 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.448+0000 7ff5537fe640 1 -- 172.21.15.12:0/49662963 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400ab980 con 0x7ff554151ed0 2023-11-28T21:56:36.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 -- 172.21.15.12:0/49662963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554151ed0 msgr2=0x7ff5540c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:36.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 --2- 172.21.15.12:0/49662963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554151ed0 0x7ff5540c7650 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff5540716f0 tx=0x7ff54001c500 comp rx=0 tx=0).stop 2023-11-28T21:56:36.454 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 -- 172.21.15.12:0/49662963 shutdown_connections 2023-11-28T21:56:36.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 --2- 172.21.15.12:0/49662963 >> v2:172.21.15.144:3300/0 conn(0x7ff55406e300 0x7ff554072070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 --2- 172.21.15.12:0/49662963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554151ed0 0x7ff5540c7650 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 -- 172.21.15.12:0/49662963 >> 172.21.15.12:0/49662963 conn(0x7ff554066360 msgr2=0x7ff554066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:36.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 -- 172.21.15.12:0/49662963 shutdown_connections 2023-11-28T21:56:36.455 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 -- 172.21.15.12:0/49662963 wait complete. 2023-11-28T21:56:36.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 Processor -- start 2023-11-28T21:56:36.456 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 -- start start 2023-11-28T21:56:36.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55406e300 0x7ff55414c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff554151af0 0x7ff554140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff55414cd50 0x7ff55413c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541f1930 con 0x7ff55414cd50 2023-11-28T21:56:36.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541f0dc0 con 0x7ff55406e300 2023-11-28T21:56:36.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff55b0bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5541f1d50 con 0x7ff554151af0 2023-11-28T21:56:36.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff559658640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff55414cd50 0x7ff55413c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.458 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff559658640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff55414cd50 0x7ff55413c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48650/0 (socket says 172.21.15.12:48650) 2023-11-28T21:56:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff559658640 1 -- 172.21.15.12:0/3489968201 learned_addr learned my addr 172.21.15.12:0/3489968201 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff558e57640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55406e300 0x7ff55414c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff553fff640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff554151af0 0x7ff554140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff559658640 1 -- 172.21.15.12:0/3489968201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff554151af0 msgr2=0x7ff554140f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff559658640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff554151af0 0x7ff554140f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff559658640 1 -- 172.21.15.12:0/3489968201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55406e300 msgr2=0x7ff55414c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff559658640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55406e300 0x7ff55414c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.459 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff559658640 1 -- 172.21.15.12:0/3489968201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400094d0 con 0x7ff55414cd50 2023-11-28T21:56:36.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.452+0000 7ff553fff640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff554151af0 0x7ff554140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:36.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff559658640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff55414cd50 0x7ff55413c910 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff54c007ec0 tx=0x7ff54c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:36.460 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff5517fa640 1 -- 172.21.15.12:0/3489968201 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c003ba0 con 0x7ff55414cd50 2023-11-28T21:56:36.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff5517fa640 1 -- 172.21.15.12:0/3489968201 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c00f040 con 0x7ff55414cd50 2023-11-28T21:56:36.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff5517fa640 1 -- 172.21.15.12:0/3489968201 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00de80 con 0x7ff55414cd50 2023-11-28T21:56:36.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff55b0bb640 1 -- 172.21.15.12:0/3489968201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5541f0fb0 con 0x7ff55414cd50 2023-11-28T21:56:36.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff55b0bb640 1 -- 172.21.15.12:0/3489968201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff554142370 con 0x7ff55414cd50 2023-11-28T21:56:36.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff55b0bb640 1 -- 172.21.15.12:0/3489968201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5541f1a70 con 0x7ff55414cd50 2023-11-28T21:56:36.461 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff5517fa640 1 -- 172.21.15.12:0/3489968201 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff54c004030 con 0x7ff55414cd50 2023-11-28T21:56:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff5517fa640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff538066470 0x7ff538068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff5517fa640 1 -- 172.21.15.12:0/3489968201 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7ff54c0848d0 con 0x7ff55414cd50 2023-11-28T21:56:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.456+0000 7ff558e57640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff538066470 0x7ff538068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.462 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.460+0000 7ff558e57640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff538066470 0x7ff538068930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff548007fd0 tx=0x7ff548009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:36.465 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.460+0000 7ff5517fa640 1 -- 172.21.15.12:0/3489968201 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff5541f1a70 con 0x7ff55414cd50 2023-11-28T21:56:36.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.620+0000 7ff55b0bb640 1 -- 172.21.15.12:0/3489968201 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5540729c0 con 0x7ff538066470 2023-11-28T21:56:36.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.632+0000 7ff5517fa640 1 -- 172.21.15.12:0/3489968201 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+374122 (secure 0 0 0) 0x7ff5480014f0 con 0x7ff538066470 2023-11-28T21:56:36.658 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:36.660 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:56:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 -- 172.21.15.12:0/3489968201 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff538066470 msgr2=0x7ff538068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff538066470 0x7ff538068930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff548007fd0 tx=0x7ff548009040 comp rx=0 tx=0).stop 2023-11-28T21:56:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 -- 172.21.15.12:0/3489968201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff55414cd50 msgr2=0x7ff55413c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff55414cd50 0x7ff55413c910 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff54c007ec0 tx=0x7ff54c00d4a0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 -- 172.21.15.12:0/3489968201 shutdown_connections 2023-11-28T21:56:36.661 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff538066470 0x7ff538068930 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff55414cd50 0x7ff55413c910 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff554151af0 0x7ff554140f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 --2- 172.21.15.12:0/3489968201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55406e300 0x7ff55414c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 -- 172.21.15.12:0/3489968201 >> 172.21.15.12:0/3489968201 conn(0x7ff554066360 msgr2=0x7ff5540c7210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:36.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 -- 172.21.15.12:0/3489968201 shutdown_connections 2023-11-28T21:56:36.662 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.636+0000 7ff55b0bb640 1 -- 172.21.15.12:0/3489968201 wait complete. 2023-11-28T21:56:36.664 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":708,"stamp":"2023-11-28T21:56:36.210879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1280114506,"num_objects":10548,"num_object_clones":0,"num_object_copies":41774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10548,"num_whiteouts":0,"num_read":9144,"num_read_kb":2407205,"num_write":56520,"num_write_kb":2354126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76052,"ondisk_log_size":76052,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3008496,"kb_used_data":1953900,"kb_used_omap":12,"kb_used_meta":1054515,"kb_avail":751966224,"statfs":{"total":773094113280,"available":770013413376,"internally_reserved":0,"allocated":2000793600,"data_stored":2656074320,"data_compressed":213714559,"data_compressed_allocated":256286720,"data_compressed_original":914001814,"omap_allocated":12751,"internal_metadata":1079823921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,1,8,13,7,4,0,1,78],"upper_bound":1024},"perf_stat":{"commit_latency_ms":267,"apply_latency_ms":267,"commit_latency_ns":267000000,"apply_latency_ns":267000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26378372,"num_objects":1446,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1557,"num_write_kb":26881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004723"},"pg_stats":[{"pgid":"2.1f","version":"51'341","reported_seq":427,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.057024+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:55:30.057024+0000","last_peered":"2023-11-28T21:55:30.057024+0000","last_clean":"2023-11-28T21:55:30.057024+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:55:30.057024+0000","last_undegraded":"2023-11-28T21:55:30.057024+0000","last_fullsized":"2023-11-28T21:55:30.057024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:46.167511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":61536,"num_write":310,"num_write_kb":8499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"50'268","reported_seq":2351,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:44.913651+0000","last_change":"2023-11-28T21:40:42.800615+0000","last_active":"2023-11-28T21:54:44.913651+0000","last_peered":"2023-11-28T21:54:44.913651+0000","last_clean":"2023-11-28T21:54:44.913651+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:54:44.913651+0000","last_undegraded":"2023-11-28T21:54:44.913651+0000","last_fullsized":"2023-11-28T21:54:44.913651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:37.956537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28950874199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"52'1318","reported_seq":3628,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.030721+0000","last_change":"2023-11-28T21:40:42.803511+0000","last_active":"2023-11-28T21:56:35.030721+0000","last_peered":"2023-11-28T21:56:35.030721+0000","last_clean":"2023-11-28T21:56:35.030721+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:56:35.030721+0000","last_undegraded":"2023-11-28T21:56:35.030721+0000","last_fullsized":"2023-11-28T21:56:35.030721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:31:55.934555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29329539100000002,"stat_sum":{"num_bytes":30193112,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":229,"num_read_kb":12319,"num_write":1291,"num_write_kb":57664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"52'465","reported_seq":539,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:26.898568+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:56:26.898568+0000","last_peered":"2023-11-28T21:56:26.898568+0000","last_clean":"2023-11-28T21:56:26.898568+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:56:26.898568+0000","last_undegraded":"2023-11-28T21:56:26.898568+0000","last_fullsized":"2023-11-28T21:56:26.898568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":81920,"num_write":420,"num_write_kb":20770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"50'251","reported_seq":2334,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.057010+0000","last_change":"2023-11-28T21:40:43.257406+0000","last_active":"2023-11-28T21:55:30.057010+0000","last_peered":"2023-11-28T21:55:30.057010+0000","last_clean":"2023-11-28T21:55:30.057010+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:55:30.057010+0000","last_undegraded":"2023-11-28T21:55:30.057010+0000","last_fullsized":"2023-11-28T21:55:30.057010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:47.663480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23094383900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"52'1401","reported_seq":3733,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.720695+0000","last_change":"2023-11-28T21:40:43.072665+0000","last_active":"2023-11-28T21:56:34.720695+0000","last_peered":"2023-11-28T21:56:34.720695+0000","last_clean":"2023-11-28T21:56:34.720695+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:56:34.720695+0000","last_undegraded":"2023-11-28T21:56:34.720695+0000","last_fullsized":"2023-11-28T21:56:34.720695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:41:20.225443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21613474099999999,"stat_sum":{"num_bytes":11729902,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":251,"num_read_kb":24615,"num_write":1375,"num_write_kb":48688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"52'761","reported_seq":854,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.356193+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:56:34.356193+0000","last_peered":"2023-11-28T21:56:34.356193+0000","last_clean":"2023-11-28T21:56:34.356193+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:56:34.356193+0000","last_undegraded":"2023-11-28T21:56:34.356193+0000","last_fullsized":"2023-11-28T21:56:34.356193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":59,"num_read_kb":131099,"num_write":714,"num_write_kb":20835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"50'280","reported_seq":2362,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147679+0000","last_change":"2023-11-28T21:40:42.830278+0000","last_active":"2023-11-28T21:54:40.147679+0000","last_peered":"2023-11-28T21:54:40.147679+0000","last_clean":"2023-11-28T21:54:40.147679+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:54:40.147679+0000","last_undegraded":"2023-11-28T21:54:40.147679+0000","last_fullsized":"2023-11-28T21:54:40.147679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:24.137854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30458173900000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"52'1324","reported_seq":3630,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.861331+0000","last_change":"2023-11-28T21:40:43.070433+0000","last_active":"2023-11-28T21:56:34.861331+0000","last_peered":"2023-11-28T21:56:34.861331+0000","last_clean":"2023-11-28T21:56:34.861331+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:56:34.861331+0000","last_undegraded":"2023-11-28T21:56:34.861331+0000","last_fullsized":"2023-11-28T21:56:34.861331+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:00.120053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23712909900000001,"stat_sum":{"num_bytes":32661416,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":226,"num_read_kb":7690,"num_write":1300,"num_write_kb":56471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"52'95","reported_seq":165,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.876669+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:56:27.876669+0000","last_peered":"2023-11-28T21:56:27.876669+0000","last_clean":"2023-11-28T21:56:27.876669+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:56:27.876669+0000","last_undegraded":"2023-11-28T21:56:27.876669+0000","last_fullsized":"2023-11-28T21:56:27.876669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":36891,"num_write":50,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"50'274","reported_seq":2356,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147734+0000","last_change":"2023-11-28T21:40:42.827917+0000","last_active":"2023-11-28T21:54:40.147734+0000","last_peered":"2023-11-28T21:54:40.147734+0000","last_clean":"2023-11-28T21:54:40.147734+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:54:40.147734+0000","last_undegraded":"2023-11-28T21:54:40.147734+0000","last_fullsized":"2023-11-28T21:54:40.147734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:32:57.982365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30416134,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"52'1294","reported_seq":3610,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.796080+0000","last_change":"2023-11-28T21:40:43.055466+0000","last_active":"2023-11-28T21:56:34.796080+0000","last_peered":"2023-11-28T21:56:34.796080+0000","last_clean":"2023-11-28T21:56:34.796080+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:56:34.796080+0000","last_undegraded":"2023-11-28T21:56:34.796080+0000","last_fullsized":"2023-11-28T21:56:34.796080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:33:55.527250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26667665699999998,"stat_sum":{"num_bytes":37281632,"num_objects":312,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":0,"num_read":236,"num_read_kb":13169,"num_write":1275,"num_write_kb":61269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"51'287","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.278456+0000","last_change":"2023-11-28T21:40:42.917342+0000","last_active":"2023-11-28T21:55:39.278456+0000","last_peered":"2023-11-28T21:55:39.278456+0000","last_clean":"2023-11-28T21:55:39.278456+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:55:39.278456+0000","last_undegraded":"2023-11-28T21:55:39.278456+0000","last_fullsized":"2023-11-28T21:55:39.278456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:39.167984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.200606007,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"52'617","reported_seq":711,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.513022+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:56:18.513022+0000","last_peered":"2023-11-28T21:56:18.513022+0000","last_clean":"2023-11-28T21:56:18.513022+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:56:18.513022+0000","last_undegraded":"2023-11-28T21:56:18.513022+0000","last_fullsized":"2023-11-28T21:56:18.513022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":114718,"num_write":571,"num_write_kb":20791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"52'1331","reported_seq":3650,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.683935+0000","last_change":"2023-11-28T21:40:43.071642+0000","last_active":"2023-11-28T21:56:34.683935+0000","last_peered":"2023-11-28T21:56:34.683935+0000","last_clean":"2023-11-28T21:56:34.683935+0000","last_became_active":"2023-11-28T21:35:25.616978+0000","last_became_peered":"2023-11-28T21:35:25.616978+0000","last_unstale":"2023-11-28T21:56:34.683935+0000","last_undegraded":"2023-11-28T21:56:34.683935+0000","last_fullsized":"2023-11-28T21:56:34.683935+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1331,"log_dups_size":0,"ondisk_log_size":1331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:05:37.470115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23129675,"stat_sum":{"num_bytes":32707084,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":229,"num_read_kb":4127,"num_write":1308,"num_write_kb":57872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"50'285","reported_seq":2367,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.047822+0000","last_change":"2023-11-28T21:40:42.798788+0000","last_active":"2023-11-28T21:54:55.047822+0000","last_peered":"2023-11-28T21:54:55.047822+0000","last_clean":"2023-11-28T21:54:55.047822+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:54:55.047822+0000","last_undegraded":"2023-11-28T21:54:55.047822+0000","last_fullsized":"2023-11-28T21:54:55.047822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:37.662225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28777871399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"52'954","reported_seq":1032,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:25.006236+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:56:25.006236+0000","last_peered":"2023-11-28T21:56:25.006236+0000","last_clean":"2023-11-28T21:56:25.006236+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:56:25.006236+0000","last_undegraded":"2023-11-28T21:56:25.006236+0000","last_fullsized":"2023-11-28T21:56:25.006236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":954,"log_dups_size":0,"ondisk_log_size":954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:53.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":75782,"num_write":932,"num_write_kb":13441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"52'1416","reported_seq":3740,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.956993+0000","last_change":"2023-11-28T21:40:43.070729+0000","last_active":"2023-11-28T21:56:34.956993+0000","last_peered":"2023-11-28T21:56:34.956993+0000","last_clean":"2023-11-28T21:56:34.956993+0000","last_became_active":"2023-11-28T21:35:25.790674+0000","last_became_peered":"2023-11-28T21:35:25.790674+0000","last_unstale":"2023-11-28T21:56:34.956993+0000","last_undegraded":"2023-11-28T21:56:34.956993+0000","last_fullsized":"2023-11-28T21:56:34.956993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1416,"log_dups_size":0,"ondisk_log_size":1416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:12:03.263147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.229623311,"stat_sum":{"num_bytes":17731185,"num_objects":354,"num_object_clones":0,"num_object_copies":1416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":234,"num_read_kb":20007,"num_write":1382,"num_write_kb":71863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"50'296","reported_seq":2379,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.272334+0000","last_change":"2023-11-28T21:40:42.800893+0000","last_active":"2023-11-28T21:55:39.272334+0000","last_peered":"2023-11-28T21:55:39.272334+0000","last_clean":"2023-11-28T21:55:39.272334+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:55:39.272334+0000","last_undegraded":"2023-11-28T21:55:39.272334+0000","last_fullsized":"2023-11-28T21:55:39.272334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:35:30.683746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29027087200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"52'722","reported_seq":790,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:33.926232+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:56:33.926232+0000","last_peered":"2023-11-28T21:56:33.926232+0000","last_clean":"2023-11-28T21:56:33.926232+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:56:33.926232+0000","last_undegraded":"2023-11-28T21:56:33.926232+0000","last_fullsized":"2023-11-28T21:56:33.926232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9810037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":32782,"num_write":677,"num_write_kb":9926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"52'1341","reported_seq":3684,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.913491+0000","last_change":"2023-11-28T21:40:42.890432+0000","last_active":"2023-11-28T21:56:34.913491+0000","last_peered":"2023-11-28T21:56:34.913491+0000","last_clean":"2023-11-28T21:56:34.913491+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:56:34.913491+0000","last_undegraded":"2023-11-28T21:56:34.913491+0000","last_fullsized":"2023-11-28T21:56:34.913491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1341,"log_dups_size":0,"ondisk_log_size":1341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:09:28.392308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24643899999999999,"stat_sum":{"num_bytes":14380393,"num_objects":336,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":262,"num_read_kb":16758,"num_write":1318,"num_write_kb":45879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"50'276","reported_seq":2359,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.687600+0000","last_change":"2023-11-28T21:40:43.071973+0000","last_active":"2023-11-28T21:55:38.687600+0000","last_peered":"2023-11-28T21:55:38.687600+0000","last_clean":"2023-11-28T21:55:38.687600+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:55:38.687600+0000","last_undegraded":"2023-11-28T21:55:38.687600+0000","last_fullsized":"2023-11-28T21:55:38.687600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:34:28.182888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21610210399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"52'341","reported_seq":411,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.377147+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:56:08.377147+0000","last_peered":"2023-11-28T21:56:08.377147+0000","last_clean":"2023-11-28T21:56:08.377147+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:56:08.377147+0000","last_undegraded":"2023-11-28T21:56:08.377147+0000","last_fullsized":"2023-11-28T21:56:08.377147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":49164,"num_write":265,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"52'1262","reported_seq":3549,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.730763+0000","last_change":"2023-11-28T21:40:43.060232+0000","last_active":"2023-11-28T21:56:34.730763+0000","last_peered":"2023-11-28T21:56:34.730763+0000","last_clean":"2023-11-28T21:56:34.730763+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:56:34.730763+0000","last_undegraded":"2023-11-28T21:56:34.730763+0000","last_fullsized":"2023-11-28T21:56:34.730763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:54:07.778886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.244586779,"stat_sum":{"num_bytes":50267474,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":206,"num_read_kb":15432,"num_write":1240,"num_write_kb":82194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"50'281","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.276659+0000","last_change":"2023-11-28T21:40:43.249048+0000","last_active":"2023-11-28T21:55:39.276659+0000","last_peered":"2023-11-28T21:55:39.276659+0000","last_clean":"2023-11-28T21:55:39.276659+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:55:39.276659+0000","last_undegraded":"2023-11-28T21:55:39.276659+0000","last_fullsized":"2023-11-28T21:55:39.276659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:35:39.660816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26728515899999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"52'168","reported_seq":240,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:20.126618+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:56:20.126618+0000","last_peered":"2023-11-28T21:56:20.126618+0000","last_clean":"2023-11-28T21:56:20.126618+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:56:20.126618+0000","last_undegraded":"2023-11-28T21:56:20.126618+0000","last_fullsized":"2023-11-28T21:56:20.126618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":45,"num_read_kb":24615,"num_write":146,"num_write_kb":4216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"52'1368","reported_seq":3700,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.654211+0000","last_change":"2023-11-28T21:40:43.072395+0000","last_active":"2023-11-28T21:56:34.654211+0000","last_peered":"2023-11-28T21:56:34.654211+0000","last_clean":"2023-11-28T21:56:34.654211+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:56:34.654211+0000","last_undegraded":"2023-11-28T21:56:34.654211+0000","last_fullsized":"2023-11-28T21:56:34.654211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:25:44.577393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21672229300000001,"stat_sum":{"num_bytes":16299977,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":250,"num_read_kb":17253,"num_write":1343,"num_write_kb":69509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'249","reported_seq":2332,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273472+0000","last_change":"2023-11-28T21:40:42.801394+0000","last_active":"2023-11-28T21:55:39.273472+0000","last_peered":"2023-11-28T21:55:39.273472+0000","last_clean":"2023-11-28T21:55:39.273472+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:55:39.273472+0000","last_undegraded":"2023-11-28T21:55:39.273472+0000","last_fullsized":"2023-11-28T21:55:39.273472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:59:35.647639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29009278799999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"52'245","reported_seq":314,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.893946+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:56:27.893946+0000","last_peered":"2023-11-28T21:56:27.893946+0000","last_clean":"2023-11-28T21:56:27.893946+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:56:27.893946+0000","last_undegraded":"2023-11-28T21:56:27.893946+0000","last_fullsized":"2023-11-28T21:56:27.893946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:36.968220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":607,"num_write":445,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"52'1386","reported_seq":3700,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.993814+0000","last_change":"2023-11-28T21:40:42.797955+0000","last_active":"2023-11-28T21:56:34.993814+0000","last_peered":"2023-11-28T21:56:34.993814+0000","last_clean":"2023-11-28T21:56:34.993814+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:56:34.993814+0000","last_undegraded":"2023-11-28T21:56:34.993814+0000","last_fullsized":"2023-11-28T21:56:34.993814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1386,"log_dups_size":0,"ondisk_log_size":1386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:27:40.256597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.287791554,"stat_sum":{"num_bytes":19942162,"num_objects":313,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":232,"num_read_kb":16373,"num_write":1352,"num_write_kb":52080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"50'287","reported_seq":2371,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:29.999480+0000","last_change":"2023-11-28T21:40:42.884411+0000","last_active":"2023-11-28T21:55:29.999480+0000","last_peered":"2023-11-28T21:55:29.999480+0000","last_clean":"2023-11-28T21:55:29.999480+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:55:29.999480+0000","last_undegraded":"2023-11-28T21:55:29.999480+0000","last_fullsized":"2023-11-28T21:55:29.999480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:48.590211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24779062600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"52'587","reported_seq":665,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.695016+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:55:43.695016+0000","last_peered":"2023-11-28T21:55:43.695016+0000","last_clean":"2023-11-28T21:55:43.695016+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:55:43.695016+0000","last_undegraded":"2023-11-28T21:55:43.695016+0000","last_fullsized":"2023-11-28T21:55:43.695016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":81538,"num_write":523,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"52'1410","reported_seq":3740,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.895421+0000","last_change":"2023-11-28T21:40:43.255749+0000","last_active":"2023-11-28T21:56:34.895421+0000","last_peered":"2023-11-28T21:56:34.895421+0000","last_clean":"2023-11-28T21:56:34.895421+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:56:34.895421+0000","last_undegraded":"2023-11-28T21:56:34.895421+0000","last_fullsized":"2023-11-28T21:56:34.895421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:38:18.061461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27155133199999998,"stat_sum":{"num_bytes":49110889,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":248,"num_read_kb":24602,"num_write":1394,"num_write_kb":89036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"50'274","reported_seq":2358,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.201955+0000","last_change":"2023-11-28T21:40:42.828660+0000","last_active":"2023-11-28T21:55:30.201955+0000","last_peered":"2023-11-28T21:55:30.201955+0000","last_clean":"2023-11-28T21:55:30.201955+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:55:30.201955+0000","last_undegraded":"2023-11-28T21:55:30.201955+0000","last_fullsized":"2023-11-28T21:55:30.201955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:55.786023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30430776599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"52'164","reported_seq":225,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.682785+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:55:38.682785+0000","last_peered":"2023-11-28T21:55:38.682785+0000","last_clean":"2023-11-28T21:55:38.682785+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:55:38.682785+0000","last_undegraded":"2023-11-28T21:55:38.682785+0000","last_fullsized":"2023-11-28T21:55:38.682785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:08:15.270352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24576,"num_write":146,"num_write_kb":4262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"52'1318","reported_seq":3622,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.840355+0000","last_change":"2023-11-28T21:40:42.887654+0000","last_active":"2023-11-28T21:56:34.840355+0000","last_peered":"2023-11-28T21:56:34.840355+0000","last_clean":"2023-11-28T21:56:34.840355+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:56:34.840355+0000","last_undegraded":"2023-11-28T21:56:34.840355+0000","last_fullsized":"2023-11-28T21:56:34.840355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:38.441656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.25659980999999998,"stat_sum":{"num_bytes":32626940,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":222,"num_read_kb":4128,"num_write":1285,"num_write_kb":60020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"50'286","reported_seq":2369,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.214610+0000","last_change":"2023-11-28T21:40:43.068901+0000","last_active":"2023-11-28T21:55:30.214610+0000","last_peered":"2023-11-28T21:55:30.214610+0000","last_clean":"2023-11-28T21:55:30.214610+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:55:30.214610+0000","last_undegraded":"2023-11-28T21:55:30.214610+0000","last_fullsized":"2023-11-28T21:55:30.214610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:05:04.092055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23142385100000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"52'365","reported_seq":439,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.694515+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:55:38.694515+0000","last_peered":"2023-11-28T21:55:38.694515+0000","last_clean":"2023-11-28T21:55:38.694515+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:55:38.694515+0000","last_undegraded":"2023-11-28T21:55:38.694515+0000","last_fullsized":"2023-11-28T21:55:38.694515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":81920,"num_write":337,"num_write_kb":16655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"52'1263","reported_seq":3555,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.958687+0000","last_change":"2023-11-28T21:40:42.883278+0000","last_active":"2023-11-28T21:56:34.958687+0000","last_peered":"2023-11-28T21:56:34.958687+0000","last_clean":"2023-11-28T21:56:34.958687+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:56:34.958687+0000","last_undegraded":"2023-11-28T21:56:34.958687+0000","last_fullsized":"2023-11-28T21:56:34.958687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:30:54.117448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24685721299999999,"stat_sum":{"num_bytes":16718463,"num_objects":288,"num_object_clones":0,"num_object_copies":1152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":210,"num_read_kb":3302,"num_write":1235,"num_write_kb":35539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"50'283","reported_seq":2367,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:44.891386+0000","last_change":"2023-11-28T21:40:42.890540+0000","last_active":"2023-11-28T21:54:44.891386+0000","last_peered":"2023-11-28T21:54:44.891386+0000","last_clean":"2023-11-28T21:54:44.891386+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:54:44.891386+0000","last_undegraded":"2023-11-28T21:54:44.891386+0000","last_fullsized":"2023-11-28T21:54:44.891386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:51.055896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24023003500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"51'934","reported_seq":1405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.724754+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:55:43.724754+0000","last_peered":"2023-11-28T21:55:43.724754+0000","last_clean":"2023-11-28T21:55:43.724754+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:55:43.724754+0000","last_undegraded":"2023-11-28T21:55:43.724754+0000","last_fullsized":"2023-11-28T21:55:43.724754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18884916,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":417,"num_read_kb":34819,"num_write":869,"num_write_kb":18975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"52'1231","reported_seq":3557,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.911522+0000","last_change":"2023-11-28T21:40:42.888191+0000","last_active":"2023-11-28T21:56:34.911522+0000","last_peered":"2023-11-28T21:56:34.911522+0000","last_clean":"2023-11-28T21:56:34.911522+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:56:34.911522+0000","last_undegraded":"2023-11-28T21:56:34.911522+0000","last_fullsized":"2023-11-28T21:56:34.911522+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:20.371038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24393632200000001,"stat_sum":{"num_bytes":26023878,"num_objects":293,"num_object_clones":0,"num_object_copies":1172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":293,"num_whiteouts":0,"num_read":244,"num_read_kb":12314,"num_write":1202,"num_write_kb":62723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"51'259","reported_seq":2342,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.056922+0000","last_change":"2023-11-28T21:40:43.256849+0000","last_active":"2023-11-28T21:55:30.056922+0000","last_peered":"2023-11-28T21:55:30.056922+0000","last_clean":"2023-11-28T21:55:30.056922+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:55:30.056922+0000","last_undegraded":"2023-11-28T21:55:30.056922+0000","last_fullsized":"2023-11-28T21:55:30.056922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:25.250732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.25105937499999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"51'313","reported_seq":395,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.725792+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:55:38.725792+0000","last_peered":"2023-11-28T21:55:38.725792+0000","last_clean":"2023-11-28T21:55:38.725792+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:55:38.725792+0000","last_undegraded":"2023-11-28T21:55:38.725792+0000","last_fullsized":"2023-11-28T21:55:38.725792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":114688,"num_write":275,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"52'1365","reported_seq":3711,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.880915+0000","last_change":"2023-11-28T21:40:43.252159+0000","last_active":"2023-11-28T21:56:34.880915+0000","last_peered":"2023-11-28T21:56:34.880915+0000","last_clean":"2023-11-28T21:56:34.880915+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:56:34.880915+0000","last_undegraded":"2023-11-28T21:56:34.880915+0000","last_fullsized":"2023-11-28T21:56:34.880915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:58:49.674343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.270607394,"stat_sum":{"num_bytes":19916106,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":264,"num_read_kb":13370,"num_write":1336,"num_write_kb":59202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"50'268","reported_seq":2351,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.051948+0000","last_change":"2023-11-28T21:40:42.802535+0000","last_active":"2023-11-28T21:55:30.051948+0000","last_peered":"2023-11-28T21:55:30.051948+0000","last_clean":"2023-11-28T21:55:30.051948+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:55:30.051948+0000","last_undegraded":"2023-11-28T21:55:30.051948+0000","last_fullsized":"2023-11-28T21:55:30.051948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:02:13.185718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29152715000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"52'311","reported_seq":375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:07.951419+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:56:07.951419+0000","last_peered":"2023-11-28T21:56:07.951419+0000","last_clean":"2023-11-28T21:56:07.951419+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:56:07.951419+0000","last_undegraded":"2023-11-28T21:56:07.951419+0000","last_fullsized":"2023-11-28T21:56:07.951419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":24594,"num_write":297,"num_write_kb":8480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.725972+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:55:38.725972+0000","last_peered":"2023-11-28T21:55:38.725972+0000","last_clean":"2023-11-28T21:55:38.725972+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:55:38.725972+0000","last_undegraded":"2023-11-28T21:55:38.725972+0000","last_fullsized":"2023-11-28T21:55:38.725972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"52'1342","reported_seq":3676,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.779903+0000","last_change":"2023-11-28T21:40:42.914002+0000","last_active":"2023-11-28T21:56:34.779903+0000","last_peered":"2023-11-28T21:56:34.779903+0000","last_clean":"2023-11-28T21:56:34.779903+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:56:34.779903+0000","last_undegraded":"2023-11-28T21:56:34.779903+0000","last_fullsized":"2023-11-28T21:56:34.779903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:47.036263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19732549099999999,"stat_sum":{"num_bytes":29762487,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":252,"num_read_kb":20508,"num_write":1310,"num_write_kb":73242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"50'256","reported_seq":2339,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.116489+0000","last_change":"2023-11-28T21:40:42.748386+0000","last_active":"2023-11-28T21:54:55.116489+0000","last_peered":"2023-11-28T21:54:55.116489+0000","last_clean":"2023-11-28T21:54:55.116489+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:54:55.116489+0000","last_undegraded":"2023-11-28T21:54:55.116489+0000","last_fullsized":"2023-11-28T21:54:55.116489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:59.734407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.219020891,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"52'475","reported_seq":534,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273615+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:55:39.273615+0000","last_peered":"2023-11-28T21:55:39.273615+0000","last_clean":"2023-11-28T21:55:39.273615+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:55:39.273615+0000","last_undegraded":"2023-11-28T21:55:39.273615+0000","last_fullsized":"2023-11-28T21:55:39.273615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":20485,"num_write":447,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"52'1372","reported_seq":3704,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.807900+0000","last_change":"2023-11-28T21:40:42.915914+0000","last_active":"2023-11-28T21:56:34.807900+0000","last_peered":"2023-11-28T21:56:34.807900+0000","last_clean":"2023-11-28T21:56:34.807900+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:56:34.807900+0000","last_undegraded":"2023-11-28T21:56:34.807900+0000","last_fullsized":"2023-11-28T21:56:34.807900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1372,"log_dups_size":0,"ondisk_log_size":1372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:52:07.090824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19929065100000001,"stat_sum":{"num_bytes":26470227,"num_objects":335,"num_object_clones":0,"num_object_copies":1340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":251,"num_read_kb":4134,"num_write":1344,"num_write_kb":47412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"51'313","reported_seq":2396,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.194876+0000","last_change":"2023-11-28T21:40:42.830692+0000","last_active":"2023-11-28T21:54:55.194876+0000","last_peered":"2023-11-28T21:54:55.194876+0000","last_clean":"2023-11-28T21:54:55.194876+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:54:55.194876+0000","last_undegraded":"2023-11-28T21:54:55.194876+0000","last_fullsized":"2023-11-28T21:54:55.194876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:33:58.325515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30458413400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"52'964","reported_seq":1044,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.896525+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:56:27.896525+0000","last_peered":"2023-11-28T21:56:27.896525+0000","last_clean":"2023-11-28T21:56:27.896525+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:56:27.896525+0000","last_undegraded":"2023-11-28T21:56:27.896525+0000","last_fullsized":"2023-11-28T21:56:27.896525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":964,"log_dups_size":0,"ondisk_log_size":964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:48.073446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777286,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":27,"num_read_kb":68346,"num_write":930,"num_write_kb":16947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"52'1319","reported_seq":3665,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.895088+0000","last_change":"2023-11-28T21:40:43.256533+0000","last_active":"2023-11-28T21:56:34.895088+0000","last_peered":"2023-11-28T21:56:34.895088+0000","last_clean":"2023-11-28T21:56:34.895088+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:56:34.895088+0000","last_undegraded":"2023-11-28T21:56:34.895088+0000","last_fullsized":"2023-11-28T21:56:34.895088+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:44.854009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27089875299999999,"stat_sum":{"num_bytes":28004524,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":265,"num_read_kb":16422,"num_write":1294,"num_write_kb":50614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"50'264","reported_seq":2340,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.274191+0000","last_change":"2023-11-28T21:40:43.090122+0000","last_active":"2023-11-28T21:55:39.274191+0000","last_peered":"2023-11-28T21:55:39.274191+0000","last_clean":"2023-11-28T21:55:39.274191+0000","last_became_active":"2023-11-28T21:35:25.594533+0000","last_became_peered":"2023-11-28T21:35:25.594533+0000","last_unstale":"2023-11-28T21:55:39.274191+0000","last_undegraded":"2023-11-28T21:55:39.274191+0000","last_fullsized":"2023-11-28T21:55:39.274191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:03.418589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14880083199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"52'303","reported_seq":378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.839543+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:56:27.839543+0000","last_peered":"2023-11-28T21:56:27.839543+0000","last_clean":"2023-11-28T21:56:27.839543+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:56:27.839543+0000","last_undegraded":"2023-11-28T21:56:27.839543+0000","last_fullsized":"2023-11-28T21:56:27.839543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":53275,"num_write":251,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"52'1307","reported_seq":3591,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.520158+0000","last_change":"2023-11-28T21:40:42.823686+0000","last_active":"2023-11-28T21:56:34.520158+0000","last_peered":"2023-11-28T21:56:34.520158+0000","last_clean":"2023-11-28T21:56:34.520158+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:56:34.520158+0000","last_undegraded":"2023-11-28T21:56:34.520158+0000","last_fullsized":"2023-11-28T21:56:34.520158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1307,"log_dups_size":0,"ondisk_log_size":1307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:34:35.818463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30648515900000001,"stat_sum":{"num_bytes":28110180,"num_objects":295,"num_object_clones":0,"num_object_copies":1180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":0,"num_read":202,"num_read_kb":8227,"num_write":1283,"num_write_kb":51039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"50'267","reported_seq":2350,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.116571+0000","last_change":"2023-11-28T21:40:42.749319+0000","last_active":"2023-11-28T21:54:55.116571+0000","last_peered":"2023-11-28T21:54:55.116571+0000","last_clean":"2023-11-28T21:54:55.116571+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:54:55.116571+0000","last_undegraded":"2023-11-28T21:54:55.116571+0000","last_fullsized":"2023-11-28T21:54:55.116571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:18:31.031890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21877398200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"52'193","reported_seq":266,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.689380+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:55:38.689380+0000","last_peered":"2023-11-28T21:55:38.689380+0000","last_clean":"2023-11-28T21:55:38.689380+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:55:38.689380+0000","last_undegraded":"2023-11-28T21:55:38.689380+0000","last_fullsized":"2023-11-28T21:55:38.689380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":21,"num_read_kb":49161,"num_write":146,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"52'1273","reported_seq":3574,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.716263+0000","last_change":"2023-11-28T21:40:43.072161+0000","last_active":"2023-11-28T21:56:34.716263+0000","last_peered":"2023-11-28T21:56:34.716263+0000","last_clean":"2023-11-28T21:56:34.716263+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:56:34.716263+0000","last_undegraded":"2023-11-28T21:56:34.716263+0000","last_fullsized":"2023-11-28T21:56:34.716263+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:39.182750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.223266511,"stat_sum":{"num_bytes":11523503,"num_objects":308,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":219,"num_read_kb":12316,"num_write":1248,"num_write_kb":38807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"50'308","reported_seq":2391,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.725792+0000","last_change":"2023-11-28T21:40:42.747053+0000","last_active":"2023-11-28T21:55:38.725792+0000","last_peered":"2023-11-28T21:55:38.725792+0000","last_clean":"2023-11-28T21:55:38.725792+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:55:38.725792+0000","last_undegraded":"2023-11-28T21:55:38.725792+0000","last_fullsized":"2023-11-28T21:55:38.725792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:54:52.864875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21978382299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"52'854","reported_seq":964,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.035796+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:56:18.035796+0000","last_peered":"2023-11-28T21:56:18.035796+0000","last_clean":"2023-11-28T21:56:18.035796+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:56:18.035796+0000","last_undegraded":"2023-11-28T21:56:18.035796+0000","last_fullsized":"2023-11-28T21:56:18.035796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":137,"num_read_kb":58709,"num_write":838,"num_write_kb":8642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"52'1274","reported_seq":3606,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.943389+0000","last_change":"2023-11-28T21:40:43.252839+0000","last_active":"2023-11-28T21:56:34.943389+0000","last_peered":"2023-11-28T21:56:34.943389+0000","last_clean":"2023-11-28T21:56:34.943389+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:56:34.943389+0000","last_undegraded":"2023-11-28T21:56:34.943389+0000","last_fullsized":"2023-11-28T21:56:34.943389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:28:08.856108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27152726799999999,"stat_sum":{"num_bytes":36307339,"num_objects":283,"num_object_clones":0,"num_object_copies":1132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":283,"num_whiteouts":0,"num_read":251,"num_read_kb":8046,"num_write":1253,"num_write_kb":60298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"51'273","reported_seq":2355,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.278420+0000","last_change":"2023-11-28T21:40:42.918241+0000","last_active":"2023-11-28T21:55:39.278420+0000","last_peered":"2023-11-28T21:55:39.278420+0000","last_clean":"2023-11-28T21:55:39.278420+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:55:39.278420+0000","last_undegraded":"2023-11-28T21:55:39.278420+0000","last_fullsized":"2023-11-28T21:55:39.278420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:57.732993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18244292200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"52'239","reported_seq":293,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.710890+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:55:43.710890+0000","last_peered":"2023-11-28T21:55:43.710890+0000","last_clean":"2023-11-28T21:55:43.710890+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:55:43.710890+0000","last_undegraded":"2023-11-28T21:55:43.710890+0000","last_fullsized":"2023-11-28T21:55:43.710890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195875,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":194,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"52'1346","reported_seq":3662,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.682315+0000","last_change":"2023-11-28T21:40:42.805322+0000","last_active":"2023-11-28T21:56:34.682315+0000","last_peered":"2023-11-28T21:56:34.682315+0000","last_clean":"2023-11-28T21:56:34.682315+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:56:34.682315+0000","last_undegraded":"2023-11-28T21:56:34.682315+0000","last_fullsized":"2023-11-28T21:56:34.682315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:22:10.703578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29439885100000002,"stat_sum":{"num_bytes":34458204,"num_objects":324,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":324,"num_whiteouts":0,"num_read":235,"num_read_kb":27797,"num_write":1320,"num_write_kb":70332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"50'300","reported_seq":2383,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.726358+0000","last_change":"2023-11-28T21:40:42.740620+0000","last_active":"2023-11-28T21:55:38.726358+0000","last_peered":"2023-11-28T21:55:38.726358+0000","last_clean":"2023-11-28T21:55:38.726358+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:55:38.726358+0000","last_undegraded":"2023-11-28T21:55:38.726358+0000","last_fullsized":"2023-11-28T21:55:38.726358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:41:24.855115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.218364582,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"51'155","reported_seq":213,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.226609+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:56:08.226609+0000","last_peered":"2023-11-28T21:56:08.226609+0000","last_clean":"2023-11-28T21:56:08.226609+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:56:08.226609+0000","last_undegraded":"2023-11-28T21:56:08.226609+0000","last_fullsized":"2023-11-28T21:56:08.226609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":107,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"52'1271","reported_seq":3570,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.041533+0000","last_change":"2023-11-28T21:40:42.806545+0000","last_active":"2023-11-28T21:56:35.041533+0000","last_peered":"2023-11-28T21:56:35.041533+0000","last_clean":"2023-11-28T21:56:35.041533+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:56:35.041533+0000","last_undegraded":"2023-11-28T21:56:35.041533+0000","last_fullsized":"2023-11-28T21:56:35.041533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:19:01.956213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29571303700000001,"stat_sum":{"num_bytes":33678171,"num_objects":350,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":350,"num_whiteouts":0,"num_read":218,"num_read_kb":4129,"num_write":1246,"num_write_kb":52865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"50'271","reported_seq":2353,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.276133+0000","last_change":"2023-11-28T21:40:43.257612+0000","last_active":"2023-11-28T21:55:39.276133+0000","last_peered":"2023-11-28T21:55:39.276133+0000","last_clean":"2023-11-28T21:55:39.276133+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:55:39.276133+0000","last_undegraded":"2023-11-28T21:55:39.276133+0000","last_fullsized":"2023-11-28T21:55:39.276133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:04.640806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21362616300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"52'304","reported_seq":397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.361916+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:56:08.361916+0000","last_peered":"2023-11-28T21:56:08.361916+0000","last_clean":"2023-11-28T21:56:08.361916+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:56:08.361916+0000","last_undegraded":"2023-11-28T21:56:08.361916+0000","last_fullsized":"2023-11-28T21:56:08.361916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:28.694184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":67,"num_read_kb":77908,"num_write":271,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"52'1265","reported_seq":3536,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.764756+0000","last_change":"2023-11-28T21:40:42.831006+0000","last_active":"2023-11-28T21:56:34.764756+0000","last_peered":"2023-11-28T21:56:34.764756+0000","last_clean":"2023-11-28T21:56:34.764756+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:56:34.764756+0000","last_undegraded":"2023-11-28T21:56:34.764756+0000","last_fullsized":"2023-11-28T21:56:34.764756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1265,"log_dups_size":0,"ondisk_log_size":1265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:03:39.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30778537500000003,"stat_sum":{"num_bytes":30037804,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":190,"num_read_kb":33,"num_write":1241,"num_write_kb":36631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"50'280","reported_seq":2363,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.726323+0000","last_change":"2023-11-28T21:40:42.748885+0000","last_active":"2023-11-28T21:55:38.726323+0000","last_peered":"2023-11-28T21:55:38.726323+0000","last_clean":"2023-11-28T21:55:38.726323+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:55:38.726323+0000","last_undegraded":"2023-11-28T21:55:38.726323+0000","last_fullsized":"2023-11-28T21:55:38.726323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:04:42.978237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21990301600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"52'747","reported_seq":861,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.900520+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:56:27.900520+0000","last_peered":"2023-11-28T21:56:27.900520+0000","last_clean":"2023-11-28T21:56:27.900520+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:56:27.900520+0000","last_undegraded":"2023-11-28T21:56:27.900520+0000","last_fullsized":"2023-11-28T21:56:27.900520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":90,"num_read_kb":122940,"num_write":681,"num_write_kb":20912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"52'1318","reported_seq":3604,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.710137+0000","last_change":"2023-11-28T21:40:42.828740+0000","last_active":"2023-11-28T21:56:34.710137+0000","last_peered":"2023-11-28T21:56:34.710137+0000","last_clean":"2023-11-28T21:56:34.710137+0000","last_became_active":"2023-11-28T21:35:25.790293+0000","last_became_peered":"2023-11-28T21:35:25.790293+0000","last_unstale":"2023-11-28T21:56:34.710137+0000","last_undegraded":"2023-11-28T21:56:34.710137+0000","last_fullsized":"2023-11-28T21:56:34.710137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:24:35.555501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30978138799999999,"stat_sum":{"num_bytes":36258609,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":208,"num_read_kb":8227,"num_write":1285,"num_write_kb":56637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"50'258","reported_seq":2340,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.689233+0000","last_change":"2023-11-28T21:40:43.068808+0000","last_active":"2023-11-28T21:55:38.689233+0000","last_peered":"2023-11-28T21:55:38.689233+0000","last_clean":"2023-11-28T21:55:38.689233+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:55:38.689233+0000","last_undegraded":"2023-11-28T21:55:38.689233+0000","last_fullsized":"2023-11-28T21:55:38.689233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:14:19.957018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22435063599999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"52'516","reported_seq":607,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.507909+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:56:18.507909+0000","last_peered":"2023-11-28T21:56:18.507909+0000","last_clean":"2023-11-28T21:56:18.507909+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:56:18.507909+0000","last_undegraded":"2023-11-28T21:56:18.507909+0000","last_fullsized":"2023-11-28T21:56:18.507909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:35.266650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":69670,"num_write":485,"num_write_kb":16676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"52'1329","reported_seq":3668,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.860899+0000","last_change":"2023-11-28T21:40:43.071209+0000","last_active":"2023-11-28T21:56:34.860899+0000","last_peered":"2023-11-28T21:56:34.860899+0000","last_clean":"2023-11-28T21:56:34.860899+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:56:34.860899+0000","last_undegraded":"2023-11-28T21:56:34.860899+0000","last_fullsized":"2023-11-28T21:56:34.860899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:12:44.679504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.2340457,"stat_sum":{"num_bytes":46614746,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":259,"num_read_kb":20536,"num_write":1294,"num_write_kb":83351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"50'255","reported_seq":2337,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.277131+0000","last_change":"2023-11-28T21:40:43.257554+0000","last_active":"2023-11-28T21:55:39.277131+0000","last_peered":"2023-11-28T21:55:39.277131+0000","last_clean":"2023-11-28T21:55:39.277131+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:55:39.277131+0000","last_undegraded":"2023-11-28T21:55:39.277131+0000","last_fullsized":"2023-11-28T21:55:39.277131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:00.431244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.228570471,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"52'990","reported_seq":1069,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.352661+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:56:08.352661+0000","last_peered":"2023-11-28T21:56:08.352661+0000","last_clean":"2023-11-28T21:56:08.352661+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:56:08.352661+0000","last_undegraded":"2023-11-28T21:56:08.352661+0000","last_fullsized":"2023-11-28T21:56:08.352661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":90124,"num_write":955,"num_write_kb":20985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"52'1334","reported_seq":3673,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.993668+0000","last_change":"2023-11-28T21:40:42.805595+0000","last_active":"2023-11-28T21:56:34.993668+0000","last_peered":"2023-11-28T21:56:34.993668+0000","last_clean":"2023-11-28T21:56:34.993668+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:56:34.993668+0000","last_undegraded":"2023-11-28T21:56:34.993668+0000","last_fullsized":"2023-11-28T21:56:34.993668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:49:48.909117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29518589499999998,"stat_sum":{"num_bytes":25457761,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":258,"num_read_kb":22214,"num_write":1317,"num_write_kb":78783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"50'290","reported_seq":2365,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:44.894160+0000","last_change":"2023-11-28T21:40:43.091522+0000","last_active":"2023-11-28T21:54:44.894160+0000","last_peered":"2023-11-28T21:54:44.894160+0000","last_clean":"2023-11-28T21:54:44.894160+0000","last_became_active":"2023-11-28T21:35:25.595386+0000","last_became_peered":"2023-11-28T21:35:25.595386+0000","last_unstale":"2023-11-28T21:54:44.894160+0000","last_undegraded":"2023-11-28T21:54:44.894160+0000","last_fullsized":"2023-11-28T21:54:44.894160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:35.597109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.148630489,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"52'109","reported_seq":167,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:25.127696+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:56:25.127696+0000","last_peered":"2023-11-28T21:56:25.127696+0000","last_clean":"2023-11-28T21:56:25.127696+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:56:25.127696+0000","last_undegraded":"2023-11-28T21:56:25.127696+0000","last_fullsized":"2023-11-28T21:56:25.127696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":67,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"52'1427","reported_seq":3771,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.994844+0000","last_change":"2023-11-28T21:40:42.795304+0000","last_active":"2023-11-28T21:56:34.994844+0000","last_peered":"2023-11-28T21:56:34.994844+0000","last_clean":"2023-11-28T21:56:34.994844+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:56:34.994844+0000","last_undegraded":"2023-11-28T21:56:34.994844+0000","last_fullsized":"2023-11-28T21:56:34.994844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:17.021758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28498848100000002,"stat_sum":{"num_bytes":33245588,"num_objects":317,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":263,"num_read_kb":28708,"num_write":1396,"num_write_kb":77612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"52'1309","reported_seq":3639,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.944713+0000","last_change":"2023-11-28T21:40:43.251187+0000","last_active":"2023-11-28T21:56:34.944713+0000","last_peered":"2023-11-28T21:56:34.944713+0000","last_clean":"2023-11-28T21:56:34.944713+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:56:34.944713+0000","last_undegraded":"2023-11-28T21:56:34.944713+0000","last_fullsized":"2023-11-28T21:56:34.944713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1309,"log_dups_size":0,"ondisk_log_size":1309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:05:16.989253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27353859400000002,"stat_sum":{"num_bytes":25285382,"num_objects":331,"num_object_clones":0,"num_object_copies":1324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":248,"num_read_kb":4117,"num_write":1287,"num_write_kb":58337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"50'274","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.681885+0000","last_change":"2023-11-28T21:40:42.876095+0000","last_active":"2023-11-28T21:55:38.681885+0000","last_peered":"2023-11-28T21:55:38.681885+0000","last_clean":"2023-11-28T21:55:38.681885+0000","last_became_active":"2023-11-28T21:35:25.589378+0000","last_became_peered":"2023-11-28T21:35:25.589378+0000","last_unstale":"2023-11-28T21:55:38.681885+0000","last_undegraded":"2023-11-28T21:55:38.681885+0000","last_fullsized":"2023-11-28T21:55:38.681885+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:58:22.995479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246426856,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"51'944","reported_seq":1031,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.709711+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:55:43.709711+0000","last_peered":"2023-11-28T21:55:43.709711+0000","last_clean":"2023-11-28T21:55:43.709711+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:55:43.709711+0000","last_undegraded":"2023-11-28T21:55:43.709711+0000","last_fullsized":"2023-11-28T21:55:43.709711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":897,"num_write_kb":25075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"52'1266","reported_seq":3553,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.472923+0000","last_change":"2023-11-28T21:40:42.918369+0000","last_active":"2023-11-28T21:56:34.472923+0000","last_peered":"2023-11-28T21:56:34.472923+0000","last_clean":"2023-11-28T21:56:34.472923+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:56:34.472923+0000","last_undegraded":"2023-11-28T21:56:34.472923+0000","last_fullsized":"2023-11-28T21:56:34.472923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1266,"log_dups_size":0,"ondisk_log_size":1266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:08:31.226473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.16839808100000001,"stat_sum":{"num_bytes":18660789,"num_objects":319,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":205,"num_read_kb":12320,"num_write":1249,"num_write_kb":51464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"51'304","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.271950+0000","last_change":"2023-11-28T21:40:42.800175+0000","last_active":"2023-11-28T21:55:39.271950+0000","last_peered":"2023-11-28T21:55:39.271950+0000","last_clean":"2023-11-28T21:55:39.271950+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:55:39.271950+0000","last_undegraded":"2023-11-28T21:55:39.271950+0000","last_fullsized":"2023-11-28T21:55:39.271950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:52.149789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28961016299999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"51'85","reported_seq":147,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273808+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:55:39.273808+0000","last_peered":"2023-11-28T21:55:39.273808+0000","last_clean":"2023-11-28T21:55:39.273808+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:55:39.273808+0000","last_undegraded":"2023-11-28T21:55:39.273808+0000","last_fullsized":"2023-11-28T21:55:39.273808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":63,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"52'1071","reported_seq":1162,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:13.995104+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:56:13.995104+0000","last_peered":"2023-11-28T21:56:13.995104+0000","last_clean":"2023-11-28T21:56:13.995104+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:56:13.995104+0000","last_undegraded":"2023-11-28T21:56:13.995104+0000","last_fullsized":"2023-11-28T21:56:13.995104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":146592,"num_write":1020,"num_write_kb":29113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"50'263","reported_seq":2344,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:39.891140+0000","last_change":"2023-11-28T21:40:43.254737+0000","last_active":"2023-11-28T21:54:39.891140+0000","last_peered":"2023-11-28T21:54:39.891140+0000","last_clean":"2023-11-28T21:54:39.891140+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:54:39.891140+0000","last_undegraded":"2023-11-28T21:54:39.891140+0000","last_fullsized":"2023-11-28T21:54:39.891140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:41.531767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26886027800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"52'1324","reported_seq":3622,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.710579+0000","last_change":"2023-11-28T21:40:43.102291+0000","last_active":"2023-11-28T21:56:34.710579+0000","last_peered":"2023-11-28T21:56:34.710579+0000","last_clean":"2023-11-28T21:56:34.710579+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:56:34.710579+0000","last_undegraded":"2023-11-28T21:56:34.710579+0000","last_fullsized":"2023-11-28T21:56:34.710579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:46.549792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.13232148399999999,"stat_sum":{"num_bytes":46545345,"num_objects":341,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":218,"num_read_kb":9401,"num_write":1295,"num_write_kb":74278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"51'467","reported_seq":534,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.709564+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:55:43.709564+0000","last_peered":"2023-11-28T21:55:43.709564+0000","last_clean":"2023-11-28T21:55:43.709564+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:55:43.709564+0000","last_undegraded":"2023-11-28T21:55:43.709564+0000","last_fullsized":"2023-11-28T21:55:43.709564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":48896,"num_write":437,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"50'268","reported_seq":2349,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:39.947213+0000","last_change":"2023-11-28T21:40:43.054453+0000","last_active":"2023-11-28T21:54:39.947213+0000","last_peered":"2023-11-28T21:54:39.947213+0000","last_clean":"2023-11-28T21:54:39.947213+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:54:39.947213+0000","last_undegraded":"2023-11-28T21:54:39.947213+0000","last_fullsized":"2023-11-28T21:54:39.947213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:59.006875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24793844200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"52'1320","reported_seq":3657,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.943074+0000","last_change":"2023-11-28T21:40:42.807134+0000","last_active":"2023-11-28T21:56:34.943074+0000","last_peered":"2023-11-28T21:56:34.943074+0000","last_clean":"2023-11-28T21:56:34.943074+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:56:34.943074+0000","last_undegraded":"2023-11-28T21:56:34.943074+0000","last_fullsized":"2023-11-28T21:56:34.943074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1320,"log_dups_size":0,"ondisk_log_size":1320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:55.847688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29562192300000001,"stat_sum":{"num_bytes":29852008,"num_objects":333,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":257,"num_read_kb":28714,"num_write":1297,"num_write_kb":83698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"51'310","reported_seq":2393,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.116742+0000","last_change":"2023-11-28T21:40:42.917455+0000","last_active":"2023-11-28T21:55:30.116742+0000","last_peered":"2023-11-28T21:55:30.116742+0000","last_clean":"2023-11-28T21:55:30.116742+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:55:30.116742+0000","last_undegraded":"2023-11-28T21:55:30.116742+0000","last_fullsized":"2023-11-28T21:55:30.116742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:29.420871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18187520500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"50'313","reported_seq":2397,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.028477+0000","last_change":"2023-11-28T21:40:43.100474+0000","last_active":"2023-11-28T21:55:30.028477+0000","last_peered":"2023-11-28T21:55:30.028477+0000","last_clean":"2023-11-28T21:55:30.028477+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:55:30.028477+0000","last_undegraded":"2023-11-28T21:55:30.028477+0000","last_fullsized":"2023-11-28T21:55:30.028477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:44:47.346946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14391041299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"50'291","reported_seq":2375,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.201786+0000","last_change":"2023-11-28T21:40:42.831609+0000","last_active":"2023-11-28T21:55:30.201786+0000","last_peered":"2023-11-28T21:55:30.201786+0000","last_clean":"2023-11-28T21:55:30.201786+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:55:30.201786+0000","last_undegraded":"2023-11-28T21:55:30.201786+0000","last_fullsized":"2023-11-28T21:55:30.201786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:49:50.710370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30621145,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"50'274","reported_seq":2357,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:35.057693+0000","last_change":"2023-11-28T21:40:43.255460+0000","last_active":"2023-11-28T21:55:35.057693+0000","last_peered":"2023-11-28T21:55:35.057693+0000","last_clean":"2023-11-28T21:55:35.057693+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:55:35.057693+0000","last_undegraded":"2023-11-28T21:55:35.057693+0000","last_fullsized":"2023-11-28T21:55:35.057693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:35:27.179438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26967866099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"50'262","reported_seq":2345,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.274127+0000","last_change":"2023-11-28T21:40:42.801894+0000","last_active":"2023-11-28T21:55:39.274127+0000","last_peered":"2023-11-28T21:55:39.274127+0000","last_clean":"2023-11-28T21:55:39.274127+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:55:39.274127+0000","last_undegraded":"2023-11-28T21:55:39.274127+0000","last_fullsized":"2023-11-28T21:55:39.274127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:46:20.949106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29058787400000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"50'280","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.682440+0000","last_change":"2023-11-28T21:40:42.886237+0000","last_active":"2023-11-28T21:55:38.682440+0000","last_peered":"2023-11-28T21:55:38.682440+0000","last_clean":"2023-11-28T21:55:38.682440+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:55:38.682440+0000","last_undegraded":"2023-11-28T21:55:38.682440+0000","last_fullsized":"2023-11-28T21:55:38.682440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:33.044587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246741036,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"50'305","reported_seq":2388,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.281023+0000","last_change":"2023-11-28T21:40:42.916517+0000","last_active":"2023-11-28T21:55:39.281023+0000","last_peered":"2023-11-28T21:55:39.281023+0000","last_clean":"2023-11-28T21:55:39.281023+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:55:39.281023+0000","last_undegraded":"2023-11-28T21:55:39.281023+0000","last_fullsized":"2023-11-28T21:55:39.281023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:26.983521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18459727300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"50'272","reported_seq":2355,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273021+0000","last_change":"2023-11-28T21:40:42.803865+0000","last_active":"2023-11-28T21:55:39.273021+0000","last_peered":"2023-11-28T21:55:39.273021+0000","last_clean":"2023-11-28T21:55:39.273021+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:55:39.273021+0000","last_undegraded":"2023-11-28T21:55:39.273021+0000","last_fullsized":"2023-11-28T21:55:39.273021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:06:41.883126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29272911699999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"50'274","reported_seq":2358,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.691065+0000","last_change":"2023-11-28T21:40:42.823937+0000","last_active":"2023-11-28T21:55:38.691065+0000","last_peered":"2023-11-28T21:55:38.691065+0000","last_clean":"2023-11-28T21:55:38.691065+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:55:38.691065+0000","last_undegraded":"2023-11-28T21:55:38.691065+0000","last_fullsized":"2023-11-28T21:55:38.691065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:18:46.016564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30579284099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"50'260","reported_seq":2342,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.047770+0000","last_change":"2023-11-28T21:40:42.799326+0000","last_active":"2023-11-28T21:54:55.047770+0000","last_peered":"2023-11-28T21:54:55.047770+0000","last_clean":"2023-11-28T21:54:55.047770+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:54:55.047770+0000","last_undegraded":"2023-11-28T21:54:55.047770+0000","last_fullsized":"2023-11-28T21:54:55.047770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:13.718530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28793565599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"50'276","reported_seq":2359,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.194766+0000","last_change":"2023-11-28T21:40:42.831259+0000","last_active":"2023-11-28T21:54:55.194766+0000","last_peered":"2023-11-28T21:54:55.194766+0000","last_clean":"2023-11-28T21:54:55.194766+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:54:55.194766+0000","last_undegraded":"2023-11-28T21:54:55.194766+0000","last_fullsized":"2023-11-28T21:54:55.194766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:47:17.850510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30636868099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"50'277","reported_seq":2359,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.052784+0000","last_change":"2023-11-28T21:40:43.253941+0000","last_active":"2023-11-28T21:54:55.052784+0000","last_peered":"2023-11-28T21:54:55.052784+0000","last_clean":"2023-11-28T21:54:55.052784+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:54:55.052784+0000","last_undegraded":"2023-11-28T21:54:55.052784+0000","last_fullsized":"2023-11-28T21:54:55.052784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:50:38.526480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26835999100000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"50'280","reported_seq":2362,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.110539+0000","last_change":"2023-11-28T21:40:42.910839+0000","last_active":"2023-11-28T21:54:55.110539+0000","last_peered":"2023-11-28T21:54:55.110539+0000","last_clean":"2023-11-28T21:54:55.110539+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:54:55.110539+0000","last_undegraded":"2023-11-28T21:54:55.110539+0000","last_fullsized":"2023-11-28T21:54:55.110539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:09:31.418117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.194469013,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"50'281","reported_seq":2364,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.116526+0000","last_change":"2023-11-28T21:40:42.749349+0000","last_active":"2023-11-28T21:54:55.116526+0000","last_peered":"2023-11-28T21:54:55.116526+0000","last_clean":"2023-11-28T21:54:55.116526+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:54:55.116526+0000","last_undegraded":"2023-11-28T21:54:55.116526+0000","last_fullsized":"2023-11-28T21:54:55.116526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:20.196676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21943309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"50'253","reported_seq":2336,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273320+0000","last_change":"2023-11-28T21:40:43.092423+0000","last_active":"2023-11-28T21:55:39.273320+0000","last_peered":"2023-11-28T21:55:39.273320+0000","last_clean":"2023-11-28T21:55:39.273320+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:55:39.273320+0000","last_undegraded":"2023-11-28T21:55:39.273320+0000","last_fullsized":"2023-11-28T21:55:39.273320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:33:34.736900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14941532399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"50'295","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.693133+0000","last_change":"2023-11-28T21:40:42.827330+0000","last_active":"2023-11-28T21:55:38.693133+0000","last_peered":"2023-11-28T21:55:38.693133+0000","last_clean":"2023-11-28T21:55:38.693133+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:55:38.693133+0000","last_undegraded":"2023-11-28T21:55:38.693133+0000","last_fullsized":"2023-11-28T21:55:38.693133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:49.288912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30374592,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"50'286","reported_seq":2369,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.690628+0000","last_change":"2023-11-28T21:40:42.820428+0000","last_active":"2023-11-28T21:55:38.690628+0000","last_peered":"2023-11-28T21:55:38.690628+0000","last_clean":"2023-11-28T21:55:38.690628+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:55:38.690628+0000","last_undegraded":"2023-11-28T21:55:38.690628+0000","last_fullsized":"2023-11-28T21:55:38.690628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:59.333901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30238551699999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"50'298","reported_seq":2381,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.682596+0000","last_change":"2023-11-28T21:40:42.876664+0000","last_active":"2023-11-28T21:55:38.682596+0000","last_peered":"2023-11-28T21:55:38.682596+0000","last_clean":"2023-11-28T21:55:38.682596+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:55:38.682596+0000","last_undegraded":"2023-11-28T21:55:38.682596+0000","last_fullsized":"2023-11-28T21:55:38.682596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:02:12.323355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246777987,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"50'284","reported_seq":2366,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.275704+0000","last_change":"2023-11-28T21:40:43.257791+0000","last_active":"2023-11-28T21:55:39.275704+0000","last_peered":"2023-11-28T21:55:39.275704+0000","last_clean":"2023-11-28T21:55:39.275704+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:55:39.275704+0000","last_undegraded":"2023-11-28T21:55:39.275704+0000","last_fullsized":"2023-11-28T21:55:39.275704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:26:58.247393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21399896700000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"50'254","reported_seq":2336,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.272944+0000","last_change":"2023-11-28T21:40:42.798449+0000","last_active":"2023-11-28T21:55:39.272944+0000","last_peered":"2023-11-28T21:55:39.272944+0000","last_clean":"2023-11-28T21:55:39.272944+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:55:39.272944+0000","last_undegraded":"2023-11-28T21:55:39.272944+0000","last_fullsized":"2023-11-28T21:55:39.272944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:01:12.073001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.287656776,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"50'245","reported_seq":2327,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.689195+0000","last_change":"2023-11-28T21:40:43.071081+0000","last_active":"2023-11-28T21:55:38.689195+0000","last_peered":"2023-11-28T21:55:38.689195+0000","last_clean":"2023-11-28T21:55:38.689195+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:55:38.689195+0000","last_undegraded":"2023-11-28T21:55:38.689195+0000","last_fullsized":"2023-11-28T21:55:38.689195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:01:10.753163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.222804104,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"50'240","reported_seq":2322,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.280173+0000","last_change":"2023-11-28T21:40:42.914448+0000","last_active":"2023-11-28T21:55:39.280173+0000","last_peered":"2023-11-28T21:55:39.280173+0000","last_clean":"2023-11-28T21:55:39.280173+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:55:39.280173+0000","last_undegraded":"2023-11-28T21:55:39.280173+0000","last_fullsized":"2023-11-28T21:55:39.280173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:33:37.008462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19768582900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"50'263","reported_seq":2345,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.280285+0000","last_change":"2023-11-28T21:40:42.916248+0000","last_active":"2023-11-28T21:55:39.280285+0000","last_peered":"2023-11-28T21:55:39.280285+0000","last_clean":"2023-11-28T21:55:39.280285+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:55:39.280285+0000","last_undegraded":"2023-11-28T21:55:39.280285+0000","last_fullsized":"2023-11-28T21:55:39.280285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:17.635492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18434383600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"50'296","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.727303+0000","last_change":"2023-11-28T21:40:42.754000+0000","last_active":"2023-11-28T21:55:38.727303+0000","last_peered":"2023-11-28T21:55:38.727303+0000","last_clean":"2023-11-28T21:55:38.727303+0000","last_became_active":"2023-11-28T21:35:25.598839+0000","last_became_peered":"2023-11-28T21:35:25.598839+0000","last_unstale":"2023-11-28T21:55:38.727303+0000","last_undegraded":"2023-11-28T21:55:38.727303+0000","last_fullsized":"2023-11-28T21:55:38.727303+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:51.550099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22244017299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"50'275","reported_seq":2356,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:39.912883+0000","last_change":"2023-11-28T21:40:42.796731+0000","last_active":"2023-11-28T21:54:39.912883+0000","last_peered":"2023-11-28T21:54:39.912883+0000","last_clean":"2023-11-28T21:54:39.912883+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:54:39.912883+0000","last_undegraded":"2023-11-28T21:54:39.912883+0000","last_fullsized":"2023-11-28T21:54:39.912883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:50.215245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28637254899999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"50'278","reported_seq":2360,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.112630+0000","last_change":"2023-11-28T21:40:42.745933+0000","last_active":"2023-11-28T21:54:40.112630+0000","last_peered":"2023-11-28T21:54:40.112630+0000","last_clean":"2023-11-28T21:54:40.112630+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:54:40.112630+0000","last_undegraded":"2023-11-28T21:54:40.112630+0000","last_fullsized":"2023-11-28T21:54:40.112630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:52.720487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.219142752,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"50'266","reported_seq":2348,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147553+0000","last_change":"2023-11-28T21:40:42.831458+0000","last_active":"2023-11-28T21:54:40.147553+0000","last_peered":"2023-11-28T21:54:40.147553+0000","last_clean":"2023-11-28T21:54:40.147553+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:54:40.147553+0000","last_undegraded":"2023-11-28T21:54:40.147553+0000","last_fullsized":"2023-11-28T21:54:40.147553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:33:21.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.307113203,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"50'282","reported_seq":2364,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.112675+0000","last_change":"2023-11-28T21:40:42.736971+0000","last_active":"2023-11-28T21:54:40.112675+0000","last_peered":"2023-11-28T21:54:40.112675+0000","last_clean":"2023-11-28T21:54:40.112675+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:54:40.112675+0000","last_undegraded":"2023-11-28T21:54:40.112675+0000","last_fullsized":"2023-11-28T21:54:40.112675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:33:12.599313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21561430200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"51'275","reported_seq":2359,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147650+0000","last_change":"2023-11-28T21:40:42.829941+0000","last_active":"2023-11-28T21:54:40.147650+0000","last_peered":"2023-11-28T21:54:40.147650+0000","last_clean":"2023-11-28T21:54:40.147650+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:54:40.147650+0000","last_undegraded":"2023-11-28T21:54:40.147650+0000","last_fullsized":"2023-11-28T21:54:40.147650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:26:40.360501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30486287499999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"50'270","reported_seq":2353,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.056979+0000","last_change":"2023-11-28T21:40:43.239342+0000","last_active":"2023-11-28T21:55:30.056979+0000","last_peered":"2023-11-28T21:55:30.056979+0000","last_clean":"2023-11-28T21:55:30.056979+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:55:30.056979+0000","last_undegraded":"2023-11-28T21:55:30.056979+0000","last_fullsized":"2023-11-28T21:55:30.056979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:08.206168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26474323500000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"50'278","reported_seq":2362,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:29.999520+0000","last_change":"2023-11-28T21:40:42.885846+0000","last_active":"2023-11-28T21:55:29.999520+0000","last_peered":"2023-11-28T21:55:29.999520+0000","last_clean":"2023-11-28T21:55:29.999520+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:55:29.999520+0000","last_undegraded":"2023-11-28T21:55:29.999520+0000","last_fullsized":"2023-11-28T21:55:29.999520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:19:35.299155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24685295800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"50'271","reported_seq":2355,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.124004+0000","last_change":"2023-11-28T21:40:42.753421+0000","last_active":"2023-11-28T21:55:30.124004+0000","last_peered":"2023-11-28T21:55:30.124004+0000","last_clean":"2023-11-28T21:55:30.124004+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:55:30.124004+0000","last_undegraded":"2023-11-28T21:55:30.124004+0000","last_fullsized":"2023-11-28T21:55:30.124004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:06:35.683601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22257470400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":927863280,"num_objects":10339,"num_object_clones":0,"num_object_copies":41356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10339,"num_whiteouts":0,"num_read":7542,"num_read_kb":441308,"num_write":41595,"num_write_kb":1955409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1822175232,"data_stored":1974226944,"data_compressed":124276003,"data_compressed_allocated":140005376,"data_compressed_original":292057088,"omap_allocated":0,"internal_metadata":0},"log_size":42424,"ondisk_log_size":42424,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17712,"ondisk_log_size":17712,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":351660858,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":207,"num_whiteouts":0,"num_read":1486,"num_read_kb":1965797,"num_write":14766,"num_write_kb":396301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192471040,"data_stored":703321716,"data_compressed":93812456,"data_compressed_allocated":121569280,"data_compressed_original":632577942,"omap_allocated":0,"internal_metadata":0},"log_size":15799,"ondisk_log_size":15799,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575118,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342780,"kb_used_data":213044,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94029060,"statfs":{"total":96636764160,"available":96285757440,"internally_reserved":0,"allocated":218157056,"data_stored":304451419,"data_compressed":25898147,"data_compressed_allocated":31465472,"data_compressed_original":118067200,"omap_allocated":1590,"internal_metadata":132839882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,10,6,4,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.97799999999999998},"min":{"1min":0.51400000000000001,"5min":0.41399999999999998,"15min":0.35299999999999998},"max":{"1min":1.232,"5min":5.056,"15min":47.475000000000001},"last":3.3719999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.82299999999999995},"min":{"1min":0.54200000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.145,"5min":4.069,"15min":5.3310000000000004},"last":1.8260000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.98999999999999999,"15min":1.099},"min":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":2.5630000000000002,"15min":47.529000000000003},"last":1.79},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.083,"15min":1.008},"min":{"1min":0.64300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.171,"5min":17.460000000000001,"15min":17.460000000000001},"last":5.734}]},{"osd":2,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.86599999999999999,"15min":0.91200000000000003},"min":{"1min":0.58399999999999996,"5min":0.504,"15min":0.47299999999999998},"max":{"1min":1.0660000000000001,"5min":2.9359999999999999,"15min":5.5140000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0229999999999999,"15min":1.022},"min":{"1min":0.70199999999999996,"5min":0.60499999999999998,"15min":0.498},"max":{"1min":1.248,"5min":2.972,"15min":5.5979999999999999},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1719999999999999,"15min":1.179},"min":{"1min":0.63800000000000001,"5min":0.57299999999999995,"15min":0.44600000000000001},"max":{"1min":1.196,"5min":16.113,"15min":47.57},"last":7.3760000000000003},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.171,"15min":1.0649999999999999},"min":{"1min":0.73599999999999999,"5min":0.65400000000000003,"15min":0.53800000000000003},"max":{"1min":1.2749999999999999,"5min":11.148999999999999,"15min":12.273},"last":7.4269999999999996}]},{"osd":4,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.1180000000000001,"15min":1.054},"min":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.35499999999999998},"max":{"1min":1.1990000000000001,"5min":17.542000000000002,"15min":17.542000000000002},"last":5.6859999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.0489999999999999,"15min":1.196},"min":{"1min":0.84699999999999998,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.363,"5min":1.5329999999999999,"15min":49.106000000000002},"last":0.95799999999999996}]},{"osd":5,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.1220000000000001,"15min":1.2909999999999999},"min":{"1min":0.80600000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.345,"5min":6.6420000000000003,"15min":64.855999999999995},"last":2.0110000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.1679999999999999,"15min":1.1970000000000001},"min":{"1min":0.73499999999999999,"5min":0.71399999999999997,"15min":0.56399999999999995},"max":{"1min":1.2689999999999999,"5min":11.327,"15min":16.753},"last":5.843}]},{"osd":6,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.194,"15min":1.1639999999999999},"min":{"1min":0.88600000000000001,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":1.1930000000000001,"5min":11.641999999999999,"15min":11.641999999999999},"last":5.7679999999999998},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1399999999999999,"15min":1.153},"min":{"1min":0.97999999999999998,"5min":0.61499999999999999,"15min":0.502},"max":{"1min":1.4199999999999999,"5min":1.474,"15min":7.4909999999999997},"last":5.4210000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575116,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":380904,"kb_used_data":255328,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":93990936,"statfs":{"total":96636764160,"available":96246718464,"internally_reserved":0,"allocated":261455872,"data_stored":340022166,"data_compressed":25161614,"data_compressed_allocated":30015488,"data_compressed_original":108883915,"omap_allocated":1594,"internal_metadata":128580038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":2.9159999999999999,"5min":1.248,"15min":1.0109999999999999},"min":{"1min":0.51500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":40.999000000000002,"5min":40.999000000000002,"15min":40.999000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.05,"15min":1.036},"min":{"1min":0.58299999999999996,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":14.132999999999999,"5min":14.132999999999999,"15min":24.399999999999999},"last":1.1000000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.89500000000000002,"15min":0.88700000000000001},"min":{"1min":0.71999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2769999999999999,"5min":2.2400000000000002,"15min":11.196},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.90800000000000003,"15min":0.95899999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.35799999999999998},"max":{"1min":1.1759999999999999,"5min":6.1550000000000002,"15min":7.9130000000000003},"last":1.0669999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.89500000000000002,"15min":0.94799999999999995},"min":{"1min":0.66300000000000003,"5min":0.502,"15min":0.39100000000000001},"max":{"1min":3.855,"5min":3.855,"15min":21.986999999999998},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.92800000000000005,"15min":0.99099999999999999},"min":{"1min":0.621,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2829999999999999,"5min":1.419,"15min":8.6170000000000009},"last":0.72099999999999997}]},{"osd":3,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.94799999999999995,"15min":0.96299999999999997},"min":{"1min":0.57099999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.8439999999999999,"5min":3.2679999999999998,"15min":11.239000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0069999999999999,"15min":1.087},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.7959999999999998,"5min":3.29,"15min":24.745000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91800000000000004,"15min":1.014},"min":{"1min":0.71599999999999997,"5min":0.32300000000000001,"15min":0.30299999999999999},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":16.015000000000001},"last":1.135},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.044,"15min":1.079},"min":{"1min":0.69499999999999995,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":13.193},"last":1.2130000000000001}]},{"osd":5,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.98099999999999998,"15min":1.109},"min":{"1min":0.65000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":2.069,"5min":2.069,"15min":25.006},"last":1.25},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0629999999999999,"15min":1.1419999999999999},"min":{"1min":0.46700000000000003,"5min":0.33800000000000002,"15min":0.33000000000000002},"max":{"1min":1.405,"5min":1.4790000000000001,"15min":7.843},"last":1.1579999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.139},"min":{"1min":0.79200000000000004,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":1.1950000000000001,"5min":1.9179999999999999,"15min":7.883},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.177,"15min":1.276},"min":{"1min":0.93300000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.3169999999999999,"5min":1.9610000000000001,"15min":27.942},"last":1.478}]}]},{"osd":5,"up_from":13,"seq":55834575116,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":366316,"kb_used_data":240740,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":94005524,"statfs":{"total":96636764160,"available":96261656576,"internally_reserved":0,"allocated":246517760,"data_stored":342867931,"data_compressed":30547202,"data_compressed_allocated":36597760,"data_compressed_original":133251072,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,3,1,0,0,0,50],"upper_bound":1024},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.99399999999999999,"15min":0.89800000000000002},"min":{"1min":0.47399999999999998,"5min":0.46500000000000002,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":15.005000000000001,"15min":15.005000000000001},"last":3.3980000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88800000000000001},"min":{"1min":0.501,"5min":0.47799999999999998,"15min":0.46700000000000003},"max":{"1min":1.2290000000000001,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":3.419}]},{"osd":1,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81100000000000005,"15min":0.83999999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.48199999999999998},"max":{"1min":0.89800000000000002,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":3.806},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.88900000000000001,"15min":0.91300000000000003},"min":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.496},"max":{"1min":1.0609999999999999,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":7.4930000000000003}]},{"osd":2,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.016,"15min":0.97399999999999998},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.105,"5min":14.775,"15min":14.775},"last":1.131},{"interface":"front","average":{"1min":1.016,"5min":1.1040000000000001,"15min":0.94599999999999995},"min":{"1min":0.72899999999999998,"5min":0.63800000000000001,"15min":0.57499999999999996},"max":{"1min":1.1919999999999999,"5min":14.901999999999999,"15min":14.901999999999999},"last":1.25}]},{"osd":3,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.034,"15min":0.98899999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.54400000000000004},"max":{"1min":1.2529999999999999,"5min":15.154,"15min":15.154},"last":1.347},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.98999999999999999,"15min":1.0109999999999999},"min":{"1min":0.63900000000000001,"5min":0.60799999999999998,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":3.4289999999999998,"15min":5.5170000000000003},"last":1.216}]},{"osd":4,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.526,"15min":1.181},"min":{"1min":0.64600000000000002,"5min":0.59299999999999997,"15min":0.51400000000000001},"max":{"1min":1.292,"5min":61.020000000000003,"15min":61.020000000000003},"last":2.98},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.143,"15min":1.0920000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2490000000000001,"5min":15.085000000000001,"15min":15.085000000000001},"last":8.6029999999999998}]},{"osd":6,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.139,"15min":1.123},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.56699999999999995},"max":{"1min":1.5469999999999999,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":1.151},{"interface":"front","average":{"1min":1.216,"5min":1.194,"15min":1.1699999999999999},"min":{"1min":1.052,"5min":0.72499999999999998,"15min":0.48899999999999999},"max":{"1min":1.6799999999999999,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.1830000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1759999999999999,"15min":1.1639999999999999},"min":{"1min":0.94099999999999995,"5min":0.75600000000000001,"15min":0.56899999999999995},"max":{"1min":1.4219999999999999,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.054,"15min":1.0189999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.33200000000000002},"max":{"1min":1.085,"5min":6.2869999999999999,"15min":6.2869999999999999},"last":1.6779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575116,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":358356,"kb_used_data":236936,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":94013484,"statfs":{"total":96636764160,"available":96269807616,"internally_reserved":0,"allocated":242622464,"data_stored":338272538,"data_compressed":27596131,"data_compressed_allocated":33427456,"data_compressed_original":129380299,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,0,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":60,"apply_latency_ms":60,"commit_latency_ns":60000000,"apply_latency_ns":60000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.155,"15min":0.97399999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0449999999999999,"5min":43.139000000000003,"15min":43.139000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84399999999999997,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.39900000000000002},"max":{"1min":1.3049999999999999,"5min":4.444,"15min":17.446000000000002},"last":3.9809999999999999}]},{"osd":1,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.98799999999999999,"15min":1.054},"min":{"1min":0.64900000000000002,"5min":0.48199999999999998,"15min":0.44},"max":{"1min":1.2849999999999999,"5min":11.557,"15min":21.763999999999999},"last":2.3780000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0229999999999999,"15min":1.4790000000000001},"min":{"1min":0.65700000000000003,"5min":0.53800000000000003,"15min":0.46500000000000002},"max":{"1min":1.452,"5min":5.3479999999999999,"15min":78.649000000000001},"last":2.4329999999999998}]},{"osd":2,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":0.96499999999999997,"15min":1.0529999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.880000000000001,"5min":10.880000000000001,"15min":28.341000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.073,"15min":1.548},"min":{"1min":0.61699999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"max":{"1min":2.069,"5min":11.614000000000001,"15min":94.998999999999995},"last":1.002}]},{"osd":3,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.208,"15min":1.103},"min":{"1min":0.86299999999999999,"5min":0.67500000000000004,"15min":0.51300000000000001},"max":{"1min":9.984,"5min":9.984,"15min":13.202999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.0609999999999999,"15min":1.056},"min":{"1min":0.69299999999999995,"5min":0.54800000000000004,"15min":0.45600000000000002},"max":{"1min":9.952,"5min":9.952,"15min":16.277999999999999},"last":2.4900000000000002}]},{"osd":5,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.0049999999999999,"15min":1.1259999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.28199999999999997},"max":{"1min":1.385,"5min":3.4409999999999998,"15min":17.033000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.3799999999999999,"15min":1.3},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.34000000000000002},"max":{"1min":1.1759999999999999,"5min":38.939,"15min":38.939},"last":0.47299999999999998}]},{"osd":6,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.046,"15min":1.1699999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.37},"max":{"1min":1.4710000000000001,"5min":4.476,"15min":17.085000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.026,"5min":1.4850000000000001,"15min":1.3380000000000001},"min":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.39000000000000001},"max":{"1min":1.631,"5min":43.420000000000002,"15min":43.420000000000002},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.121,"15min":1.6160000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.309},"max":{"1min":1.5269999999999999,"5min":3.2109999999999999,"15min":77.638999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.198,"5min":1.24,"15min":1.175},"min":{"1min":0.90200000000000002,"5min":0.75900000000000001,"15min":0.51500000000000001},"max":{"1min":1.798,"5min":3.613,"15min":5.6200000000000001},"last":0.61699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575117,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457756,"kb_used_data":304404,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":93914084,"statfs":{"total":96636764160,"available":96168022016,"internally_reserved":0,"allocated":311709696,"data_stored":408700527,"data_compressed":36249891,"data_compressed_allocated":42946560,"data_compressed_original":140238848,"omap_allocated":1592,"internal_metadata":157022664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.89500000000000002,"15min":0.70999999999999996},"min":{"1min":0.246,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.67900000000000005,"5min":36.585000000000001,"15min":36.585000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.89500000000000002,"15min":0.67800000000000005},"min":{"1min":0.379,"5min":0.34300000000000003,"15min":0.33100000000000002},"max":{"1min":0.68400000000000005,"5min":30.065000000000001,"15min":30.065000000000001},"last":0.46500000000000002}]},{"osd":1,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.59999999999999998},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.26300000000000001},"max":{"1min":0.876,"5min":4.8099999999999996,"15min":10.449},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.61799999999999999,"15min":0.67400000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.98599999999999999,"5min":4.8520000000000003,"15min":10.481999999999999},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59099999999999997,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.27900000000000003},"max":{"1min":0.70299999999999996,"5min":1.25,"15min":10.369},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.56799999999999995,"15min":0.69599999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78600000000000003,"5min":1.1240000000000001,"15min":10.406000000000001},"last":0.40400000000000003}]},{"osd":4,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91700000000000004,"15min":1.361},"min":{"1min":0.71799999999999997,"5min":0.50700000000000001,"15min":0.376},"max":{"1min":1.0129999999999999,"5min":4.2119999999999997,"15min":110.39700000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.147,"5min":7.2030000000000003,"15min":108.777},"last":0.83699999999999997}]},{"osd":5,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.86199999999999999,"15min":0.92600000000000005},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.39900000000000002},"max":{"1min":0.98799999999999999,"5min":7.2679999999999998,"15min":17.513000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.873,"15min":0.92900000000000005},"min":{"1min":0.621,"5min":0.621,"15min":0.52300000000000002},"max":{"1min":0.998,"5min":1.7070000000000001,"15min":10.516999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82399999999999995,"15min":1.4019999999999999},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.365,"15min":138.91300000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.875,"15min":0.94499999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.48299999999999998},"max":{"1min":1.0980000000000001,"5min":1.766,"15min":10.302},"last":0.88600000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.91900000000000004,"15min":0.95799999999999996},"min":{"1min":0.63100000000000001,"5min":0.60999999999999999,"15min":0.45200000000000001},"max":{"1min":1.1990000000000001,"5min":7.3040000000000003,"15min":17.263000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.97799999999999998,"15min":1.034},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.49299999999999999},"max":{"1min":1.0289999999999999,"5min":7.3339999999999996,"15min":28.613},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575115,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":285968,"kb_used_data":174280,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94085872,"statfs":{"total":96636764160,"available":96343932928,"internally_reserved":0,"allocated":178462720,"data_stored":223929989,"data_compressed":15805945,"data_compressed_allocated":18817024,"data_compressed_original":64589824,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,0,0,0,0,26],"upper_bound":1024},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.623,"15min":0.57999999999999996},"min":{"1min":0.29199999999999998,"5min":0.26800000000000002,"15min":0.23599999999999999},"max":{"1min":0.58299999999999996,"5min":13.599,"15min":13.599},"last":1.6679999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.60699999999999998},"min":{"1min":0.34100000000000003,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":8.0630000000000006,"15min":8.0630000000000006},"last":7.1159999999999997}]},{"osd":1,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.53300000000000003,"15min":0.57499999999999996},"min":{"1min":0.33400000000000002,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.61799999999999999,"5min":3.0760000000000001,"15min":9.3260000000000005},"last":7.8970000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.70699999999999996,"15min":0.67000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":5.6660000000000004,"15min":7.4779999999999998},"last":10.657}]},{"osd":3,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.95199999999999996,"15min":0.79200000000000004},"min":{"1min":0.42099999999999999,"5min":0.32300000000000001,"15min":0.307},"max":{"1min":0.878,"5min":29.123999999999999,"15min":29.123999999999999},"last":7.1639999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.63100000000000001,"15min":0.59699999999999998},"min":{"1min":0.40000000000000002,"5min":0.312,"15min":0.28000000000000003},"max":{"1min":0.754,"5min":5.5410000000000004,"15min":7.4390000000000001},"last":8.8780000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.94099999999999995,"15min":0.92300000000000004},"min":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.49099999999999999},"max":{"1min":1.0489999999999999,"5min":5.6289999999999996,"15min":7.3630000000000004},"last":2.5630000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.87,"15min":0.879},"min":{"1min":0.60599999999999998,"5min":0.51200000000000001,"15min":0.41399999999999998},"max":{"1min":0.98599999999999999,"5min":5.5490000000000004,"15min":9.2289999999999992},"last":1.518}]},{"osd":5,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.85499999999999998,"15min":0.84299999999999997},"min":{"1min":0.622,"5min":0.501,"15min":0.501},"max":{"1min":1.01,"5min":3.3610000000000002,"15min":3.3610000000000002},"last":6.875},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.0109999999999999,"15min":0.93700000000000006},"min":{"1min":0.67300000000000004,"5min":0.59399999999999997,"15min":0.48399999999999999},"max":{"1min":1.266,"5min":6.343,"15min":7.2649999999999997},"last":8.7110000000000003}]},{"osd":6,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.0489999999999999,"15min":0.91800000000000004},"min":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":1.044,"5min":18.959,"15min":18.959},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.1339999999999999,"15min":0.97499999999999998},"min":{"1min":0.68300000000000005,"5min":0.622,"15min":0.52200000000000002},"max":{"1min":1.0700000000000001,"5min":21.957999999999998,"15min":21.957999999999998},"last":8.8420000000000005}]},{"osd":7,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89700000000000002,"15min":0.92100000000000004},"min":{"1min":0.68400000000000005,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":0.999,"5min":3.5299999999999998,"15min":9.2799999999999994},"last":8.7949999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.93300000000000005,"15min":0.90900000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.57099999999999995},"max":{"1min":1.042,"5min":2.6379999999999999,"15min":3.4470000000000001},"last":8.7569999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575115,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472320,"kb_used_data":314808,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93899520,"statfs":{"total":96636764160,"available":96153108480,"internally_reserved":0,"allocated":322363392,"data_stored":416685636,"data_compressed":32711569,"data_compressed_allocated":39284736,"data_compressed_original":133918720,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":0.75700000000000001,"15min":0.69599999999999995},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.246},"max":{"1min":10.096,"5min":12.311999999999999,"15min":14.538},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.623,"15min":0.65100000000000002},"min":{"1min":0.39000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.4119999999999999,"5min":8.2479999999999993,"15min":14.618},"last":0.57199999999999995}]},{"osd":2,"last update":"Tue Nov 28 21:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.66100000000000003,"15min":0.63100000000000001},"min":{"1min":0.40799999999999997,"5min":0.34899999999999998,"15min":0.31900000000000001},"max":{"1min":1.7150000000000001,"5min":5.1689999999999996,"15min":8.4749999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":4.7030000000000003,"15min":8.5150000000000006},"last":0.40600000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.75900000000000001,"15min":0.84699999999999998},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.27600000000000002},"max":{"1min":3.028,"5min":13.385,"15min":43.536999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.84699999999999998},"min":{"1min":0.501,"5min":0.443,"15min":0.375},"max":{"1min":5.6050000000000004,"5min":8.2899999999999991,"15min":18.308},"last":0.67900000000000005}]},{"osd":4,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":0.997,"15min":0.95699999999999996},"min":{"1min":0.63500000000000001,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":9.9849999999999994,"5min":9.9849999999999994,"15min":9.9849999999999994},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.4670000000000001,"5min":0.97899999999999998,"15min":1.079},"min":{"1min":0.61699999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":10.287000000000001,"5min":10.287000000000001,"15min":43.497999999999998},"last":0.69899999999999995}]},{"osd":5,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.93100000000000005,"15min":0.91600000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":5.5350000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.016,"15min":0.99399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.448,"5min":3.448,"15min":5.7409999999999997},"last":0.92600000000000005}]},{"osd":6,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.0449999999999999,"15min":1.0109999999999999},"min":{"1min":0.69899999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":10.023,"5min":10.023,"15min":10.385},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.05,"15min":1.2470000000000001},"min":{"1min":0.77700000000000002,"5min":0.65100000000000002,"15min":0.55200000000000005},"max":{"1min":3.7480000000000002,"5min":5.9349999999999996,"15min":43.448999999999998},"last":0.86299999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0469999999999999,"15min":1.024},"min":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":5.4109999999999996,"5min":5.9779999999999998,"15min":7.96},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.516,"5min":1.1899999999999999,"15min":1.1639999999999999},"min":{"1min":0.66500000000000004,"5min":0.55800000000000005,"15min":0.51500000000000001},"max":{"1min":10.063000000000001,"5min":15.672000000000001,"15min":42.473999999999997},"last":0.66100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575116,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":344096,"kb_used_data":214360,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94027744,"statfs":{"total":96636764160,"available":96284409856,"internally_reserved":0,"allocated":219504640,"data_stored":281144114,"data_compressed":19744060,"data_compressed_allocated":23732224,"data_compressed_original":85671936,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"min":{"1min":0.29499999999999998,"5min":0.27800000000000002,"15min":0.215},"max":{"1min":1.581,"5min":1.958,"15min":3.3929999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.60199999999999998,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.27700000000000002,"15min":0.248},"max":{"1min":4.5739999999999998,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.64900000000000002,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.4489999999999998,"5min":5.4489999999999998,"15min":5.8559999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":3.0840000000000001,"15min":5.4210000000000003},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.77700000000000002,"15min":0.623},"min":{"1min":0.30599999999999999,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":5.0019999999999998,"5min":16.187999999999999,"15min":16.187999999999999},"last":0.495},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.69199999999999995},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.5410000000000004,"5min":15.004,"15min":15.004},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":0.93999999999999995,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.42899999999999999},"max":{"1min":4.9119999999999999,"5min":5.2069999999999999,"15min":5.734},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.873,"15min":0.90500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0940000000000001,"5min":3.2160000000000002,"15min":5.3730000000000002},"last":0.753}]},{"osd":5,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.91900000000000004,"15min":0.84299999999999997},"min":{"1min":0.67300000000000004,"5min":0.59899999999999998,"15min":0.40699999999999997},"max":{"1min":4.4939999999999998,"5min":4.4939999999999998,"15min":4.4939999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.88500000000000001},"min":{"1min":0.65000000000000002,"5min":0.60799999999999998,"15min":0.50900000000000001},"max":{"1min":1.7350000000000001,"5min":4.7910000000000004,"15min":4.7910000000000004},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.01,"15min":0.89900000000000002},"min":{"1min":0.65500000000000003,"5min":0.60999999999999999,"15min":0.48699999999999999},"max":{"1min":4.8630000000000004,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.63},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.88100000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.55100000000000005},"max":{"1min":1.667,"5min":3.1339999999999999,"15min":3.4710000000000001},"last":0.90900000000000003}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.90800000000000003},"min":{"1min":0.627,"5min":0.627,"15min":0.48999999999999999},"max":{"1min":1.6950000000000001,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.95299999999999996,"15min":0.89600000000000002},"min":{"1min":0.68799999999999994,"5min":0.63,"15min":0.55700000000000005},"max":{"1min":1.627,"5min":3.1819999999999999,"15min":3.806},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16297984,"data_stored":62915220,"data_compressed":8677157,"data_compressed_allocated":11337728,"data_compressed_original":57970688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28188672,"data_stored":97891275,"data_compressed":12584510,"data_compressed_allocated":16609280,"data_compressed_original":86339584,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16445440,"data_stored":52440076,"data_compressed":6685552,"data_compressed_allocated":8757248,"data_compressed_original":44773376,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26685440,"data_stored":98577398,"data_compressed":13459583,"data_compressed_allocated":17428480,"data_compressed_original":89337856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":27463680,"data_stored":104859297,"data_compressed":13833556,"data_compressed_allocated":17862656,"data_compressed_original":95277003,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26279936,"data_stored":100664674,"data_compressed":13681157,"data_compressed_allocated":17580032,"data_compressed_original":91983872,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23711744,"data_stored":88082717,"data_compressed":11602354,"data_compressed_allocated":14856192,"data_compressed_original":79245259,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":27398144,"data_stored":97891059,"data_compressed":13288587,"data_compressed_allocated":17137664,"data_compressed_original":87650304,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":206610432,"data_stored":222244864,"data_compressed":12071885,"data_compressed_allocated":13537280,"data_compressed_original":29171712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294195200,"data_stored":319320064,"data_compressed":20251866,"data_compressed_allocated":22855680,"data_compressed_original":47980544,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164610048,"data_stored":175325184,"data_compressed":9845077,"data_compressed_allocated":10899456,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":288477184,"data_stored":316018688,"data_compressed":23971937,"data_compressed_allocated":27017216,"data_compressed_original":54558720,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":217907200,"data_stored":237305856,"data_compressed":14188788,"data_compressed_allocated":16142336,"data_compressed_original":35540992,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":220975104,"data_stored":243601408,"data_compressed":17073954,"data_compressed_allocated":19296256,"data_compressed_original":41922560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":237195264,"data_stored":251670528,"data_compressed":13556545,"data_compressed_allocated":15155200,"data_compressed_original":29630464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":192204800,"data_stored":208740352,"data_compressed":13315951,"data_compressed_allocated":15101952,"data_compressed_original":31637504,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:56:36.672 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:56:36.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc939e6640 1 Processor -- start 2023-11-28T21:56:36.831 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc939e6640 1 -- start start 2023-11-28T21:56:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc939e6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc8c158420 0x7fbc8c06d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc939e6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc8c0c38f0 0x7fbc8c0c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc939e6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8c0c3460 con 0x7fbc8c15e010 2023-11-28T21:56:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc939e6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8c158800 con 0x7fbc8c0c38f0 2023-11-28T21:56:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc939e6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c162610 con 0x7fbc8c158420 2023-11-28T21:56:36.833 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc90f81640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbc8c0c38f0 0x7fbc8c0c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc91782640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc8c158420 0x7fbc8c06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc91f83640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fbc8c15e010 0x7fbc8c15e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:41902/0 (socket says 172.21.15.12:41902) 2023-11-28T21:56:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc91f83640 1 -- 172.21.15.12:0/1517661649 learned_addr learned my addr 172.21.15.12:0/1517661649 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:36.834 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc90f81640 1 -- 172.21.15.12:0/1517661649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc8c158420 msgr2=0x7fbc8c06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc90f81640 1 --2- 172.21.15.12:0/1517661649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc8c158420 0x7fbc8c06d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc90f81640 1 -- 172.21.15.12:0/1517661649 >> v1:172.21.15.144:6789/0 conn(0x7fbc8c15e010 legacy=0x7fbc8c15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:36.835 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.828+0000 7fbc90f81640 1 -- 172.21.15.12:0/1517661649 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c0c2b70 con 0x7fbc8c0c38f0 2023-11-28T21:56:36.836 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.832+0000 7fbc90f81640 1 --2- 172.21.15.12:0/1517661649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c0c38f0 0x7fbc8c0c3d60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0097c0 tx=0x7fbc7c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59cc5ff70a30851b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:36.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.832+0000 7fbc83fff640 1 -- 172.21.15.12:0/1517661649 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0ab600 con 0x7fbc8c0c38f0 2023-11-28T21:56:36.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.832+0000 7fbc83fff640 1 -- 172.21.15.12:0/1517661649 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0a7050 con 0x7fbc8c0c38f0 2023-11-28T21:56:36.837 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.832+0000 7fbc83fff640 1 -- 172.21.15.12:0/1517661649 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0ab980 con 0x7fbc8c0c38f0 2023-11-28T21:56:36.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.832+0000 7fbc939e6640 1 -- 172.21.15.12:0/1517661649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c0c38f0 msgr2=0x7fbc8c0c3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:36.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.832+0000 7fbc939e6640 1 --2- 172.21.15.12:0/1517661649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c0c38f0 0x7fbc8c0c3d60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0097c0 tx=0x7fbc7c01c480 comp rx=0 tx=0).stop 2023-11-28T21:56:36.838 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.832+0000 7fbc939e6640 1 -- 172.21.15.12:0/1517661649 shutdown_connections 2023-11-28T21:56:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.832+0000 7fbc939e6640 1 --2- 172.21.15.12:0/1517661649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c0c38f0 0x7fbc8c0c3d60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.832+0000 7fbc939e6640 1 --2- 172.21.15.12:0/1517661649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc8c158420 0x7fbc8c06d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.832+0000 7fbc939e6640 1 -- 172.21.15.12:0/1517661649 >> 172.21.15.12:0/1517661649 conn(0x7fbc8c066360 msgr2=0x7fbc8c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 -- 172.21.15.12:0/1517661649 shutdown_connections 2023-11-28T21:56:36.839 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 -- 172.21.15.12:0/1517661649 wait complete. 2023-11-28T21:56:36.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 Processor -- start 2023-11-28T21:56:36.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 -- start start 2023-11-28T21:56:36.841 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc8c0c38f0 0x7fbc8c14ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc8c158420 0x7fbc8c0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c15e010 0x7fbc8c0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c158b90 con 0x7fbc8c158420 2023-11-28T21:56:36.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c157470 con 0x7fbc8c15e010 2023-11-28T21:56:36.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0c96c0 con 0x7fbc8c0c38f0 2023-11-28T21:56:36.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc90f81640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc8c158420 0x7fbc8c0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.842 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc90f81640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc8c158420 0x7fbc8c0ca6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48664/0 (socket says 172.21.15.12:48664) 2023-11-28T21:56:36.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc90f81640 1 -- 172.21.15.12:0/1641707842 learned_addr learned my addr 172.21.15.12:0/1641707842 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:36.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc91f83640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c15e010 0x7fbc8c0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc91782640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc8c0c38f0 0x7fbc8c14ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc90f81640 1 -- 172.21.15.12:0/1641707842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc8c0c38f0 msgr2=0x7fbc8c14ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:36.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc90f81640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc8c0c38f0 0x7fbc8c14ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc90f81640 1 -- 172.21.15.12:0/1641707842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c15e010 msgr2=0x7fbc8c0cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:36.843 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc90f81640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c15e010 0x7fbc8c0cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:36.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc90f81640 1 -- 172.21.15.12:0/1641707842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c009420 con 0x7fbc8c158420 2023-11-28T21:56:36.844 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc90f81640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc8c158420 0x7fbc8c0ca6c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0ae040 tx=0x7fbc7c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:36.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc81ffb640 1 -- 172.21.15.12:0/1641707842 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0b4430 con 0x7fbc8c158420 2023-11-28T21:56:36.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc81ffb640 1 -- 172.21.15.12:0/1641707842 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0b4590 con 0x7fbc8c158420 2023-11-28T21:56:36.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc81ffb640 1 -- 172.21.15.12:0/1641707842 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0b4940 con 0x7fbc8c158420 2023-11-28T21:56:36.845 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.836+0000 7fbc939e6640 1 -- 172.21.15.12:0/1641707842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8c0c9920 con 0x7fbc8c158420 2023-11-28T21:56:36.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.840+0000 7fbc939e6640 1 -- 172.21.15.12:0/1641707842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8c0c7220 con 0x7fbc8c158420 2023-11-28T21:56:36.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.840+0000 7fbc939e6640 1 -- 172.21.15.12:0/1641707842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8c074f60 con 0x7fbc8c158420 2023-11-28T21:56:36.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.840+0000 7fbc81ffb640 1 -- 172.21.15.12:0/1641707842 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fbc7c0b2040 con 0x7fbc8c158420 2023-11-28T21:56:36.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.840+0000 7fbc81ffb640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc70066470 0x7fbc70068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:36.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.840+0000 7fbc81ffb640 1 -- 172.21.15.12:0/1641707842 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fbc7c11e2e0 con 0x7fbc8c158420 2023-11-28T21:56:36.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.840+0000 7fbc91782640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc70066470 0x7fbc70068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:36.846 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.840+0000 7fbc91782640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc70066470 0x7fbc70068930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009b60 tx=0x7fbc88009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:36.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:36.844+0000 7fbc81ffb640 1 -- 172.21.15.12:0/1641707842 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc7c0eb180 con 0x7fbc8c158420 2023-11-28T21:56:37.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.000+0000 7fbc939e6640 1 -- 172.21.15.12:0/1641707842 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc8c0c22a0 con 0x7fbc70066470 2023-11-28T21:56:37.013 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.008+0000 7fbc81ffb640 1 -- 172.21.15.12:0/1641707842 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+374122 (secure 0 0 0) 0x7fbc8c0c22a0 con 0x7fbc70066470 2023-11-28T21:56:37.017 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:56:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 -- 172.21.15.12:0/1641707842 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc70066470 msgr2=0x7fbc70068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc70066470 0x7fbc70068930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009b60 tx=0x7fbc88009190 comp rx=0 tx=0).stop 2023-11-28T21:56:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 -- 172.21.15.12:0/1641707842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc8c158420 msgr2=0x7fbc8c0ca6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc8c158420 0x7fbc8c0ca6c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0ae040 tx=0x7fbc7c019220 comp rx=0 tx=0).stop 2023-11-28T21:56:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 -- 172.21.15.12:0/1641707842 shutdown_connections 2023-11-28T21:56:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fbc70066470 0x7fbc70068930 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc8c15e010 0x7fbc8c0cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc8c158420 0x7fbc8c0ca6c0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 --2- 172.21.15.12:0/1641707842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc8c0c38f0 0x7fbc8c14ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 -- 172.21.15.12:0/1641707842 >> 172.21.15.12:0/1641707842 conn(0x7fbc8c066360 msgr2=0x7fbc8c070ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 -- 172.21.15.12:0/1641707842 shutdown_connections 2023-11-28T21:56:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.012+0000 7fbc939e6640 1 -- 172.21.15.12:0/1641707842 wait complete. 2023-11-28T21:56:37.036 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":708,"stamp":"2023-11-28T21:56:36.210879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1280114506,"num_objects":10548,"num_object_clones":0,"num_object_copies":41774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10548,"num_whiteouts":0,"num_read":9144,"num_read_kb":2407205,"num_write":56520,"num_write_kb":2354126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76052,"ondisk_log_size":76052,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3008496,"kb_used_data":1953900,"kb_used_omap":12,"kb_used_meta":1054515,"kb_avail":751966224,"statfs":{"total":773094113280,"available":770013413376,"internally_reserved":0,"allocated":2000793600,"data_stored":2656074320,"data_compressed":213714559,"data_compressed_allocated":256286720,"data_compressed_original":914001814,"omap_allocated":12751,"internal_metadata":1079823921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,1,8,13,7,4,0,1,78],"upper_bound":1024},"perf_stat":{"commit_latency_ms":267,"apply_latency_ms":267,"commit_latency_ns":267000000,"apply_latency_ns":267000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26378372,"num_objects":1446,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1557,"num_write_kb":26881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004723"},"pg_stats":[{"pgid":"2.1f","version":"51'341","reported_seq":427,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.057024+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:55:30.057024+0000","last_peered":"2023-11-28T21:55:30.057024+0000","last_clean":"2023-11-28T21:55:30.057024+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:55:30.057024+0000","last_undegraded":"2023-11-28T21:55:30.057024+0000","last_fullsized":"2023-11-28T21:55:30.057024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:46.167511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":61536,"num_write":310,"num_write_kb":8499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"50'268","reported_seq":2351,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:44.913651+0000","last_change":"2023-11-28T21:40:42.800615+0000","last_active":"2023-11-28T21:54:44.913651+0000","last_peered":"2023-11-28T21:54:44.913651+0000","last_clean":"2023-11-28T21:54:44.913651+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:54:44.913651+0000","last_undegraded":"2023-11-28T21:54:44.913651+0000","last_fullsized":"2023-11-28T21:54:44.913651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:37.956537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28950874199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"52'1318","reported_seq":3628,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.030721+0000","last_change":"2023-11-28T21:40:42.803511+0000","last_active":"2023-11-28T21:56:35.030721+0000","last_peered":"2023-11-28T21:56:35.030721+0000","last_clean":"2023-11-28T21:56:35.030721+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:56:35.030721+0000","last_undegraded":"2023-11-28T21:56:35.030721+0000","last_fullsized":"2023-11-28T21:56:35.030721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:31:55.934555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29329539100000002,"stat_sum":{"num_bytes":30193112,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":229,"num_read_kb":12319,"num_write":1291,"num_write_kb":57664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"52'465","reported_seq":539,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:26.898568+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:56:26.898568+0000","last_peered":"2023-11-28T21:56:26.898568+0000","last_clean":"2023-11-28T21:56:26.898568+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:56:26.898568+0000","last_undegraded":"2023-11-28T21:56:26.898568+0000","last_fullsized":"2023-11-28T21:56:26.898568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":81920,"num_write":420,"num_write_kb":20770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"50'251","reported_seq":2334,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.057010+0000","last_change":"2023-11-28T21:40:43.257406+0000","last_active":"2023-11-28T21:55:30.057010+0000","last_peered":"2023-11-28T21:55:30.057010+0000","last_clean":"2023-11-28T21:55:30.057010+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:55:30.057010+0000","last_undegraded":"2023-11-28T21:55:30.057010+0000","last_fullsized":"2023-11-28T21:55:30.057010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:47.663480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23094383900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"52'1401","reported_seq":3733,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.720695+0000","last_change":"2023-11-28T21:40:43.072665+0000","last_active":"2023-11-28T21:56:34.720695+0000","last_peered":"2023-11-28T21:56:34.720695+0000","last_clean":"2023-11-28T21:56:34.720695+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:56:34.720695+0000","last_undegraded":"2023-11-28T21:56:34.720695+0000","last_fullsized":"2023-11-28T21:56:34.720695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:41:20.225443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21613474099999999,"stat_sum":{"num_bytes":11729902,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":251,"num_read_kb":24615,"num_write":1375,"num_write_kb":48688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"52'761","reported_seq":854,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.356193+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:56:34.356193+0000","last_peered":"2023-11-28T21:56:34.356193+0000","last_clean":"2023-11-28T21:56:34.356193+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:56:34.356193+0000","last_undegraded":"2023-11-28T21:56:34.356193+0000","last_fullsized":"2023-11-28T21:56:34.356193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":59,"num_read_kb":131099,"num_write":714,"num_write_kb":20835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"50'280","reported_seq":2362,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147679+0000","last_change":"2023-11-28T21:40:42.830278+0000","last_active":"2023-11-28T21:54:40.147679+0000","last_peered":"2023-11-28T21:54:40.147679+0000","last_clean":"2023-11-28T21:54:40.147679+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:54:40.147679+0000","last_undegraded":"2023-11-28T21:54:40.147679+0000","last_fullsized":"2023-11-28T21:54:40.147679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:24.137854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30458173900000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"52'1324","reported_seq":3630,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.861331+0000","last_change":"2023-11-28T21:40:43.070433+0000","last_active":"2023-11-28T21:56:34.861331+0000","last_peered":"2023-11-28T21:56:34.861331+0000","last_clean":"2023-11-28T21:56:34.861331+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:56:34.861331+0000","last_undegraded":"2023-11-28T21:56:34.861331+0000","last_fullsized":"2023-11-28T21:56:34.861331+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:00.120053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23712909900000001,"stat_sum":{"num_bytes":32661416,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":226,"num_read_kb":7690,"num_write":1300,"num_write_kb":56471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"52'95","reported_seq":165,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.876669+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:56:27.876669+0000","last_peered":"2023-11-28T21:56:27.876669+0000","last_clean":"2023-11-28T21:56:27.876669+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:56:27.876669+0000","last_undegraded":"2023-11-28T21:56:27.876669+0000","last_fullsized":"2023-11-28T21:56:27.876669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":36891,"num_write":50,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"50'274","reported_seq":2356,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147734+0000","last_change":"2023-11-28T21:40:42.827917+0000","last_active":"2023-11-28T21:54:40.147734+0000","last_peered":"2023-11-28T21:54:40.147734+0000","last_clean":"2023-11-28T21:54:40.147734+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:54:40.147734+0000","last_undegraded":"2023-11-28T21:54:40.147734+0000","last_fullsized":"2023-11-28T21:54:40.147734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:32:57.982365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30416134,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"52'1294","reported_seq":3610,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.796080+0000","last_change":"2023-11-28T21:40:43.055466+0000","last_active":"2023-11-28T21:56:34.796080+0000","last_peered":"2023-11-28T21:56:34.796080+0000","last_clean":"2023-11-28T21:56:34.796080+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:56:34.796080+0000","last_undegraded":"2023-11-28T21:56:34.796080+0000","last_fullsized":"2023-11-28T21:56:34.796080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:33:55.527250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26667665699999998,"stat_sum":{"num_bytes":37281632,"num_objects":312,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":0,"num_read":236,"num_read_kb":13169,"num_write":1275,"num_write_kb":61269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"51'287","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.278456+0000","last_change":"2023-11-28T21:40:42.917342+0000","last_active":"2023-11-28T21:55:39.278456+0000","last_peered":"2023-11-28T21:55:39.278456+0000","last_clean":"2023-11-28T21:55:39.278456+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:55:39.278456+0000","last_undegraded":"2023-11-28T21:55:39.278456+0000","last_fullsized":"2023-11-28T21:55:39.278456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:39.167984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.200606007,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"52'617","reported_seq":711,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.513022+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:56:18.513022+0000","last_peered":"2023-11-28T21:56:18.513022+0000","last_clean":"2023-11-28T21:56:18.513022+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:56:18.513022+0000","last_undegraded":"2023-11-28T21:56:18.513022+0000","last_fullsized":"2023-11-28T21:56:18.513022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":114718,"num_write":571,"num_write_kb":20791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"52'1331","reported_seq":3650,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.683935+0000","last_change":"2023-11-28T21:40:43.071642+0000","last_active":"2023-11-28T21:56:34.683935+0000","last_peered":"2023-11-28T21:56:34.683935+0000","last_clean":"2023-11-28T21:56:34.683935+0000","last_became_active":"2023-11-28T21:35:25.616978+0000","last_became_peered":"2023-11-28T21:35:25.616978+0000","last_unstale":"2023-11-28T21:56:34.683935+0000","last_undegraded":"2023-11-28T21:56:34.683935+0000","last_fullsized":"2023-11-28T21:56:34.683935+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1331,"log_dups_size":0,"ondisk_log_size":1331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:05:37.470115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23129675,"stat_sum":{"num_bytes":32707084,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":229,"num_read_kb":4127,"num_write":1308,"num_write_kb":57872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"50'285","reported_seq":2367,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.047822+0000","last_change":"2023-11-28T21:40:42.798788+0000","last_active":"2023-11-28T21:54:55.047822+0000","last_peered":"2023-11-28T21:54:55.047822+0000","last_clean":"2023-11-28T21:54:55.047822+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:54:55.047822+0000","last_undegraded":"2023-11-28T21:54:55.047822+0000","last_fullsized":"2023-11-28T21:54:55.047822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:37.662225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28777871399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"52'954","reported_seq":1032,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:25.006236+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:56:25.006236+0000","last_peered":"2023-11-28T21:56:25.006236+0000","last_clean":"2023-11-28T21:56:25.006236+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:56:25.006236+0000","last_undegraded":"2023-11-28T21:56:25.006236+0000","last_fullsized":"2023-11-28T21:56:25.006236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":954,"log_dups_size":0,"ondisk_log_size":954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:53.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":75782,"num_write":932,"num_write_kb":13441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"52'1416","reported_seq":3740,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.956993+0000","last_change":"2023-11-28T21:40:43.070729+0000","last_active":"2023-11-28T21:56:34.956993+0000","last_peered":"2023-11-28T21:56:34.956993+0000","last_clean":"2023-11-28T21:56:34.956993+0000","last_became_active":"2023-11-28T21:35:25.790674+0000","last_became_peered":"2023-11-28T21:35:25.790674+0000","last_unstale":"2023-11-28T21:56:34.956993+0000","last_undegraded":"2023-11-28T21:56:34.956993+0000","last_fullsized":"2023-11-28T21:56:34.956993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1416,"log_dups_size":0,"ondisk_log_size":1416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:12:03.263147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.229623311,"stat_sum":{"num_bytes":17731185,"num_objects":354,"num_object_clones":0,"num_object_copies":1416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":234,"num_read_kb":20007,"num_write":1382,"num_write_kb":71863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"50'296","reported_seq":2379,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.272334+0000","last_change":"2023-11-28T21:40:42.800893+0000","last_active":"2023-11-28T21:55:39.272334+0000","last_peered":"2023-11-28T21:55:39.272334+0000","last_clean":"2023-11-28T21:55:39.272334+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:55:39.272334+0000","last_undegraded":"2023-11-28T21:55:39.272334+0000","last_fullsized":"2023-11-28T21:55:39.272334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:35:30.683746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29027087200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"52'722","reported_seq":790,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:33.926232+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:56:33.926232+0000","last_peered":"2023-11-28T21:56:33.926232+0000","last_clean":"2023-11-28T21:56:33.926232+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:56:33.926232+0000","last_undegraded":"2023-11-28T21:56:33.926232+0000","last_fullsized":"2023-11-28T21:56:33.926232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9810037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":32782,"num_write":677,"num_write_kb":9926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"52'1341","reported_seq":3684,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.913491+0000","last_change":"2023-11-28T21:40:42.890432+0000","last_active":"2023-11-28T21:56:34.913491+0000","last_peered":"2023-11-28T21:56:34.913491+0000","last_clean":"2023-11-28T21:56:34.913491+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:56:34.913491+0000","last_undegraded":"2023-11-28T21:56:34.913491+0000","last_fullsized":"2023-11-28T21:56:34.913491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1341,"log_dups_size":0,"ondisk_log_size":1341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:09:28.392308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24643899999999999,"stat_sum":{"num_bytes":14380393,"num_objects":336,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":262,"num_read_kb":16758,"num_write":1318,"num_write_kb":45879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"50'276","reported_seq":2359,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.687600+0000","last_change":"2023-11-28T21:40:43.071973+0000","last_active":"2023-11-28T21:55:38.687600+0000","last_peered":"2023-11-28T21:55:38.687600+0000","last_clean":"2023-11-28T21:55:38.687600+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:55:38.687600+0000","last_undegraded":"2023-11-28T21:55:38.687600+0000","last_fullsized":"2023-11-28T21:55:38.687600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:34:28.182888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21610210399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"52'341","reported_seq":411,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.377147+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:56:08.377147+0000","last_peered":"2023-11-28T21:56:08.377147+0000","last_clean":"2023-11-28T21:56:08.377147+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:56:08.377147+0000","last_undegraded":"2023-11-28T21:56:08.377147+0000","last_fullsized":"2023-11-28T21:56:08.377147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":49164,"num_write":265,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"52'1262","reported_seq":3549,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.730763+0000","last_change":"2023-11-28T21:40:43.060232+0000","last_active":"2023-11-28T21:56:34.730763+0000","last_peered":"2023-11-28T21:56:34.730763+0000","last_clean":"2023-11-28T21:56:34.730763+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:56:34.730763+0000","last_undegraded":"2023-11-28T21:56:34.730763+0000","last_fullsized":"2023-11-28T21:56:34.730763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:54:07.778886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.244586779,"stat_sum":{"num_bytes":50267474,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":206,"num_read_kb":15432,"num_write":1240,"num_write_kb":82194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"50'281","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.276659+0000","last_change":"2023-11-28T21:40:43.249048+0000","last_active":"2023-11-28T21:55:39.276659+0000","last_peered":"2023-11-28T21:55:39.276659+0000","last_clean":"2023-11-28T21:55:39.276659+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:55:39.276659+0000","last_undegraded":"2023-11-28T21:55:39.276659+0000","last_fullsized":"2023-11-28T21:55:39.276659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:35:39.660816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26728515899999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"52'168","reported_seq":240,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:20.126618+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:56:20.126618+0000","last_peered":"2023-11-28T21:56:20.126618+0000","last_clean":"2023-11-28T21:56:20.126618+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:56:20.126618+0000","last_undegraded":"2023-11-28T21:56:20.126618+0000","last_fullsized":"2023-11-28T21:56:20.126618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":45,"num_read_kb":24615,"num_write":146,"num_write_kb":4216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"52'1368","reported_seq":3700,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.654211+0000","last_change":"2023-11-28T21:40:43.072395+0000","last_active":"2023-11-28T21:56:34.654211+0000","last_peered":"2023-11-28T21:56:34.654211+0000","last_clean":"2023-11-28T21:56:34.654211+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:56:34.654211+0000","last_undegraded":"2023-11-28T21:56:34.654211+0000","last_fullsized":"2023-11-28T21:56:34.654211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:25:44.577393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21672229300000001,"stat_sum":{"num_bytes":16299977,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":250,"num_read_kb":17253,"num_write":1343,"num_write_kb":69509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'249","reported_seq":2332,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273472+0000","last_change":"2023-11-28T21:40:42.801394+0000","last_active":"2023-11-28T21:55:39.273472+0000","last_peered":"2023-11-28T21:55:39.273472+0000","last_clean":"2023-11-28T21:55:39.273472+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:55:39.273472+0000","last_undegraded":"2023-11-28T21:55:39.273472+0000","last_fullsized":"2023-11-28T21:55:39.273472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:59:35.647639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29009278799999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"52'245","reported_seq":314,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.893946+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:56:27.893946+0000","last_peered":"2023-11-28T21:56:27.893946+0000","last_clean":"2023-11-28T21:56:27.893946+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:56:27.893946+0000","last_undegraded":"2023-11-28T21:56:27.893946+0000","last_fullsized":"2023-11-28T21:56:27.893946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:36.968220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":607,"num_write":445,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"52'1386","reported_seq":3700,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.993814+0000","last_change":"2023-11-28T21:40:42.797955+0000","last_active":"2023-11-28T21:56:34.993814+0000","last_peered":"2023-11-28T21:56:34.993814+0000","last_clean":"2023-11-28T21:56:34.993814+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:56:34.993814+0000","last_undegraded":"2023-11-28T21:56:34.993814+0000","last_fullsized":"2023-11-28T21:56:34.993814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1386,"log_dups_size":0,"ondisk_log_size":1386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:27:40.256597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.287791554,"stat_sum":{"num_bytes":19942162,"num_objects":313,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":232,"num_read_kb":16373,"num_write":1352,"num_write_kb":52080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"50'287","reported_seq":2371,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:29.999480+0000","last_change":"2023-11-28T21:40:42.884411+0000","last_active":"2023-11-28T21:55:29.999480+0000","last_peered":"2023-11-28T21:55:29.999480+0000","last_clean":"2023-11-28T21:55:29.999480+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:55:29.999480+0000","last_undegraded":"2023-11-28T21:55:29.999480+0000","last_fullsized":"2023-11-28T21:55:29.999480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:48.590211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24779062600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"52'587","reported_seq":665,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.695016+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:55:43.695016+0000","last_peered":"2023-11-28T21:55:43.695016+0000","last_clean":"2023-11-28T21:55:43.695016+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:55:43.695016+0000","last_undegraded":"2023-11-28T21:55:43.695016+0000","last_fullsized":"2023-11-28T21:55:43.695016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":81538,"num_write":523,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"52'1410","reported_seq":3740,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.895421+0000","last_change":"2023-11-28T21:40:43.255749+0000","last_active":"2023-11-28T21:56:34.895421+0000","last_peered":"2023-11-28T21:56:34.895421+0000","last_clean":"2023-11-28T21:56:34.895421+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:56:34.895421+0000","last_undegraded":"2023-11-28T21:56:34.895421+0000","last_fullsized":"2023-11-28T21:56:34.895421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:38:18.061461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27155133199999998,"stat_sum":{"num_bytes":49110889,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":248,"num_read_kb":24602,"num_write":1394,"num_write_kb":89036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"50'274","reported_seq":2358,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.201955+0000","last_change":"2023-11-28T21:40:42.828660+0000","last_active":"2023-11-28T21:55:30.201955+0000","last_peered":"2023-11-28T21:55:30.201955+0000","last_clean":"2023-11-28T21:55:30.201955+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:55:30.201955+0000","last_undegraded":"2023-11-28T21:55:30.201955+0000","last_fullsized":"2023-11-28T21:55:30.201955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:55.786023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30430776599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"52'164","reported_seq":225,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.682785+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:55:38.682785+0000","last_peered":"2023-11-28T21:55:38.682785+0000","last_clean":"2023-11-28T21:55:38.682785+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:55:38.682785+0000","last_undegraded":"2023-11-28T21:55:38.682785+0000","last_fullsized":"2023-11-28T21:55:38.682785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:08:15.270352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24576,"num_write":146,"num_write_kb":4262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"52'1318","reported_seq":3622,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.840355+0000","last_change":"2023-11-28T21:40:42.887654+0000","last_active":"2023-11-28T21:56:34.840355+0000","last_peered":"2023-11-28T21:56:34.840355+0000","last_clean":"2023-11-28T21:56:34.840355+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:56:34.840355+0000","last_undegraded":"2023-11-28T21:56:34.840355+0000","last_fullsized":"2023-11-28T21:56:34.840355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:38.441656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.25659980999999998,"stat_sum":{"num_bytes":32626940,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":222,"num_read_kb":4128,"num_write":1285,"num_write_kb":60020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"50'286","reported_seq":2369,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.214610+0000","last_change":"2023-11-28T21:40:43.068901+0000","last_active":"2023-11-28T21:55:30.214610+0000","last_peered":"2023-11-28T21:55:30.214610+0000","last_clean":"2023-11-28T21:55:30.214610+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:55:30.214610+0000","last_undegraded":"2023-11-28T21:55:30.214610+0000","last_fullsized":"2023-11-28T21:55:30.214610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:05:04.092055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23142385100000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"52'365","reported_seq":439,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.694515+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:55:38.694515+0000","last_peered":"2023-11-28T21:55:38.694515+0000","last_clean":"2023-11-28T21:55:38.694515+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:55:38.694515+0000","last_undegraded":"2023-11-28T21:55:38.694515+0000","last_fullsized":"2023-11-28T21:55:38.694515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":81920,"num_write":337,"num_write_kb":16655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"52'1263","reported_seq":3555,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.958687+0000","last_change":"2023-11-28T21:40:42.883278+0000","last_active":"2023-11-28T21:56:34.958687+0000","last_peered":"2023-11-28T21:56:34.958687+0000","last_clean":"2023-11-28T21:56:34.958687+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:56:34.958687+0000","last_undegraded":"2023-11-28T21:56:34.958687+0000","last_fullsized":"2023-11-28T21:56:34.958687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:30:54.117448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24685721299999999,"stat_sum":{"num_bytes":16718463,"num_objects":288,"num_object_clones":0,"num_object_copies":1152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":210,"num_read_kb":3302,"num_write":1235,"num_write_kb":35539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"50'283","reported_seq":2367,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:44.891386+0000","last_change":"2023-11-28T21:40:42.890540+0000","last_active":"2023-11-28T21:54:44.891386+0000","last_peered":"2023-11-28T21:54:44.891386+0000","last_clean":"2023-11-28T21:54:44.891386+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:54:44.891386+0000","last_undegraded":"2023-11-28T21:54:44.891386+0000","last_fullsized":"2023-11-28T21:54:44.891386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:51.055896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24023003500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"51'934","reported_seq":1405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.724754+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:55:43.724754+0000","last_peered":"2023-11-28T21:55:43.724754+0000","last_clean":"2023-11-28T21:55:43.724754+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:55:43.724754+0000","last_undegraded":"2023-11-28T21:55:43.724754+0000","last_fullsized":"2023-11-28T21:55:43.724754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18884916,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":417,"num_read_kb":34819,"num_write":869,"num_write_kb":18975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"52'1231","reported_seq":3557,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.911522+0000","last_change":"2023-11-28T21:40:42.888191+0000","last_active":"2023-11-28T21:56:34.911522+0000","last_peered":"2023-11-28T21:56:34.911522+0000","last_clean":"2023-11-28T21:56:34.911522+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:56:34.911522+0000","last_undegraded":"2023-11-28T21:56:34.911522+0000","last_fullsized":"2023-11-28T21:56:34.911522+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:20.371038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24393632200000001,"stat_sum":{"num_bytes":26023878,"num_objects":293,"num_object_clones":0,"num_object_copies":1172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":293,"num_whiteouts":0,"num_read":244,"num_read_kb":12314,"num_write":1202,"num_write_kb":62723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"51'259","reported_seq":2342,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.056922+0000","last_change":"2023-11-28T21:40:43.256849+0000","last_active":"2023-11-28T21:55:30.056922+0000","last_peered":"2023-11-28T21:55:30.056922+0000","last_clean":"2023-11-28T21:55:30.056922+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:55:30.056922+0000","last_undegraded":"2023-11-28T21:55:30.056922+0000","last_fullsized":"2023-11-28T21:55:30.056922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:25.250732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.25105937499999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"51'313","reported_seq":395,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.725792+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:55:38.725792+0000","last_peered":"2023-11-28T21:55:38.725792+0000","last_clean":"2023-11-28T21:55:38.725792+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:55:38.725792+0000","last_undegraded":"2023-11-28T21:55:38.725792+0000","last_fullsized":"2023-11-28T21:55:38.725792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":114688,"num_write":275,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"52'1365","reported_seq":3711,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.880915+0000","last_change":"2023-11-28T21:40:43.252159+0000","last_active":"2023-11-28T21:56:34.880915+0000","last_peered":"2023-11-28T21:56:34.880915+0000","last_clean":"2023-11-28T21:56:34.880915+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:56:34.880915+0000","last_undegraded":"2023-11-28T21:56:34.880915+0000","last_fullsized":"2023-11-28T21:56:34.880915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:58:49.674343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.270607394,"stat_sum":{"num_bytes":19916106,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":264,"num_read_kb":13370,"num_write":1336,"num_write_kb":59202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"50'268","reported_seq":2351,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.051948+0000","last_change":"2023-11-28T21:40:42.802535+0000","last_active":"2023-11-28T21:55:30.051948+0000","last_peered":"2023-11-28T21:55:30.051948+0000","last_clean":"2023-11-28T21:55:30.051948+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:55:30.051948+0000","last_undegraded":"2023-11-28T21:55:30.051948+0000","last_fullsized":"2023-11-28T21:55:30.051948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:02:13.185718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29152715000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"52'311","reported_seq":375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:07.951419+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:56:07.951419+0000","last_peered":"2023-11-28T21:56:07.951419+0000","last_clean":"2023-11-28T21:56:07.951419+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:56:07.951419+0000","last_undegraded":"2023-11-28T21:56:07.951419+0000","last_fullsized":"2023-11-28T21:56:07.951419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":24594,"num_write":297,"num_write_kb":8480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.725972+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:55:38.725972+0000","last_peered":"2023-11-28T21:55:38.725972+0000","last_clean":"2023-11-28T21:55:38.725972+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:55:38.725972+0000","last_undegraded":"2023-11-28T21:55:38.725972+0000","last_fullsized":"2023-11-28T21:55:38.725972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"52'1342","reported_seq":3676,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.779903+0000","last_change":"2023-11-28T21:40:42.914002+0000","last_active":"2023-11-28T21:56:34.779903+0000","last_peered":"2023-11-28T21:56:34.779903+0000","last_clean":"2023-11-28T21:56:34.779903+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:56:34.779903+0000","last_undegraded":"2023-11-28T21:56:34.779903+0000","last_fullsized":"2023-11-28T21:56:34.779903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:47.036263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19732549099999999,"stat_sum":{"num_bytes":29762487,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":252,"num_read_kb":20508,"num_write":1310,"num_write_kb":73242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"50'256","reported_seq":2339,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.116489+0000","last_change":"2023-11-28T21:40:42.748386+0000","last_active":"2023-11-28T21:54:55.116489+0000","last_peered":"2023-11-28T21:54:55.116489+0000","last_clean":"2023-11-28T21:54:55.116489+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:54:55.116489+0000","last_undegraded":"2023-11-28T21:54:55.116489+0000","last_fullsized":"2023-11-28T21:54:55.116489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:59.734407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.219020891,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"52'475","reported_seq":534,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273615+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:55:39.273615+0000","last_peered":"2023-11-28T21:55:39.273615+0000","last_clean":"2023-11-28T21:55:39.273615+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:55:39.273615+0000","last_undegraded":"2023-11-28T21:55:39.273615+0000","last_fullsized":"2023-11-28T21:55:39.273615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":20485,"num_write":447,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"52'1372","reported_seq":3704,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.807900+0000","last_change":"2023-11-28T21:40:42.915914+0000","last_active":"2023-11-28T21:56:34.807900+0000","last_peered":"2023-11-28T21:56:34.807900+0000","last_clean":"2023-11-28T21:56:34.807900+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:56:34.807900+0000","last_undegraded":"2023-11-28T21:56:34.807900+0000","last_fullsized":"2023-11-28T21:56:34.807900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1372,"log_dups_size":0,"ondisk_log_size":1372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:52:07.090824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19929065100000001,"stat_sum":{"num_bytes":26470227,"num_objects":335,"num_object_clones":0,"num_object_copies":1340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":251,"num_read_kb":4134,"num_write":1344,"num_write_kb":47412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"51'313","reported_seq":2396,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.194876+0000","last_change":"2023-11-28T21:40:42.830692+0000","last_active":"2023-11-28T21:54:55.194876+0000","last_peered":"2023-11-28T21:54:55.194876+0000","last_clean":"2023-11-28T21:54:55.194876+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:54:55.194876+0000","last_undegraded":"2023-11-28T21:54:55.194876+0000","last_fullsized":"2023-11-28T21:54:55.194876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:33:58.325515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30458413400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"52'964","reported_seq":1044,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.896525+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:56:27.896525+0000","last_peered":"2023-11-28T21:56:27.896525+0000","last_clean":"2023-11-28T21:56:27.896525+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:56:27.896525+0000","last_undegraded":"2023-11-28T21:56:27.896525+0000","last_fullsized":"2023-11-28T21:56:27.896525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":964,"log_dups_size":0,"ondisk_log_size":964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:48.073446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777286,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":27,"num_read_kb":68346,"num_write":930,"num_write_kb":16947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"52'1319","reported_seq":3665,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.895088+0000","last_change":"2023-11-28T21:40:43.256533+0000","last_active":"2023-11-28T21:56:34.895088+0000","last_peered":"2023-11-28T21:56:34.895088+0000","last_clean":"2023-11-28T21:56:34.895088+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:56:34.895088+0000","last_undegraded":"2023-11-28T21:56:34.895088+0000","last_fullsized":"2023-11-28T21:56:34.895088+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:44.854009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27089875299999999,"stat_sum":{"num_bytes":28004524,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":265,"num_read_kb":16422,"num_write":1294,"num_write_kb":50614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"50'264","reported_seq":2340,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.274191+0000","last_change":"2023-11-28T21:40:43.090122+0000","last_active":"2023-11-28T21:55:39.274191+0000","last_peered":"2023-11-28T21:55:39.274191+0000","last_clean":"2023-11-28T21:55:39.274191+0000","last_became_active":"2023-11-28T21:35:25.594533+0000","last_became_peered":"2023-11-28T21:35:25.594533+0000","last_unstale":"2023-11-28T21:55:39.274191+0000","last_undegraded":"2023-11-28T21:55:39.274191+0000","last_fullsized":"2023-11-28T21:55:39.274191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:03.418589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14880083199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"52'303","reported_seq":378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.839543+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:56:27.839543+0000","last_peered":"2023-11-28T21:56:27.839543+0000","last_clean":"2023-11-28T21:56:27.839543+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:56:27.839543+0000","last_undegraded":"2023-11-28T21:56:27.839543+0000","last_fullsized":"2023-11-28T21:56:27.839543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":53275,"num_write":251,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"52'1307","reported_seq":3591,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.520158+0000","last_change":"2023-11-28T21:40:42.823686+0000","last_active":"2023-11-28T21:56:34.520158+0000","last_peered":"2023-11-28T21:56:34.520158+0000","last_clean":"2023-11-28T21:56:34.520158+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:56:34.520158+0000","last_undegraded":"2023-11-28T21:56:34.520158+0000","last_fullsized":"2023-11-28T21:56:34.520158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1307,"log_dups_size":0,"ondisk_log_size":1307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:34:35.818463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30648515900000001,"stat_sum":{"num_bytes":28110180,"num_objects":295,"num_object_clones":0,"num_object_copies":1180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":0,"num_read":202,"num_read_kb":8227,"num_write":1283,"num_write_kb":51039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"50'267","reported_seq":2350,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.116571+0000","last_change":"2023-11-28T21:40:42.749319+0000","last_active":"2023-11-28T21:54:55.116571+0000","last_peered":"2023-11-28T21:54:55.116571+0000","last_clean":"2023-11-28T21:54:55.116571+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:54:55.116571+0000","last_undegraded":"2023-11-28T21:54:55.116571+0000","last_fullsized":"2023-11-28T21:54:55.116571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:18:31.031890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21877398200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"52'193","reported_seq":266,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.689380+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:55:38.689380+0000","last_peered":"2023-11-28T21:55:38.689380+0000","last_clean":"2023-11-28T21:55:38.689380+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:55:38.689380+0000","last_undegraded":"2023-11-28T21:55:38.689380+0000","last_fullsized":"2023-11-28T21:55:38.689380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":21,"num_read_kb":49161,"num_write":146,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"52'1273","reported_seq":3574,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.716263+0000","last_change":"2023-11-28T21:40:43.072161+0000","last_active":"2023-11-28T21:56:34.716263+0000","last_peered":"2023-11-28T21:56:34.716263+0000","last_clean":"2023-11-28T21:56:34.716263+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:56:34.716263+0000","last_undegraded":"2023-11-28T21:56:34.716263+0000","last_fullsized":"2023-11-28T21:56:34.716263+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:39.182750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.223266511,"stat_sum":{"num_bytes":11523503,"num_objects":308,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":219,"num_read_kb":12316,"num_write":1248,"num_write_kb":38807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"50'308","reported_seq":2391,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.725792+0000","last_change":"2023-11-28T21:40:42.747053+0000","last_active":"2023-11-28T21:55:38.725792+0000","last_peered":"2023-11-28T21:55:38.725792+0000","last_clean":"2023-11-28T21:55:38.725792+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:55:38.725792+0000","last_undegraded":"2023-11-28T21:55:38.725792+0000","last_fullsized":"2023-11-28T21:55:38.725792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:54:52.864875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21978382299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"52'854","reported_seq":964,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.035796+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:56:18.035796+0000","last_peered":"2023-11-28T21:56:18.035796+0000","last_clean":"2023-11-28T21:56:18.035796+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:56:18.035796+0000","last_undegraded":"2023-11-28T21:56:18.035796+0000","last_fullsized":"2023-11-28T21:56:18.035796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":137,"num_read_kb":58709,"num_write":838,"num_write_kb":8642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"52'1274","reported_seq":3606,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.943389+0000","last_change":"2023-11-28T21:40:43.252839+0000","last_active":"2023-11-28T21:56:34.943389+0000","last_peered":"2023-11-28T21:56:34.943389+0000","last_clean":"2023-11-28T21:56:34.943389+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:56:34.943389+0000","last_undegraded":"2023-11-28T21:56:34.943389+0000","last_fullsized":"2023-11-28T21:56:34.943389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:28:08.856108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27152726799999999,"stat_sum":{"num_bytes":36307339,"num_objects":283,"num_object_clones":0,"num_object_copies":1132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":283,"num_whiteouts":0,"num_read":251,"num_read_kb":8046,"num_write":1253,"num_write_kb":60298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"51'273","reported_seq":2355,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.278420+0000","last_change":"2023-11-28T21:40:42.918241+0000","last_active":"2023-11-28T21:55:39.278420+0000","last_peered":"2023-11-28T21:55:39.278420+0000","last_clean":"2023-11-28T21:55:39.278420+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:55:39.278420+0000","last_undegraded":"2023-11-28T21:55:39.278420+0000","last_fullsized":"2023-11-28T21:55:39.278420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:57.732993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18244292200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"52'239","reported_seq":293,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.710890+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:55:43.710890+0000","last_peered":"2023-11-28T21:55:43.710890+0000","last_clean":"2023-11-28T21:55:43.710890+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:55:43.710890+0000","last_undegraded":"2023-11-28T21:55:43.710890+0000","last_fullsized":"2023-11-28T21:55:43.710890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195875,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":194,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"52'1346","reported_seq":3662,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.682315+0000","last_change":"2023-11-28T21:40:42.805322+0000","last_active":"2023-11-28T21:56:34.682315+0000","last_peered":"2023-11-28T21:56:34.682315+0000","last_clean":"2023-11-28T21:56:34.682315+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:56:34.682315+0000","last_undegraded":"2023-11-28T21:56:34.682315+0000","last_fullsized":"2023-11-28T21:56:34.682315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:22:10.703578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29439885100000002,"stat_sum":{"num_bytes":34458204,"num_objects":324,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":324,"num_whiteouts":0,"num_read":235,"num_read_kb":27797,"num_write":1320,"num_write_kb":70332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"50'300","reported_seq":2383,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.726358+0000","last_change":"2023-11-28T21:40:42.740620+0000","last_active":"2023-11-28T21:55:38.726358+0000","last_peered":"2023-11-28T21:55:38.726358+0000","last_clean":"2023-11-28T21:55:38.726358+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:55:38.726358+0000","last_undegraded":"2023-11-28T21:55:38.726358+0000","last_fullsized":"2023-11-28T21:55:38.726358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:41:24.855115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.218364582,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"51'155","reported_seq":213,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.226609+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:56:08.226609+0000","last_peered":"2023-11-28T21:56:08.226609+0000","last_clean":"2023-11-28T21:56:08.226609+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:56:08.226609+0000","last_undegraded":"2023-11-28T21:56:08.226609+0000","last_fullsized":"2023-11-28T21:56:08.226609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":107,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"52'1271","reported_seq":3570,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.041533+0000","last_change":"2023-11-28T21:40:42.806545+0000","last_active":"2023-11-28T21:56:35.041533+0000","last_peered":"2023-11-28T21:56:35.041533+0000","last_clean":"2023-11-28T21:56:35.041533+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:56:35.041533+0000","last_undegraded":"2023-11-28T21:56:35.041533+0000","last_fullsized":"2023-11-28T21:56:35.041533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:19:01.956213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29571303700000001,"stat_sum":{"num_bytes":33678171,"num_objects":350,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":350,"num_whiteouts":0,"num_read":218,"num_read_kb":4129,"num_write":1246,"num_write_kb":52865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"50'271","reported_seq":2353,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.276133+0000","last_change":"2023-11-28T21:40:43.257612+0000","last_active":"2023-11-28T21:55:39.276133+0000","last_peered":"2023-11-28T21:55:39.276133+0000","last_clean":"2023-11-28T21:55:39.276133+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:55:39.276133+0000","last_undegraded":"2023-11-28T21:55:39.276133+0000","last_fullsized":"2023-11-28T21:55:39.276133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:04.640806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21362616300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"52'304","reported_seq":397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.361916+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:56:08.361916+0000","last_peered":"2023-11-28T21:56:08.361916+0000","last_clean":"2023-11-28T21:56:08.361916+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:56:08.361916+0000","last_undegraded":"2023-11-28T21:56:08.361916+0000","last_fullsized":"2023-11-28T21:56:08.361916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:28.694184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":67,"num_read_kb":77908,"num_write":271,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"52'1265","reported_seq":3536,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.764756+0000","last_change":"2023-11-28T21:40:42.831006+0000","last_active":"2023-11-28T21:56:34.764756+0000","last_peered":"2023-11-28T21:56:34.764756+0000","last_clean":"2023-11-28T21:56:34.764756+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:56:34.764756+0000","last_undegraded":"2023-11-28T21:56:34.764756+0000","last_fullsized":"2023-11-28T21:56:34.764756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1265,"log_dups_size":0,"ondisk_log_size":1265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:03:39.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30778537500000003,"stat_sum":{"num_bytes":30037804,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":190,"num_read_kb":33,"num_write":1241,"num_write_kb":36631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"50'280","reported_seq":2363,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.726323+0000","last_change":"2023-11-28T21:40:42.748885+0000","last_active":"2023-11-28T21:55:38.726323+0000","last_peered":"2023-11-28T21:55:38.726323+0000","last_clean":"2023-11-28T21:55:38.726323+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:55:38.726323+0000","last_undegraded":"2023-11-28T21:55:38.726323+0000","last_fullsized":"2023-11-28T21:55:38.726323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:04:42.978237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21990301600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"52'747","reported_seq":861,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.900520+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:56:27.900520+0000","last_peered":"2023-11-28T21:56:27.900520+0000","last_clean":"2023-11-28T21:56:27.900520+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:56:27.900520+0000","last_undegraded":"2023-11-28T21:56:27.900520+0000","last_fullsized":"2023-11-28T21:56:27.900520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":90,"num_read_kb":122940,"num_write":681,"num_write_kb":20912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"52'1318","reported_seq":3604,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.710137+0000","last_change":"2023-11-28T21:40:42.828740+0000","last_active":"2023-11-28T21:56:34.710137+0000","last_peered":"2023-11-28T21:56:34.710137+0000","last_clean":"2023-11-28T21:56:34.710137+0000","last_became_active":"2023-11-28T21:35:25.790293+0000","last_became_peered":"2023-11-28T21:35:25.790293+0000","last_unstale":"2023-11-28T21:56:34.710137+0000","last_undegraded":"2023-11-28T21:56:34.710137+0000","last_fullsized":"2023-11-28T21:56:34.710137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:24:35.555501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30978138799999999,"stat_sum":{"num_bytes":36258609,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":208,"num_read_kb":8227,"num_write":1285,"num_write_kb":56637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"50'258","reported_seq":2340,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.689233+0000","last_change":"2023-11-28T21:40:43.068808+0000","last_active":"2023-11-28T21:55:38.689233+0000","last_peered":"2023-11-28T21:55:38.689233+0000","last_clean":"2023-11-28T21:55:38.689233+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:55:38.689233+0000","last_undegraded":"2023-11-28T21:55:38.689233+0000","last_fullsized":"2023-11-28T21:55:38.689233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:14:19.957018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22435063599999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"52'516","reported_seq":607,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.507909+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:56:18.507909+0000","last_peered":"2023-11-28T21:56:18.507909+0000","last_clean":"2023-11-28T21:56:18.507909+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:56:18.507909+0000","last_undegraded":"2023-11-28T21:56:18.507909+0000","last_fullsized":"2023-11-28T21:56:18.507909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:35.266650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":69670,"num_write":485,"num_write_kb":16676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"52'1329","reported_seq":3668,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.860899+0000","last_change":"2023-11-28T21:40:43.071209+0000","last_active":"2023-11-28T21:56:34.860899+0000","last_peered":"2023-11-28T21:56:34.860899+0000","last_clean":"2023-11-28T21:56:34.860899+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:56:34.860899+0000","last_undegraded":"2023-11-28T21:56:34.860899+0000","last_fullsized":"2023-11-28T21:56:34.860899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:12:44.679504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.2340457,"stat_sum":{"num_bytes":46614746,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":259,"num_read_kb":20536,"num_write":1294,"num_write_kb":83351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"50'255","reported_seq":2337,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.277131+0000","last_change":"2023-11-28T21:40:43.257554+0000","last_active":"2023-11-28T21:55:39.277131+0000","last_peered":"2023-11-28T21:55:39.277131+0000","last_clean":"2023-11-28T21:55:39.277131+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:55:39.277131+0000","last_undegraded":"2023-11-28T21:55:39.277131+0000","last_fullsized":"2023-11-28T21:55:39.277131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:00.431244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.228570471,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"52'990","reported_seq":1069,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.352661+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:56:08.352661+0000","last_peered":"2023-11-28T21:56:08.352661+0000","last_clean":"2023-11-28T21:56:08.352661+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:56:08.352661+0000","last_undegraded":"2023-11-28T21:56:08.352661+0000","last_fullsized":"2023-11-28T21:56:08.352661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":90124,"num_write":955,"num_write_kb":20985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"52'1334","reported_seq":3673,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.993668+0000","last_change":"2023-11-28T21:40:42.805595+0000","last_active":"2023-11-28T21:56:34.993668+0000","last_peered":"2023-11-28T21:56:34.993668+0000","last_clean":"2023-11-28T21:56:34.993668+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:56:34.993668+0000","last_undegraded":"2023-11-28T21:56:34.993668+0000","last_fullsized":"2023-11-28T21:56:34.993668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:49:48.909117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29518589499999998,"stat_sum":{"num_bytes":25457761,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":258,"num_read_kb":22214,"num_write":1317,"num_write_kb":78783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"50'290","reported_seq":2365,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:44.894160+0000","last_change":"2023-11-28T21:40:43.091522+0000","last_active":"2023-11-28T21:54:44.894160+0000","last_peered":"2023-11-28T21:54:44.894160+0000","last_clean":"2023-11-28T21:54:44.894160+0000","last_became_active":"2023-11-28T21:35:25.595386+0000","last_became_peered":"2023-11-28T21:35:25.595386+0000","last_unstale":"2023-11-28T21:54:44.894160+0000","last_undegraded":"2023-11-28T21:54:44.894160+0000","last_fullsized":"2023-11-28T21:54:44.894160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:35.597109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.148630489,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"52'109","reported_seq":167,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:25.127696+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:56:25.127696+0000","last_peered":"2023-11-28T21:56:25.127696+0000","last_clean":"2023-11-28T21:56:25.127696+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:56:25.127696+0000","last_undegraded":"2023-11-28T21:56:25.127696+0000","last_fullsized":"2023-11-28T21:56:25.127696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":67,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"52'1427","reported_seq":3771,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.994844+0000","last_change":"2023-11-28T21:40:42.795304+0000","last_active":"2023-11-28T21:56:34.994844+0000","last_peered":"2023-11-28T21:56:34.994844+0000","last_clean":"2023-11-28T21:56:34.994844+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:56:34.994844+0000","last_undegraded":"2023-11-28T21:56:34.994844+0000","last_fullsized":"2023-11-28T21:56:34.994844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:17.021758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28498848100000002,"stat_sum":{"num_bytes":33245588,"num_objects":317,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":263,"num_read_kb":28708,"num_write":1396,"num_write_kb":77612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"52'1309","reported_seq":3639,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.944713+0000","last_change":"2023-11-28T21:40:43.251187+0000","last_active":"2023-11-28T21:56:34.944713+0000","last_peered":"2023-11-28T21:56:34.944713+0000","last_clean":"2023-11-28T21:56:34.944713+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:56:34.944713+0000","last_undegraded":"2023-11-28T21:56:34.944713+0000","last_fullsized":"2023-11-28T21:56:34.944713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1309,"log_dups_size":0,"ondisk_log_size":1309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:05:16.989253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27353859400000002,"stat_sum":{"num_bytes":25285382,"num_objects":331,"num_object_clones":0,"num_object_copies":1324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":248,"num_read_kb":4117,"num_write":1287,"num_write_kb":58337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"50'274","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.681885+0000","last_change":"2023-11-28T21:40:42.876095+0000","last_active":"2023-11-28T21:55:38.681885+0000","last_peered":"2023-11-28T21:55:38.681885+0000","last_clean":"2023-11-28T21:55:38.681885+0000","last_became_active":"2023-11-28T21:35:25.589378+0000","last_became_peered":"2023-11-28T21:35:25.589378+0000","last_unstale":"2023-11-28T21:55:38.681885+0000","last_undegraded":"2023-11-28T21:55:38.681885+0000","last_fullsized":"2023-11-28T21:55:38.681885+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:58:22.995479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246426856,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"51'944","reported_seq":1031,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.709711+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:55:43.709711+0000","last_peered":"2023-11-28T21:55:43.709711+0000","last_clean":"2023-11-28T21:55:43.709711+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:55:43.709711+0000","last_undegraded":"2023-11-28T21:55:43.709711+0000","last_fullsized":"2023-11-28T21:55:43.709711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":897,"num_write_kb":25075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"52'1266","reported_seq":3553,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.472923+0000","last_change":"2023-11-28T21:40:42.918369+0000","last_active":"2023-11-28T21:56:34.472923+0000","last_peered":"2023-11-28T21:56:34.472923+0000","last_clean":"2023-11-28T21:56:34.472923+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:56:34.472923+0000","last_undegraded":"2023-11-28T21:56:34.472923+0000","last_fullsized":"2023-11-28T21:56:34.472923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1266,"log_dups_size":0,"ondisk_log_size":1266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:08:31.226473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.16839808100000001,"stat_sum":{"num_bytes":18660789,"num_objects":319,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":205,"num_read_kb":12320,"num_write":1249,"num_write_kb":51464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"51'304","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.271950+0000","last_change":"2023-11-28T21:40:42.800175+0000","last_active":"2023-11-28T21:55:39.271950+0000","last_peered":"2023-11-28T21:55:39.271950+0000","last_clean":"2023-11-28T21:55:39.271950+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:55:39.271950+0000","last_undegraded":"2023-11-28T21:55:39.271950+0000","last_fullsized":"2023-11-28T21:55:39.271950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:52.149789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28961016299999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"51'85","reported_seq":147,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273808+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:55:39.273808+0000","last_peered":"2023-11-28T21:55:39.273808+0000","last_clean":"2023-11-28T21:55:39.273808+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:55:39.273808+0000","last_undegraded":"2023-11-28T21:55:39.273808+0000","last_fullsized":"2023-11-28T21:55:39.273808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":63,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"52'1071","reported_seq":1162,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:13.995104+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:56:13.995104+0000","last_peered":"2023-11-28T21:56:13.995104+0000","last_clean":"2023-11-28T21:56:13.995104+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:56:13.995104+0000","last_undegraded":"2023-11-28T21:56:13.995104+0000","last_fullsized":"2023-11-28T21:56:13.995104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":146592,"num_write":1020,"num_write_kb":29113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"50'263","reported_seq":2344,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:39.891140+0000","last_change":"2023-11-28T21:40:43.254737+0000","last_active":"2023-11-28T21:54:39.891140+0000","last_peered":"2023-11-28T21:54:39.891140+0000","last_clean":"2023-11-28T21:54:39.891140+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:54:39.891140+0000","last_undegraded":"2023-11-28T21:54:39.891140+0000","last_fullsized":"2023-11-28T21:54:39.891140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:41.531767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26886027800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"52'1324","reported_seq":3622,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.710579+0000","last_change":"2023-11-28T21:40:43.102291+0000","last_active":"2023-11-28T21:56:34.710579+0000","last_peered":"2023-11-28T21:56:34.710579+0000","last_clean":"2023-11-28T21:56:34.710579+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:56:34.710579+0000","last_undegraded":"2023-11-28T21:56:34.710579+0000","last_fullsized":"2023-11-28T21:56:34.710579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:46.549792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.13232148399999999,"stat_sum":{"num_bytes":46545345,"num_objects":341,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":218,"num_read_kb":9401,"num_write":1295,"num_write_kb":74278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"51'467","reported_seq":534,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.709564+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:55:43.709564+0000","last_peered":"2023-11-28T21:55:43.709564+0000","last_clean":"2023-11-28T21:55:43.709564+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:55:43.709564+0000","last_undegraded":"2023-11-28T21:55:43.709564+0000","last_fullsized":"2023-11-28T21:55:43.709564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":48896,"num_write":437,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"50'268","reported_seq":2349,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:39.947213+0000","last_change":"2023-11-28T21:40:43.054453+0000","last_active":"2023-11-28T21:54:39.947213+0000","last_peered":"2023-11-28T21:54:39.947213+0000","last_clean":"2023-11-28T21:54:39.947213+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:54:39.947213+0000","last_undegraded":"2023-11-28T21:54:39.947213+0000","last_fullsized":"2023-11-28T21:54:39.947213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:59.006875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24793844200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"52'1320","reported_seq":3657,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.943074+0000","last_change":"2023-11-28T21:40:42.807134+0000","last_active":"2023-11-28T21:56:34.943074+0000","last_peered":"2023-11-28T21:56:34.943074+0000","last_clean":"2023-11-28T21:56:34.943074+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:56:34.943074+0000","last_undegraded":"2023-11-28T21:56:34.943074+0000","last_fullsized":"2023-11-28T21:56:34.943074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1320,"log_dups_size":0,"ondisk_log_size":1320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:55.847688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29562192300000001,"stat_sum":{"num_bytes":29852008,"num_objects":333,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":257,"num_read_kb":28714,"num_write":1297,"num_write_kb":83698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"51'310","reported_seq":2393,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.116742+0000","last_change":"2023-11-28T21:40:42.917455+0000","last_active":"2023-11-28T21:55:30.116742+0000","last_peered":"2023-11-28T21:55:30.116742+0000","last_clean":"2023-11-28T21:55:30.116742+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:55:30.116742+0000","last_undegraded":"2023-11-28T21:55:30.116742+0000","last_fullsized":"2023-11-28T21:55:30.116742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:29.420871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18187520500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"50'313","reported_seq":2397,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.028477+0000","last_change":"2023-11-28T21:40:43.100474+0000","last_active":"2023-11-28T21:55:30.028477+0000","last_peered":"2023-11-28T21:55:30.028477+0000","last_clean":"2023-11-28T21:55:30.028477+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:55:30.028477+0000","last_undegraded":"2023-11-28T21:55:30.028477+0000","last_fullsized":"2023-11-28T21:55:30.028477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:44:47.346946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14391041299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"50'291","reported_seq":2375,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.201786+0000","last_change":"2023-11-28T21:40:42.831609+0000","last_active":"2023-11-28T21:55:30.201786+0000","last_peered":"2023-11-28T21:55:30.201786+0000","last_clean":"2023-11-28T21:55:30.201786+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:55:30.201786+0000","last_undegraded":"2023-11-28T21:55:30.201786+0000","last_fullsized":"2023-11-28T21:55:30.201786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:49:50.710370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30621145,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"50'274","reported_seq":2357,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:35.057693+0000","last_change":"2023-11-28T21:40:43.255460+0000","last_active":"2023-11-28T21:55:35.057693+0000","last_peered":"2023-11-28T21:55:35.057693+0000","last_clean":"2023-11-28T21:55:35.057693+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:55:35.057693+0000","last_undegraded":"2023-11-28T21:55:35.057693+0000","last_fullsized":"2023-11-28T21:55:35.057693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:35:27.179438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26967866099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"50'262","reported_seq":2345,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.274127+0000","last_change":"2023-11-28T21:40:42.801894+0000","last_active":"2023-11-28T21:55:39.274127+0000","last_peered":"2023-11-28T21:55:39.274127+0000","last_clean":"2023-11-28T21:55:39.274127+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:55:39.274127+0000","last_undegraded":"2023-11-28T21:55:39.274127+0000","last_fullsized":"2023-11-28T21:55:39.274127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:46:20.949106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29058787400000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"50'280","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.682440+0000","last_change":"2023-11-28T21:40:42.886237+0000","last_active":"2023-11-28T21:55:38.682440+0000","last_peered":"2023-11-28T21:55:38.682440+0000","last_clean":"2023-11-28T21:55:38.682440+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:55:38.682440+0000","last_undegraded":"2023-11-28T21:55:38.682440+0000","last_fullsized":"2023-11-28T21:55:38.682440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:33.044587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246741036,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"50'305","reported_seq":2388,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.281023+0000","last_change":"2023-11-28T21:40:42.916517+0000","last_active":"2023-11-28T21:55:39.281023+0000","last_peered":"2023-11-28T21:55:39.281023+0000","last_clean":"2023-11-28T21:55:39.281023+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:55:39.281023+0000","last_undegraded":"2023-11-28T21:55:39.281023+0000","last_fullsized":"2023-11-28T21:55:39.281023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:26.983521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18459727300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"50'272","reported_seq":2355,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273021+0000","last_change":"2023-11-28T21:40:42.803865+0000","last_active":"2023-11-28T21:55:39.273021+0000","last_peered":"2023-11-28T21:55:39.273021+0000","last_clean":"2023-11-28T21:55:39.273021+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:55:39.273021+0000","last_undegraded":"2023-11-28T21:55:39.273021+0000","last_fullsized":"2023-11-28T21:55:39.273021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:06:41.883126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29272911699999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"50'274","reported_seq":2358,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.691065+0000","last_change":"2023-11-28T21:40:42.823937+0000","last_active":"2023-11-28T21:55:38.691065+0000","last_peered":"2023-11-28T21:55:38.691065+0000","last_clean":"2023-11-28T21:55:38.691065+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:55:38.691065+0000","last_undegraded":"2023-11-28T21:55:38.691065+0000","last_fullsized":"2023-11-28T21:55:38.691065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:18:46.016564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30579284099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"50'260","reported_seq":2342,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.047770+0000","last_change":"2023-11-28T21:40:42.799326+0000","last_active":"2023-11-28T21:54:55.047770+0000","last_peered":"2023-11-28T21:54:55.047770+0000","last_clean":"2023-11-28T21:54:55.047770+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:54:55.047770+0000","last_undegraded":"2023-11-28T21:54:55.047770+0000","last_fullsized":"2023-11-28T21:54:55.047770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:13.718530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28793565599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"50'276","reported_seq":2359,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.194766+0000","last_change":"2023-11-28T21:40:42.831259+0000","last_active":"2023-11-28T21:54:55.194766+0000","last_peered":"2023-11-28T21:54:55.194766+0000","last_clean":"2023-11-28T21:54:55.194766+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:54:55.194766+0000","last_undegraded":"2023-11-28T21:54:55.194766+0000","last_fullsized":"2023-11-28T21:54:55.194766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:47:17.850510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30636868099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"50'277","reported_seq":2359,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.052784+0000","last_change":"2023-11-28T21:40:43.253941+0000","last_active":"2023-11-28T21:54:55.052784+0000","last_peered":"2023-11-28T21:54:55.052784+0000","last_clean":"2023-11-28T21:54:55.052784+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:54:55.052784+0000","last_undegraded":"2023-11-28T21:54:55.052784+0000","last_fullsized":"2023-11-28T21:54:55.052784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:50:38.526480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26835999100000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"50'280","reported_seq":2362,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.110539+0000","last_change":"2023-11-28T21:40:42.910839+0000","last_active":"2023-11-28T21:54:55.110539+0000","last_peered":"2023-11-28T21:54:55.110539+0000","last_clean":"2023-11-28T21:54:55.110539+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:54:55.110539+0000","last_undegraded":"2023-11-28T21:54:55.110539+0000","last_fullsized":"2023-11-28T21:54:55.110539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:09:31.418117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.194469013,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"50'281","reported_seq":2364,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.116526+0000","last_change":"2023-11-28T21:40:42.749349+0000","last_active":"2023-11-28T21:54:55.116526+0000","last_peered":"2023-11-28T21:54:55.116526+0000","last_clean":"2023-11-28T21:54:55.116526+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:54:55.116526+0000","last_undegraded":"2023-11-28T21:54:55.116526+0000","last_fullsized":"2023-11-28T21:54:55.116526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:20.196676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21943309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"50'253","reported_seq":2336,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273320+0000","last_change":"2023-11-28T21:40:43.092423+0000","last_active":"2023-11-28T21:55:39.273320+0000","last_peered":"2023-11-28T21:55:39.273320+0000","last_clean":"2023-11-28T21:55:39.273320+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:55:39.273320+0000","last_undegraded":"2023-11-28T21:55:39.273320+0000","last_fullsized":"2023-11-28T21:55:39.273320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:33:34.736900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14941532399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"50'295","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.693133+0000","last_change":"2023-11-28T21:40:42.827330+0000","last_active":"2023-11-28T21:55:38.693133+0000","last_peered":"2023-11-28T21:55:38.693133+0000","last_clean":"2023-11-28T21:55:38.693133+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:55:38.693133+0000","last_undegraded":"2023-11-28T21:55:38.693133+0000","last_fullsized":"2023-11-28T21:55:38.693133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:49.288912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30374592,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"50'286","reported_seq":2369,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.690628+0000","last_change":"2023-11-28T21:40:42.820428+0000","last_active":"2023-11-28T21:55:38.690628+0000","last_peered":"2023-11-28T21:55:38.690628+0000","last_clean":"2023-11-28T21:55:38.690628+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:55:38.690628+0000","last_undegraded":"2023-11-28T21:55:38.690628+0000","last_fullsized":"2023-11-28T21:55:38.690628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:59.333901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30238551699999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"50'298","reported_seq":2381,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.682596+0000","last_change":"2023-11-28T21:40:42.876664+0000","last_active":"2023-11-28T21:55:38.682596+0000","last_peered":"2023-11-28T21:55:38.682596+0000","last_clean":"2023-11-28T21:55:38.682596+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:55:38.682596+0000","last_undegraded":"2023-11-28T21:55:38.682596+0000","last_fullsized":"2023-11-28T21:55:38.682596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:02:12.323355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246777987,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"50'284","reported_seq":2366,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.275704+0000","last_change":"2023-11-28T21:40:43.257791+0000","last_active":"2023-11-28T21:55:39.275704+0000","last_peered":"2023-11-28T21:55:39.275704+0000","last_clean":"2023-11-28T21:55:39.275704+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:55:39.275704+0000","last_undegraded":"2023-11-28T21:55:39.275704+0000","last_fullsized":"2023-11-28T21:55:39.275704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:26:58.247393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21399896700000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"50'254","reported_seq":2336,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.272944+0000","last_change":"2023-11-28T21:40:42.798449+0000","last_active":"2023-11-28T21:55:39.272944+0000","last_peered":"2023-11-28T21:55:39.272944+0000","last_clean":"2023-11-28T21:55:39.272944+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:55:39.272944+0000","last_undegraded":"2023-11-28T21:55:39.272944+0000","last_fullsized":"2023-11-28T21:55:39.272944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:01:12.073001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.287656776,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"50'245","reported_seq":2327,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.689195+0000","last_change":"2023-11-28T21:40:43.071081+0000","last_active":"2023-11-28T21:55:38.689195+0000","last_peered":"2023-11-28T21:55:38.689195+0000","last_clean":"2023-11-28T21:55:38.689195+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:55:38.689195+0000","last_undegraded":"2023-11-28T21:55:38.689195+0000","last_fullsized":"2023-11-28T21:55:38.689195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:01:10.753163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.222804104,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"50'240","reported_seq":2322,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.280173+0000","last_change":"2023-11-28T21:40:42.914448+0000","last_active":"2023-11-28T21:55:39.280173+0000","last_peered":"2023-11-28T21:55:39.280173+0000","last_clean":"2023-11-28T21:55:39.280173+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:55:39.280173+0000","last_undegraded":"2023-11-28T21:55:39.280173+0000","last_fullsized":"2023-11-28T21:55:39.280173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:33:37.008462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19768582900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"50'263","reported_seq":2345,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.280285+0000","last_change":"2023-11-28T21:40:42.916248+0000","last_active":"2023-11-28T21:55:39.280285+0000","last_peered":"2023-11-28T21:55:39.280285+0000","last_clean":"2023-11-28T21:55:39.280285+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:55:39.280285+0000","last_undegraded":"2023-11-28T21:55:39.280285+0000","last_fullsized":"2023-11-28T21:55:39.280285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:17.635492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18434383600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"50'296","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.727303+0000","last_change":"2023-11-28T21:40:42.754000+0000","last_active":"2023-11-28T21:55:38.727303+0000","last_peered":"2023-11-28T21:55:38.727303+0000","last_clean":"2023-11-28T21:55:38.727303+0000","last_became_active":"2023-11-28T21:35:25.598839+0000","last_became_peered":"2023-11-28T21:35:25.598839+0000","last_unstale":"2023-11-28T21:55:38.727303+0000","last_undegraded":"2023-11-28T21:55:38.727303+0000","last_fullsized":"2023-11-28T21:55:38.727303+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:51.550099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22244017299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"50'275","reported_seq":2356,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:39.912883+0000","last_change":"2023-11-28T21:40:42.796731+0000","last_active":"2023-11-28T21:54:39.912883+0000","last_peered":"2023-11-28T21:54:39.912883+0000","last_clean":"2023-11-28T21:54:39.912883+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:54:39.912883+0000","last_undegraded":"2023-11-28T21:54:39.912883+0000","last_fullsized":"2023-11-28T21:54:39.912883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:50.215245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28637254899999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"50'278","reported_seq":2360,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.112630+0000","last_change":"2023-11-28T21:40:42.745933+0000","last_active":"2023-11-28T21:54:40.112630+0000","last_peered":"2023-11-28T21:54:40.112630+0000","last_clean":"2023-11-28T21:54:40.112630+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:54:40.112630+0000","last_undegraded":"2023-11-28T21:54:40.112630+0000","last_fullsized":"2023-11-28T21:54:40.112630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:52.720487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.219142752,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"50'266","reported_seq":2348,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147553+0000","last_change":"2023-11-28T21:40:42.831458+0000","last_active":"2023-11-28T21:54:40.147553+0000","last_peered":"2023-11-28T21:54:40.147553+0000","last_clean":"2023-11-28T21:54:40.147553+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:54:40.147553+0000","last_undegraded":"2023-11-28T21:54:40.147553+0000","last_fullsized":"2023-11-28T21:54:40.147553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:33:21.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.307113203,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"50'282","reported_seq":2364,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.112675+0000","last_change":"2023-11-28T21:40:42.736971+0000","last_active":"2023-11-28T21:54:40.112675+0000","last_peered":"2023-11-28T21:54:40.112675+0000","last_clean":"2023-11-28T21:54:40.112675+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:54:40.112675+0000","last_undegraded":"2023-11-28T21:54:40.112675+0000","last_fullsized":"2023-11-28T21:54:40.112675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:33:12.599313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21561430200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"51'275","reported_seq":2359,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147650+0000","last_change":"2023-11-28T21:40:42.829941+0000","last_active":"2023-11-28T21:54:40.147650+0000","last_peered":"2023-11-28T21:54:40.147650+0000","last_clean":"2023-11-28T21:54:40.147650+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:54:40.147650+0000","last_undegraded":"2023-11-28T21:54:40.147650+0000","last_fullsized":"2023-11-28T21:54:40.147650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:26:40.360501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30486287499999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"50'270","reported_seq":2353,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.056979+0000","last_change":"2023-11-28T21:40:43.239342+0000","last_active":"2023-11-28T21:55:30.056979+0000","last_peered":"2023-11-28T21:55:30.056979+0000","last_clean":"2023-11-28T21:55:30.056979+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:55:30.056979+0000","last_undegraded":"2023-11-28T21:55:30.056979+0000","last_fullsized":"2023-11-28T21:55:30.056979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:08.206168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26474323500000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"50'278","reported_seq":2362,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:29.999520+0000","last_change":"2023-11-28T21:40:42.885846+0000","last_active":"2023-11-28T21:55:29.999520+0000","last_peered":"2023-11-28T21:55:29.999520+0000","last_clean":"2023-11-28T21:55:29.999520+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:55:29.999520+0000","last_undegraded":"2023-11-28T21:55:29.999520+0000","last_fullsized":"2023-11-28T21:55:29.999520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:19:35.299155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24685295800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"50'271","reported_seq":2355,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.124004+0000","last_change":"2023-11-28T21:40:42.753421+0000","last_active":"2023-11-28T21:55:30.124004+0000","last_peered":"2023-11-28T21:55:30.124004+0000","last_clean":"2023-11-28T21:55:30.124004+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:55:30.124004+0000","last_undegraded":"2023-11-28T21:55:30.124004+0000","last_fullsized":"2023-11-28T21:55:30.124004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:06:35.683601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22257470400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":927863280,"num_objects":10339,"num_object_clones":0,"num_object_copies":41356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10339,"num_whiteouts":0,"num_read":7542,"num_read_kb":441308,"num_write":41595,"num_write_kb":1955409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1822175232,"data_stored":1974226944,"data_compressed":124276003,"data_compressed_allocated":140005376,"data_compressed_original":292057088,"omap_allocated":0,"internal_metadata":0},"log_size":42424,"ondisk_log_size":42424,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17712,"ondisk_log_size":17712,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":351660858,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":207,"num_whiteouts":0,"num_read":1486,"num_read_kb":1965797,"num_write":14766,"num_write_kb":396301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192471040,"data_stored":703321716,"data_compressed":93812456,"data_compressed_allocated":121569280,"data_compressed_original":632577942,"omap_allocated":0,"internal_metadata":0},"log_size":15799,"ondisk_log_size":15799,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575118,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342780,"kb_used_data":213044,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94029060,"statfs":{"total":96636764160,"available":96285757440,"internally_reserved":0,"allocated":218157056,"data_stored":304451419,"data_compressed":25898147,"data_compressed_allocated":31465472,"data_compressed_original":118067200,"omap_allocated":1590,"internal_metadata":132839882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,10,6,4,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.97799999999999998},"min":{"1min":0.51400000000000001,"5min":0.41399999999999998,"15min":0.35299999999999998},"max":{"1min":1.232,"5min":5.056,"15min":47.475000000000001},"last":3.3719999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.82299999999999995},"min":{"1min":0.54200000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.145,"5min":4.069,"15min":5.3310000000000004},"last":1.8260000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.98999999999999999,"15min":1.099},"min":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":2.5630000000000002,"15min":47.529000000000003},"last":1.79},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.083,"15min":1.008},"min":{"1min":0.64300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.171,"5min":17.460000000000001,"15min":17.460000000000001},"last":5.734}]},{"osd":2,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.86599999999999999,"15min":0.91200000000000003},"min":{"1min":0.58399999999999996,"5min":0.504,"15min":0.47299999999999998},"max":{"1min":1.0660000000000001,"5min":2.9359999999999999,"15min":5.5140000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0229999999999999,"15min":1.022},"min":{"1min":0.70199999999999996,"5min":0.60499999999999998,"15min":0.498},"max":{"1min":1.248,"5min":2.972,"15min":5.5979999999999999},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1719999999999999,"15min":1.179},"min":{"1min":0.63800000000000001,"5min":0.57299999999999995,"15min":0.44600000000000001},"max":{"1min":1.196,"5min":16.113,"15min":47.57},"last":7.3760000000000003},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.171,"15min":1.0649999999999999},"min":{"1min":0.73599999999999999,"5min":0.65400000000000003,"15min":0.53800000000000003},"max":{"1min":1.2749999999999999,"5min":11.148999999999999,"15min":12.273},"last":7.4269999999999996}]},{"osd":4,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.1180000000000001,"15min":1.054},"min":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.35499999999999998},"max":{"1min":1.1990000000000001,"5min":17.542000000000002,"15min":17.542000000000002},"last":5.6859999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.0489999999999999,"15min":1.196},"min":{"1min":0.84699999999999998,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.363,"5min":1.5329999999999999,"15min":49.106000000000002},"last":0.95799999999999996}]},{"osd":5,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.1220000000000001,"15min":1.2909999999999999},"min":{"1min":0.80600000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.345,"5min":6.6420000000000003,"15min":64.855999999999995},"last":2.0110000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.1679999999999999,"15min":1.1970000000000001},"min":{"1min":0.73499999999999999,"5min":0.71399999999999997,"15min":0.56399999999999995},"max":{"1min":1.2689999999999999,"5min":11.327,"15min":16.753},"last":5.843}]},{"osd":6,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.194,"15min":1.1639999999999999},"min":{"1min":0.88600000000000001,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":1.1930000000000001,"5min":11.641999999999999,"15min":11.641999999999999},"last":5.7679999999999998},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1399999999999999,"15min":1.153},"min":{"1min":0.97999999999999998,"5min":0.61499999999999999,"15min":0.502},"max":{"1min":1.4199999999999999,"5min":1.474,"15min":7.4909999999999997},"last":5.4210000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575116,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":380904,"kb_used_data":255328,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":93990936,"statfs":{"total":96636764160,"available":96246718464,"internally_reserved":0,"allocated":261455872,"data_stored":340022166,"data_compressed":25161614,"data_compressed_allocated":30015488,"data_compressed_original":108883915,"omap_allocated":1594,"internal_metadata":128580038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":2.9159999999999999,"5min":1.248,"15min":1.0109999999999999},"min":{"1min":0.51500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":40.999000000000002,"5min":40.999000000000002,"15min":40.999000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.05,"15min":1.036},"min":{"1min":0.58299999999999996,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":14.132999999999999,"5min":14.132999999999999,"15min":24.399999999999999},"last":1.1000000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.89500000000000002,"15min":0.88700000000000001},"min":{"1min":0.71999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2769999999999999,"5min":2.2400000000000002,"15min":11.196},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.90800000000000003,"15min":0.95899999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.35799999999999998},"max":{"1min":1.1759999999999999,"5min":6.1550000000000002,"15min":7.9130000000000003},"last":1.0669999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.89500000000000002,"15min":0.94799999999999995},"min":{"1min":0.66300000000000003,"5min":0.502,"15min":0.39100000000000001},"max":{"1min":3.855,"5min":3.855,"15min":21.986999999999998},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.92800000000000005,"15min":0.99099999999999999},"min":{"1min":0.621,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2829999999999999,"5min":1.419,"15min":8.6170000000000009},"last":0.72099999999999997}]},{"osd":3,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.94799999999999995,"15min":0.96299999999999997},"min":{"1min":0.57099999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.8439999999999999,"5min":3.2679999999999998,"15min":11.239000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0069999999999999,"15min":1.087},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.7959999999999998,"5min":3.29,"15min":24.745000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91800000000000004,"15min":1.014},"min":{"1min":0.71599999999999997,"5min":0.32300000000000001,"15min":0.30299999999999999},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":16.015000000000001},"last":1.135},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.044,"15min":1.079},"min":{"1min":0.69499999999999995,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":13.193},"last":1.2130000000000001}]},{"osd":5,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.98099999999999998,"15min":1.109},"min":{"1min":0.65000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":2.069,"5min":2.069,"15min":25.006},"last":1.25},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0629999999999999,"15min":1.1419999999999999},"min":{"1min":0.46700000000000003,"5min":0.33800000000000002,"15min":0.33000000000000002},"max":{"1min":1.405,"5min":1.4790000000000001,"15min":7.843},"last":1.1579999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.139},"min":{"1min":0.79200000000000004,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":1.1950000000000001,"5min":1.9179999999999999,"15min":7.883},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.177,"15min":1.276},"min":{"1min":0.93300000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.3169999999999999,"5min":1.9610000000000001,"15min":27.942},"last":1.478}]}]},{"osd":5,"up_from":13,"seq":55834575116,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":366316,"kb_used_data":240740,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":94005524,"statfs":{"total":96636764160,"available":96261656576,"internally_reserved":0,"allocated":246517760,"data_stored":342867931,"data_compressed":30547202,"data_compressed_allocated":36597760,"data_compressed_original":133251072,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,3,1,0,0,0,50],"upper_bound":1024},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.99399999999999999,"15min":0.89800000000000002},"min":{"1min":0.47399999999999998,"5min":0.46500000000000002,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":15.005000000000001,"15min":15.005000000000001},"last":3.3980000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88800000000000001},"min":{"1min":0.501,"5min":0.47799999999999998,"15min":0.46700000000000003},"max":{"1min":1.2290000000000001,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":3.419}]},{"osd":1,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81100000000000005,"15min":0.83999999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.48199999999999998},"max":{"1min":0.89800000000000002,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":3.806},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.88900000000000001,"15min":0.91300000000000003},"min":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.496},"max":{"1min":1.0609999999999999,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":7.4930000000000003}]},{"osd":2,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.016,"15min":0.97399999999999998},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.105,"5min":14.775,"15min":14.775},"last":1.131},{"interface":"front","average":{"1min":1.016,"5min":1.1040000000000001,"15min":0.94599999999999995},"min":{"1min":0.72899999999999998,"5min":0.63800000000000001,"15min":0.57499999999999996},"max":{"1min":1.1919999999999999,"5min":14.901999999999999,"15min":14.901999999999999},"last":1.25}]},{"osd":3,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.034,"15min":0.98899999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.54400000000000004},"max":{"1min":1.2529999999999999,"5min":15.154,"15min":15.154},"last":1.347},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.98999999999999999,"15min":1.0109999999999999},"min":{"1min":0.63900000000000001,"5min":0.60799999999999998,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":3.4289999999999998,"15min":5.5170000000000003},"last":1.216}]},{"osd":4,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.526,"15min":1.181},"min":{"1min":0.64600000000000002,"5min":0.59299999999999997,"15min":0.51400000000000001},"max":{"1min":1.292,"5min":61.020000000000003,"15min":61.020000000000003},"last":2.98},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.143,"15min":1.0920000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2490000000000001,"5min":15.085000000000001,"15min":15.085000000000001},"last":8.6029999999999998}]},{"osd":6,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.139,"15min":1.123},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.56699999999999995},"max":{"1min":1.5469999999999999,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":1.151},{"interface":"front","average":{"1min":1.216,"5min":1.194,"15min":1.1699999999999999},"min":{"1min":1.052,"5min":0.72499999999999998,"15min":0.48899999999999999},"max":{"1min":1.6799999999999999,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.1830000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1759999999999999,"15min":1.1639999999999999},"min":{"1min":0.94099999999999995,"5min":0.75600000000000001,"15min":0.56899999999999995},"max":{"1min":1.4219999999999999,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.054,"15min":1.0189999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.33200000000000002},"max":{"1min":1.085,"5min":6.2869999999999999,"15min":6.2869999999999999},"last":1.6779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575116,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":358356,"kb_used_data":236936,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":94013484,"statfs":{"total":96636764160,"available":96269807616,"internally_reserved":0,"allocated":242622464,"data_stored":338272538,"data_compressed":27596131,"data_compressed_allocated":33427456,"data_compressed_original":129380299,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,0,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":60,"apply_latency_ms":60,"commit_latency_ns":60000000,"apply_latency_ns":60000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.155,"15min":0.97399999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0449999999999999,"5min":43.139000000000003,"15min":43.139000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84399999999999997,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.39900000000000002},"max":{"1min":1.3049999999999999,"5min":4.444,"15min":17.446000000000002},"last":3.9809999999999999}]},{"osd":1,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.98799999999999999,"15min":1.054},"min":{"1min":0.64900000000000002,"5min":0.48199999999999998,"15min":0.44},"max":{"1min":1.2849999999999999,"5min":11.557,"15min":21.763999999999999},"last":2.3780000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0229999999999999,"15min":1.4790000000000001},"min":{"1min":0.65700000000000003,"5min":0.53800000000000003,"15min":0.46500000000000002},"max":{"1min":1.452,"5min":5.3479999999999999,"15min":78.649000000000001},"last":2.4329999999999998}]},{"osd":2,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":0.96499999999999997,"15min":1.0529999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.880000000000001,"5min":10.880000000000001,"15min":28.341000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.073,"15min":1.548},"min":{"1min":0.61699999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"max":{"1min":2.069,"5min":11.614000000000001,"15min":94.998999999999995},"last":1.002}]},{"osd":3,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.208,"15min":1.103},"min":{"1min":0.86299999999999999,"5min":0.67500000000000004,"15min":0.51300000000000001},"max":{"1min":9.984,"5min":9.984,"15min":13.202999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.0609999999999999,"15min":1.056},"min":{"1min":0.69299999999999995,"5min":0.54800000000000004,"15min":0.45600000000000002},"max":{"1min":9.952,"5min":9.952,"15min":16.277999999999999},"last":2.4900000000000002}]},{"osd":5,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.0049999999999999,"15min":1.1259999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.28199999999999997},"max":{"1min":1.385,"5min":3.4409999999999998,"15min":17.033000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.3799999999999999,"15min":1.3},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.34000000000000002},"max":{"1min":1.1759999999999999,"5min":38.939,"15min":38.939},"last":0.47299999999999998}]},{"osd":6,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.046,"15min":1.1699999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.37},"max":{"1min":1.4710000000000001,"5min":4.476,"15min":17.085000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.026,"5min":1.4850000000000001,"15min":1.3380000000000001},"min":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.39000000000000001},"max":{"1min":1.631,"5min":43.420000000000002,"15min":43.420000000000002},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.121,"15min":1.6160000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.309},"max":{"1min":1.5269999999999999,"5min":3.2109999999999999,"15min":77.638999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.198,"5min":1.24,"15min":1.175},"min":{"1min":0.90200000000000002,"5min":0.75900000000000001,"15min":0.51500000000000001},"max":{"1min":1.798,"5min":3.613,"15min":5.6200000000000001},"last":0.61699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575117,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457756,"kb_used_data":304404,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":93914084,"statfs":{"total":96636764160,"available":96168022016,"internally_reserved":0,"allocated":311709696,"data_stored":408700527,"data_compressed":36249891,"data_compressed_allocated":42946560,"data_compressed_original":140238848,"omap_allocated":1592,"internal_metadata":157022664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.89500000000000002,"15min":0.70999999999999996},"min":{"1min":0.246,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.67900000000000005,"5min":36.585000000000001,"15min":36.585000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.89500000000000002,"15min":0.67800000000000005},"min":{"1min":0.379,"5min":0.34300000000000003,"15min":0.33100000000000002},"max":{"1min":0.68400000000000005,"5min":30.065000000000001,"15min":30.065000000000001},"last":0.46500000000000002}]},{"osd":1,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.59999999999999998},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.26300000000000001},"max":{"1min":0.876,"5min":4.8099999999999996,"15min":10.449},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.61799999999999999,"15min":0.67400000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.98599999999999999,"5min":4.8520000000000003,"15min":10.481999999999999},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59099999999999997,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.27900000000000003},"max":{"1min":0.70299999999999996,"5min":1.25,"15min":10.369},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.56799999999999995,"15min":0.69599999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78600000000000003,"5min":1.1240000000000001,"15min":10.406000000000001},"last":0.40400000000000003}]},{"osd":4,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91700000000000004,"15min":1.361},"min":{"1min":0.71799999999999997,"5min":0.50700000000000001,"15min":0.376},"max":{"1min":1.0129999999999999,"5min":4.2119999999999997,"15min":110.39700000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.147,"5min":7.2030000000000003,"15min":108.777},"last":0.83699999999999997}]},{"osd":5,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.86199999999999999,"15min":0.92600000000000005},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.39900000000000002},"max":{"1min":0.98799999999999999,"5min":7.2679999999999998,"15min":17.513000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.873,"15min":0.92900000000000005},"min":{"1min":0.621,"5min":0.621,"15min":0.52300000000000002},"max":{"1min":0.998,"5min":1.7070000000000001,"15min":10.516999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82399999999999995,"15min":1.4019999999999999},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.365,"15min":138.91300000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.875,"15min":0.94499999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.48299999999999998},"max":{"1min":1.0980000000000001,"5min":1.766,"15min":10.302},"last":0.88600000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.91900000000000004,"15min":0.95799999999999996},"min":{"1min":0.63100000000000001,"5min":0.60999999999999999,"15min":0.45200000000000001},"max":{"1min":1.1990000000000001,"5min":7.3040000000000003,"15min":17.263000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.97799999999999998,"15min":1.034},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.49299999999999999},"max":{"1min":1.0289999999999999,"5min":7.3339999999999996,"15min":28.613},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575115,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":285968,"kb_used_data":174280,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94085872,"statfs":{"total":96636764160,"available":96343932928,"internally_reserved":0,"allocated":178462720,"data_stored":223929989,"data_compressed":15805945,"data_compressed_allocated":18817024,"data_compressed_original":64589824,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,0,0,0,0,26],"upper_bound":1024},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.623,"15min":0.57999999999999996},"min":{"1min":0.29199999999999998,"5min":0.26800000000000002,"15min":0.23599999999999999},"max":{"1min":0.58299999999999996,"5min":13.599,"15min":13.599},"last":1.6679999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.60699999999999998},"min":{"1min":0.34100000000000003,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":8.0630000000000006,"15min":8.0630000000000006},"last":7.1159999999999997}]},{"osd":1,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.53300000000000003,"15min":0.57499999999999996},"min":{"1min":0.33400000000000002,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.61799999999999999,"5min":3.0760000000000001,"15min":9.3260000000000005},"last":7.8970000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.70699999999999996,"15min":0.67000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":5.6660000000000004,"15min":7.4779999999999998},"last":10.657}]},{"osd":3,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.95199999999999996,"15min":0.79200000000000004},"min":{"1min":0.42099999999999999,"5min":0.32300000000000001,"15min":0.307},"max":{"1min":0.878,"5min":29.123999999999999,"15min":29.123999999999999},"last":7.1639999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.63100000000000001,"15min":0.59699999999999998},"min":{"1min":0.40000000000000002,"5min":0.312,"15min":0.28000000000000003},"max":{"1min":0.754,"5min":5.5410000000000004,"15min":7.4390000000000001},"last":8.8780000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.94099999999999995,"15min":0.92300000000000004},"min":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.49099999999999999},"max":{"1min":1.0489999999999999,"5min":5.6289999999999996,"15min":7.3630000000000004},"last":2.5630000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.87,"15min":0.879},"min":{"1min":0.60599999999999998,"5min":0.51200000000000001,"15min":0.41399999999999998},"max":{"1min":0.98599999999999999,"5min":5.5490000000000004,"15min":9.2289999999999992},"last":1.518}]},{"osd":5,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.85499999999999998,"15min":0.84299999999999997},"min":{"1min":0.622,"5min":0.501,"15min":0.501},"max":{"1min":1.01,"5min":3.3610000000000002,"15min":3.3610000000000002},"last":6.875},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.0109999999999999,"15min":0.93700000000000006},"min":{"1min":0.67300000000000004,"5min":0.59399999999999997,"15min":0.48399999999999999},"max":{"1min":1.266,"5min":6.343,"15min":7.2649999999999997},"last":8.7110000000000003}]},{"osd":6,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.0489999999999999,"15min":0.91800000000000004},"min":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":1.044,"5min":18.959,"15min":18.959},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.1339999999999999,"15min":0.97499999999999998},"min":{"1min":0.68300000000000005,"5min":0.622,"15min":0.52200000000000002},"max":{"1min":1.0700000000000001,"5min":21.957999999999998,"15min":21.957999999999998},"last":8.8420000000000005}]},{"osd":7,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89700000000000002,"15min":0.92100000000000004},"min":{"1min":0.68400000000000005,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":0.999,"5min":3.5299999999999998,"15min":9.2799999999999994},"last":8.7949999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.93300000000000005,"15min":0.90900000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.57099999999999995},"max":{"1min":1.042,"5min":2.6379999999999999,"15min":3.4470000000000001},"last":8.7569999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575115,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472320,"kb_used_data":314808,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93899520,"statfs":{"total":96636764160,"available":96153108480,"internally_reserved":0,"allocated":322363392,"data_stored":416685636,"data_compressed":32711569,"data_compressed_allocated":39284736,"data_compressed_original":133918720,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":0.75700000000000001,"15min":0.69599999999999995},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.246},"max":{"1min":10.096,"5min":12.311999999999999,"15min":14.538},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.623,"15min":0.65100000000000002},"min":{"1min":0.39000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.4119999999999999,"5min":8.2479999999999993,"15min":14.618},"last":0.57199999999999995}]},{"osd":2,"last update":"Tue Nov 28 21:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.66100000000000003,"15min":0.63100000000000001},"min":{"1min":0.40799999999999997,"5min":0.34899999999999998,"15min":0.31900000000000001},"max":{"1min":1.7150000000000001,"5min":5.1689999999999996,"15min":8.4749999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":4.7030000000000003,"15min":8.5150000000000006},"last":0.40600000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.75900000000000001,"15min":0.84699999999999998},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.27600000000000002},"max":{"1min":3.028,"5min":13.385,"15min":43.536999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.84699999999999998},"min":{"1min":0.501,"5min":0.443,"15min":0.375},"max":{"1min":5.6050000000000004,"5min":8.2899999999999991,"15min":18.308},"last":0.67900000000000005}]},{"osd":4,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":0.997,"15min":0.95699999999999996},"min":{"1min":0.63500000000000001,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":9.9849999999999994,"5min":9.9849999999999994,"15min":9.9849999999999994},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.4670000000000001,"5min":0.97899999999999998,"15min":1.079},"min":{"1min":0.61699999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":10.287000000000001,"5min":10.287000000000001,"15min":43.497999999999998},"last":0.69899999999999995}]},{"osd":5,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.93100000000000005,"15min":0.91600000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":5.5350000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.016,"15min":0.99399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.448,"5min":3.448,"15min":5.7409999999999997},"last":0.92600000000000005}]},{"osd":6,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.0449999999999999,"15min":1.0109999999999999},"min":{"1min":0.69899999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":10.023,"5min":10.023,"15min":10.385},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.05,"15min":1.2470000000000001},"min":{"1min":0.77700000000000002,"5min":0.65100000000000002,"15min":0.55200000000000005},"max":{"1min":3.7480000000000002,"5min":5.9349999999999996,"15min":43.448999999999998},"last":0.86299999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0469999999999999,"15min":1.024},"min":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":5.4109999999999996,"5min":5.9779999999999998,"15min":7.96},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.516,"5min":1.1899999999999999,"15min":1.1639999999999999},"min":{"1min":0.66500000000000004,"5min":0.55800000000000005,"15min":0.51500000000000001},"max":{"1min":10.063000000000001,"5min":15.672000000000001,"15min":42.473999999999997},"last":0.66100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575116,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":344096,"kb_used_data":214360,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94027744,"statfs":{"total":96636764160,"available":96284409856,"internally_reserved":0,"allocated":219504640,"data_stored":281144114,"data_compressed":19744060,"data_compressed_allocated":23732224,"data_compressed_original":85671936,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"min":{"1min":0.29499999999999998,"5min":0.27800000000000002,"15min":0.215},"max":{"1min":1.581,"5min":1.958,"15min":3.3929999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.60199999999999998,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.27700000000000002,"15min":0.248},"max":{"1min":4.5739999999999998,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.64900000000000002,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.4489999999999998,"5min":5.4489999999999998,"15min":5.8559999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":3.0840000000000001,"15min":5.4210000000000003},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.77700000000000002,"15min":0.623},"min":{"1min":0.30599999999999999,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":5.0019999999999998,"5min":16.187999999999999,"15min":16.187999999999999},"last":0.495},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.69199999999999995},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.5410000000000004,"5min":15.004,"15min":15.004},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":0.93999999999999995,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.42899999999999999},"max":{"1min":4.9119999999999999,"5min":5.2069999999999999,"15min":5.734},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.873,"15min":0.90500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0940000000000001,"5min":3.2160000000000002,"15min":5.3730000000000002},"last":0.753}]},{"osd":5,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.91900000000000004,"15min":0.84299999999999997},"min":{"1min":0.67300000000000004,"5min":0.59899999999999998,"15min":0.40699999999999997},"max":{"1min":4.4939999999999998,"5min":4.4939999999999998,"15min":4.4939999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.88500000000000001},"min":{"1min":0.65000000000000002,"5min":0.60799999999999998,"15min":0.50900000000000001},"max":{"1min":1.7350000000000001,"5min":4.7910000000000004,"15min":4.7910000000000004},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.01,"15min":0.89900000000000002},"min":{"1min":0.65500000000000003,"5min":0.60999999999999999,"15min":0.48699999999999999},"max":{"1min":4.8630000000000004,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.63},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.88100000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.55100000000000005},"max":{"1min":1.667,"5min":3.1339999999999999,"15min":3.4710000000000001},"last":0.90900000000000003}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.90800000000000003},"min":{"1min":0.627,"5min":0.627,"15min":0.48999999999999999},"max":{"1min":1.6950000000000001,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.95299999999999996,"15min":0.89600000000000002},"min":{"1min":0.68799999999999994,"5min":0.63,"15min":0.55700000000000005},"max":{"1min":1.627,"5min":3.1819999999999999,"15min":3.806},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16297984,"data_stored":62915220,"data_compressed":8677157,"data_compressed_allocated":11337728,"data_compressed_original":57970688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28188672,"data_stored":97891275,"data_compressed":12584510,"data_compressed_allocated":16609280,"data_compressed_original":86339584,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16445440,"data_stored":52440076,"data_compressed":6685552,"data_compressed_allocated":8757248,"data_compressed_original":44773376,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26685440,"data_stored":98577398,"data_compressed":13459583,"data_compressed_allocated":17428480,"data_compressed_original":89337856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":27463680,"data_stored":104859297,"data_compressed":13833556,"data_compressed_allocated":17862656,"data_compressed_original":95277003,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26279936,"data_stored":100664674,"data_compressed":13681157,"data_compressed_allocated":17580032,"data_compressed_original":91983872,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23711744,"data_stored":88082717,"data_compressed":11602354,"data_compressed_allocated":14856192,"data_compressed_original":79245259,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":27398144,"data_stored":97891059,"data_compressed":13288587,"data_compressed_allocated":17137664,"data_compressed_original":87650304,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":206610432,"data_stored":222244864,"data_compressed":12071885,"data_compressed_allocated":13537280,"data_compressed_original":29171712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294195200,"data_stored":319320064,"data_compressed":20251866,"data_compressed_allocated":22855680,"data_compressed_original":47980544,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164610048,"data_stored":175325184,"data_compressed":9845077,"data_compressed_allocated":10899456,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":288477184,"data_stored":316018688,"data_compressed":23971937,"data_compressed_allocated":27017216,"data_compressed_original":54558720,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":217907200,"data_stored":237305856,"data_compressed":14188788,"data_compressed_allocated":16142336,"data_compressed_original":35540992,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":220975104,"data_stored":243601408,"data_compressed":17073954,"data_compressed_allocated":19296256,"data_compressed_original":41922560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":237195264,"data_stored":251670528,"data_compressed":13556545,"data_compressed_allocated":15155200,"data_compressed_original":29630464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":192204800,"data_stored":208740352,"data_compressed":13315951,"data_compressed_allocated":15101952,"data_compressed_original":31637504,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:56:37.042 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T21:56:37.043 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:56:37.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.216+0000 7f218271d640 1 Processor -- start 2023-11-28T21:56:37.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.216+0000 7f218271d640 1 -- start start 2023-11-28T21:56:37.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.216+0000 7f218271d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f217c158b20 0x7f217c156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:37.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.216+0000 7f218271d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c067410 con 0x7f217c150c20 2023-11-28T21:56:37.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.216+0000 7f218271d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c06e090 con 0x7f217c06e300 2023-11-28T21:56:37.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.216+0000 7f218271d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c157f30 con 0x7f217c158b20 2023-11-28T21:56:37.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.216+0000 7f217bfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f217c150c20 0x7f217c0c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58074/0 (socket says 172.21.15.12:58074) 2023-11-28T21:56:37.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.216+0000 7f217bfff640 1 -- 172.21.15.12:0/363770888 learned_addr learned my addr 172.21.15.12:0/363770888 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:37.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.216+0000 7f2180cba640 1 --2- 172.21.15.12:0/363770888 >> v2:172.21.15.144:3300/0 conn(0x7f217c158b20 0x7f217c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.220+0000 7f217affd640 1 -- 172.21.15.12:0/363770888 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1780376204 0 0) 0x7f217c067410 con 0x7f217c150c20 2023-11-28T21:56:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.220+0000 7f217affd640 1 -- 172.21.15.12:0/363770888 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2164003530 con 0x7f217c150c20 2023-11-28T21:56:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.220+0000 7f2180cba640 1 -- 172.21.15.12:0/363770888 >> v1:172.21.15.144:6789/0 conn(0x7f217c06e300 legacy=0x7f217c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.220+0000 7f217affd640 1 -- 172.21.15.12:0/363770888 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2318089824 0 0) 0x7f2164003530 con 0x7f217c150c20 2023-11-28T21:56:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.220+0000 7f2180cba640 1 -- 172.21.15.12:0/363770888 >> v1:172.21.15.12:6789/0 conn(0x7f217c150c20 legacy=0x7f217c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.220+0000 7f2180cba640 1 -- 172.21.15.12:0/363770888 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c151000 con 0x7f217c158b20 2023-11-28T21:56:37.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.220+0000 7f2180cba640 1 --2- 172.21.15.12:0/363770888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c158b20 0x7f217c156910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2174009d90 tx=0x7f217401ea80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a72a6ec9090051d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:37.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.220+0000 7f217affd640 1 -- 172.21.15.12:0/363770888 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c001680 con 0x7f217c158b20 2023-11-28T21:56:37.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.220+0000 7f217affd640 1 -- 172.21.15.12:0/363770888 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21740a76e0 con 0x7f217c158b20 2023-11-28T21:56:37.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.220+0000 7f217affd640 1 -- 172.21.15.12:0/363770888 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740a7a50 con 0x7f217c158b20 2023-11-28T21:56:37.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 -- 172.21.15.12:0/363770888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c158b20 msgr2=0x7f217c156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 --2- 172.21.15.12:0/363770888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c158b20 0x7f217c156910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2174009d90 tx=0x7f217401ea80 comp rx=0 tx=0).stop 2023-11-28T21:56:37.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 -- 172.21.15.12:0/363770888 shutdown_connections 2023-11-28T21:56:37.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 --2- 172.21.15.12:0/363770888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c158b20 0x7f217c156910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 -- 172.21.15.12:0/363770888 >> 172.21.15.12:0/363770888 conn(0x7f217c066360 msgr2=0x7f217c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:37.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 -- 172.21.15.12:0/363770888 shutdown_connections 2023-11-28T21:56:37.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 -- 172.21.15.12:0/363770888 wait complete. 2023-11-28T21:56:37.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 Processor -- start 2023-11-28T21:56:37.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 -- start start 2023-11-28T21:56:37.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c06e300 0x7f217c0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:37.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c150c20 0x7f217c14dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f217c158b20 0x7f217c14e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c14da50 con 0x7f217c158b20 2023-11-28T21:56:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c15f380 con 0x7f217c150c20 2023-11-28T21:56:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f218271d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c1f2d70 con 0x7f217c06e300 2023-11-28T21:56:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f2180cba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f217c158b20 0x7f217c14e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f2180cba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f217c158b20 0x7f217c14e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48676/0 (socket says 172.21.15.12:48676) 2023-11-28T21:56:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f2180cba640 1 -- 172.21.15.12:0/546291356 learned_addr learned my addr 172.21.15.12:0/546291356 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:37.230 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f217b7fe640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c150c20 0x7f217c14dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f217bfff640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c06e300 0x7f217c0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f2180cba640 1 -- 172.21.15.12:0/546291356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c06e300 msgr2=0x7f217c0c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f2180cba640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c06e300 0x7f217c0c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f2180cba640 1 -- 172.21.15.12:0/546291356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c150c20 msgr2=0x7f217c14dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f2180cba640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c150c20 0x7f217c14dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f2180cba640 1 -- 172.21.15.12:0/546291356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2174009a70 con 0x7f217c158b20 2023-11-28T21:56:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f217b7fe640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c150c20 0x7f217c14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:37.231 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.224+0000 7f217bfff640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c06e300 0x7f217c0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.228+0000 7f2180cba640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f217c158b20 0x7f217c14e420 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f217401c6b0 tx=0x7f217401b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:37.232 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.228+0000 7f2178ff9640 1 -- 172.21.15.12:0/546291356 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740a7c60 con 0x7f217c158b20 2023-11-28T21:56:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.228+0000 7f2178ff9640 1 -- 172.21.15.12:0/546291356 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21740a7dc0 con 0x7f217c158b20 2023-11-28T21:56:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.228+0000 7f2178ff9640 1 -- 172.21.15.12:0/546291356 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174004240 con 0x7f217c158b20 2023-11-28T21:56:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.228+0000 7f218271d640 1 -- 172.21.15.12:0/546291356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c1f3050 con 0x7f217c158b20 2023-11-28T21:56:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.228+0000 7f218271d640 1 -- 172.21.15.12:0/546291356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c1f3610 con 0x7f217c158b20 2023-11-28T21:56:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.228+0000 7f218271d640 1 -- 172.21.15.12:0/546291356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c14dc00 con 0x7f217c158b20 2023-11-28T21:56:37.233 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.228+0000 7f2178ff9640 1 -- 172.21.15.12:0/546291356 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f21740043a0 con 0x7f217c158b20 2023-11-28T21:56:37.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.232+0000 7f2178ff9640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2168066470 0x7f2168068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:37.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.232+0000 7f2178ff9640 1 -- 172.21.15.12:0/546291356 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f217411e750 con 0x7f217c158b20 2023-11-28T21:56:37.234 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.232+0000 7f217bfff640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2168066470 0x7f2168068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:37.235 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.232+0000 7f217bfff640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2168066470 0x7f2168068930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f216c005890 tx=0x7f216c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:37.237 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.236+0000 7f2178ff9640 1 -- 172.21.15.12:0/546291356 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f217c14dc00 con 0x7f217c158b20 2023-11-28T21:56:37.395 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.392+0000 7f218271d640 1 -- 172.21.15.12:0/546291356 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f217c06ddb0 con 0x7f2168066470 2023-11-28T21:56:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.400+0000 7f2178ff9640 1 -- 172.21.15.12:0/546291356 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+374122 (secure 0 0 0) 0x7f216c0058e0 con 0x7f2168066470 2023-11-28T21:56:37.412 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:37.414 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:56:37.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 -- 172.21.15.12:0/546291356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2168066470 msgr2=0x7f2168068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2168066470 0x7f2168068930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f216c005890 tx=0x7f216c009040 comp rx=0 tx=0).stop 2023-11-28T21:56:37.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 -- 172.21.15.12:0/546291356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f217c158b20 msgr2=0x7f217c14e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f217c158b20 0x7f217c14e420 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f217401c6b0 tx=0x7f217401b820 comp rx=0 tx=0).stop 2023-11-28T21:56:37.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 -- 172.21.15.12:0/546291356 shutdown_connections 2023-11-28T21:56:37.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2168066470 0x7f2168068930 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f217c158b20 0x7f217c14e420 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.416 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c150c20 0x7f217c14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 --2- 172.21.15.12:0/546291356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c06e300 0x7f217c0c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 -- 172.21.15.12:0/546291356 >> 172.21.15.12:0/546291356 conn(0x7f217c066360 msgr2=0x7f217c14ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:37.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 -- 172.21.15.12:0/546291356 shutdown_connections 2023-11-28T21:56:37.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.404+0000 7f218271d640 1 -- 172.21.15.12:0/546291356 wait complete. 2023-11-28T21:56:37.429 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":708,"stamp":"2023-11-28T21:56:36.210879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1280114506,"num_objects":10548,"num_object_clones":0,"num_object_copies":41774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10548,"num_whiteouts":0,"num_read":9144,"num_read_kb":2407205,"num_write":56520,"num_write_kb":2354126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76052,"ondisk_log_size":76052,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3008496,"kb_used_data":1953900,"kb_used_omap":12,"kb_used_meta":1054515,"kb_avail":751966224,"statfs":{"total":773094113280,"available":770013413376,"internally_reserved":0,"allocated":2000793600,"data_stored":2656074320,"data_compressed":213714559,"data_compressed_allocated":256286720,"data_compressed_original":914001814,"omap_allocated":12751,"internal_metadata":1079823921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,1,8,13,7,4,0,1,78],"upper_bound":1024},"perf_stat":{"commit_latency_ms":267,"apply_latency_ms":267,"commit_latency_ns":267000000,"apply_latency_ns":267000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26378372,"num_objects":1446,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1557,"num_write_kb":26881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004723"},"pg_stats":[{"pgid":"2.1f","version":"51'341","reported_seq":427,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.057024+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:55:30.057024+0000","last_peered":"2023-11-28T21:55:30.057024+0000","last_clean":"2023-11-28T21:55:30.057024+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:55:30.057024+0000","last_undegraded":"2023-11-28T21:55:30.057024+0000","last_fullsized":"2023-11-28T21:55:30.057024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:24:46.167511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":61536,"num_write":310,"num_write_kb":8499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"50'268","reported_seq":2351,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:44.913651+0000","last_change":"2023-11-28T21:40:42.800615+0000","last_active":"2023-11-28T21:54:44.913651+0000","last_peered":"2023-11-28T21:54:44.913651+0000","last_clean":"2023-11-28T21:54:44.913651+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:54:44.913651+0000","last_undegraded":"2023-11-28T21:54:44.913651+0000","last_fullsized":"2023-11-28T21:54:44.913651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:37.956537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28950874199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"52'1318","reported_seq":3628,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.030721+0000","last_change":"2023-11-28T21:40:42.803511+0000","last_active":"2023-11-28T21:56:35.030721+0000","last_peered":"2023-11-28T21:56:35.030721+0000","last_clean":"2023-11-28T21:56:35.030721+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:56:35.030721+0000","last_undegraded":"2023-11-28T21:56:35.030721+0000","last_fullsized":"2023-11-28T21:56:35.030721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:31:55.934555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29329539100000002,"stat_sum":{"num_bytes":30193112,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":229,"num_read_kb":12319,"num_write":1291,"num_write_kb":57664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"52'465","reported_seq":539,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:26.898568+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:56:26.898568+0000","last_peered":"2023-11-28T21:56:26.898568+0000","last_clean":"2023-11-28T21:56:26.898568+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:56:26.898568+0000","last_undegraded":"2023-11-28T21:56:26.898568+0000","last_fullsized":"2023-11-28T21:56:26.898568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":81920,"num_write":420,"num_write_kb":20770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"50'251","reported_seq":2334,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.057010+0000","last_change":"2023-11-28T21:40:43.257406+0000","last_active":"2023-11-28T21:55:30.057010+0000","last_peered":"2023-11-28T21:55:30.057010+0000","last_clean":"2023-11-28T21:55:30.057010+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:55:30.057010+0000","last_undegraded":"2023-11-28T21:55:30.057010+0000","last_fullsized":"2023-11-28T21:55:30.057010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:47.663480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23094383900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"52'1401","reported_seq":3733,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.720695+0000","last_change":"2023-11-28T21:40:43.072665+0000","last_active":"2023-11-28T21:56:34.720695+0000","last_peered":"2023-11-28T21:56:34.720695+0000","last_clean":"2023-11-28T21:56:34.720695+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:56:34.720695+0000","last_undegraded":"2023-11-28T21:56:34.720695+0000","last_fullsized":"2023-11-28T21:56:34.720695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:41:20.225443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21613474099999999,"stat_sum":{"num_bytes":11729902,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":251,"num_read_kb":24615,"num_write":1375,"num_write_kb":48688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"52'761","reported_seq":854,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.356193+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:56:34.356193+0000","last_peered":"2023-11-28T21:56:34.356193+0000","last_clean":"2023-11-28T21:56:34.356193+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:56:34.356193+0000","last_undegraded":"2023-11-28T21:56:34.356193+0000","last_fullsized":"2023-11-28T21:56:34.356193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":59,"num_read_kb":131099,"num_write":714,"num_write_kb":20835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"50'280","reported_seq":2362,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147679+0000","last_change":"2023-11-28T21:40:42.830278+0000","last_active":"2023-11-28T21:54:40.147679+0000","last_peered":"2023-11-28T21:54:40.147679+0000","last_clean":"2023-11-28T21:54:40.147679+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:54:40.147679+0000","last_undegraded":"2023-11-28T21:54:40.147679+0000","last_fullsized":"2023-11-28T21:54:40.147679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:24.137854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30458173900000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"52'1324","reported_seq":3630,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.861331+0000","last_change":"2023-11-28T21:40:43.070433+0000","last_active":"2023-11-28T21:56:34.861331+0000","last_peered":"2023-11-28T21:56:34.861331+0000","last_clean":"2023-11-28T21:56:34.861331+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:56:34.861331+0000","last_undegraded":"2023-11-28T21:56:34.861331+0000","last_fullsized":"2023-11-28T21:56:34.861331+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:00.120053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23712909900000001,"stat_sum":{"num_bytes":32661416,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":226,"num_read_kb":7690,"num_write":1300,"num_write_kb":56471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"52'95","reported_seq":165,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.876669+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:56:27.876669+0000","last_peered":"2023-11-28T21:56:27.876669+0000","last_clean":"2023-11-28T21:56:27.876669+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:56:27.876669+0000","last_undegraded":"2023-11-28T21:56:27.876669+0000","last_fullsized":"2023-11-28T21:56:27.876669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":36891,"num_write":50,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"50'274","reported_seq":2356,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147734+0000","last_change":"2023-11-28T21:40:42.827917+0000","last_active":"2023-11-28T21:54:40.147734+0000","last_peered":"2023-11-28T21:54:40.147734+0000","last_clean":"2023-11-28T21:54:40.147734+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:54:40.147734+0000","last_undegraded":"2023-11-28T21:54:40.147734+0000","last_fullsized":"2023-11-28T21:54:40.147734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:32:57.982365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30416134,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"52'1294","reported_seq":3610,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.796080+0000","last_change":"2023-11-28T21:40:43.055466+0000","last_active":"2023-11-28T21:56:34.796080+0000","last_peered":"2023-11-28T21:56:34.796080+0000","last_clean":"2023-11-28T21:56:34.796080+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:56:34.796080+0000","last_undegraded":"2023-11-28T21:56:34.796080+0000","last_fullsized":"2023-11-28T21:56:34.796080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:33:55.527250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26667665699999998,"stat_sum":{"num_bytes":37281632,"num_objects":312,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":0,"num_read":236,"num_read_kb":13169,"num_write":1275,"num_write_kb":61269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"51'287","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.278456+0000","last_change":"2023-11-28T21:40:42.917342+0000","last_active":"2023-11-28T21:55:39.278456+0000","last_peered":"2023-11-28T21:55:39.278456+0000","last_clean":"2023-11-28T21:55:39.278456+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:55:39.278456+0000","last_undegraded":"2023-11-28T21:55:39.278456+0000","last_fullsized":"2023-11-28T21:55:39.278456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:39.167984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.200606007,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"52'617","reported_seq":711,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.513022+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:56:18.513022+0000","last_peered":"2023-11-28T21:56:18.513022+0000","last_clean":"2023-11-28T21:56:18.513022+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:56:18.513022+0000","last_undegraded":"2023-11-28T21:56:18.513022+0000","last_fullsized":"2023-11-28T21:56:18.513022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":114718,"num_write":571,"num_write_kb":20791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"52'1331","reported_seq":3650,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.683935+0000","last_change":"2023-11-28T21:40:43.071642+0000","last_active":"2023-11-28T21:56:34.683935+0000","last_peered":"2023-11-28T21:56:34.683935+0000","last_clean":"2023-11-28T21:56:34.683935+0000","last_became_active":"2023-11-28T21:35:25.616978+0000","last_became_peered":"2023-11-28T21:35:25.616978+0000","last_unstale":"2023-11-28T21:56:34.683935+0000","last_undegraded":"2023-11-28T21:56:34.683935+0000","last_fullsized":"2023-11-28T21:56:34.683935+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1331,"log_dups_size":0,"ondisk_log_size":1331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:05:37.470115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23129675,"stat_sum":{"num_bytes":32707084,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":229,"num_read_kb":4127,"num_write":1308,"num_write_kb":57872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"50'285","reported_seq":2367,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.047822+0000","last_change":"2023-11-28T21:40:42.798788+0000","last_active":"2023-11-28T21:54:55.047822+0000","last_peered":"2023-11-28T21:54:55.047822+0000","last_clean":"2023-11-28T21:54:55.047822+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:54:55.047822+0000","last_undegraded":"2023-11-28T21:54:55.047822+0000","last_fullsized":"2023-11-28T21:54:55.047822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:37.662225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28777871399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"52'954","reported_seq":1032,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:25.006236+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:56:25.006236+0000","last_peered":"2023-11-28T21:56:25.006236+0000","last_clean":"2023-11-28T21:56:25.006236+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:56:25.006236+0000","last_undegraded":"2023-11-28T21:56:25.006236+0000","last_fullsized":"2023-11-28T21:56:25.006236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":954,"log_dups_size":0,"ondisk_log_size":954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:07:53.339651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":75782,"num_write":932,"num_write_kb":13441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"52'1416","reported_seq":3740,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.956993+0000","last_change":"2023-11-28T21:40:43.070729+0000","last_active":"2023-11-28T21:56:34.956993+0000","last_peered":"2023-11-28T21:56:34.956993+0000","last_clean":"2023-11-28T21:56:34.956993+0000","last_became_active":"2023-11-28T21:35:25.790674+0000","last_became_peered":"2023-11-28T21:35:25.790674+0000","last_unstale":"2023-11-28T21:56:34.956993+0000","last_undegraded":"2023-11-28T21:56:34.956993+0000","last_fullsized":"2023-11-28T21:56:34.956993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1416,"log_dups_size":0,"ondisk_log_size":1416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:12:03.263147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.229623311,"stat_sum":{"num_bytes":17731185,"num_objects":354,"num_object_clones":0,"num_object_copies":1416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":234,"num_read_kb":20007,"num_write":1382,"num_write_kb":71863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"50'296","reported_seq":2379,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.272334+0000","last_change":"2023-11-28T21:40:42.800893+0000","last_active":"2023-11-28T21:55:39.272334+0000","last_peered":"2023-11-28T21:55:39.272334+0000","last_clean":"2023-11-28T21:55:39.272334+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:55:39.272334+0000","last_undegraded":"2023-11-28T21:55:39.272334+0000","last_fullsized":"2023-11-28T21:55:39.272334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:35:30.683746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29027087200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"52'722","reported_seq":790,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:33.926232+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:56:33.926232+0000","last_peered":"2023-11-28T21:56:33.926232+0000","last_clean":"2023-11-28T21:56:33.926232+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:56:33.926232+0000","last_undegraded":"2023-11-28T21:56:33.926232+0000","last_fullsized":"2023-11-28T21:56:33.926232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9810037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":32782,"num_write":677,"num_write_kb":9926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"52'1341","reported_seq":3684,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.913491+0000","last_change":"2023-11-28T21:40:42.890432+0000","last_active":"2023-11-28T21:56:34.913491+0000","last_peered":"2023-11-28T21:56:34.913491+0000","last_clean":"2023-11-28T21:56:34.913491+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:56:34.913491+0000","last_undegraded":"2023-11-28T21:56:34.913491+0000","last_fullsized":"2023-11-28T21:56:34.913491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1341,"log_dups_size":0,"ondisk_log_size":1341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:09:28.392308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24643899999999999,"stat_sum":{"num_bytes":14380393,"num_objects":336,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":262,"num_read_kb":16758,"num_write":1318,"num_write_kb":45879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"50'276","reported_seq":2359,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.687600+0000","last_change":"2023-11-28T21:40:43.071973+0000","last_active":"2023-11-28T21:55:38.687600+0000","last_peered":"2023-11-28T21:55:38.687600+0000","last_clean":"2023-11-28T21:55:38.687600+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:55:38.687600+0000","last_undegraded":"2023-11-28T21:55:38.687600+0000","last_fullsized":"2023-11-28T21:55:38.687600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:34:28.182888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21610210399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"52'341","reported_seq":411,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.377147+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:56:08.377147+0000","last_peered":"2023-11-28T21:56:08.377147+0000","last_clean":"2023-11-28T21:56:08.377147+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:56:08.377147+0000","last_undegraded":"2023-11-28T21:56:08.377147+0000","last_fullsized":"2023-11-28T21:56:08.377147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":49164,"num_write":265,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"52'1262","reported_seq":3549,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.730763+0000","last_change":"2023-11-28T21:40:43.060232+0000","last_active":"2023-11-28T21:56:34.730763+0000","last_peered":"2023-11-28T21:56:34.730763+0000","last_clean":"2023-11-28T21:56:34.730763+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:56:34.730763+0000","last_undegraded":"2023-11-28T21:56:34.730763+0000","last_fullsized":"2023-11-28T21:56:34.730763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:54:07.778886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.244586779,"stat_sum":{"num_bytes":50267474,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":206,"num_read_kb":15432,"num_write":1240,"num_write_kb":82194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"50'281","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.276659+0000","last_change":"2023-11-28T21:40:43.249048+0000","last_active":"2023-11-28T21:55:39.276659+0000","last_peered":"2023-11-28T21:55:39.276659+0000","last_clean":"2023-11-28T21:55:39.276659+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:55:39.276659+0000","last_undegraded":"2023-11-28T21:55:39.276659+0000","last_fullsized":"2023-11-28T21:55:39.276659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:35:39.660816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26728515899999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"52'168","reported_seq":240,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:20.126618+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:56:20.126618+0000","last_peered":"2023-11-28T21:56:20.126618+0000","last_clean":"2023-11-28T21:56:20.126618+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:56:20.126618+0000","last_undegraded":"2023-11-28T21:56:20.126618+0000","last_fullsized":"2023-11-28T21:56:20.126618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":45,"num_read_kb":24615,"num_write":146,"num_write_kb":4216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"52'1368","reported_seq":3700,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.654211+0000","last_change":"2023-11-28T21:40:43.072395+0000","last_active":"2023-11-28T21:56:34.654211+0000","last_peered":"2023-11-28T21:56:34.654211+0000","last_clean":"2023-11-28T21:56:34.654211+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:56:34.654211+0000","last_undegraded":"2023-11-28T21:56:34.654211+0000","last_fullsized":"2023-11-28T21:56:34.654211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:25:44.577393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21672229300000001,"stat_sum":{"num_bytes":16299977,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":250,"num_read_kb":17253,"num_write":1343,"num_write_kb":69509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"50'249","reported_seq":2332,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273472+0000","last_change":"2023-11-28T21:40:42.801394+0000","last_active":"2023-11-28T21:55:39.273472+0000","last_peered":"2023-11-28T21:55:39.273472+0000","last_clean":"2023-11-28T21:55:39.273472+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:55:39.273472+0000","last_undegraded":"2023-11-28T21:55:39.273472+0000","last_fullsized":"2023-11-28T21:55:39.273472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:59:35.647639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29009278799999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"52'245","reported_seq":314,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.893946+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:56:27.893946+0000","last_peered":"2023-11-28T21:56:27.893946+0000","last_clean":"2023-11-28T21:56:27.893946+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:56:27.893946+0000","last_undegraded":"2023-11-28T21:56:27.893946+0000","last_fullsized":"2023-11-28T21:56:27.893946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:36.968220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":607,"num_write":445,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"52'1386","reported_seq":3700,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.993814+0000","last_change":"2023-11-28T21:40:42.797955+0000","last_active":"2023-11-28T21:56:34.993814+0000","last_peered":"2023-11-28T21:56:34.993814+0000","last_clean":"2023-11-28T21:56:34.993814+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:56:34.993814+0000","last_undegraded":"2023-11-28T21:56:34.993814+0000","last_fullsized":"2023-11-28T21:56:34.993814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1386,"log_dups_size":0,"ondisk_log_size":1386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:27:40.256597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.287791554,"stat_sum":{"num_bytes":19942162,"num_objects":313,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":232,"num_read_kb":16373,"num_write":1352,"num_write_kb":52080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"50'287","reported_seq":2371,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:29.999480+0000","last_change":"2023-11-28T21:40:42.884411+0000","last_active":"2023-11-28T21:55:29.999480+0000","last_peered":"2023-11-28T21:55:29.999480+0000","last_clean":"2023-11-28T21:55:29.999480+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:55:29.999480+0000","last_undegraded":"2023-11-28T21:55:29.999480+0000","last_fullsized":"2023-11-28T21:55:29.999480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:48.590211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24779062600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"52'587","reported_seq":665,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.695016+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:55:43.695016+0000","last_peered":"2023-11-28T21:55:43.695016+0000","last_clean":"2023-11-28T21:55:43.695016+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:55:43.695016+0000","last_undegraded":"2023-11-28T21:55:43.695016+0000","last_fullsized":"2023-11-28T21:55:43.695016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":81538,"num_write":523,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"52'1410","reported_seq":3740,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.895421+0000","last_change":"2023-11-28T21:40:43.255749+0000","last_active":"2023-11-28T21:56:34.895421+0000","last_peered":"2023-11-28T21:56:34.895421+0000","last_clean":"2023-11-28T21:56:34.895421+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:56:34.895421+0000","last_undegraded":"2023-11-28T21:56:34.895421+0000","last_fullsized":"2023-11-28T21:56:34.895421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:38:18.061461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27155133199999998,"stat_sum":{"num_bytes":49110889,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":248,"num_read_kb":24602,"num_write":1394,"num_write_kb":89036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"50'274","reported_seq":2358,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.201955+0000","last_change":"2023-11-28T21:40:42.828660+0000","last_active":"2023-11-28T21:55:30.201955+0000","last_peered":"2023-11-28T21:55:30.201955+0000","last_clean":"2023-11-28T21:55:30.201955+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:55:30.201955+0000","last_undegraded":"2023-11-28T21:55:30.201955+0000","last_fullsized":"2023-11-28T21:55:30.201955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:55.786023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30430776599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"52'164","reported_seq":225,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.682785+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:55:38.682785+0000","last_peered":"2023-11-28T21:55:38.682785+0000","last_clean":"2023-11-28T21:55:38.682785+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:55:38.682785+0000","last_undegraded":"2023-11-28T21:55:38.682785+0000","last_fullsized":"2023-11-28T21:55:38.682785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:08:15.270352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24576,"num_write":146,"num_write_kb":4262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"52'1318","reported_seq":3622,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.840355+0000","last_change":"2023-11-28T21:40:42.887654+0000","last_active":"2023-11-28T21:56:34.840355+0000","last_peered":"2023-11-28T21:56:34.840355+0000","last_clean":"2023-11-28T21:56:34.840355+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:56:34.840355+0000","last_undegraded":"2023-11-28T21:56:34.840355+0000","last_fullsized":"2023-11-28T21:56:34.840355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:38.441656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.25659980999999998,"stat_sum":{"num_bytes":32626940,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":222,"num_read_kb":4128,"num_write":1285,"num_write_kb":60020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"50'286","reported_seq":2369,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.214610+0000","last_change":"2023-11-28T21:40:43.068901+0000","last_active":"2023-11-28T21:55:30.214610+0000","last_peered":"2023-11-28T21:55:30.214610+0000","last_clean":"2023-11-28T21:55:30.214610+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:55:30.214610+0000","last_undegraded":"2023-11-28T21:55:30.214610+0000","last_fullsized":"2023-11-28T21:55:30.214610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:05:04.092055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23142385100000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"52'365","reported_seq":439,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.694515+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:55:38.694515+0000","last_peered":"2023-11-28T21:55:38.694515+0000","last_clean":"2023-11-28T21:55:38.694515+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:55:38.694515+0000","last_undegraded":"2023-11-28T21:55:38.694515+0000","last_fullsized":"2023-11-28T21:55:38.694515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":81920,"num_write":337,"num_write_kb":16655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"52'1263","reported_seq":3555,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.958687+0000","last_change":"2023-11-28T21:40:42.883278+0000","last_active":"2023-11-28T21:56:34.958687+0000","last_peered":"2023-11-28T21:56:34.958687+0000","last_clean":"2023-11-28T21:56:34.958687+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:56:34.958687+0000","last_undegraded":"2023-11-28T21:56:34.958687+0000","last_fullsized":"2023-11-28T21:56:34.958687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:30:54.117448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24685721299999999,"stat_sum":{"num_bytes":16718463,"num_objects":288,"num_object_clones":0,"num_object_copies":1152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":210,"num_read_kb":3302,"num_write":1235,"num_write_kb":35539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"50'283","reported_seq":2367,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:44.891386+0000","last_change":"2023-11-28T21:40:42.890540+0000","last_active":"2023-11-28T21:54:44.891386+0000","last_peered":"2023-11-28T21:54:44.891386+0000","last_clean":"2023-11-28T21:54:44.891386+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:54:44.891386+0000","last_undegraded":"2023-11-28T21:54:44.891386+0000","last_fullsized":"2023-11-28T21:54:44.891386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:05:51.055896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24023003500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"51'934","reported_seq":1405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.724754+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:55:43.724754+0000","last_peered":"2023-11-28T21:55:43.724754+0000","last_clean":"2023-11-28T21:55:43.724754+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:55:43.724754+0000","last_undegraded":"2023-11-28T21:55:43.724754+0000","last_fullsized":"2023-11-28T21:55:43.724754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18884916,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":417,"num_read_kb":34819,"num_write":869,"num_write_kb":18975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"52'1231","reported_seq":3557,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.911522+0000","last_change":"2023-11-28T21:40:42.888191+0000","last_active":"2023-11-28T21:56:34.911522+0000","last_peered":"2023-11-28T21:56:34.911522+0000","last_clean":"2023-11-28T21:56:34.911522+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:56:34.911522+0000","last_undegraded":"2023-11-28T21:56:34.911522+0000","last_fullsized":"2023-11-28T21:56:34.911522+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:20.371038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24393632200000001,"stat_sum":{"num_bytes":26023878,"num_objects":293,"num_object_clones":0,"num_object_copies":1172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":293,"num_whiteouts":0,"num_read":244,"num_read_kb":12314,"num_write":1202,"num_write_kb":62723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"51'259","reported_seq":2342,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.056922+0000","last_change":"2023-11-28T21:40:43.256849+0000","last_active":"2023-11-28T21:55:30.056922+0000","last_peered":"2023-11-28T21:55:30.056922+0000","last_clean":"2023-11-28T21:55:30.056922+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:55:30.056922+0000","last_undegraded":"2023-11-28T21:55:30.056922+0000","last_fullsized":"2023-11-28T21:55:30.056922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:11:25.250732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.25105937499999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"51'313","reported_seq":395,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.725792+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:55:38.725792+0000","last_peered":"2023-11-28T21:55:38.725792+0000","last_clean":"2023-11-28T21:55:38.725792+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:55:38.725792+0000","last_undegraded":"2023-11-28T21:55:38.725792+0000","last_fullsized":"2023-11-28T21:55:38.725792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":114688,"num_write":275,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"52'1365","reported_seq":3711,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.880915+0000","last_change":"2023-11-28T21:40:43.252159+0000","last_active":"2023-11-28T21:56:34.880915+0000","last_peered":"2023-11-28T21:56:34.880915+0000","last_clean":"2023-11-28T21:56:34.880915+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:56:34.880915+0000","last_undegraded":"2023-11-28T21:56:34.880915+0000","last_fullsized":"2023-11-28T21:56:34.880915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:58:49.674343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.270607394,"stat_sum":{"num_bytes":19916106,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":264,"num_read_kb":13370,"num_write":1336,"num_write_kb":59202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"50'268","reported_seq":2351,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.051948+0000","last_change":"2023-11-28T21:40:42.802535+0000","last_active":"2023-11-28T21:55:30.051948+0000","last_peered":"2023-11-28T21:55:30.051948+0000","last_clean":"2023-11-28T21:55:30.051948+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:55:30.051948+0000","last_undegraded":"2023-11-28T21:55:30.051948+0000","last_fullsized":"2023-11-28T21:55:30.051948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:02:13.185718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29152715000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"52'311","reported_seq":375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:07.951419+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:56:07.951419+0000","last_peered":"2023-11-28T21:56:07.951419+0000","last_clean":"2023-11-28T21:56:07.951419+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:56:07.951419+0000","last_undegraded":"2023-11-28T21:56:07.951419+0000","last_fullsized":"2023-11-28T21:56:07.951419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":24594,"num_write":297,"num_write_kb":8480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.725972+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:55:38.725972+0000","last_peered":"2023-11-28T21:55:38.725972+0000","last_clean":"2023-11-28T21:55:38.725972+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:55:38.725972+0000","last_undegraded":"2023-11-28T21:55:38.725972+0000","last_fullsized":"2023-11-28T21:55:38.725972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"52'1342","reported_seq":3676,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.779903+0000","last_change":"2023-11-28T21:40:42.914002+0000","last_active":"2023-11-28T21:56:34.779903+0000","last_peered":"2023-11-28T21:56:34.779903+0000","last_clean":"2023-11-28T21:56:34.779903+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:56:34.779903+0000","last_undegraded":"2023-11-28T21:56:34.779903+0000","last_fullsized":"2023-11-28T21:56:34.779903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:47.036263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19732549099999999,"stat_sum":{"num_bytes":29762487,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":252,"num_read_kb":20508,"num_write":1310,"num_write_kb":73242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"50'256","reported_seq":2339,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.116489+0000","last_change":"2023-11-28T21:40:42.748386+0000","last_active":"2023-11-28T21:54:55.116489+0000","last_peered":"2023-11-28T21:54:55.116489+0000","last_clean":"2023-11-28T21:54:55.116489+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:54:55.116489+0000","last_undegraded":"2023-11-28T21:54:55.116489+0000","last_fullsized":"2023-11-28T21:54:55.116489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:59.734407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.219020891,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"52'475","reported_seq":534,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273615+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:55:39.273615+0000","last_peered":"2023-11-28T21:55:39.273615+0000","last_clean":"2023-11-28T21:55:39.273615+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:55:39.273615+0000","last_undegraded":"2023-11-28T21:55:39.273615+0000","last_fullsized":"2023-11-28T21:55:39.273615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":20485,"num_write":447,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"52'1372","reported_seq":3704,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.807900+0000","last_change":"2023-11-28T21:40:42.915914+0000","last_active":"2023-11-28T21:56:34.807900+0000","last_peered":"2023-11-28T21:56:34.807900+0000","last_clean":"2023-11-28T21:56:34.807900+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:56:34.807900+0000","last_undegraded":"2023-11-28T21:56:34.807900+0000","last_fullsized":"2023-11-28T21:56:34.807900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1372,"log_dups_size":0,"ondisk_log_size":1372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:52:07.090824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19929065100000001,"stat_sum":{"num_bytes":26470227,"num_objects":335,"num_object_clones":0,"num_object_copies":1340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":251,"num_read_kb":4134,"num_write":1344,"num_write_kb":47412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"51'313","reported_seq":2396,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.194876+0000","last_change":"2023-11-28T21:40:42.830692+0000","last_active":"2023-11-28T21:54:55.194876+0000","last_peered":"2023-11-28T21:54:55.194876+0000","last_clean":"2023-11-28T21:54:55.194876+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:54:55.194876+0000","last_undegraded":"2023-11-28T21:54:55.194876+0000","last_fullsized":"2023-11-28T21:54:55.194876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:33:58.325515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30458413400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"52'964","reported_seq":1044,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.896525+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:56:27.896525+0000","last_peered":"2023-11-28T21:56:27.896525+0000","last_clean":"2023-11-28T21:56:27.896525+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:56:27.896525+0000","last_undegraded":"2023-11-28T21:56:27.896525+0000","last_fullsized":"2023-11-28T21:56:27.896525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":964,"log_dups_size":0,"ondisk_log_size":964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:22:48.073446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777286,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":27,"num_read_kb":68346,"num_write":930,"num_write_kb":16947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"52'1319","reported_seq":3665,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.895088+0000","last_change":"2023-11-28T21:40:43.256533+0000","last_active":"2023-11-28T21:56:34.895088+0000","last_peered":"2023-11-28T21:56:34.895088+0000","last_clean":"2023-11-28T21:56:34.895088+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:56:34.895088+0000","last_undegraded":"2023-11-28T21:56:34.895088+0000","last_fullsized":"2023-11-28T21:56:34.895088+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:07:44.854009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27089875299999999,"stat_sum":{"num_bytes":28004524,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":265,"num_read_kb":16422,"num_write":1294,"num_write_kb":50614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"50'264","reported_seq":2340,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.274191+0000","last_change":"2023-11-28T21:40:43.090122+0000","last_active":"2023-11-28T21:55:39.274191+0000","last_peered":"2023-11-28T21:55:39.274191+0000","last_clean":"2023-11-28T21:55:39.274191+0000","last_became_active":"2023-11-28T21:35:25.594533+0000","last_became_peered":"2023-11-28T21:35:25.594533+0000","last_unstale":"2023-11-28T21:55:39.274191+0000","last_undegraded":"2023-11-28T21:55:39.274191+0000","last_fullsized":"2023-11-28T21:55:39.274191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:03.418589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14880083199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"52'303","reported_seq":378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.839543+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:56:27.839543+0000","last_peered":"2023-11-28T21:56:27.839543+0000","last_clean":"2023-11-28T21:56:27.839543+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:56:27.839543+0000","last_undegraded":"2023-11-28T21:56:27.839543+0000","last_fullsized":"2023-11-28T21:56:27.839543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":53275,"num_write":251,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"52'1307","reported_seq":3591,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.520158+0000","last_change":"2023-11-28T21:40:42.823686+0000","last_active":"2023-11-28T21:56:34.520158+0000","last_peered":"2023-11-28T21:56:34.520158+0000","last_clean":"2023-11-28T21:56:34.520158+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:56:34.520158+0000","last_undegraded":"2023-11-28T21:56:34.520158+0000","last_fullsized":"2023-11-28T21:56:34.520158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1307,"log_dups_size":0,"ondisk_log_size":1307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:34:35.818463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30648515900000001,"stat_sum":{"num_bytes":28110180,"num_objects":295,"num_object_clones":0,"num_object_copies":1180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":0,"num_read":202,"num_read_kb":8227,"num_write":1283,"num_write_kb":51039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"50'267","reported_seq":2350,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.116571+0000","last_change":"2023-11-28T21:40:42.749319+0000","last_active":"2023-11-28T21:54:55.116571+0000","last_peered":"2023-11-28T21:54:55.116571+0000","last_clean":"2023-11-28T21:54:55.116571+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:54:55.116571+0000","last_undegraded":"2023-11-28T21:54:55.116571+0000","last_fullsized":"2023-11-28T21:54:55.116571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:18:31.031890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21877398200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"52'193","reported_seq":266,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.689380+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:55:38.689380+0000","last_peered":"2023-11-28T21:55:38.689380+0000","last_clean":"2023-11-28T21:55:38.689380+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:55:38.689380+0000","last_undegraded":"2023-11-28T21:55:38.689380+0000","last_fullsized":"2023-11-28T21:55:38.689380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":21,"num_read_kb":49161,"num_write":146,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"52'1273","reported_seq":3574,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.716263+0000","last_change":"2023-11-28T21:40:43.072161+0000","last_active":"2023-11-28T21:56:34.716263+0000","last_peered":"2023-11-28T21:56:34.716263+0000","last_clean":"2023-11-28T21:56:34.716263+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:56:34.716263+0000","last_undegraded":"2023-11-28T21:56:34.716263+0000","last_fullsized":"2023-11-28T21:56:34.716263+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:39.182750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.223266511,"stat_sum":{"num_bytes":11523503,"num_objects":308,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":219,"num_read_kb":12316,"num_write":1248,"num_write_kb":38807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"50'308","reported_seq":2391,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.725792+0000","last_change":"2023-11-28T21:40:42.747053+0000","last_active":"2023-11-28T21:55:38.725792+0000","last_peered":"2023-11-28T21:55:38.725792+0000","last_clean":"2023-11-28T21:55:38.725792+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:55:38.725792+0000","last_undegraded":"2023-11-28T21:55:38.725792+0000","last_fullsized":"2023-11-28T21:55:38.725792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:54:52.864875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21978382299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"52'854","reported_seq":964,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.035796+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:56:18.035796+0000","last_peered":"2023-11-28T21:56:18.035796+0000","last_clean":"2023-11-28T21:56:18.035796+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:56:18.035796+0000","last_undegraded":"2023-11-28T21:56:18.035796+0000","last_fullsized":"2023-11-28T21:56:18.035796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":137,"num_read_kb":58709,"num_write":838,"num_write_kb":8642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"52'1274","reported_seq":3606,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.943389+0000","last_change":"2023-11-28T21:40:43.252839+0000","last_active":"2023-11-28T21:56:34.943389+0000","last_peered":"2023-11-28T21:56:34.943389+0000","last_clean":"2023-11-28T21:56:34.943389+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:56:34.943389+0000","last_undegraded":"2023-11-28T21:56:34.943389+0000","last_fullsized":"2023-11-28T21:56:34.943389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:28:08.856108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27152726799999999,"stat_sum":{"num_bytes":36307339,"num_objects":283,"num_object_clones":0,"num_object_copies":1132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":283,"num_whiteouts":0,"num_read":251,"num_read_kb":8046,"num_write":1253,"num_write_kb":60298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"51'273","reported_seq":2355,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.278420+0000","last_change":"2023-11-28T21:40:42.918241+0000","last_active":"2023-11-28T21:55:39.278420+0000","last_peered":"2023-11-28T21:55:39.278420+0000","last_clean":"2023-11-28T21:55:39.278420+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:55:39.278420+0000","last_undegraded":"2023-11-28T21:55:39.278420+0000","last_fullsized":"2023-11-28T21:55:39.278420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:57.732993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18244292200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"52'239","reported_seq":293,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.710890+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:55:43.710890+0000","last_peered":"2023-11-28T21:55:43.710890+0000","last_clean":"2023-11-28T21:55:43.710890+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:55:43.710890+0000","last_undegraded":"2023-11-28T21:55:43.710890+0000","last_fullsized":"2023-11-28T21:55:43.710890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195875,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":194,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"52'1346","reported_seq":3662,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.682315+0000","last_change":"2023-11-28T21:40:42.805322+0000","last_active":"2023-11-28T21:56:34.682315+0000","last_peered":"2023-11-28T21:56:34.682315+0000","last_clean":"2023-11-28T21:56:34.682315+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:56:34.682315+0000","last_undegraded":"2023-11-28T21:56:34.682315+0000","last_fullsized":"2023-11-28T21:56:34.682315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:22:10.703578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29439885100000002,"stat_sum":{"num_bytes":34458204,"num_objects":324,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":324,"num_whiteouts":0,"num_read":235,"num_read_kb":27797,"num_write":1320,"num_write_kb":70332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"50'300","reported_seq":2383,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.726358+0000","last_change":"2023-11-28T21:40:42.740620+0000","last_active":"2023-11-28T21:55:38.726358+0000","last_peered":"2023-11-28T21:55:38.726358+0000","last_clean":"2023-11-28T21:55:38.726358+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:55:38.726358+0000","last_undegraded":"2023-11-28T21:55:38.726358+0000","last_fullsized":"2023-11-28T21:55:38.726358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:41:24.855115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.218364582,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"51'155","reported_seq":213,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.226609+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:56:08.226609+0000","last_peered":"2023-11-28T21:56:08.226609+0000","last_clean":"2023-11-28T21:56:08.226609+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:56:08.226609+0000","last_undegraded":"2023-11-28T21:56:08.226609+0000","last_fullsized":"2023-11-28T21:56:08.226609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":107,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"52'1271","reported_seq":3570,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.041533+0000","last_change":"2023-11-28T21:40:42.806545+0000","last_active":"2023-11-28T21:56:35.041533+0000","last_peered":"2023-11-28T21:56:35.041533+0000","last_clean":"2023-11-28T21:56:35.041533+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:56:35.041533+0000","last_undegraded":"2023-11-28T21:56:35.041533+0000","last_fullsized":"2023-11-28T21:56:35.041533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:19:01.956213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29571303700000001,"stat_sum":{"num_bytes":33678171,"num_objects":350,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":350,"num_whiteouts":0,"num_read":218,"num_read_kb":4129,"num_write":1246,"num_write_kb":52865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"50'271","reported_seq":2353,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.276133+0000","last_change":"2023-11-28T21:40:43.257612+0000","last_active":"2023-11-28T21:55:39.276133+0000","last_peered":"2023-11-28T21:55:39.276133+0000","last_clean":"2023-11-28T21:55:39.276133+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:55:39.276133+0000","last_undegraded":"2023-11-28T21:55:39.276133+0000","last_fullsized":"2023-11-28T21:55:39.276133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:04.640806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21362616300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"52'304","reported_seq":397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.361916+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:56:08.361916+0000","last_peered":"2023-11-28T21:56:08.361916+0000","last_clean":"2023-11-28T21:56:08.361916+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:56:08.361916+0000","last_undegraded":"2023-11-28T21:56:08.361916+0000","last_fullsized":"2023-11-28T21:56:08.361916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:21:28.694184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":67,"num_read_kb":77908,"num_write":271,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"52'1265","reported_seq":3536,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.764756+0000","last_change":"2023-11-28T21:40:42.831006+0000","last_active":"2023-11-28T21:56:34.764756+0000","last_peered":"2023-11-28T21:56:34.764756+0000","last_clean":"2023-11-28T21:56:34.764756+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:56:34.764756+0000","last_undegraded":"2023-11-28T21:56:34.764756+0000","last_fullsized":"2023-11-28T21:56:34.764756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1265,"log_dups_size":0,"ondisk_log_size":1265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:03:39.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30778537500000003,"stat_sum":{"num_bytes":30037804,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":190,"num_read_kb":33,"num_write":1241,"num_write_kb":36631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"50'280","reported_seq":2363,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.726323+0000","last_change":"2023-11-28T21:40:42.748885+0000","last_active":"2023-11-28T21:55:38.726323+0000","last_peered":"2023-11-28T21:55:38.726323+0000","last_clean":"2023-11-28T21:55:38.726323+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:55:38.726323+0000","last_undegraded":"2023-11-28T21:55:38.726323+0000","last_fullsized":"2023-11-28T21:55:38.726323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:04:42.978237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21990301600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"52'747","reported_seq":861,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.900520+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:56:27.900520+0000","last_peered":"2023-11-28T21:56:27.900520+0000","last_clean":"2023-11-28T21:56:27.900520+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:56:27.900520+0000","last_undegraded":"2023-11-28T21:56:27.900520+0000","last_fullsized":"2023-11-28T21:56:27.900520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":90,"num_read_kb":122940,"num_write":681,"num_write_kb":20912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"52'1318","reported_seq":3604,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.710137+0000","last_change":"2023-11-28T21:40:42.828740+0000","last_active":"2023-11-28T21:56:34.710137+0000","last_peered":"2023-11-28T21:56:34.710137+0000","last_clean":"2023-11-28T21:56:34.710137+0000","last_became_active":"2023-11-28T21:35:25.790293+0000","last_became_peered":"2023-11-28T21:35:25.790293+0000","last_unstale":"2023-11-28T21:56:34.710137+0000","last_undegraded":"2023-11-28T21:56:34.710137+0000","last_fullsized":"2023-11-28T21:56:34.710137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:24:35.555501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30978138799999999,"stat_sum":{"num_bytes":36258609,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":208,"num_read_kb":8227,"num_write":1285,"num_write_kb":56637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"50'258","reported_seq":2340,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.689233+0000","last_change":"2023-11-28T21:40:43.068808+0000","last_active":"2023-11-28T21:55:38.689233+0000","last_peered":"2023-11-28T21:55:38.689233+0000","last_clean":"2023-11-28T21:55:38.689233+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:55:38.689233+0000","last_undegraded":"2023-11-28T21:55:38.689233+0000","last_fullsized":"2023-11-28T21:55:38.689233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:14:19.957018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22435063599999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"52'516","reported_seq":607,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.507909+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:56:18.507909+0000","last_peered":"2023-11-28T21:56:18.507909+0000","last_clean":"2023-11-28T21:56:18.507909+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:56:18.507909+0000","last_undegraded":"2023-11-28T21:56:18.507909+0000","last_fullsized":"2023-11-28T21:56:18.507909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:35.266650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":69670,"num_write":485,"num_write_kb":16676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"52'1329","reported_seq":3668,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.860899+0000","last_change":"2023-11-28T21:40:43.071209+0000","last_active":"2023-11-28T21:56:34.860899+0000","last_peered":"2023-11-28T21:56:34.860899+0000","last_clean":"2023-11-28T21:56:34.860899+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:56:34.860899+0000","last_undegraded":"2023-11-28T21:56:34.860899+0000","last_fullsized":"2023-11-28T21:56:34.860899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:12:44.679504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.2340457,"stat_sum":{"num_bytes":46614746,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":259,"num_read_kb":20536,"num_write":1294,"num_write_kb":83351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"50'255","reported_seq":2337,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.277131+0000","last_change":"2023-11-28T21:40:43.257554+0000","last_active":"2023-11-28T21:55:39.277131+0000","last_peered":"2023-11-28T21:55:39.277131+0000","last_clean":"2023-11-28T21:55:39.277131+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:55:39.277131+0000","last_undegraded":"2023-11-28T21:55:39.277131+0000","last_fullsized":"2023-11-28T21:55:39.277131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:57:00.431244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.228570471,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"52'990","reported_seq":1069,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.352661+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:56:08.352661+0000","last_peered":"2023-11-28T21:56:08.352661+0000","last_clean":"2023-11-28T21:56:08.352661+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:56:08.352661+0000","last_undegraded":"2023-11-28T21:56:08.352661+0000","last_fullsized":"2023-11-28T21:56:08.352661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":90124,"num_write":955,"num_write_kb":20985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"52'1334","reported_seq":3673,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.993668+0000","last_change":"2023-11-28T21:40:42.805595+0000","last_active":"2023-11-28T21:56:34.993668+0000","last_peered":"2023-11-28T21:56:34.993668+0000","last_clean":"2023-11-28T21:56:34.993668+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:56:34.993668+0000","last_undegraded":"2023-11-28T21:56:34.993668+0000","last_fullsized":"2023-11-28T21:56:34.993668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:49:48.909117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29518589499999998,"stat_sum":{"num_bytes":25457761,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":258,"num_read_kb":22214,"num_write":1317,"num_write_kb":78783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"50'290","reported_seq":2365,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:44.894160+0000","last_change":"2023-11-28T21:40:43.091522+0000","last_active":"2023-11-28T21:54:44.894160+0000","last_peered":"2023-11-28T21:54:44.894160+0000","last_clean":"2023-11-28T21:54:44.894160+0000","last_became_active":"2023-11-28T21:35:25.595386+0000","last_became_peered":"2023-11-28T21:35:25.595386+0000","last_unstale":"2023-11-28T21:54:44.894160+0000","last_undegraded":"2023-11-28T21:54:44.894160+0000","last_fullsized":"2023-11-28T21:54:44.894160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:35.597109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.148630489,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"52'109","reported_seq":167,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:25.127696+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:56:25.127696+0000","last_peered":"2023-11-28T21:56:25.127696+0000","last_clean":"2023-11-28T21:56:25.127696+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:56:25.127696+0000","last_undegraded":"2023-11-28T21:56:25.127696+0000","last_fullsized":"2023-11-28T21:56:25.127696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":67,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"52'1427","reported_seq":3771,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.994844+0000","last_change":"2023-11-28T21:40:42.795304+0000","last_active":"2023-11-28T21:56:34.994844+0000","last_peered":"2023-11-28T21:56:34.994844+0000","last_clean":"2023-11-28T21:56:34.994844+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:56:34.994844+0000","last_undegraded":"2023-11-28T21:56:34.994844+0000","last_fullsized":"2023-11-28T21:56:34.994844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:17.021758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28498848100000002,"stat_sum":{"num_bytes":33245588,"num_objects":317,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":263,"num_read_kb":28708,"num_write":1396,"num_write_kb":77612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"52'1309","reported_seq":3639,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.944713+0000","last_change":"2023-11-28T21:40:43.251187+0000","last_active":"2023-11-28T21:56:34.944713+0000","last_peered":"2023-11-28T21:56:34.944713+0000","last_clean":"2023-11-28T21:56:34.944713+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:56:34.944713+0000","last_undegraded":"2023-11-28T21:56:34.944713+0000","last_fullsized":"2023-11-28T21:56:34.944713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1309,"log_dups_size":0,"ondisk_log_size":1309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:05:16.989253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27353859400000002,"stat_sum":{"num_bytes":25285382,"num_objects":331,"num_object_clones":0,"num_object_copies":1324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":248,"num_read_kb":4117,"num_write":1287,"num_write_kb":58337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"50'274","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.681885+0000","last_change":"2023-11-28T21:40:42.876095+0000","last_active":"2023-11-28T21:55:38.681885+0000","last_peered":"2023-11-28T21:55:38.681885+0000","last_clean":"2023-11-28T21:55:38.681885+0000","last_became_active":"2023-11-28T21:35:25.589378+0000","last_became_peered":"2023-11-28T21:35:25.589378+0000","last_unstale":"2023-11-28T21:55:38.681885+0000","last_undegraded":"2023-11-28T21:55:38.681885+0000","last_fullsized":"2023-11-28T21:55:38.681885+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:58:22.995479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246426856,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"51'944","reported_seq":1031,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.709711+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:55:43.709711+0000","last_peered":"2023-11-28T21:55:43.709711+0000","last_clean":"2023-11-28T21:55:43.709711+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:55:43.709711+0000","last_undegraded":"2023-11-28T21:55:43.709711+0000","last_fullsized":"2023-11-28T21:55:43.709711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":897,"num_write_kb":25075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"52'1266","reported_seq":3553,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.472923+0000","last_change":"2023-11-28T21:40:42.918369+0000","last_active":"2023-11-28T21:56:34.472923+0000","last_peered":"2023-11-28T21:56:34.472923+0000","last_clean":"2023-11-28T21:56:34.472923+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:56:34.472923+0000","last_undegraded":"2023-11-28T21:56:34.472923+0000","last_fullsized":"2023-11-28T21:56:34.472923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1266,"log_dups_size":0,"ondisk_log_size":1266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:08:31.226473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.16839808100000001,"stat_sum":{"num_bytes":18660789,"num_objects":319,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":205,"num_read_kb":12320,"num_write":1249,"num_write_kb":51464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"51'304","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.271950+0000","last_change":"2023-11-28T21:40:42.800175+0000","last_active":"2023-11-28T21:55:39.271950+0000","last_peered":"2023-11-28T21:55:39.271950+0000","last_clean":"2023-11-28T21:55:39.271950+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:55:39.271950+0000","last_undegraded":"2023-11-28T21:55:39.271950+0000","last_fullsized":"2023-11-28T21:55:39.271950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:52.149789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28961016299999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"51'85","reported_seq":147,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273808+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:55:39.273808+0000","last_peered":"2023-11-28T21:55:39.273808+0000","last_clean":"2023-11-28T21:55:39.273808+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:55:39.273808+0000","last_undegraded":"2023-11-28T21:55:39.273808+0000","last_fullsized":"2023-11-28T21:55:39.273808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":63,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"52'1071","reported_seq":1162,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:13.995104+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:56:13.995104+0000","last_peered":"2023-11-28T21:56:13.995104+0000","last_clean":"2023-11-28T21:56:13.995104+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:56:13.995104+0000","last_undegraded":"2023-11-28T21:56:13.995104+0000","last_fullsized":"2023-11-28T21:56:13.995104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":146592,"num_write":1020,"num_write_kb":29113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"50'263","reported_seq":2344,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:39.891140+0000","last_change":"2023-11-28T21:40:43.254737+0000","last_active":"2023-11-28T21:54:39.891140+0000","last_peered":"2023-11-28T21:54:39.891140+0000","last_clean":"2023-11-28T21:54:39.891140+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:54:39.891140+0000","last_undegraded":"2023-11-28T21:54:39.891140+0000","last_fullsized":"2023-11-28T21:54:39.891140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:31:41.531767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26886027800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"52'1324","reported_seq":3622,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.710579+0000","last_change":"2023-11-28T21:40:43.102291+0000","last_active":"2023-11-28T21:56:34.710579+0000","last_peered":"2023-11-28T21:56:34.710579+0000","last_clean":"2023-11-28T21:56:34.710579+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:56:34.710579+0000","last_undegraded":"2023-11-28T21:56:34.710579+0000","last_fullsized":"2023-11-28T21:56:34.710579+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:46.549792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.13232148399999999,"stat_sum":{"num_bytes":46545345,"num_objects":341,"num_object_clones":0,"num_object_copies":1364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":218,"num_read_kb":9401,"num_write":1295,"num_write_kb":74278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"51'467","reported_seq":534,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.709564+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:55:43.709564+0000","last_peered":"2023-11-28T21:55:43.709564+0000","last_clean":"2023-11-28T21:55:43.709564+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:55:43.709564+0000","last_undegraded":"2023-11-28T21:55:43.709564+0000","last_fullsized":"2023-11-28T21:55:43.709564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":48896,"num_write":437,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"50'268","reported_seq":2349,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:39.947213+0000","last_change":"2023-11-28T21:40:43.054453+0000","last_active":"2023-11-28T21:54:39.947213+0000","last_peered":"2023-11-28T21:54:39.947213+0000","last_clean":"2023-11-28T21:54:39.947213+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:54:39.947213+0000","last_undegraded":"2023-11-28T21:54:39.947213+0000","last_fullsized":"2023-11-28T21:54:39.947213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:59.006875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24793844200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"52'1320","reported_seq":3657,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.943074+0000","last_change":"2023-11-28T21:40:42.807134+0000","last_active":"2023-11-28T21:56:34.943074+0000","last_peered":"2023-11-28T21:56:34.943074+0000","last_clean":"2023-11-28T21:56:34.943074+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:56:34.943074+0000","last_undegraded":"2023-11-28T21:56:34.943074+0000","last_fullsized":"2023-11-28T21:56:34.943074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1320,"log_dups_size":0,"ondisk_log_size":1320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:55.847688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29562192300000001,"stat_sum":{"num_bytes":29852008,"num_objects":333,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":257,"num_read_kb":28714,"num_write":1297,"num_write_kb":83698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"51'310","reported_seq":2393,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.116742+0000","last_change":"2023-11-28T21:40:42.917455+0000","last_active":"2023-11-28T21:55:30.116742+0000","last_peered":"2023-11-28T21:55:30.116742+0000","last_clean":"2023-11-28T21:55:30.116742+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:55:30.116742+0000","last_undegraded":"2023-11-28T21:55:30.116742+0000","last_fullsized":"2023-11-28T21:55:30.116742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:29.420871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18187520500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"50'313","reported_seq":2397,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.028477+0000","last_change":"2023-11-28T21:40:43.100474+0000","last_active":"2023-11-28T21:55:30.028477+0000","last_peered":"2023-11-28T21:55:30.028477+0000","last_clean":"2023-11-28T21:55:30.028477+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:55:30.028477+0000","last_undegraded":"2023-11-28T21:55:30.028477+0000","last_fullsized":"2023-11-28T21:55:30.028477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:44:47.346946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14391041299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"50'291","reported_seq":2375,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.201786+0000","last_change":"2023-11-28T21:40:42.831609+0000","last_active":"2023-11-28T21:55:30.201786+0000","last_peered":"2023-11-28T21:55:30.201786+0000","last_clean":"2023-11-28T21:55:30.201786+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:55:30.201786+0000","last_undegraded":"2023-11-28T21:55:30.201786+0000","last_fullsized":"2023-11-28T21:55:30.201786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:49:50.710370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30621145,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"50'274","reported_seq":2357,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:35.057693+0000","last_change":"2023-11-28T21:40:43.255460+0000","last_active":"2023-11-28T21:55:35.057693+0000","last_peered":"2023-11-28T21:55:35.057693+0000","last_clean":"2023-11-28T21:55:35.057693+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:55:35.057693+0000","last_undegraded":"2023-11-28T21:55:35.057693+0000","last_fullsized":"2023-11-28T21:55:35.057693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:35:27.179438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26967866099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"50'262","reported_seq":2345,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.274127+0000","last_change":"2023-11-28T21:40:42.801894+0000","last_active":"2023-11-28T21:55:39.274127+0000","last_peered":"2023-11-28T21:55:39.274127+0000","last_clean":"2023-11-28T21:55:39.274127+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:55:39.274127+0000","last_undegraded":"2023-11-28T21:55:39.274127+0000","last_fullsized":"2023-11-28T21:55:39.274127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:46:20.949106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29058787400000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"50'280","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.682440+0000","last_change":"2023-11-28T21:40:42.886237+0000","last_active":"2023-11-28T21:55:38.682440+0000","last_peered":"2023-11-28T21:55:38.682440+0000","last_clean":"2023-11-28T21:55:38.682440+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:55:38.682440+0000","last_undegraded":"2023-11-28T21:55:38.682440+0000","last_fullsized":"2023-11-28T21:55:38.682440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:33.044587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246741036,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"50'305","reported_seq":2388,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.281023+0000","last_change":"2023-11-28T21:40:42.916517+0000","last_active":"2023-11-28T21:55:39.281023+0000","last_peered":"2023-11-28T21:55:39.281023+0000","last_clean":"2023-11-28T21:55:39.281023+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:55:39.281023+0000","last_undegraded":"2023-11-28T21:55:39.281023+0000","last_fullsized":"2023-11-28T21:55:39.281023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:26.983521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18459727300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"50'272","reported_seq":2355,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273021+0000","last_change":"2023-11-28T21:40:42.803865+0000","last_active":"2023-11-28T21:55:39.273021+0000","last_peered":"2023-11-28T21:55:39.273021+0000","last_clean":"2023-11-28T21:55:39.273021+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:55:39.273021+0000","last_undegraded":"2023-11-28T21:55:39.273021+0000","last_fullsized":"2023-11-28T21:55:39.273021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:06:41.883126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29272911699999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"50'274","reported_seq":2358,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.691065+0000","last_change":"2023-11-28T21:40:42.823937+0000","last_active":"2023-11-28T21:55:38.691065+0000","last_peered":"2023-11-28T21:55:38.691065+0000","last_clean":"2023-11-28T21:55:38.691065+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:55:38.691065+0000","last_undegraded":"2023-11-28T21:55:38.691065+0000","last_fullsized":"2023-11-28T21:55:38.691065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:18:46.016564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30579284099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"50'260","reported_seq":2342,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.047770+0000","last_change":"2023-11-28T21:40:42.799326+0000","last_active":"2023-11-28T21:54:55.047770+0000","last_peered":"2023-11-28T21:54:55.047770+0000","last_clean":"2023-11-28T21:54:55.047770+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:54:55.047770+0000","last_undegraded":"2023-11-28T21:54:55.047770+0000","last_fullsized":"2023-11-28T21:54:55.047770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:13.718530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28793565599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"50'276","reported_seq":2359,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.194766+0000","last_change":"2023-11-28T21:40:42.831259+0000","last_active":"2023-11-28T21:54:55.194766+0000","last_peered":"2023-11-28T21:54:55.194766+0000","last_clean":"2023-11-28T21:54:55.194766+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:54:55.194766+0000","last_undegraded":"2023-11-28T21:54:55.194766+0000","last_fullsized":"2023-11-28T21:54:55.194766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:47:17.850510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30636868099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"50'277","reported_seq":2359,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.052784+0000","last_change":"2023-11-28T21:40:43.253941+0000","last_active":"2023-11-28T21:54:55.052784+0000","last_peered":"2023-11-28T21:54:55.052784+0000","last_clean":"2023-11-28T21:54:55.052784+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:54:55.052784+0000","last_undegraded":"2023-11-28T21:54:55.052784+0000","last_fullsized":"2023-11-28T21:54:55.052784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:50:38.526480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26835999100000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"50'280","reported_seq":2362,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.110539+0000","last_change":"2023-11-28T21:40:42.910839+0000","last_active":"2023-11-28T21:54:55.110539+0000","last_peered":"2023-11-28T21:54:55.110539+0000","last_clean":"2023-11-28T21:54:55.110539+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:54:55.110539+0000","last_undegraded":"2023-11-28T21:54:55.110539+0000","last_fullsized":"2023-11-28T21:54:55.110539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:09:31.418117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.194469013,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"50'281","reported_seq":2364,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:55.116526+0000","last_change":"2023-11-28T21:40:42.749349+0000","last_active":"2023-11-28T21:54:55.116526+0000","last_peered":"2023-11-28T21:54:55.116526+0000","last_clean":"2023-11-28T21:54:55.116526+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:54:55.116526+0000","last_undegraded":"2023-11-28T21:54:55.116526+0000","last_fullsized":"2023-11-28T21:54:55.116526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:20.196676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21943309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"50'253","reported_seq":2336,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273320+0000","last_change":"2023-11-28T21:40:43.092423+0000","last_active":"2023-11-28T21:55:39.273320+0000","last_peered":"2023-11-28T21:55:39.273320+0000","last_clean":"2023-11-28T21:55:39.273320+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:55:39.273320+0000","last_undegraded":"2023-11-28T21:55:39.273320+0000","last_fullsized":"2023-11-28T21:55:39.273320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:33:34.736900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14941532399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"50'295","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.693133+0000","last_change":"2023-11-28T21:40:42.827330+0000","last_active":"2023-11-28T21:55:38.693133+0000","last_peered":"2023-11-28T21:55:38.693133+0000","last_clean":"2023-11-28T21:55:38.693133+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:55:38.693133+0000","last_undegraded":"2023-11-28T21:55:38.693133+0000","last_fullsized":"2023-11-28T21:55:38.693133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:49.288912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30374592,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"50'286","reported_seq":2369,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.690628+0000","last_change":"2023-11-28T21:40:42.820428+0000","last_active":"2023-11-28T21:55:38.690628+0000","last_peered":"2023-11-28T21:55:38.690628+0000","last_clean":"2023-11-28T21:55:38.690628+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:55:38.690628+0000","last_undegraded":"2023-11-28T21:55:38.690628+0000","last_fullsized":"2023-11-28T21:55:38.690628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:59.333901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30238551699999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"50'298","reported_seq":2381,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.682596+0000","last_change":"2023-11-28T21:40:42.876664+0000","last_active":"2023-11-28T21:55:38.682596+0000","last_peered":"2023-11-28T21:55:38.682596+0000","last_clean":"2023-11-28T21:55:38.682596+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:55:38.682596+0000","last_undegraded":"2023-11-28T21:55:38.682596+0000","last_fullsized":"2023-11-28T21:55:38.682596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:02:12.323355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246777987,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"50'284","reported_seq":2366,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.275704+0000","last_change":"2023-11-28T21:40:43.257791+0000","last_active":"2023-11-28T21:55:39.275704+0000","last_peered":"2023-11-28T21:55:39.275704+0000","last_clean":"2023-11-28T21:55:39.275704+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:55:39.275704+0000","last_undegraded":"2023-11-28T21:55:39.275704+0000","last_fullsized":"2023-11-28T21:55:39.275704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:26:58.247393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21399896700000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"50'254","reported_seq":2336,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.272944+0000","last_change":"2023-11-28T21:40:42.798449+0000","last_active":"2023-11-28T21:55:39.272944+0000","last_peered":"2023-11-28T21:55:39.272944+0000","last_clean":"2023-11-28T21:55:39.272944+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:55:39.272944+0000","last_undegraded":"2023-11-28T21:55:39.272944+0000","last_fullsized":"2023-11-28T21:55:39.272944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:01:12.073001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.287656776,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"50'245","reported_seq":2327,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.689195+0000","last_change":"2023-11-28T21:40:43.071081+0000","last_active":"2023-11-28T21:55:38.689195+0000","last_peered":"2023-11-28T21:55:38.689195+0000","last_clean":"2023-11-28T21:55:38.689195+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:55:38.689195+0000","last_undegraded":"2023-11-28T21:55:38.689195+0000","last_fullsized":"2023-11-28T21:55:38.689195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:01:10.753163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.222804104,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"50'240","reported_seq":2322,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.280173+0000","last_change":"2023-11-28T21:40:42.914448+0000","last_active":"2023-11-28T21:55:39.280173+0000","last_peered":"2023-11-28T21:55:39.280173+0000","last_clean":"2023-11-28T21:55:39.280173+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:55:39.280173+0000","last_undegraded":"2023-11-28T21:55:39.280173+0000","last_fullsized":"2023-11-28T21:55:39.280173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:33:37.008462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19768582900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"50'263","reported_seq":2345,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.280285+0000","last_change":"2023-11-28T21:40:42.916248+0000","last_active":"2023-11-28T21:55:39.280285+0000","last_peered":"2023-11-28T21:55:39.280285+0000","last_clean":"2023-11-28T21:55:39.280285+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:55:39.280285+0000","last_undegraded":"2023-11-28T21:55:39.280285+0000","last_fullsized":"2023-11-28T21:55:39.280285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:17.635492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18434383600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"50'296","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.727303+0000","last_change":"2023-11-28T21:40:42.754000+0000","last_active":"2023-11-28T21:55:38.727303+0000","last_peered":"2023-11-28T21:55:38.727303+0000","last_clean":"2023-11-28T21:55:38.727303+0000","last_became_active":"2023-11-28T21:35:25.598839+0000","last_became_peered":"2023-11-28T21:35:25.598839+0000","last_unstale":"2023-11-28T21:55:38.727303+0000","last_undegraded":"2023-11-28T21:55:38.727303+0000","last_fullsized":"2023-11-28T21:55:38.727303+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:51.550099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22244017299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"50'275","reported_seq":2356,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:39.912883+0000","last_change":"2023-11-28T21:40:42.796731+0000","last_active":"2023-11-28T21:54:39.912883+0000","last_peered":"2023-11-28T21:54:39.912883+0000","last_clean":"2023-11-28T21:54:39.912883+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:54:39.912883+0000","last_undegraded":"2023-11-28T21:54:39.912883+0000","last_fullsized":"2023-11-28T21:54:39.912883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:50.215245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28637254899999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"50'278","reported_seq":2360,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.112630+0000","last_change":"2023-11-28T21:40:42.745933+0000","last_active":"2023-11-28T21:54:40.112630+0000","last_peered":"2023-11-28T21:54:40.112630+0000","last_clean":"2023-11-28T21:54:40.112630+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:54:40.112630+0000","last_undegraded":"2023-11-28T21:54:40.112630+0000","last_fullsized":"2023-11-28T21:54:40.112630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:52.720487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.219142752,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"50'266","reported_seq":2348,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147553+0000","last_change":"2023-11-28T21:40:42.831458+0000","last_active":"2023-11-28T21:54:40.147553+0000","last_peered":"2023-11-28T21:54:40.147553+0000","last_clean":"2023-11-28T21:54:40.147553+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:54:40.147553+0000","last_undegraded":"2023-11-28T21:54:40.147553+0000","last_fullsized":"2023-11-28T21:54:40.147553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:33:21.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.307113203,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"50'282","reported_seq":2364,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.112675+0000","last_change":"2023-11-28T21:40:42.736971+0000","last_active":"2023-11-28T21:54:40.112675+0000","last_peered":"2023-11-28T21:54:40.112675+0000","last_clean":"2023-11-28T21:54:40.112675+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:54:40.112675+0000","last_undegraded":"2023-11-28T21:54:40.112675+0000","last_fullsized":"2023-11-28T21:54:40.112675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:33:12.599313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21561430200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"51'275","reported_seq":2359,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:54:40.147650+0000","last_change":"2023-11-28T21:40:42.829941+0000","last_active":"2023-11-28T21:54:40.147650+0000","last_peered":"2023-11-28T21:54:40.147650+0000","last_clean":"2023-11-28T21:54:40.147650+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:54:40.147650+0000","last_undegraded":"2023-11-28T21:54:40.147650+0000","last_fullsized":"2023-11-28T21:54:40.147650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:26:40.360501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30486287499999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"50'270","reported_seq":2353,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.056979+0000","last_change":"2023-11-28T21:40:43.239342+0000","last_active":"2023-11-28T21:55:30.056979+0000","last_peered":"2023-11-28T21:55:30.056979+0000","last_clean":"2023-11-28T21:55:30.056979+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:55:30.056979+0000","last_undegraded":"2023-11-28T21:55:30.056979+0000","last_fullsized":"2023-11-28T21:55:30.056979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:08.206168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26474323500000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"50'278","reported_seq":2362,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:29.999520+0000","last_change":"2023-11-28T21:40:42.885846+0000","last_active":"2023-11-28T21:55:29.999520+0000","last_peered":"2023-11-28T21:55:29.999520+0000","last_clean":"2023-11-28T21:55:29.999520+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:55:29.999520+0000","last_undegraded":"2023-11-28T21:55:29.999520+0000","last_fullsized":"2023-11-28T21:55:29.999520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:19:35.299155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24685295800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"50'271","reported_seq":2355,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-11-28T21:55:30.124004+0000","last_change":"2023-11-28T21:40:42.753421+0000","last_active":"2023-11-28T21:55:30.124004+0000","last_peered":"2023-11-28T21:55:30.124004+0000","last_clean":"2023-11-28T21:55:30.124004+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:55:30.124004+0000","last_undegraded":"2023-11-28T21:55:30.124004+0000","last_fullsized":"2023-11-28T21:55:30.124004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:06:35.683601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22257470400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":927863280,"num_objects":10339,"num_object_clones":0,"num_object_copies":41356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10339,"num_whiteouts":0,"num_read":7542,"num_read_kb":441308,"num_write":41595,"num_write_kb":1955409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1822175232,"data_stored":1974226944,"data_compressed":124276003,"data_compressed_allocated":140005376,"data_compressed_original":292057088,"omap_allocated":0,"internal_metadata":0},"log_size":42424,"ondisk_log_size":42424,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17712,"ondisk_log_size":17712,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":351660858,"num_objects":207,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":207,"num_whiteouts":0,"num_read":1486,"num_read_kb":1965797,"num_write":14766,"num_write_kb":396301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192471040,"data_stored":703321716,"data_compressed":93812456,"data_compressed_allocated":121569280,"data_compressed_original":632577942,"omap_allocated":0,"internal_metadata":0},"log_size":15799,"ondisk_log_size":15799,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575118,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342780,"kb_used_data":213044,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94029060,"statfs":{"total":96636764160,"available":96285757440,"internally_reserved":0,"allocated":218157056,"data_stored":304451419,"data_compressed":25898147,"data_compressed_allocated":31465472,"data_compressed_original":118067200,"omap_allocated":1590,"internal_metadata":132839882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,10,6,4,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.97799999999999998},"min":{"1min":0.51400000000000001,"5min":0.41399999999999998,"15min":0.35299999999999998},"max":{"1min":1.232,"5min":5.056,"15min":47.475000000000001},"last":3.3719999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.82299999999999995},"min":{"1min":0.54200000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.145,"5min":4.069,"15min":5.3310000000000004},"last":1.8260000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.98999999999999999,"15min":1.099},"min":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":2.5630000000000002,"15min":47.529000000000003},"last":1.79},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.083,"15min":1.008},"min":{"1min":0.64300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.171,"5min":17.460000000000001,"15min":17.460000000000001},"last":5.734}]},{"osd":2,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.86599999999999999,"15min":0.91200000000000003},"min":{"1min":0.58399999999999996,"5min":0.504,"15min":0.47299999999999998},"max":{"1min":1.0660000000000001,"5min":2.9359999999999999,"15min":5.5140000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0229999999999999,"15min":1.022},"min":{"1min":0.70199999999999996,"5min":0.60499999999999998,"15min":0.498},"max":{"1min":1.248,"5min":2.972,"15min":5.5979999999999999},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1719999999999999,"15min":1.179},"min":{"1min":0.63800000000000001,"5min":0.57299999999999995,"15min":0.44600000000000001},"max":{"1min":1.196,"5min":16.113,"15min":47.57},"last":7.3760000000000003},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.171,"15min":1.0649999999999999},"min":{"1min":0.73599999999999999,"5min":0.65400000000000003,"15min":0.53800000000000003},"max":{"1min":1.2749999999999999,"5min":11.148999999999999,"15min":12.273},"last":7.4269999999999996}]},{"osd":4,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.1180000000000001,"15min":1.054},"min":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.35499999999999998},"max":{"1min":1.1990000000000001,"5min":17.542000000000002,"15min":17.542000000000002},"last":5.6859999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.0489999999999999,"15min":1.196},"min":{"1min":0.84699999999999998,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.363,"5min":1.5329999999999999,"15min":49.106000000000002},"last":0.95799999999999996}]},{"osd":5,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.1220000000000001,"15min":1.2909999999999999},"min":{"1min":0.80600000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.345,"5min":6.6420000000000003,"15min":64.855999999999995},"last":2.0110000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.1679999999999999,"15min":1.1970000000000001},"min":{"1min":0.73499999999999999,"5min":0.71399999999999997,"15min":0.56399999999999995},"max":{"1min":1.2689999999999999,"5min":11.327,"15min":16.753},"last":5.843}]},{"osd":6,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.194,"15min":1.1639999999999999},"min":{"1min":0.88600000000000001,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":1.1930000000000001,"5min":11.641999999999999,"15min":11.641999999999999},"last":5.7679999999999998},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1399999999999999,"15min":1.153},"min":{"1min":0.97999999999999998,"5min":0.61499999999999999,"15min":0.502},"max":{"1min":1.4199999999999999,"5min":1.474,"15min":7.4909999999999997},"last":5.4210000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575116,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":380904,"kb_used_data":255328,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":93990936,"statfs":{"total":96636764160,"available":96246718464,"internally_reserved":0,"allocated":261455872,"data_stored":340022166,"data_compressed":25161614,"data_compressed_allocated":30015488,"data_compressed_original":108883915,"omap_allocated":1594,"internal_metadata":128580038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":2.9159999999999999,"5min":1.248,"15min":1.0109999999999999},"min":{"1min":0.51500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":40.999000000000002,"5min":40.999000000000002,"15min":40.999000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.05,"15min":1.036},"min":{"1min":0.58299999999999996,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":14.132999999999999,"5min":14.132999999999999,"15min":24.399999999999999},"last":1.1000000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.89500000000000002,"15min":0.88700000000000001},"min":{"1min":0.71999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2769999999999999,"5min":2.2400000000000002,"15min":11.196},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.90800000000000003,"15min":0.95899999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.35799999999999998},"max":{"1min":1.1759999999999999,"5min":6.1550000000000002,"15min":7.9130000000000003},"last":1.0669999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.89500000000000002,"15min":0.94799999999999995},"min":{"1min":0.66300000000000003,"5min":0.502,"15min":0.39100000000000001},"max":{"1min":3.855,"5min":3.855,"15min":21.986999999999998},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.92800000000000005,"15min":0.99099999999999999},"min":{"1min":0.621,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2829999999999999,"5min":1.419,"15min":8.6170000000000009},"last":0.72099999999999997}]},{"osd":3,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.94799999999999995,"15min":0.96299999999999997},"min":{"1min":0.57099999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.8439999999999999,"5min":3.2679999999999998,"15min":11.239000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0069999999999999,"15min":1.087},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.7959999999999998,"5min":3.29,"15min":24.745000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91800000000000004,"15min":1.014},"min":{"1min":0.71599999999999997,"5min":0.32300000000000001,"15min":0.30299999999999999},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":16.015000000000001},"last":1.135},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.044,"15min":1.079},"min":{"1min":0.69499999999999995,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":13.193},"last":1.2130000000000001}]},{"osd":5,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.98099999999999998,"15min":1.109},"min":{"1min":0.65000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":2.069,"5min":2.069,"15min":25.006},"last":1.25},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0629999999999999,"15min":1.1419999999999999},"min":{"1min":0.46700000000000003,"5min":0.33800000000000002,"15min":0.33000000000000002},"max":{"1min":1.405,"5min":1.4790000000000001,"15min":7.843},"last":1.1579999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.139},"min":{"1min":0.79200000000000004,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":1.1950000000000001,"5min":1.9179999999999999,"15min":7.883},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.177,"15min":1.276},"min":{"1min":0.93300000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.3169999999999999,"5min":1.9610000000000001,"15min":27.942},"last":1.478}]}]},{"osd":5,"up_from":13,"seq":55834575116,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":366316,"kb_used_data":240740,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":94005524,"statfs":{"total":96636764160,"available":96261656576,"internally_reserved":0,"allocated":246517760,"data_stored":342867931,"data_compressed":30547202,"data_compressed_allocated":36597760,"data_compressed_original":133251072,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,3,1,0,0,0,50],"upper_bound":1024},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.99399999999999999,"15min":0.89800000000000002},"min":{"1min":0.47399999999999998,"5min":0.46500000000000002,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":15.005000000000001,"15min":15.005000000000001},"last":3.3980000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88800000000000001},"min":{"1min":0.501,"5min":0.47799999999999998,"15min":0.46700000000000003},"max":{"1min":1.2290000000000001,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":3.419}]},{"osd":1,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81100000000000005,"15min":0.83999999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.48199999999999998},"max":{"1min":0.89800000000000002,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":3.806},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.88900000000000001,"15min":0.91300000000000003},"min":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.496},"max":{"1min":1.0609999999999999,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":7.4930000000000003}]},{"osd":2,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.016,"15min":0.97399999999999998},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.105,"5min":14.775,"15min":14.775},"last":1.131},{"interface":"front","average":{"1min":1.016,"5min":1.1040000000000001,"15min":0.94599999999999995},"min":{"1min":0.72899999999999998,"5min":0.63800000000000001,"15min":0.57499999999999996},"max":{"1min":1.1919999999999999,"5min":14.901999999999999,"15min":14.901999999999999},"last":1.25}]},{"osd":3,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.034,"15min":0.98899999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.54400000000000004},"max":{"1min":1.2529999999999999,"5min":15.154,"15min":15.154},"last":1.347},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.98999999999999999,"15min":1.0109999999999999},"min":{"1min":0.63900000000000001,"5min":0.60799999999999998,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":3.4289999999999998,"15min":5.5170000000000003},"last":1.216}]},{"osd":4,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.526,"15min":1.181},"min":{"1min":0.64600000000000002,"5min":0.59299999999999997,"15min":0.51400000000000001},"max":{"1min":1.292,"5min":61.020000000000003,"15min":61.020000000000003},"last":2.98},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.143,"15min":1.0920000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2490000000000001,"5min":15.085000000000001,"15min":15.085000000000001},"last":8.6029999999999998}]},{"osd":6,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.139,"15min":1.123},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.56699999999999995},"max":{"1min":1.5469999999999999,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":1.151},{"interface":"front","average":{"1min":1.216,"5min":1.194,"15min":1.1699999999999999},"min":{"1min":1.052,"5min":0.72499999999999998,"15min":0.48899999999999999},"max":{"1min":1.6799999999999999,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.1830000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1759999999999999,"15min":1.1639999999999999},"min":{"1min":0.94099999999999995,"5min":0.75600000000000001,"15min":0.56899999999999995},"max":{"1min":1.4219999999999999,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.3200000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.054,"15min":1.0189999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.33200000000000002},"max":{"1min":1.085,"5min":6.2869999999999999,"15min":6.2869999999999999},"last":1.6779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575116,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":358356,"kb_used_data":236936,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":94013484,"statfs":{"total":96636764160,"available":96269807616,"internally_reserved":0,"allocated":242622464,"data_stored":338272538,"data_compressed":27596131,"data_compressed_allocated":33427456,"data_compressed_original":129380299,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,0,0,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":60,"apply_latency_ms":60,"commit_latency_ns":60000000,"apply_latency_ns":60000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.155,"15min":0.97399999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0449999999999999,"5min":43.139000000000003,"15min":43.139000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84399999999999997,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.39900000000000002},"max":{"1min":1.3049999999999999,"5min":4.444,"15min":17.446000000000002},"last":3.9809999999999999}]},{"osd":1,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.98799999999999999,"15min":1.054},"min":{"1min":0.64900000000000002,"5min":0.48199999999999998,"15min":0.44},"max":{"1min":1.2849999999999999,"5min":11.557,"15min":21.763999999999999},"last":2.3780000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0229999999999999,"15min":1.4790000000000001},"min":{"1min":0.65700000000000003,"5min":0.53800000000000003,"15min":0.46500000000000002},"max":{"1min":1.452,"5min":5.3479999999999999,"15min":78.649000000000001},"last":2.4329999999999998}]},{"osd":2,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":0.96499999999999997,"15min":1.0529999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.880000000000001,"5min":10.880000000000001,"15min":28.341000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.073,"15min":1.548},"min":{"1min":0.61699999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"max":{"1min":2.069,"5min":11.614000000000001,"15min":94.998999999999995},"last":1.002}]},{"osd":3,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.208,"15min":1.103},"min":{"1min":0.86299999999999999,"5min":0.67500000000000004,"15min":0.51300000000000001},"max":{"1min":9.984,"5min":9.984,"15min":13.202999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.0609999999999999,"15min":1.056},"min":{"1min":0.69299999999999995,"5min":0.54800000000000004,"15min":0.45600000000000002},"max":{"1min":9.952,"5min":9.952,"15min":16.277999999999999},"last":2.4900000000000002}]},{"osd":5,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.0049999999999999,"15min":1.1259999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.28199999999999997},"max":{"1min":1.385,"5min":3.4409999999999998,"15min":17.033000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.3799999999999999,"15min":1.3},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.34000000000000002},"max":{"1min":1.1759999999999999,"5min":38.939,"15min":38.939},"last":0.47299999999999998}]},{"osd":6,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.046,"15min":1.1699999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.37},"max":{"1min":1.4710000000000001,"5min":4.476,"15min":17.085000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.026,"5min":1.4850000000000001,"15min":1.3380000000000001},"min":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.39000000000000001},"max":{"1min":1.631,"5min":43.420000000000002,"15min":43.420000000000002},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.121,"15min":1.6160000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.309},"max":{"1min":1.5269999999999999,"5min":3.2109999999999999,"15min":77.638999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.198,"5min":1.24,"15min":1.175},"min":{"1min":0.90200000000000002,"5min":0.75900000000000001,"15min":0.51500000000000001},"max":{"1min":1.798,"5min":3.613,"15min":5.6200000000000001},"last":0.61699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575117,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457756,"kb_used_data":304404,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":93914084,"statfs":{"total":96636764160,"available":96168022016,"internally_reserved":0,"allocated":311709696,"data_stored":408700527,"data_compressed":36249891,"data_compressed_allocated":42946560,"data_compressed_original":140238848,"omap_allocated":1592,"internal_metadata":157022664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.89500000000000002,"15min":0.70999999999999996},"min":{"1min":0.246,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.67900000000000005,"5min":36.585000000000001,"15min":36.585000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.89500000000000002,"15min":0.67800000000000005},"min":{"1min":0.379,"5min":0.34300000000000003,"15min":0.33100000000000002},"max":{"1min":0.68400000000000005,"5min":30.065000000000001,"15min":30.065000000000001},"last":0.46500000000000002}]},{"osd":1,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.59999999999999998},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.26300000000000001},"max":{"1min":0.876,"5min":4.8099999999999996,"15min":10.449},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.61799999999999999,"15min":0.67400000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.98599999999999999,"5min":4.8520000000000003,"15min":10.481999999999999},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59099999999999997,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.27900000000000003},"max":{"1min":0.70299999999999996,"5min":1.25,"15min":10.369},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.56799999999999995,"15min":0.69599999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78600000000000003,"5min":1.1240000000000001,"15min":10.406000000000001},"last":0.40400000000000003}]},{"osd":4,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91700000000000004,"15min":1.361},"min":{"1min":0.71799999999999997,"5min":0.50700000000000001,"15min":0.376},"max":{"1min":1.0129999999999999,"5min":4.2119999999999997,"15min":110.39700000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.147,"5min":7.2030000000000003,"15min":108.777},"last":0.83699999999999997}]},{"osd":5,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.86199999999999999,"15min":0.92600000000000005},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.39900000000000002},"max":{"1min":0.98799999999999999,"5min":7.2679999999999998,"15min":17.513000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.873,"15min":0.92900000000000005},"min":{"1min":0.621,"5min":0.621,"15min":0.52300000000000002},"max":{"1min":0.998,"5min":1.7070000000000001,"15min":10.516999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82399999999999995,"15min":1.4019999999999999},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.365,"15min":138.91300000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.875,"15min":0.94499999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.48299999999999998},"max":{"1min":1.0980000000000001,"5min":1.766,"15min":10.302},"last":0.88600000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.91900000000000004,"15min":0.95799999999999996},"min":{"1min":0.63100000000000001,"5min":0.60999999999999999,"15min":0.45200000000000001},"max":{"1min":1.1990000000000001,"5min":7.3040000000000003,"15min":17.263000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.97799999999999998,"15min":1.034},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.49299999999999999},"max":{"1min":1.0289999999999999,"5min":7.3339999999999996,"15min":28.613},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575115,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":285968,"kb_used_data":174280,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94085872,"statfs":{"total":96636764160,"available":96343932928,"internally_reserved":0,"allocated":178462720,"data_stored":223929989,"data_compressed":15805945,"data_compressed_allocated":18817024,"data_compressed_original":64589824,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,0,0,0,0,26],"upper_bound":1024},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.623,"15min":0.57999999999999996},"min":{"1min":0.29199999999999998,"5min":0.26800000000000002,"15min":0.23599999999999999},"max":{"1min":0.58299999999999996,"5min":13.599,"15min":13.599},"last":1.6679999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.60699999999999998},"min":{"1min":0.34100000000000003,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":8.0630000000000006,"15min":8.0630000000000006},"last":7.1159999999999997}]},{"osd":1,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.53300000000000003,"15min":0.57499999999999996},"min":{"1min":0.33400000000000002,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.61799999999999999,"5min":3.0760000000000001,"15min":9.3260000000000005},"last":7.8970000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.70699999999999996,"15min":0.67000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":5.6660000000000004,"15min":7.4779999999999998},"last":10.657}]},{"osd":3,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.95199999999999996,"15min":0.79200000000000004},"min":{"1min":0.42099999999999999,"5min":0.32300000000000001,"15min":0.307},"max":{"1min":0.878,"5min":29.123999999999999,"15min":29.123999999999999},"last":7.1639999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.63100000000000001,"15min":0.59699999999999998},"min":{"1min":0.40000000000000002,"5min":0.312,"15min":0.28000000000000003},"max":{"1min":0.754,"5min":5.5410000000000004,"15min":7.4390000000000001},"last":8.8780000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.94099999999999995,"15min":0.92300000000000004},"min":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.49099999999999999},"max":{"1min":1.0489999999999999,"5min":5.6289999999999996,"15min":7.3630000000000004},"last":2.5630000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.87,"15min":0.879},"min":{"1min":0.60599999999999998,"5min":0.51200000000000001,"15min":0.41399999999999998},"max":{"1min":0.98599999999999999,"5min":5.5490000000000004,"15min":9.2289999999999992},"last":1.518}]},{"osd":5,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.85499999999999998,"15min":0.84299999999999997},"min":{"1min":0.622,"5min":0.501,"15min":0.501},"max":{"1min":1.01,"5min":3.3610000000000002,"15min":3.3610000000000002},"last":6.875},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.0109999999999999,"15min":0.93700000000000006},"min":{"1min":0.67300000000000004,"5min":0.59399999999999997,"15min":0.48399999999999999},"max":{"1min":1.266,"5min":6.343,"15min":7.2649999999999997},"last":8.7110000000000003}]},{"osd":6,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.0489999999999999,"15min":0.91800000000000004},"min":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":1.044,"5min":18.959,"15min":18.959},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.1339999999999999,"15min":0.97499999999999998},"min":{"1min":0.68300000000000005,"5min":0.622,"15min":0.52200000000000002},"max":{"1min":1.0700000000000001,"5min":21.957999999999998,"15min":21.957999999999998},"last":8.8420000000000005}]},{"osd":7,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89700000000000002,"15min":0.92100000000000004},"min":{"1min":0.68400000000000005,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":0.999,"5min":3.5299999999999998,"15min":9.2799999999999994},"last":8.7949999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.93300000000000005,"15min":0.90900000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.57099999999999995},"max":{"1min":1.042,"5min":2.6379999999999999,"15min":3.4470000000000001},"last":8.7569999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575115,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472320,"kb_used_data":314808,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93899520,"statfs":{"total":96636764160,"available":96153108480,"internally_reserved":0,"allocated":322363392,"data_stored":416685636,"data_compressed":32711569,"data_compressed_allocated":39284736,"data_compressed_original":133918720,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":0.75700000000000001,"15min":0.69599999999999995},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.246},"max":{"1min":10.096,"5min":12.311999999999999,"15min":14.538},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.623,"15min":0.65100000000000002},"min":{"1min":0.39000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.4119999999999999,"5min":8.2479999999999993,"15min":14.618},"last":0.57199999999999995}]},{"osd":2,"last update":"Tue Nov 28 21:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.66100000000000003,"15min":0.63100000000000001},"min":{"1min":0.40799999999999997,"5min":0.34899999999999998,"15min":0.31900000000000001},"max":{"1min":1.7150000000000001,"5min":5.1689999999999996,"15min":8.4749999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":4.7030000000000003,"15min":8.5150000000000006},"last":0.40600000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.75900000000000001,"15min":0.84699999999999998},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.27600000000000002},"max":{"1min":3.028,"5min":13.385,"15min":43.536999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.84699999999999998},"min":{"1min":0.501,"5min":0.443,"15min":0.375},"max":{"1min":5.6050000000000004,"5min":8.2899999999999991,"15min":18.308},"last":0.67900000000000005}]},{"osd":4,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":0.997,"15min":0.95699999999999996},"min":{"1min":0.63500000000000001,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":9.9849999999999994,"5min":9.9849999999999994,"15min":9.9849999999999994},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.4670000000000001,"5min":0.97899999999999998,"15min":1.079},"min":{"1min":0.61699999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":10.287000000000001,"5min":10.287000000000001,"15min":43.497999999999998},"last":0.69899999999999995}]},{"osd":5,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.93100000000000005,"15min":0.91600000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":5.5350000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.016,"15min":0.99399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.448,"5min":3.448,"15min":5.7409999999999997},"last":0.92600000000000005}]},{"osd":6,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.0449999999999999,"15min":1.0109999999999999},"min":{"1min":0.69899999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":10.023,"5min":10.023,"15min":10.385},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.05,"15min":1.2470000000000001},"min":{"1min":0.77700000000000002,"5min":0.65100000000000002,"15min":0.55200000000000005},"max":{"1min":3.7480000000000002,"5min":5.9349999999999996,"15min":43.448999999999998},"last":0.86299999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0469999999999999,"15min":1.024},"min":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":5.4109999999999996,"5min":5.9779999999999998,"15min":7.96},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.516,"5min":1.1899999999999999,"15min":1.1639999999999999},"min":{"1min":0.66500000000000004,"5min":0.55800000000000005,"15min":0.51500000000000001},"max":{"1min":10.063000000000001,"5min":15.672000000000001,"15min":42.473999999999997},"last":0.66100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575116,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":344096,"kb_used_data":214360,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94027744,"statfs":{"total":96636764160,"available":96284409856,"internally_reserved":0,"allocated":219504640,"data_stored":281144114,"data_compressed":19744060,"data_compressed_allocated":23732224,"data_compressed_original":85671936,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"min":{"1min":0.29499999999999998,"5min":0.27800000000000002,"15min":0.215},"max":{"1min":1.581,"5min":1.958,"15min":3.3929999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.60199999999999998,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.27700000000000002,"15min":0.248},"max":{"1min":4.5739999999999998,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.64900000000000002,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.4489999999999998,"5min":5.4489999999999998,"15min":5.8559999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":3.0840000000000001,"15min":5.4210000000000003},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.77700000000000002,"15min":0.623},"min":{"1min":0.30599999999999999,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":5.0019999999999998,"5min":16.187999999999999,"15min":16.187999999999999},"last":0.495},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.69199999999999995},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.5410000000000004,"5min":15.004,"15min":15.004},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":0.93999999999999995,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.42899999999999999},"max":{"1min":4.9119999999999999,"5min":5.2069999999999999,"15min":5.734},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.873,"15min":0.90500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0940000000000001,"5min":3.2160000000000002,"15min":5.3730000000000002},"last":0.753}]},{"osd":5,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.91900000000000004,"15min":0.84299999999999997},"min":{"1min":0.67300000000000004,"5min":0.59899999999999998,"15min":0.40699999999999997},"max":{"1min":4.4939999999999998,"5min":4.4939999999999998,"15min":4.4939999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.88500000000000001},"min":{"1min":0.65000000000000002,"5min":0.60799999999999998,"15min":0.50900000000000001},"max":{"1min":1.7350000000000001,"5min":4.7910000000000004,"15min":4.7910000000000004},"last":0.57599999999999996}]},{"osd":6,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.01,"15min":0.89900000000000002},"min":{"1min":0.65500000000000003,"5min":0.60999999999999999,"15min":0.48699999999999999},"max":{"1min":4.8630000000000004,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.63},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.88100000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.55100000000000005},"max":{"1min":1.667,"5min":3.1339999999999999,"15min":3.4710000000000001},"last":0.90900000000000003}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.90800000000000003},"min":{"1min":0.627,"5min":0.627,"15min":0.48999999999999999},"max":{"1min":1.6950000000000001,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.95299999999999996,"15min":0.89600000000000002},"min":{"1min":0.68799999999999994,"5min":0.63,"15min":0.55700000000000005},"max":{"1min":1.627,"5min":3.1819999999999999,"15min":3.806},"last":0.61499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16297984,"data_stored":62915220,"data_compressed":8677157,"data_compressed_allocated":11337728,"data_compressed_original":57970688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28188672,"data_stored":97891275,"data_compressed":12584510,"data_compressed_allocated":16609280,"data_compressed_original":86339584,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16445440,"data_stored":52440076,"data_compressed":6685552,"data_compressed_allocated":8757248,"data_compressed_original":44773376,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26685440,"data_stored":98577398,"data_compressed":13459583,"data_compressed_allocated":17428480,"data_compressed_original":89337856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":27463680,"data_stored":104859297,"data_compressed":13833556,"data_compressed_allocated":17862656,"data_compressed_original":95277003,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":26279936,"data_stored":100664674,"data_compressed":13681157,"data_compressed_allocated":17580032,"data_compressed_original":91983872,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":23711744,"data_stored":88082717,"data_compressed":11602354,"data_compressed_allocated":14856192,"data_compressed_original":79245259,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":27398144,"data_stored":97891059,"data_compressed":13288587,"data_compressed_allocated":17137664,"data_compressed_original":87650304,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":206610432,"data_stored":222244864,"data_compressed":12071885,"data_compressed_allocated":13537280,"data_compressed_original":29171712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294195200,"data_stored":319320064,"data_compressed":20251866,"data_compressed_allocated":22855680,"data_compressed_original":47980544,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164610048,"data_stored":175325184,"data_compressed":9845077,"data_compressed_allocated":10899456,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":288477184,"data_stored":316018688,"data_compressed":23971937,"data_compressed_allocated":27017216,"data_compressed_original":54558720,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":217907200,"data_stored":237305856,"data_compressed":14188788,"data_compressed_allocated":16142336,"data_compressed_original":35540992,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":220975104,"data_stored":243601408,"data_compressed":17073954,"data_compressed_allocated":19296256,"data_compressed_original":41922560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":237195264,"data_stored":251670528,"data_compressed":13556545,"data_compressed_allocated":15155200,"data_compressed_original":29630464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":192204800,"data_stored":208740352,"data_compressed":13315951,"data_compressed_allocated":15101952,"data_compressed_original":31637504,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:56:37.436 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:56:37.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e0419d640 1 Processor -- start 2023-11-28T21:56:37.617 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e0419d640 1 -- start start 2023-11-28T21:56:37.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e0419d640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1dfc0c27c0 0x7f1dfc156af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e0419d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc0c6ce0 con 0x7f1dfc06d7d0 2023-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e0419d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc0c5df0 con 0x7f1dfc1500d0 2023-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e0419d640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc157030 con 0x7f1dfc0c27c0 2023-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e01738640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1dfc0c27c0 0x7f1dfc156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e01738640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f1dfc0c27c0 0x7f1dfc156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48682/0 (socket says 172.21.15.12:48682) 2023-11-28T21:56:37.619 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e01738640 1 -- 172.21.15.12:0/4248168618 learned_addr learned my addr 172.21.15.12:0/4248168618 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:37.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e0273a640 1 --1- 172.21.15.12:0/4248168618 >> v1:172.21.15.12:6789/0 conn(0x7f1dfc06d7d0 0x7f1dfc1609f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58078/0 (socket says 172.21.15.12:58078) 2023-11-28T21:56:37.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e01738640 1 -- 172.21.15.12:0/4248168618 >> v1:172.21.15.144:6789/0 conn(0x7f1dfc1500d0 legacy=0x7f1dfc0c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e01738640 1 -- 172.21.15.12:0/4248168618 >> v1:172.21.15.12:6789/0 conn(0x7f1dfc06d7d0 legacy=0x7f1dfc1609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.612+0000 7f1e01738640 1 -- 172.21.15.12:0/4248168618 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc0c60d0 con 0x7f1dfc0c27c0 2023-11-28T21:56:37.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e01738640 1 --2- 172.21.15.12:0/4248168618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc0c27c0 0x7f1dfc156af0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1dec008120 tx=0x7f1dec01c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ff834c520d7766 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:37.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e00f37640 1 -- 172.21.15.12:0/4248168618 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec004230 con 0x7f1dfc0c27c0 2023-11-28T21:56:37.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e00f37640 1 -- 172.21.15.12:0/4248168618 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec004390 con 0x7f1dfc0c27c0 2023-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e00f37640 1 -- 172.21.15.12:0/4248168618 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec003820 con 0x7f1dfc0c27c0 2023-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e0419d640 1 -- 172.21.15.12:0/4248168618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc0c27c0 msgr2=0x7f1dfc156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e0419d640 1 --2- 172.21.15.12:0/4248168618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc0c27c0 0x7f1dfc156af0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1dec008120 tx=0x7f1dec01c7e0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e0419d640 1 -- 172.21.15.12:0/4248168618 shutdown_connections 2023-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e0419d640 1 --2- 172.21.15.12:0/4248168618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc0c27c0 0x7f1dfc156af0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e0419d640 1 -- 172.21.15.12:0/4248168618 >> 172.21.15.12:0/4248168618 conn(0x7f1dfc066360 msgr2=0x7f1dfc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:37.622 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e0419d640 1 -- 172.21.15.12:0/4248168618 shutdown_connections 2023-11-28T21:56:37.623 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.616+0000 7f1e0419d640 1 -- 172.21.15.12:0/4248168618 wait complete. 2023-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0419d640 1 Processor -- start 2023-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0419d640 1 -- start start 2023-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0419d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dfc06d7d0 0x7f1dfc15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0419d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dfc0c27c0 0x7f1dfc0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0419d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc1500d0 0x7f1dfc148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0419d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1f2a70 con 0x7f1dfc1500d0 2023-11-28T21:56:37.625 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0419d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1f2d50 con 0x7f1dfc06d7d0 2023-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0419d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1f3030 con 0x7f1dfc0c27c0 2023-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0273a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc1500d0 0x7f1dfc148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0273a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc1500d0 0x7f1dfc148b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48684/0 (socket says 172.21.15.12:48684) 2023-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0273a640 1 -- 172.21.15.12:0/2739019111 learned_addr learned my addr 172.21.15.12:0/2739019111 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e01f39640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dfc06d7d0 0x7f1dfc15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e01738640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dfc0c27c0 0x7f1dfc0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:37.626 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0273a640 1 -- 172.21.15.12:0/2739019111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dfc0c27c0 msgr2=0x7f1dfc0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0273a640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dfc0c27c0 0x7f1dfc0ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0273a640 1 -- 172.21.15.12:0/2739019111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dfc06d7d0 msgr2=0x7f1dfc15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0273a640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dfc06d7d0 0x7f1dfc15eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0273a640 1 -- 172.21.15.12:0/2739019111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec00a580 con 0x7f1dfc1500d0 2023-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0273a640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc1500d0 0x7f1dfc148b40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1df400caa0 tx=0x7f1df400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1df27fc640 1 -- 172.21.15.12:0/2739019111 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df400a730 con 0x7f1dfc1500d0 2023-11-28T21:56:37.627 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1df27fc640 1 -- 172.21.15.12:0/2739019111 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df400a890 con 0x7f1dfc1500d0 2023-11-28T21:56:37.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1df27fc640 1 -- 172.21.15.12:0/2739019111 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df400ac40 con 0x7f1dfc1500d0 2023-11-28T21:56:37.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0419d640 1 -- 172.21.15.12:0/2739019111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc1f3310 con 0x7f1dfc1500d0 2023-11-28T21:56:37.628 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.620+0000 7f1e0419d640 1 -- 172.21.15.12:0/2739019111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc157230 con 0x7f1dfc1500d0 2023-11-28T21:56:37.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.624+0000 7f1e0419d640 1 -- 172.21.15.12:0/2739019111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc157420 con 0x7f1dfc1500d0 2023-11-28T21:56:37.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.624+0000 7f1df27fc640 1 -- 172.21.15.12:0/2739019111 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1df4006070 con 0x7f1dfc1500d0 2023-11-28T21:56:37.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.624+0000 7f1df27fc640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1de0066470 0x7f1de0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:37.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.624+0000 7f1df27fc640 1 -- 172.21.15.12:0/2739019111 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f1df40840a0 con 0x7f1dfc1500d0 2023-11-28T21:56:37.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.624+0000 7f1e01f39640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1de0066470 0x7f1de0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:37.629 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.624+0000 7f1e01f39640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1de0066470 0x7f1de0068930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000f70 tx=0x7f1df8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:37.631 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.628+0000 7f1df27fc640 1 -- 172.21.15.12:0/2739019111 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1df4050d80 con 0x7f1dfc1500d0 2023-11-28T21:56:37.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.788+0000 7f1e0419d640 1 -- 172.21.15.12:0/2739019111 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dfc074f60 con 0x7f1dfc1500d0 2023-11-28T21:56:37.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.788+0000 7f1df27fc640 1 -- 172.21.15.12:0/2739019111 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+20091 (secure 0 0 0) 0x7f1df4017070 con 0x7f1dfc1500d0 2023-11-28T21:56:37.792 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:37.792 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":52,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:55:38.673845+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6839/754378087":"2023-11-29T21:55:38.673802+0000","172.21.15.12:6836/754378087":"2023-11-29T21:55:38.673802+0000","172.21.15.144:6832/3053317078":"2023-11-29T21:39:58.381227+0000","172.21.15.144:6833/3053317078":"2023-11-29T21:39:58.381227+0000","172.21.15.12:6839/2705421041":"2023-11-29T21:48:53.535998+0000","172.21.15.12:6834/3741155090":"2023-11-29T21:41:03.398724+0000","172.21.15.12:6838/3474424840":"2023-11-29T21:46:48.503169+0000","172.21.15.144:6832/126572048":"2023-11-29T21:52:53.611902+0000","172.21.15.12:6838/3741155090":"2023-11-29T21:41:03.398724+0000","172.21.15.12:6834/3474424840":"2023-11-29T21:46:48.503169+0000","172.21.15.12:6834/1551825785":"2023-11-29T21:50:38.562185+0000","172.21.15.12:6836/3009112102":"2023-11-29T21:42:38.424201+0000","172.21.15.12:6834/291731973":"2023-11-29T21:44:08.460474+0000","172.21.15.144:6833/126572048":"2023-11-29T21:52:53.611902+0000","172.21.15.12:6838/291731973":"2023-11-29T21:44:08.460474+0000","172.21.15.12:6836/3299030262":"2023-11-29T21:45:23.480028+0000","172.21.15.12:6839/3009112102":"2023-11-29T21:42:38.424201+0000","172.21.15.12:6839/3299030262":"2023-11-29T21:45:23.480028+0000","172.21.15.12:6838/1551825785":"2023-11-29T21:50:38.562185+0000","172.21.15.12:6836/2705421041":"2023-11-29T21:48:53.535998+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:56:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.788+0000 7f1e0419d640 1 -- 172.21.15.12:0/2739019111 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1de0066470 msgr2=0x7f1de0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.788+0000 7f1e0419d640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1de0066470 0x7f1de0068930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000f70 tx=0x7f1df8009040 comp rx=0 tx=0).stop 2023-11-28T21:56:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.788+0000 7f1e0419d640 1 -- 172.21.15.12:0/2739019111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc1500d0 msgr2=0x7f1dfc148b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:37.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.792+0000 7f1e0419d640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc1500d0 0x7f1dfc148b40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1df400caa0 tx=0x7f1df400cea0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.792+0000 7f1e0419d640 1 -- 172.21.15.12:0/2739019111 shutdown_connections 2023-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.792+0000 7f1e0419d640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f1de0066470 0x7f1de0068930 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.792+0000 7f1e0419d640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1dfc1500d0 0x7f1dfc148b40 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.792+0000 7f1e0419d640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dfc0c27c0 0x7f1dfc0ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.792+0000 7f1e0419d640 1 --2- 172.21.15.12:0/2739019111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dfc06d7d0 0x7f1dfc15eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.792+0000 7f1e0419d640 1 -- 172.21.15.12:0/2739019111 >> 172.21.15.12:0/2739019111 conn(0x7f1dfc066360 msgr2=0x7f1dfc14e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.792+0000 7f1e0419d640 1 -- 172.21.15.12:0/2739019111 shutdown_connections 2023-11-28T21:56:37.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:37.792+0000 7f1e0419d640 1 -- 172.21.15.12:0/2739019111 wait complete. 2023-11-28T21:56:38.815 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T21:56:38.816 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T21:56:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f25c5a640 1 Processor -- start 2023-11-28T21:56:38.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f25c5a640 1 -- start start 2023-11-28T21:56:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f25c5a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 0x7f0f201569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f25c5a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0f2006e100 0x7f0f200c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:38.976 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f25c5a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2006e560 con 0x7f0f200c5180 2023-11-28T21:56:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f25c5a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f200c4820 con 0x7f0f2006e100 2023-11-28T21:56:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f25c5a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200c3570 con 0x7f0f20150c00 2023-11-28T21:56:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f1ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f0f200c5180 0x7f0f200c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58094/0 (socket says 172.21.15.12:58094) 2023-11-28T21:56:38.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f1ffff640 1 -- 172.21.15.12:0/4084272071 learned_addr learned my addr 172.21.15.12:0/4084272071 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f17fff640 1 --2- 172.21.15.12:0/4084272071 >> v2:172.21.15.144:3300/0 conn(0x7f0f2006e100 0x7f0f200c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f1f7fe640 1 --2- 172.21.15.12:0/4084272071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 0x7f0f201569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f1effd640 1 -- 172.21.15.12:0/4084272071 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521897532 0 0) 0x7f0f2006e560 con 0x7f0f200c5180 2023-11-28T21:56:38.978 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f1effd640 1 -- 172.21.15.12:0/4084272071 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f0c003530 con 0x7f0f200c5180 2023-11-28T21:56:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f17fff640 1 -- 172.21.15.12:0/4084272071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 msgr2=0x7f0f201569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f17fff640 1 --2- 172.21.15.12:0/4084272071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 0x7f0f201569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f17fff640 1 -- 172.21.15.12:0/4084272071 >> v1:172.21.15.12:6789/0 conn(0x7f0f200c5180 legacy=0x7f0f200c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f1effd640 1 -- 172.21.15.12:0/4084272071 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3317048107 0 0) 0x7f0f0c003530 con 0x7f0f200c5180 2023-11-28T21:56:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f17fff640 1 -- 172.21.15.12:0/4084272071 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f200c3830 con 0x7f0f2006e100 2023-11-28T21:56:38.979 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.972+0000 7f0f1f7fe640 1 --2- 172.21.15.12:0/4084272071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 0x7f0f201569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:56:38.980 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f17fff640 1 --2- 172.21.15.12:0/4084272071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f2006e100 0x7f0f200c4c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0f080097c0 tx=0x7f0f0801c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f19a51a84e689e2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f1effd640 1 -- 172.21.15.12:0/4084272071 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18001a00 con 0x7f0f2006e100 2023-11-28T21:56:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f1effd640 1 -- 172.21.15.12:0/4084272071 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f080a7050 con 0x7f0f2006e100 2023-11-28T21:56:38.981 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f1effd640 1 -- 172.21.15.12:0/4084272071 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f080ab600 con 0x7f0f2006e100 2023-11-28T21:56:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f25c5a640 1 -- 172.21.15.12:0/4084272071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f2006e100 msgr2=0x7f0f200c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f25c5a640 1 --2- 172.21.15.12:0/4084272071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f2006e100 0x7f0f200c4c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0f080097c0 tx=0x7f0f0801c370 comp rx=0 tx=0).stop 2023-11-28T21:56:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f25c5a640 1 -- 172.21.15.12:0/4084272071 shutdown_connections 2023-11-28T21:56:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f25c5a640 1 --2- 172.21.15.12:0/4084272071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f2006e100 0x7f0f200c4c40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:38.982 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f25c5a640 1 --2- 172.21.15.12:0/4084272071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 0x7f0f201569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f25c5a640 1 -- 172.21.15.12:0/4084272071 >> 172.21.15.12:0/4084272071 conn(0x7f0f2006d5e0 msgr2=0x7f0f2006d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f25c5a640 1 -- 172.21.15.12:0/4084272071 shutdown_connections 2023-11-28T21:56:38.983 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.976+0000 7f0f25c5a640 1 -- 172.21.15.12:0/4084272071 wait complete. 2023-11-28T21:56:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f25c5a640 1 Processor -- start 2023-11-28T21:56:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f25c5a640 1 -- start start 2023-11-28T21:56:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f25c5a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f2006e100 0x7f0f201f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f25c5a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f200c5180 0x7f0f201463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:38.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f25c5a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 0x7f0f200ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f25c5a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20071710 con 0x7f0f200c5180 2023-11-28T21:56:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f25c5a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f20145f20 con 0x7f0f2006e100 2023-11-28T21:56:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f25c5a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2014ec40 con 0x7f0f20150c00 2023-11-28T21:56:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f200c5180 0x7f0f201463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f200c5180 0x7f0f201463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48690/0 (socket says 172.21.15.12:48690) 2023-11-28T21:56:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f17fff640 1 -- 172.21.15.12:0/809554953 learned_addr learned my addr 172.21.15.12:0/809554953 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f1f7fe640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f2006e100 0x7f0f201f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:38.986 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f1ffff640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 0x7f0f200ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:38.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f17fff640 1 -- 172.21.15.12:0/809554953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 msgr2=0x7f0f200ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:38.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f17fff640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 0x7f0f200ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:38.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f17fff640 1 -- 172.21.15.12:0/809554953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f2006e100 msgr2=0x7f0f201f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:38.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f17fff640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f2006e100 0x7f0f201f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:38.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f17fff640 1 -- 172.21.15.12:0/809554953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f10009420 con 0x7f0f200c5180 2023-11-28T21:56:38.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f1ffff640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f20150c00 0x7f0f200ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:38.987 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f17fff640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f200c5180 0x7f0f201463b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0f0801aa20 tx=0x7f0f08019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:38.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f1cff9640 1 -- 172.21.15.12:0/809554953 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08006e20 con 0x7f0f200c5180 2023-11-28T21:56:38.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f1cff9640 1 -- 172.21.15.12:0/809554953 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f080b4400 con 0x7f0f200c5180 2023-11-28T21:56:38.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f08009420 con 0x7f0f200c5180 2023-11-28T21:56:38.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f1cff9640 1 -- 172.21.15.12:0/809554953 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f080b4770 con 0x7f0f200c5180 2023-11-28T21:56:38.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.980+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f20157c90 con 0x7f0f200c5180 2023-11-28T21:56:38.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f200c3260 con 0x7f0f200c5180 2023-11-28T21:56:38.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f1cff9640 1 -- 172.21.15.12:0/809554953 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0f080b2040 con 0x7f0f200c5180 2023-11-28T21:56:38.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f1cff9640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0f00066470 0x7f0f00068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:38.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f1cff9640 1 -- 172.21.15.12:0/809554953 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f0f0811e2e0 con 0x7f0f200c5180 2023-11-28T21:56:38.990 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f1f7fe640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0f00066470 0x7f0f00068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:38.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f1cff9640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] conn(0x7f0f000704f0 0x7f0f00072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:38.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f1cff9640 1 -- 172.21.15.12:0/809554953 --> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f00073060 con 0x7f0f000704f0 2023-11-28T21:56:38.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f1cff9640 1 -- 172.21.15.12:0/809554953 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f081208c0 con 0x7f0f200c5180 2023-11-28T21:56:38.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f1ffff640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] conn(0x7f0f000704f0 0x7f0f00072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:38.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f1f7fe640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0f00066470 0x7f0f00068930 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009cf0 tx=0x7f0f10009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:38.991 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.984+0000 7f0f1ffff640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] conn(0x7f0f000704f0 0x7f0f00072990 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:38.992 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:38.988+0000 7f0f1cff9640 1 -- 172.21.15.12:0/809554953 <== osd.0 v2:172.21.15.12:6816/3923148634 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0f00073060 con 0x7f0f000704f0 2023-11-28T21:56:39.003 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 --> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0f200c4440 con 0x7f0f000704f0 2023-11-28T21:56:39.004 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f1cff9640 1 -- 172.21.15.12:0/809554953 <== osd.0 v2:172.21.15.12:6816/3923148634 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0f180070e0 con 0x7f0f000704f0 2023-11-28T21:56:39.004 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:56:39.004 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-28T21:56:39.004 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:56:39.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] conn(0x7f0f000704f0 msgr2=0x7f0f00072990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.005 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:6816/3923148634,v1:172.21.15.12:6817/3923148634] conn(0x7f0f000704f0 0x7f0f00072990 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0f00066470 msgr2=0x7f0f00068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0f00066470 0x7f0f00068930 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009cf0 tx=0x7f0f10009190 comp rx=0 tx=0).stop 2023-11-28T21:56:39.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f200c5180 msgr2=0x7f0f201463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 --2- 172.21.15.12:0/809554953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0f200c5180 0x7f0f201463b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0f0801aa20 tx=0x7f0f08019110 comp rx=0 tx=0).stop 2023-11-28T21:56:39.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f1ffff640 1 -- 172.21.15.12:0/809554953 reap_dead start 2023-11-28T21:56:39.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 shutdown_connections 2023-11-28T21:56:39.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 >> 172.21.15.12:0/809554953 conn(0x7f0f2006d5e0 msgr2=0x7f0f2006aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:39.006 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 shutdown_connections 2023-11-28T21:56:39.007 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.000+0000 7f0f25c5a640 1 -- 172.21.15.12:0/809554953 wait complete. 2023-11-28T21:56:39.022 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T21:56:39.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9efad1d640 1 Processor -- start 2023-11-28T21:56:39.167 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9efad1d640 1 -- start start 2023-11-28T21:56:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9efad1d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9ef4066e80 0x7f9ef406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9efad1d640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef40c2ed0 con 0x7f9ef4150e40 2023-11-28T21:56:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9efad1d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef40708e0 con 0x7f9ef406ffa0 2023-11-28T21:56:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9efad1d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef41571a0 con 0x7f9ef4066e80 2023-11-28T21:56:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9ef8ab9640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9ef4150e40 0x7f9ef4156a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58104/0 (socket says 172.21.15.12:58104) 2023-11-28T21:56:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9ef8ab9640 1 -- 172.21.15.12:0/3294492195 learned_addr learned my addr 172.21.15.12:0/3294492195 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eebfff640 1 --2- 172.21.15.12:0/3294492195 >> v2:172.21.15.144:3300/0 conn(0x7f9ef4066e80 0x7f9ef406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578103574 0 0) 0x7f9ef40c2ed0 con 0x7f9ef4150e40 2023-11-28T21:56:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9edc003530 con 0x7f9ef4150e40 2023-11-28T21:56:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2225474829 0 0) 0x7f9edc003530 con 0x7f9ef4150e40 2023-11-28T21:56:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 >> v2:172.21.15.144:3300/0 conn(0x7f9ef4066e80 msgr2=0x7f9ef406fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 --2- 172.21.15.12:0/3294492195 >> v2:172.21.15.144:3300/0 conn(0x7f9ef4066e80 0x7f9ef406fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 >> v1:172.21.15.144:6789/0 conn(0x7f9ef406ffa0 legacy=0x7f9ef4161330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4151220 con 0x7f9ef4150e40 2023-11-28T21:56:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f9ef0001680 con 0x7f9ef4150e40 2023-11-28T21:56:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 >> v1:172.21.15.12:6789/0 conn(0x7f9ef4150e40 legacy=0x7f9ef4156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9edc0039f0 0x7f9edc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc0047d0 0x7f9edc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9edc005190 0x7f9edc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef41571a0 con 0x7f9edc0039f0 2023-11-28T21:56:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc00bfb0 con 0x7f9edc0047d0 2023-11-28T21:56:39.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc00c170 con 0x7f9edc005190 2023-11-28T21:56:39.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9ef8ab9640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9edc0039f0 0x7f9edc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9eebfff640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc0047d0 0x7f9edc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.164+0000 7f9ef92ba640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9edc005190 0x7f9edc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9ef8ab9640 1 -- 172.21.15.12:0/3294492195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9edc005190 msgr2=0x7f9edc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:39.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9ef8ab9640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9edc005190 0x7f9edc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9ef8ab9640 1 -- 172.21.15.12:0/3294492195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc0047d0 msgr2=0x7f9edc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:39.173 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9ef8ab9640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9edc0047d0 0x7f9edc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9ef8ab9640 1 -- 172.21.15.12:0/3294492195 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc003530 con 0x7f9edc0039f0 2023-11-28T21:56:39.174 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9ef92ba640 1 -- 172.21.15.12:0/3294492195 reap_dead start 2023-11-28T21:56:39.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9ef8ab9640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9edc0039f0 0x7f9edc003dd0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0007bf0 tx=0x7f9ef001ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8a171d27c0e429 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:39.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00a7c90 con 0x7f9edc0039f0 2023-11-28T21:56:39.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef00a7df0 con 0x7f9edc0039f0 2023-11-28T21:56:39.175 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9eeb7fe640 1 -- 172.21.15.12:0/3294492195 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0005b20 con 0x7f9edc0039f0 2023-11-28T21:56:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9efad1d640 1 -- 172.21.15.12:0/3294492195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9edc0039f0 msgr2=0x7f9edc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9efad1d640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9edc0039f0 0x7f9edc003dd0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0007bf0 tx=0x7f9ef001ed90 comp rx=0 tx=0).stop 2023-11-28T21:56:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9efad1d640 1 -- 172.21.15.12:0/3294492195 shutdown_connections 2023-11-28T21:56:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9efad1d640 1 --2- 172.21.15.12:0/3294492195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9edc0039f0 0x7f9edc003dd0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9efad1d640 1 -- 172.21.15.12:0/3294492195 >> 172.21.15.12:0/3294492195 conn(0x7f9ef4066360 msgr2=0x7f9ef4066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9efad1d640 1 -- 172.21.15.12:0/3294492195 shutdown_connections 2023-11-28T21:56:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.168+0000 7f9efad1d640 1 -- 172.21.15.12:0/3294492195 wait complete. 2023-11-28T21:56:39.176 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9efad1d640 1 Processor -- start 2023-11-28T21:56:39.177 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9efad1d640 1 -- start start 2023-11-28T21:56:39.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9efad1d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9edc0039f0 0x7f9ef406cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.178 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9efad1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ef40717e0 0x7f9ef414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9efad1d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef414def0 0x7f9ef4152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9efad1d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40cb100 con 0x7f9ef40717e0 2023-11-28T21:56:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9efad1d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef414e370 con 0x7f9ef414def0 2023-11-28T21:56:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9efad1d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef41ef5d0 con 0x7f9edc0039f0 2023-11-28T21:56:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9eebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ef40717e0 0x7f9ef414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9eebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ef40717e0 0x7f9ef414d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48720/0 (socket says 172.21.15.12:48720) 2023-11-28T21:56:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9eebfff640 1 -- 172.21.15.12:0/2968492589 learned_addr learned my addr 172.21.15.12:0/2968492589 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:39.179 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9ef8ab9640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9edc0039f0 0x7f9ef406cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9ef92ba640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef414def0 0x7f9ef4152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9eebfff640 1 -- 172.21.15.12:0/2968492589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9edc0039f0 msgr2=0x7f9ef406cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9eebfff640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9edc0039f0 0x7f9ef406cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9eebfff640 1 -- 172.21.15.12:0/2968492589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef414def0 msgr2=0x7f9ef4152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9eebfff640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef414def0 0x7f9ef4152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9eebfff640 1 -- 172.21.15.12:0/2968492589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef41f03f0 con 0x7f9ef40717e0 2023-11-28T21:56:39.180 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9ef92ba640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef414def0 0x7f9ef4152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9eebfff640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ef40717e0 0x7f9ef414d9b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4011530 tx=0x7f9ee40119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9ee97fa640 1 -- 172.21.15.12:0/2968492589 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4015870 con 0x7f9ef40717e0 2023-11-28T21:56:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9ee97fa640 1 -- 172.21.15.12:0/2968492589 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee40159d0 con 0x7f9ef40717e0 2023-11-28T21:56:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9ee97fa640 1 -- 172.21.15.12:0/2968492589 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4015d40 con 0x7f9ef40717e0 2023-11-28T21:56:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.172+0000 7f9efad1d640 1 -- 172.21.15.12:0/2968492589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef40c6040 con 0x7f9ef40717e0 2023-11-28T21:56:39.181 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.176+0000 7f9efad1d640 1 -- 172.21.15.12:0/2968492589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef41efac0 con 0x7f9ef40717e0 2023-11-28T21:56:39.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.176+0000 7f9efad1d640 1 -- 172.21.15.12:0/2968492589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef4074f60 con 0x7f9ef40717e0 2023-11-28T21:56:39.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.176+0000 7f9ee97fa640 1 -- 172.21.15.12:0/2968492589 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9ee4013400 con 0x7f9ef40717e0 2023-11-28T21:56:39.182 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.176+0000 7f9ee97fa640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ee0066470 0x7f9ee0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.176+0000 7f9ee97fa640 1 -- 172.21.15.12:0/2968492589 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f9ee4084c10 con 0x7f9ef40717e0 2023-11-28T21:56:39.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.176+0000 7f9ef8ab9640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ee0066470 0x7f9ee0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.183 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.180+0000 7f9ef8ab9640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ee0066470 0x7f9ee0068930 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00ad000 tx=0x7f9ef00b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.180+0000 7f9ee97fa640 1 -- 172.21.15.12:0/2968492589 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9ee40518d0 con 0x7f9ef40717e0 2023-11-28T21:56:39.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 -- 172.21.15.12:0/2968492589 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ef415a000 con 0x7f9ee0066470 2023-11-28T21:56:39.342 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9ee97fa640 1 -- 172.21.15.12:0/2968492589 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9ef00a7a90 con 0x7f9ee0066470 2023-11-28T21:56:39.343 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T21:56:39.344 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 -- 172.21.15.12:0/2968492589 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ee0066470 msgr2=0x7f9ee0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ee0066470 0x7f9ee0068930 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00ad000 tx=0x7f9ef00b2040 comp rx=0 tx=0).stop 2023-11-28T21:56:39.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 -- 172.21.15.12:0/2968492589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ef40717e0 msgr2=0x7f9ef414d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ef40717e0 0x7f9ef414d9b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4011530 tx=0x7f9ee40119f0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 -- 172.21.15.12:0/2968492589 shutdown_connections 2023-11-28T21:56:39.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9ee0066470 0x7f9ee0068930 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.345 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef414def0 0x7f9ef4152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9ef40717e0 0x7f9ef414d9b0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 --2- 172.21.15.12:0/2968492589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9edc0039f0 0x7f9ef406cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 -- 172.21.15.12:0/2968492589 >> 172.21.15.12:0/2968492589 conn(0x7f9ef4066360 msgr2=0x7f9ef414fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:39.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 -- 172.21.15.12:0/2968492589 shutdown_connections 2023-11-28T21:56:39.346 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.340+0000 7f9efad1d640 1 -- 172.21.15.12:0/2968492589 wait complete. 2023-11-28T21:56:39.364 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T21:56:39.364 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T21:56:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d47f8640 1 Processor -- start 2023-11-28T21:56:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d47f8640 1 -- start start 2023-11-28T21:56:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d47f8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd9cc150c00 0x7fd9cc1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d47f8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd9cc06e100 0x7fd9cc0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d47f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9cc0c5180 0x7fd9cc0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d47f8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9cc0c4820 con 0x7fd9cc150c00 2023-11-28T21:56:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d47f8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9cc06e590 con 0x7fd9cc06e100 2023-11-28T21:56:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d47f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc0c3570 con 0x7fd9cc0c5180 2023-11-28T21:56:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d2594640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd9cc150c00 0x7fd9cc1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d2594640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd9cc150c00 0x7fd9cc1569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48734/0 (socket says 172.21.15.12:48734) 2023-11-28T21:56:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d2594640 1 -- 172.21.15.12:0/148400675 learned_addr learned my addr 172.21.15.12:0/148400675 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d2d95640 1 --2- 172.21.15.12:0/148400675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9cc0c5180 0x7fd9cc0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d1d93640 1 --2- 172.21.15.12:0/148400675 >> v2:172.21.15.144:3300/0 conn(0x7fd9cc06e100 0x7fd9cc0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d2594640 1 -- 172.21.15.12:0/148400675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9cc0c5180 msgr2=0x7fd9cc0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d2594640 1 --2- 172.21.15.12:0/148400675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9cc0c5180 0x7fd9cc0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d2594640 1 -- 172.21.15.12:0/148400675 >> v2:172.21.15.144:3300/0 conn(0x7fd9cc06e100 msgr2=0x7fd9cc0c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:39.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d2594640 1 --2- 172.21.15.12:0/148400675 >> v2:172.21.15.144:3300/0 conn(0x7fd9cc06e100 0x7fd9cc0c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d2594640 1 -- 172.21.15.12:0/148400675 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0c3830 con 0x7fd9cc150c00 2023-11-28T21:56:39.515 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.508+0000 7fd9d1d93640 1 --2- 172.21.15.12:0/148400675 >> v2:172.21.15.144:3300/0 conn(0x7fd9cc06e100 0x7fd9cc0c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T21:56:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d2594640 1 --2- 172.21.15.12:0/148400675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9cc150c00 0x7fd9cc1569c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc06bae0 tx=0x7fd9c801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849e08c2b42c634c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d1592640 1 -- 172.21.15.12:0/148400675 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80ac600 con 0x7fd9cc150c00 2023-11-28T21:56:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d1592640 1 -- 172.21.15.12:0/148400675 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c80a8050 con 0x7fd9cc150c00 2023-11-28T21:56:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d1592640 1 -- 172.21.15.12:0/148400675 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80ac980 con 0x7fd9cc150c00 2023-11-28T21:56:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d47f8640 1 -- 172.21.15.12:0/148400675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9cc150c00 msgr2=0x7fd9cc1569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d47f8640 1 --2- 172.21.15.12:0/148400675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9cc150c00 0x7fd9cc1569c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc06bae0 tx=0x7fd9c801c4d0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d47f8640 1 -- 172.21.15.12:0/148400675 shutdown_connections 2023-11-28T21:56:39.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d47f8640 1 --2- 172.21.15.12:0/148400675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9cc0c5180 0x7fd9cc0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.517 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d47f8640 1 --2- 172.21.15.12:0/148400675 >> v2:172.21.15.144:3300/0 conn(0x7fd9cc06e100 0x7fd9cc0c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d47f8640 1 --2- 172.21.15.12:0/148400675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9cc150c00 0x7fd9cc1569c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d47f8640 1 -- 172.21.15.12:0/148400675 >> 172.21.15.12:0/148400675 conn(0x7fd9cc06d5e0 msgr2=0x7fd9cc06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:39.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d47f8640 1 -- 172.21.15.12:0/148400675 shutdown_connections 2023-11-28T21:56:39.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.512+0000 7fd9d47f8640 1 -- 172.21.15.12:0/148400675 wait complete. 2023-11-28T21:56:39.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 Processor -- start 2023-11-28T21:56:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 -- start start 2023-11-28T21:56:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9cc06e100 0x7fd9cc152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9cc0c5180 0x7fd9cc1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9cc150c00 0x7fd9cc158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc14e0f0 con 0x7fd9cc06e100 2023-11-28T21:56:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc0c2300 con 0x7fd9cc0c5180 2023-11-28T21:56:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc14e340 con 0x7fd9cc150c00 2023-11-28T21:56:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d2594640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9cc06e100 0x7fd9cc152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d2594640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9cc06e100 0x7fd9cc152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48748/0 (socket says 172.21.15.12:48748) 2023-11-28T21:56:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d2594640 1 -- 172.21.15.12:0/2869306096 learned_addr learned my addr 172.21.15.12:0/2869306096 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d2d95640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9cc150c00 0x7fd9cc158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d1d93640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9cc0c5180 0x7fd9cc1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d2594640 1 -- 172.21.15.12:0/2869306096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9cc150c00 msgr2=0x7fd9cc158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d2594640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9cc150c00 0x7fd9cc158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d2594640 1 -- 172.21.15.12:0/2869306096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9cc0c5180 msgr2=0x7fd9cc1529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d2594640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9cc0c5180 0x7fd9cc1529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d2594640 1 -- 172.21.15.12:0/2869306096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c80094d0 con 0x7fd9cc06e100 2023-11-28T21:56:39.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d2594640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9cc06e100 0x7fd9cc152460 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd9c801ab80 tx=0x7fd9c801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:39.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9c2ffd640 1 -- 172.21.15.12:0/2869306096 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80bc070 con 0x7fd9cc06e100 2023-11-28T21:56:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9c2ffd640 1 -- 172.21.15.12:0/2869306096 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c80ace90 con 0x7fd9cc06e100 2023-11-28T21:56:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9cc159bd0 con 0x7fd9cc06e100 2023-11-28T21:56:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9c2ffd640 1 -- 172.21.15.12:0/2869306096 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80b7650 con 0x7fd9cc06e100 2023-11-28T21:56:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9cc151f40 con 0x7fd9cc06e100 2023-11-28T21:56:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.516+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9cc0c3260 con 0x7fd9cc06e100 2023-11-28T21:56:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9c2ffd640 1 -- 172.21.15.12:0/2869306096 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd9c80b3070 con 0x7fd9cc06e100 2023-11-28T21:56:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9c2ffd640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd9b0066470 0x7fd9b0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9c2ffd640 1 -- 172.21.15.12:0/2869306096 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fd9c811ef20 con 0x7fd9cc06e100 2023-11-28T21:56:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9d1d93640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd9b0066470 0x7fd9b0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9c2ffd640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] conn(0x7fd9b00704f0 0x7fd9b0072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9c2ffd640 1 -- 172.21.15.12:0/2869306096 --> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9b0073060 con 0x7fd9b00704f0 2023-11-28T21:56:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9c2ffd640 1 -- 172.21.15.12:0/2869306096 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9c811f290 con 0x7fd9cc06e100 2023-11-28T21:56:39.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9d2d95640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] conn(0x7fd9b00704f0 0x7fd9b0072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9d1d93640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd9b0066470 0x7fd9b0068930 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0c4bb0 tx=0x7fd9bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:39.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9d2d95640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] conn(0x7fd9b00704f0 0x7fd9b0072990 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:39.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.520+0000 7fd9c2ffd640 1 -- 172.21.15.12:0/2869306096 <== osd.1 v2:172.21.15.12:6802/1716423303 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd9b0073060 con 0x7fd9b00704f0 2023-11-28T21:56:39.538 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 --> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd9cc0c4440 con 0x7fd9b00704f0 2023-11-28T21:56:39.539 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9c2ffd640 1 -- 172.21.15.12:0/2869306096 <== osd.1 v2:172.21.15.12:6802/1716423303 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd9c4009ad0 con 0x7fd9b00704f0 2023-11-28T21:56:39.539 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:56:39.539 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-28T21:56:39.539 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:56:39.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] conn(0x7fd9b00704f0 msgr2=0x7fd9b0072990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.540 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:6802/1716423303,v1:172.21.15.12:6805/1716423303] conn(0x7fd9b00704f0 0x7fd9b0072990 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd9b0066470 msgr2=0x7fd9b0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd9b0066470 0x7fd9b0068930 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0c4bb0 tx=0x7fd9bc008040 comp rx=0 tx=0).stop 2023-11-28T21:56:39.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9cc06e100 msgr2=0x7fd9cc152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 --2- 172.21.15.12:0/2869306096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9cc06e100 0x7fd9cc152460 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd9c801ab80 tx=0x7fd9c801ad10 comp rx=0 tx=0).stop 2023-11-28T21:56:39.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d2d95640 1 -- 172.21.15.12:0/2869306096 reap_dead start 2023-11-28T21:56:39.541 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 shutdown_connections 2023-11-28T21:56:39.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 >> 172.21.15.12:0/2869306096 conn(0x7fd9cc06d5e0 msgr2=0x7fd9cc14ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:39.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 shutdown_connections 2023-11-28T21:56:39.542 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.536+0000 7fd9d47f8640 1 -- 172.21.15.12:0/2869306096 wait complete. 2023-11-28T21:56:39.557 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T21:56:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.696+0000 7f2270e9e640 1 Processor -- start 2023-11-28T21:56:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.696+0000 7f2270e9e640 1 -- start start 2023-11-28T21:56:39.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f2270e9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c153e00 0x7f226c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f2270e9e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f226c15d7f0 0x7f226c15dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f2270e9e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226c0c7710 con 0x7f226c06bc00 2023-11-28T21:56:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f2270e9e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226c1541e0 con 0x7f226c15d7f0 2023-11-28T21:56:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f2270e9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c15e180 con 0x7f226c153e00 2023-11-28T21:56:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f2263fff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f226c06bc00 0x7f226c0cd410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58118/0 (socket says 172.21.15.12:58118) 2023-11-28T21:56:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f2263fff640 1 -- 172.21.15.12:0/2512414869 learned_addr learned my addr 172.21.15.12:0/2512414869 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f226ad9d640 1 --2- 172.21.15.12:0/2512414869 >> v2:172.21.15.144:3300/0 conn(0x7f226c15d7f0 0x7f226c15dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f226a59c640 1 --2- 172.21.15.12:0/2512414869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c153e00 0x7f226c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f2269d9b640 1 -- 172.21.15.12:0/2512414869 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793431169 0 0) 0x7f226c0c7710 con 0x7f226c06bc00 2023-11-28T21:56:39.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f2269d9b640 1 -- 172.21.15.12:0/2512414869 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2258003530 con 0x7f226c06bc00 2023-11-28T21:56:39.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f226ad9d640 1 -- 172.21.15.12:0/2512414869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c153e00 msgr2=0x7f226c06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:39.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f226ad9d640 1 --2- 172.21.15.12:0/2512414869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c153e00 0x7f226c06b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f226ad9d640 1 -- 172.21.15.12:0/2512414869 >> v1:172.21.15.12:6789/0 conn(0x7f226c06bc00 legacy=0x7f226c0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.700+0000 7f226ad9d640 1 -- 172.21.15.12:0/2512414869 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c15a010 con 0x7f226c15d7f0 2023-11-28T21:56:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f226ad9d640 1 --2- 172.21.15.12:0/2512414869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c15d7f0 0x7f226c15dc40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2264009d90 tx=0x7f226401f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ff22157e33422b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2269d9b640 1 -- 172.21.15.12:0/2512414869 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22640af600 con 0x7f226c15d7f0 2023-11-28T21:56:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2269d9b640 1 -- 172.21.15.12:0/2512414869 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22640ab050 con 0x7f226c15d7f0 2023-11-28T21:56:39.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2269d9b640 1 -- 172.21.15.12:0/2512414869 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22640af980 con 0x7f226c15d7f0 2023-11-28T21:56:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2270e9e640 1 -- 172.21.15.12:0/2512414869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c15d7f0 msgr2=0x7f226c15dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2270e9e640 1 --2- 172.21.15.12:0/2512414869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c15d7f0 0x7f226c15dc40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2264009d90 tx=0x7f226401f130 comp rx=0 tx=0).stop 2023-11-28T21:56:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2270e9e640 1 -- 172.21.15.12:0/2512414869 shutdown_connections 2023-11-28T21:56:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2270e9e640 1 --2- 172.21.15.12:0/2512414869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c15d7f0 0x7f226c15dc40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2270e9e640 1 --2- 172.21.15.12:0/2512414869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c153e00 0x7f226c06b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2270e9e640 1 -- 172.21.15.12:0/2512414869 >> 172.21.15.12:0/2512414869 conn(0x7f226c066360 msgr2=0x7f226c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2270e9e640 1 -- 172.21.15.12:0/2512414869 shutdown_connections 2023-11-28T21:56:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2270e9e640 1 -- 172.21.15.12:0/2512414869 wait complete. 2023-11-28T21:56:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.704+0000 7f2270e9e640 1 Processor -- start 2023-11-28T21:56:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2270e9e640 1 -- start start 2023-11-28T21:56:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2270e9e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c06bc00 0x7f226c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2270e9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c153e00 0x7f226c14eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2270e9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c15d7f0 0x7f226c14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2270e9e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0caec0 con 0x7f226c06bc00 2023-11-28T21:56:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2270e9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c14e6f0 con 0x7f226c15d7f0 2023-11-28T21:56:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2270e9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c1ef730 con 0x7f226c153e00 2023-11-28T21:56:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f226a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c06bc00 0x7f226c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f226a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c06bc00 0x7f226c0ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48760/0 (socket says 172.21.15.12:48760) 2023-11-28T21:56:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f226a59c640 1 -- 172.21.15.12:0/620837402 learned_addr learned my addr 172.21.15.12:0/620837402 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2263fff640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c153e00 0x7f226c14eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f226ad9d640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c15d7f0 0x7f226c14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f226a59c640 1 -- 172.21.15.12:0/620837402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c153e00 msgr2=0x7f226c14eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f226a59c640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c153e00 0x7f226c14eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f226a59c640 1 -- 172.21.15.12:0/620837402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c15d7f0 msgr2=0x7f226c14f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f226a59c640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c15d7f0 0x7f226c14f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f226a59c640 1 -- 172.21.15.12:0/620837402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009a70 con 0x7f226c06bc00 2023-11-28T21:56:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f226a59c640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c06bc00 0x7f226c0ca980 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e930 tx=0x7f225c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2262ffd640 1 -- 172.21.15.12:0/620837402 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c009690 con 0x7f226c06bc00 2023-11-28T21:56:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2262ffd640 1 -- 172.21.15.12:0/620837402 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c00ca00 con 0x7f226c06bc00 2023-11-28T21:56:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2270e9e640 1 -- 172.21.15.12:0/620837402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226c14ddc0 con 0x7f226c06bc00 2023-11-28T21:56:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2262ffd640 1 -- 172.21.15.12:0/620837402 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c00cd70 con 0x7f226c06bc00 2023-11-28T21:56:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2270e9e640 1 -- 172.21.15.12:0/620837402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c14e380 con 0x7f226c06bc00 2023-11-28T21:56:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.708+0000 7f2270e9e640 1 -- 172.21.15.12:0/620837402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c06c620 con 0x7f226c06bc00 2023-11-28T21:56:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.712+0000 7f2262ffd640 1 -- 172.21.15.12:0/620837402 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f225c013470 con 0x7f226c06bc00 2023-11-28T21:56:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.712+0000 7f2262ffd640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f224c066470 0x7f224c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.712+0000 7f2262ffd640 1 -- 172.21.15.12:0/620837402 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f225c084740 con 0x7f226c06bc00 2023-11-28T21:56:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.712+0000 7f2263fff640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f224c066470 0x7f224c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.712+0000 7f2263fff640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f224c066470 0x7f224c068930 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f226c1546f0 tx=0x7f2254007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.716+0000 7f2262ffd640 1 -- 172.21.15.12:0/620837402 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f225c051600 con 0x7f226c06bc00 2023-11-28T21:56:39.877 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.872+0000 7f2270e9e640 1 -- 172.21.15.12:0/620837402 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f226c04e850 con 0x7f224c066470 2023-11-28T21:56:39.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.872+0000 7f2262ffd640 1 -- 172.21.15.12:0/620837402 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f226c04e850 con 0x7f224c066470 2023-11-28T21:56:39.878 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T21:56:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 -- 172.21.15.12:0/620837402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f224c066470 msgr2=0x7f224c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f224c066470 0x7f224c068930 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f226c1546f0 tx=0x7f2254007d40 comp rx=0 tx=0).stop 2023-11-28T21:56:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 -- 172.21.15.12:0/620837402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c06bc00 msgr2=0x7f226c0ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c06bc00 0x7f226c0ca980 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e930 tx=0x7f225c00edf0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 -- 172.21.15.12:0/620837402 shutdown_connections 2023-11-28T21:56:39.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f224c066470 0x7f224c068930 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f226c15d7f0 0x7f226c14f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f226c153e00 0x7f226c14eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 --2- 172.21.15.12:0/620837402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f226c06bc00 0x7f226c0ca980 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:39.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 -- 172.21.15.12:0/620837402 >> 172.21.15.12:0/620837402 conn(0x7f226c066360 msgr2=0x7f226c0c5750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:39.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 -- 172.21.15.12:0/620837402 shutdown_connections 2023-11-28T21:56:39.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:39.876+0000 7f2270e9e640 1 -- 172.21.15.12:0/620837402 wait complete. 2023-11-28T21:56:39.899 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T21:56:39.899 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T21:56:40.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.044+0000 7f097d6d6640 1 Processor -- start 2023-11-28T21:56:40.049 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.044+0000 7f097d6d6640 1 -- start start 2023-11-28T21:56:40.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.048+0000 7f097d6d6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0978140390 0x7f09780725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.048+0000 7f097d6d6640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09780c6920 con 0x7f097815b0c0 2023-11-28T21:56:40.050 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.048+0000 7f097d6d6640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0978071e20 con 0x7f0978072af0 2023-11-28T21:56:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.048+0000 7f097d6d6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f097815b5b0 con 0x7f0978140390 2023-11-28T21:56:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.048+0000 7f0976ffd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f0978140390 0x7f09780725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.048+0000 7f096ffff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f097815b0c0 0x7f09780c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58132/0 (socket says 172.21.15.12:58132) 2023-11-28T21:56:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.048+0000 7f096ffff640 1 -- 172.21.15.12:0/3870215806 learned_addr learned my addr 172.21.15.12:0/3870215806 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:40.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.048+0000 7f0976ffd640 1 -- 172.21.15.12:0/3870215806 >> v1:172.21.15.144:6789/0 conn(0x7f0978072af0 legacy=0x7f0978162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:40.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.048+0000 7f0976ffd640 1 -- 172.21.15.12:0/3870215806 >> v1:172.21.15.12:6789/0 conn(0x7f097815b0c0 legacy=0x7f09780c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.052 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.048+0000 7f0976ffd640 1 -- 172.21.15.12:0/3870215806 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097815b890 con 0x7f0978140390 2023-11-28T21:56:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f0976ffd640 1 --2- 172.21.15.12:0/3870215806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0978140390 0x7f09780725b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f09680097c0 tx=0x7f096801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16d806df2627f0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f09767fc640 1 -- 172.21.15.12:0/3870215806 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09680ab610 con 0x7f0978140390 2023-11-28T21:56:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f09767fc640 1 -- 172.21.15.12:0/3870215806 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09680ab770 con 0x7f0978140390 2023-11-28T21:56:40.054 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f09767fc640 1 -- 172.21.15.12:0/3870215806 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09680abb00 con 0x7f0978140390 2023-11-28T21:56:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 -- 172.21.15.12:0/3870215806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0978140390 msgr2=0x7f09780725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 --2- 172.21.15.12:0/3870215806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0978140390 0x7f09780725b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f09680097c0 tx=0x7f096801c410 comp rx=0 tx=0).stop 2023-11-28T21:56:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 -- 172.21.15.12:0/3870215806 shutdown_connections 2023-11-28T21:56:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 --2- 172.21.15.12:0/3870215806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0978140390 0x7f09780725b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.055 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 -- 172.21.15.12:0/3870215806 >> 172.21.15.12:0/3870215806 conn(0x7f097806d5e0 msgr2=0x7f097806d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 -- 172.21.15.12:0/3870215806 shutdown_connections 2023-11-28T21:56:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 -- 172.21.15.12:0/3870215806 wait complete. 2023-11-28T21:56:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 Processor -- start 2023-11-28T21:56:40.056 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 -- start start 2023-11-28T21:56:40.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0978072af0 0x7f09780c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0978140390 0x7f0978154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f097815b0c0 0x7f0978162a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09781619b0 con 0x7f0978072af0 2023-11-28T21:56:40.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09781545e0 con 0x7f097815b0c0 2023-11-28T21:56:40.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f097d6d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09781548c0 con 0x7f0978140390 2023-11-28T21:56:40.058 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f0976ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0978072af0 0x7f09780c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f0976ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0978072af0 0x7f09780c27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48786/0 (socket says 172.21.15.12:48786) 2023-11-28T21:56:40.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.052+0000 7f0976ffd640 1 -- 172.21.15.12:0/3844262556 learned_addr learned my addr 172.21.15.12:0/3844262556 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:40.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f09777fe640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f097815b0c0 0x7f0978162a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f096ffff640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0978140390 0x7f0978154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f0976ffd640 1 -- 172.21.15.12:0/3844262556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0978140390 msgr2=0x7f0978154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f0976ffd640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0978140390 0x7f0978154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.059 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f0976ffd640 1 -- 172.21.15.12:0/3844262556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f097815b0c0 msgr2=0x7f0978162a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f0976ffd640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f097815b0c0 0x7f0978162a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.060 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f0976ffd640 1 -- 172.21.15.12:0/3844262556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968009420 con 0x7f0978072af0 2023-11-28T21:56:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f0976ffd640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0978072af0 0x7f09780c27c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f09680ad040 tx=0x7f096801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f096f7fe640 1 -- 172.21.15.12:0/3844262556 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09680b5810 con 0x7f0978072af0 2023-11-28T21:56:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f096f7fe640 1 -- 172.21.15.12:0/3844262556 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09680b5990 con 0x7f0978072af0 2023-11-28T21:56:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0978161080 con 0x7f0978072af0 2023-11-28T21:56:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f096f7fe640 1 -- 172.21.15.12:0/3844262556 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09680b5d40 con 0x7f0978072af0 2023-11-28T21:56:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0978161640 con 0x7f0978072af0 2023-11-28T21:56:40.061 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0978067ed0 con 0x7f0978072af0 2023-11-28T21:56:40.062 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f096f7fe640 1 -- 172.21.15.12:0/3844262556 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f09680bf410 con 0x7f0978072af0 2023-11-28T21:56:40.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f096f7fe640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0958066470 0x7f0958068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.056+0000 7f096f7fe640 1 -- 172.21.15.12:0/3844262556 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f096811fba0 con 0x7f0978072af0 2023-11-28T21:56:40.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.060+0000 7f096ffff640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0958066470 0x7f0958068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.060+0000 7f096f7fe640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] conn(0x7f09580704f0 0x7f0958072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.060+0000 7f096f7fe640 1 -- 172.21.15.12:0/3844262556 --> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0958073060 con 0x7f09580704f0 2023-11-28T21:56:40.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.060+0000 7f096f7fe640 1 -- 172.21.15.12:0/3844262556 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f096811ff10 con 0x7f0978072af0 2023-11-28T21:56:40.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.060+0000 7f09777fe640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] conn(0x7f09580704f0 0x7f0958072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.063 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.060+0000 7f096ffff640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0958066470 0x7f0958068930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0978151370 tx=0x7f0960009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.060+0000 7f09777fe640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] conn(0x7f09580704f0 0x7f0958072990 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.064 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.060+0000 7f096f7fe640 1 -- 172.21.15.12:0/3844262556 <== osd.2 v2:172.21.15.12:6800/2573246535 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0958073060 con 0x7f09580704f0 2023-11-28T21:56:40.077 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 --> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0978070790 con 0x7f09580704f0 2023-11-28T21:56:40.078 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f096f7fe640 1 -- 172.21.15.12:0/3844262556 <== osd.2 v2:172.21.15.12:6800/2573246535 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0970005610 con 0x7f09580704f0 2023-11-28T21:56:40.078 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:56:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-28T21:56:40.078 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:56:40.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] conn(0x7f09580704f0 msgr2=0x7f0958072990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.079 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f097d6d6640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:6800/2573246535,v1:172.21.15.12:6801/2573246535] conn(0x7f09580704f0 0x7f0958072990 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0958066470 msgr2=0x7f0958068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f097d6d6640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f0958066470 0x7f0958068930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0978151370 tx=0x7f0960009040 comp rx=0 tx=0).stop 2023-11-28T21:56:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0978072af0 msgr2=0x7f09780c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f097d6d6640 1 --2- 172.21.15.12:0/3844262556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0978072af0 0x7f09780c27c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f09680ad040 tx=0x7f096801ac50 comp rx=0 tx=0).stop 2023-11-28T21:56:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f09777fe640 1 -- 172.21.15.12:0/3844262556 reap_dead start 2023-11-28T21:56:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 shutdown_connections 2023-11-28T21:56:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.072+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 >> 172.21.15.12:0/3844262556 conn(0x7f097806d5e0 msgr2=0x7f09780c75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.076+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 shutdown_connections 2023-11-28T21:56:40.081 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.076+0000 7f097d6d6640 1 -- 172.21.15.12:0/3844262556 wait complete. 2023-11-28T21:56:40.096 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T21:56:40.241 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.236+0000 7ff4b5cfd640 1 Processor -- start 2023-11-28T21:56:40.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.236+0000 7ff4b5cfd640 1 -- start start 2023-11-28T21:56:40.244 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4b5cfd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4b006e300 0x7ff4b00c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4b5cfd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b0157b70 con 0x7ff4b0151af0 2023-11-28T21:56:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4b5cfd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b0157e50 con 0x7ff4b0151ed0 2023-11-28T21:56:40.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4b5cfd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00707f0 con 0x7ff4b006e300 2023-11-28T21:56:40.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4af7fe640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff4b0151af0 0x7ff4b00c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58146/0 (socket says 172.21.15.12:58146) 2023-11-28T21:56:40.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4af7fe640 1 -- 172.21.15.12:0/1876526865 learned_addr learned my addr 172.21.15.12:0/1876526865 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:40.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4affff640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4b006e300 0x7ff4b00c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127641599 0 0) 0x7ff4b0157b70 con 0x7ff4b0151af0 2023-11-28T21:56:40.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff49c003530 con 0x7ff4b0151af0 2023-11-28T21:56:40.247 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3699761547 0 0) 0x7ff49c003530 con 0x7ff4b0151af0 2023-11-28T21:56:40.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4b006e300 msgr2=0x7ff4b00c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:40.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4b006e300 0x7ff4b00c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 >> v1:172.21.15.144:6789/0 conn(0x7ff4b0151ed0 legacy=0x7ff4b0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b0158130 con 0x7ff4b0151af0 2023-11-28T21:56:40.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7ff4a0001680 con 0x7ff4b0151af0 2023-11-28T21:56:40.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 >> v1:172.21.15.12:6789/0 conn(0x7ff4b0151af0 legacy=0x7ff4b00c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.248 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4b006e300 0x7ff49c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c003fa0 0x7ff49c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c004d10 0x7ff49c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00707f0 con 0x7ff49c003fa0 2023-11-28T21:56:40.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c00baf0 con 0x7ff49c004d10 2023-11-28T21:56:40.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c00bcb0 con 0x7ff4b006e300 2023-11-28T21:56:40.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4a7fff640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c003fa0 0x7ff49c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.249 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4af7fe640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4b006e300 0x7ff49c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4affff640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c004d10 0x7ff49c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4a7fff640 1 -- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4b006e300 msgr2=0x7ff49c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:40.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4a7fff640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4b006e300 0x7ff49c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4a7fff640 1 -- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c004d10 msgr2=0x7ff49c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:40.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4a7fff640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c004d10 0x7ff49c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.250 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.240+0000 7ff4a7fff640 1 -- 172.21.15.12:0/1876526865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c003530 con 0x7ff49c003fa0 2023-11-28T21:56:40.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4a7fff640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c003fa0 0x7ff49c0047d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0068c10 tx=0x7ff49801c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733c1ae51124b3a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.253 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4980052d0 con 0x7ff49c003fa0 2023-11-28T21:56:40.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4980a7050 con 0x7ff49c003fa0 2023-11-28T21:56:40.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4aeffd640 1 -- 172.21.15.12:0/1876526865 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498003600 con 0x7ff49c003fa0 2023-11-28T21:56:40.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1876526865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c003fa0 msgr2=0x7ff49c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4b5cfd640 1 --2- 172.21.15.12:0/1876526865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff49c003fa0 0x7ff49c0047d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0068c10 tx=0x7ff49801c2e0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4affff640 1 -- 172.21.15.12:0/1876526865 reap_dead start 2023-11-28T21:56:40.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1876526865 shutdown_connections 2023-11-28T21:56:40.254 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1876526865 >> 172.21.15.12:0/1876526865 conn(0x7ff4b0066360 msgr2=0x7ff4b0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:40.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1876526865 shutdown_connections 2023-11-28T21:56:40.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.244+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1876526865 wait complete. 2023-11-28T21:56:40.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 Processor -- start 2023-11-28T21:56:40.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 -- start start 2023-11-28T21:56:40.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c003fa0 0x7ff4b01ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4b00c23c0 0x7ff4b014c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.255 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4b01451f0 0x7ff4b0150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b014bcd0 con 0x7ff4b01451f0 2023-11-28T21:56:40.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c90c0 con 0x7ff4b00c23c0 2023-11-28T21:56:40.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c93a0 con 0x7ff49c003fa0 2023-11-28T21:56:40.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4affff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4b01451f0 0x7ff4b0150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4affff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4b01451f0 0x7ff4b0150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48806/0 (socket says 172.21.15.12:48806) 2023-11-28T21:56:40.256 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4affff640 1 -- 172.21.15.12:0/1005229044 learned_addr learned my addr 172.21.15.12:0/1005229044 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4af7fe640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c003fa0 0x7ff4b01ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4a7fff640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4b00c23c0 0x7ff4b014c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4affff640 1 -- 172.21.15.12:0/1005229044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c003fa0 msgr2=0x7ff4b01ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4affff640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c003fa0 0x7ff4b01ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4affff640 1 -- 172.21.15.12:0/1005229044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4b00c23c0 msgr2=0x7ff4b014c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4affff640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4b00c23c0 0x7ff4b014c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.257 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4affff640 1 -- 172.21.15.12:0/1005229044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b014bf70 con 0x7ff4b01451f0 2023-11-28T21:56:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4affff640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4b01451f0 0x7ff4b0150840 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff4b006e250 tx=0x7ff4a80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4acff9640 1 -- 172.21.15.12:0/1005229044 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800e070 con 0x7ff4b01451f0 2023-11-28T21:56:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4acff9640 1 -- 172.21.15.12:0/1005229044 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a8011e10 con 0x7ff4b01451f0 2023-11-28T21:56:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1005229044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b014c250 con 0x7ff4b01451f0 2023-11-28T21:56:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4acff9640 1 -- 172.21.15.12:0/1005229044 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80039a0 con 0x7ff4b01451f0 2023-11-28T21:56:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1005229044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b014ccb0 con 0x7ff4b01451f0 2023-11-28T21:56:40.258 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.248+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1005229044 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b014c440 con 0x7ff4b01451f0 2023-11-28T21:56:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.252+0000 7ff4acff9640 1 -- 172.21.15.12:0/1005229044 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff4a8003b00 con 0x7ff4b01451f0 2023-11-28T21:56:40.259 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.252+0000 7ff4acff9640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff490066470 0x7ff490068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.252+0000 7ff4acff9640 1 -- 172.21.15.12:0/1005229044 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7ff4a8084cf0 con 0x7ff4b01451f0 2023-11-28T21:56:40.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.252+0000 7ff4af7fe640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff490066470 0x7ff490068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.252+0000 7ff4af7fe640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff490066470 0x7ff490068930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0001060 tx=0x7ff4a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.260 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.256+0000 7ff4acff9640 1 -- 172.21.15.12:0/1005229044 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff4a80519b0 con 0x7ff4b01451f0 2023-11-28T21:56:40.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1005229044 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4b01586f0 con 0x7ff490066470 2023-11-28T21:56:40.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4acff9640 1 -- 172.21.15.12:0/1005229044 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff4a0005610 con 0x7ff490066470 2023-11-28T21:56:40.419 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T21:56:40.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1005229044 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff490066470 msgr2=0x7ff490068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff490066470 0x7ff490068930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0001060 tx=0x7ff4a0009040 comp rx=0 tx=0).stop 2023-11-28T21:56:40.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1005229044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4b01451f0 msgr2=0x7ff4b0150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4b01451f0 0x7ff4b0150840 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff4b006e250 tx=0x7ff4a80115a0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1005229044 shutdown_connections 2023-11-28T21:56:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff490066470 0x7ff490068930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4b01451f0 0x7ff4b0150840 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4b00c23c0 0x7ff4b014c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.423 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 --2- 172.21.15.12:0/1005229044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff49c003fa0 0x7ff4b01ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1005229044 >> 172.21.15.12:0/1005229044 conn(0x7ff4b0066360 msgr2=0x7ff4b0147d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1005229044 shutdown_connections 2023-11-28T21:56:40.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.416+0000 7ff4b5cfd640 1 -- 172.21.15.12:0/1005229044 wait complete. 2023-11-28T21:56:40.441 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T21:56:40.442 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T21:56:40.589 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.584+0000 7f138fcc9640 1 Processor -- start 2023-11-28T21:56:40.590 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.584+0000 7f138fcc9640 1 -- start start 2023-11-28T21:56:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138fcc9640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1388150c00 0x7f13881569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138fcc9640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138806e590 con 0x7f138806e100 2023-11-28T21:56:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138fcc9640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880c3570 con 0x7f13880c6dd0 2023-11-28T21:56:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138fcc9640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880c4820 con 0x7f1388150c00 2023-11-28T21:56:40.591 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138d264640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f138806e100 0x7f13880c4c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58150/0 (socket says 172.21.15.12:58150) 2023-11-28T21:56:40.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138d264640 1 -- 172.21.15.12:0/3106185062 learned_addr learned my addr 172.21.15.12:0/3106185062 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:40.592 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138da65640 1 --2- 172.21.15.12:0/3106185062 >> v2:172.21.15.144:3300/0 conn(0x7f1388150c00 0x7f13881569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138ca63640 1 -- 172.21.15.12:0/3106185062 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867934551 0 0) 0x7f138806e590 con 0x7f138806e100 2023-11-28T21:56:40.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138ca63640 1 -- 172.21.15.12:0/3106185062 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1374003530 con 0x7f138806e100 2023-11-28T21:56:40.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138da65640 1 -- 172.21.15.12:0/3106185062 >> v1:172.21.15.144:6789/0 conn(0x7f13880c6dd0 legacy=0x7f1388161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138da65640 1 -- 172.21.15.12:0/3106185062 >> v1:172.21.15.12:6789/0 conn(0x7f138806e100 legacy=0x7f13880c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138ca63640 1 -- 172.21.15.12:0/3106185062 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1061929052 0 0) 0x7f1374003530 con 0x7f138806e100 2023-11-28T21:56:40.593 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.588+0000 7f138da65640 1 -- 172.21.15.12:0/3106185062 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880c3830 con 0x7f1388150c00 2023-11-28T21:56:40.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138da65640 1 --2- 172.21.15.12:0/3106185062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1388150c00 0x7f13881569c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f13840097c0 tx=0x7f138401c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6d263d801ec33c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138ca63640 1 -- 172.21.15.12:0/3106185062 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378001680 con 0x7f1388150c00 2023-11-28T21:56:40.595 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138ca63640 1 -- 172.21.15.12:0/3106185062 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13840a7050 con 0x7f1388150c00 2023-11-28T21:56:40.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138ca63640 1 -- 172.21.15.12:0/3106185062 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840ab600 con 0x7f1388150c00 2023-11-28T21:56:40.596 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138fcc9640 1 -- 172.21.15.12:0/3106185062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1388150c00 msgr2=0x7f13881569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138fcc9640 1 --2- 172.21.15.12:0/3106185062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1388150c00 0x7f13881569c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f13840097c0 tx=0x7f138401c370 comp rx=0 tx=0).stop 2023-11-28T21:56:40.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138fcc9640 1 -- 172.21.15.12:0/3106185062 shutdown_connections 2023-11-28T21:56:40.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138fcc9640 1 --2- 172.21.15.12:0/3106185062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1388150c00 0x7f13881569c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138fcc9640 1 -- 172.21.15.12:0/3106185062 >> 172.21.15.12:0/3106185062 conn(0x7f138806d5e0 msgr2=0x7f138806d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:40.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138fcc9640 1 -- 172.21.15.12:0/3106185062 shutdown_connections 2023-11-28T21:56:40.597 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.592+0000 7f138fcc9640 1 -- 172.21.15.12:0/3106185062 wait complete. 2023-11-28T21:56:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 Processor -- start 2023-11-28T21:56:40.598 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 -- start start 2023-11-28T21:56:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f138806e100 0x7f1388140a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13880c6dd0 0x7f13881f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388150c00 0x7f1388158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.599 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138815ae70 con 0x7f1388150c00 2023-11-28T21:56:40.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1388140fd0 con 0x7f13880c6dd0 2023-11-28T21:56:40.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1388152cf0 con 0x7f138806e100 2023-11-28T21:56:40.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138e266640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388150c00 0x7f1388158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138e266640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388150c00 0x7f1388158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48814/0 (socket says 172.21.15.12:48814) 2023-11-28T21:56:40.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138e266640 1 -- 172.21.15.12:0/2225388621 learned_addr learned my addr 172.21.15.12:0/2225388621 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:40.600 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138da65640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f138806e100 0x7f1388140a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.601 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138d264640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13880c6dd0 0x7f13881f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138e266640 1 -- 172.21.15.12:0/2225388621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f138806e100 msgr2=0x7f1388140a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138e266640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f138806e100 0x7f1388140a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138e266640 1 -- 172.21.15.12:0/2225388621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13880c6dd0 msgr2=0x7f13881f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138e266640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13880c6dd0 0x7f13881f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.602 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138e266640 1 -- 172.21.15.12:0/2225388621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1384009420 con 0x7f1388150c00 2023-11-28T21:56:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138da65640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f138806e100 0x7f1388140a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138d264640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13880c6dd0 0x7f13881f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138e266640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388150c00 0x7f1388158520 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f138000c760 tx=0x7f138000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f137e7fc640 1 -- 172.21.15.12:0/2225388621 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380003b80 con 0x7f1388150c00 2023-11-28T21:56:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f137e7fc640 1 -- 172.21.15.12:0/2225388621 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1380003ce0 con 0x7f1388150c00 2023-11-28T21:56:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f137e7fc640 1 -- 172.21.15.12:0/2225388621 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000aa30 con 0x7f1388150c00 2023-11-28T21:56:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13881f4060 con 0x7f1388150c00 2023-11-28T21:56:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138815a5d0 con 0x7f1388150c00 2023-11-28T21:56:40.603 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.596+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13880c9230 con 0x7f1388150c00 2023-11-28T21:56:40.604 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f137e7fc640 1 -- 172.21.15.12:0/2225388621 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f138000ab90 con 0x7f1388150c00 2023-11-28T21:56:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f137e7fc640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f136c066470 0x7f136c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f137e7fc640 1 -- 172.21.15.12:0/2225388621 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f1380084460 con 0x7f1388150c00 2023-11-28T21:56:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f138da65640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f136c066470 0x7f136c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f137e7fc640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] conn(0x7f136c0704f0 0x7f136c072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f137e7fc640 1 -- 172.21.15.12:0/2225388621 --> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f136c073060 con 0x7f136c0704f0 2023-11-28T21:56:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f137e7fc640 1 -- 172.21.15.12:0/2225388621 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1380086a40 con 0x7f1388150c00 2023-11-28T21:56:40.605 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f138d264640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] conn(0x7f136c0704f0 0x7f136c072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f138da65640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f136c066470 0x7f136c068930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f13840ad000 tx=0x7f13840a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f138d264640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] conn(0x7f136c0704f0 0x7f136c072990 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.606 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.600+0000 7f137e7fc640 1 -- 172.21.15.12:0/2225388621 <== osd.3 v2:172.21.15.12:6824/1952033258 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f136c073060 con 0x7f136c0704f0 2023-11-28T21:56:40.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 --> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f13880c3260 con 0x7f136c0704f0 2023-11-28T21:56:40.618 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f137e7fc640 1 -- 172.21.15.12:0/2225388621 <== osd.3 v2:172.21.15.12:6824/1952033258 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1378007b00 con 0x7f136c0704f0 2023-11-28T21:56:40.619 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:56:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-28T21:56:40.619 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:56:40.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] conn(0x7f136c0704f0 msgr2=0x7f136c072990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:6824/1952033258,v1:172.21.15.12:6825/1952033258] conn(0x7f136c0704f0 0x7f136c072990 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f136c066470 msgr2=0x7f136c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f136c066470 0x7f136c068930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f13840ad000 tx=0x7f13840a6000 comp rx=0 tx=0).stop 2023-11-28T21:56:40.620 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388150c00 msgr2=0x7f1388158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 --2- 172.21.15.12:0/2225388621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1388150c00 0x7f1388158520 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f138000c760 tx=0x7f138000cc20 comp rx=0 tx=0).stop 2023-11-28T21:56:40.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138e266640 1 -- 172.21.15.12:0/2225388621 reap_dead start 2023-11-28T21:56:40.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 shutdown_connections 2023-11-28T21:56:40.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 >> 172.21.15.12:0/2225388621 conn(0x7f138806d5e0 msgr2=0x7f138814f910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:40.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 shutdown_connections 2023-11-28T21:56:40.621 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.616+0000 7f138fcc9640 1 -- 172.21.15.12:0/2225388621 wait complete. 2023-11-28T21:56:40.637 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T21:56:40.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.776+0000 7f036ffe6640 1 Processor -- start 2023-11-28T21:56:40.782 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036ffe6640 1 -- start start 2023-11-28T21:56:40.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036ffe6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 0x7f03680c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036ffe6640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0368151ed0 0x7f03680c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036ffe6640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f036806e300 0x7f03680c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036ffe6640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03680707f0 con 0x7f036806e300 2023-11-28T21:56:40.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036ffe6640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03680c7b90 con 0x7f0368151ed0 2023-11-28T21:56:40.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036ffe6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0368071c20 con 0x7f0368151af0 2023-11-28T21:56:40.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036e583640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f036806e300 0x7f03680c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036e583640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f036806e300 0x7f03680c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48826/0 (socket says 172.21.15.12:48826) 2023-11-28T21:56:40.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036e583640 1 -- 172.21.15.12:0/2740895543 learned_addr learned my addr 172.21.15.12:0/2740895543 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:40.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036dd82640 1 --2- 172.21.15.12:0/2740895543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 0x7f03680c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036d581640 1 --2- 172.21.15.12:0/2740895543 >> v2:172.21.15.144:3300/0 conn(0x7f0368151ed0 0x7f03680c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036e583640 1 -- 172.21.15.12:0/2740895543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 msgr2=0x7f03680c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:40.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036e583640 1 --2- 172.21.15.12:0/2740895543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 0x7f03680c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.785 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036e583640 1 -- 172.21.15.12:0/2740895543 >> v2:172.21.15.144:3300/0 conn(0x7f0368151ed0 msgr2=0x7f03680c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:40.786 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036e583640 1 --2- 172.21.15.12:0/2740895543 >> v2:172.21.15.144:3300/0 conn(0x7f0368151ed0 0x7f03680c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036e583640 1 -- 172.21.15.12:0/2740895543 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0368157b70 con 0x7f036806e300 2023-11-28T21:56:40.787 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.780+0000 7f036d581640 1 --2- 172.21.15.12:0/2740895543 >> v2:172.21.15.144:3300/0 conn(0x7f0368151ed0 0x7f03680c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:40.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036e583640 1 --2- 172.21.15.12:0/2740895543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f036806e300 0x7f03680c4f10 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f036806c7c0 tx=0x7f036001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba53dd84469ba21e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036cd80640 1 -- 172.21.15.12:0/2740895543 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600a5740 con 0x7f036806e300 2023-11-28T21:56:40.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036cd80640 1 -- 172.21.15.12:0/2740895543 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03600a58a0 con 0x7f036806e300 2023-11-28T21:56:40.788 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036cd80640 1 -- 172.21.15.12:0/2740895543 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600a5c30 con 0x7f036806e300 2023-11-28T21:56:40.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036ffe6640 1 -- 172.21.15.12:0/2740895543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f036806e300 msgr2=0x7f03680c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036ffe6640 1 --2- 172.21.15.12:0/2740895543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f036806e300 0x7f03680c4f10 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f036806c7c0 tx=0x7f036001c9d0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036ffe6640 1 -- 172.21.15.12:0/2740895543 shutdown_connections 2023-11-28T21:56:40.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036ffe6640 1 --2- 172.21.15.12:0/2740895543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f036806e300 0x7f03680c4f10 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036ffe6640 1 --2- 172.21.15.12:0/2740895543 >> v2:172.21.15.144:3300/0 conn(0x7f0368151ed0 0x7f03680c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036ffe6640 1 --2- 172.21.15.12:0/2740895543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 0x7f03680c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.789 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036ffe6640 1 -- 172.21.15.12:0/2740895543 >> 172.21.15.12:0/2740895543 conn(0x7f0368066360 msgr2=0x7f0368066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:40.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036ffe6640 1 -- 172.21.15.12:0/2740895543 shutdown_connections 2023-11-28T21:56:40.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.784+0000 7f036ffe6640 1 -- 172.21.15.12:0/2740895543 wait complete. 2023-11-28T21:56:40.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036ffe6640 1 Processor -- start 2023-11-28T21:56:40.790 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036ffe6640 1 -- start start 2023-11-28T21:56:40.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036ffe6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f036806e300 0x7f0368141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036ffe6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 0x7f0368159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036ffe6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03680c95e0 0x7f036806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036ffe6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681f6220 con 0x7f03680c95e0 2023-11-28T21:56:40.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036ffe6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03681f64c0 con 0x7f036806e300 2023-11-28T21:56:40.792 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036ffe6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03681f67a0 con 0x7f0368151af0 2023-11-28T21:56:40.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036e583640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03680c95e0 0x7f036806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036e583640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03680c95e0 0x7f036806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48828/0 (socket says 172.21.15.12:48828) 2023-11-28T21:56:40.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036e583640 1 -- 172.21.15.12:0/44444650 learned_addr learned my addr 172.21.15.12:0/44444650 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:40.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036dd82640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f036806e300 0x7f0368141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036d581640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 0x7f0368159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036e583640 1 -- 172.21.15.12:0/44444650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 msgr2=0x7f0368159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036e583640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 0x7f0368159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.793 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036e583640 1 -- 172.21.15.12:0/44444650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f036806e300 msgr2=0x7f0368141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036e583640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f036806e300 0x7f0368141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036e583640 1 -- 172.21.15.12:0/44444650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0360009b30 con 0x7f03680c95e0 2023-11-28T21:56:40.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036d581640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 0x7f0368159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:40.794 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.788+0000 7f036e583640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03680c95e0 0x7f036806d5d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f03600af040 tx=0x7f0360019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.795 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.792+0000 7f035e7fc640 1 -- 172.21.15.12:0/44444650 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600b8670 con 0x7f03680c95e0 2023-11-28T21:56:40.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.792+0000 7f036ffe6640 1 -- 172.21.15.12:0/44444650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03681f6a80 con 0x7f03680c95e0 2023-11-28T21:56:40.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.792+0000 7f035e7fc640 1 -- 172.21.15.12:0/44444650 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03600b87d0 con 0x7f03680c95e0 2023-11-28T21:56:40.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.792+0000 7f035e7fc640 1 -- 172.21.15.12:0/44444650 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600b8b80 con 0x7f03680c95e0 2023-11-28T21:56:40.796 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.792+0000 7f036ffe6640 1 -- 172.21.15.12:0/44444650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036815a600 con 0x7f03680c95e0 2023-11-28T21:56:40.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.792+0000 7f036ffe6640 1 -- 172.21.15.12:0/44444650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03681f6fa0 con 0x7f03680c95e0 2023-11-28T21:56:40.797 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.792+0000 7f035e7fc640 1 -- 172.21.15.12:0/44444650 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f03600b8ce0 con 0x7f03680c95e0 2023-11-28T21:56:40.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.796+0000 7f035e7fc640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f034c066470 0x7f034c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:40.798 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.796+0000 7f035e7fc640 1 -- 172.21.15.12:0/44444650 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f036011fb10 con 0x7f03680c95e0 2023-11-28T21:56:40.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.796+0000 7f036dd82640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f034c066470 0x7f034c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:40.799 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.796+0000 7f036dd82640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f034c066470 0x7f034c068930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0364009ae0 tx=0x7f0364009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:40.802 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.800+0000 7f035e7fc640 1 -- 172.21.15.12:0/44444650 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03681f6fa0 con 0x7f03680c95e0 2023-11-28T21:56:40.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.956+0000 7f036ffe6640 1 -- 172.21.15.12:0/44444650 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f03681586f0 con 0x7f034c066470 2023-11-28T21:56:40.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.956+0000 7f035e7fc640 1 -- 172.21.15.12:0/44444650 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0364011a00 con 0x7f034c066470 2023-11-28T21:56:40.960 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T21:56:40.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 -- 172.21.15.12:0/44444650 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f034c066470 msgr2=0x7f034c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f034c066470 0x7f034c068930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0364009ae0 tx=0x7f0364009190 comp rx=0 tx=0).stop 2023-11-28T21:56:40.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 -- 172.21.15.12:0/44444650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03680c95e0 msgr2=0x7f036806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:40.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03680c95e0 0x7f036806d5d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f03600af040 tx=0x7f0360019770 comp rx=0 tx=0).stop 2023-11-28T21:56:40.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 -- 172.21.15.12:0/44444650 shutdown_connections 2023-11-28T21:56:40.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f034c066470 0x7f034c068930 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f03680c95e0 0x7f036806d5d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0368151af0 0x7f0368159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 --2- 172.21.15.12:0/44444650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f036806e300 0x7f0368141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:40.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 -- 172.21.15.12:0/44444650 >> 172.21.15.12:0/44444650 conn(0x7f0368066360 msgr2=0x7f0368150900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:40.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 -- 172.21.15.12:0/44444650 shutdown_connections 2023-11-28T21:56:40.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:40.960+0000 7f036ffe6640 1 -- 172.21.15.12:0/44444650 wait complete. 2023-11-28T21:56:40.982 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T21:56:40.982 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T21:56:41.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.124+0000 7ff801993640 1 Processor -- start 2023-11-28T21:56:41.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.124+0000 7ff801993640 1 -- start start 2023-11-28T21:56:41.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff801993640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff7fc1543e0 0x7ff7fc071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff801993640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff7fc0ccdb0 0x7ff7fc0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff801993640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7fc072370 0x7ff7fc0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff801993640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7fc1547c0 con 0x7ff7fc1543e0 2023-11-28T21:56:41.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff801993640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7fc072850 con 0x7ff7fc0ccdb0 2023-11-28T21:56:41.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff801993640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc159ff0 con 0x7ff7fc072370 2023-11-28T21:56:41.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff800991640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff7fc1543e0 0x7ff7fc071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff800991640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff7fc1543e0 0x7ff7fc071e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48838/0 (socket says 172.21.15.12:48838) 2023-11-28T21:56:41.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff800991640 1 -- 172.21.15.12:0/4028170675 learned_addr learned my addr 172.21.15.12:0/4028170675 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:41.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff7fbfff640 1 --2- 172.21.15.12:0/4028170675 >> v2:172.21.15.144:3300/0 conn(0x7ff7fc0ccdb0 0x7ff7fc0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff801192640 1 --2- 172.21.15.12:0/4028170675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7fc072370 0x7ff7fc0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff800991640 1 -- 172.21.15.12:0/4028170675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7fc072370 msgr2=0x7ff7fc0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:41.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff800991640 1 --2- 172.21.15.12:0/4028170675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7fc072370 0x7ff7fc0cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff800991640 1 -- 172.21.15.12:0/4028170675 >> v2:172.21.15.144:3300/0 conn(0x7ff7fc0ccdb0 msgr2=0x7ff7fc0cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:41.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff800991640 1 --2- 172.21.15.12:0/4028170675 >> v2:172.21.15.144:3300/0 conn(0x7ff7fc0ccdb0 0x7ff7fc0cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.128+0000 7ff800991640 1 -- 172.21.15.12:0/4028170675 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc15a200 con 0x7ff7fc1543e0 2023-11-28T21:56:41.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff800991640 1 --2- 172.21.15.12:0/4028170675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7fc1543e0 0x7ff7fc071e30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc08d880 tx=0x7ff7f001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43e2182651f43f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff7fb7fe640 1 -- 172.21.15.12:0/4028170675 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00a89b0 con 0x7ff7fc1543e0 2023-11-28T21:56:41.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff7fb7fe640 1 -- 172.21.15.12:0/4028170675 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f00a8b10 con 0x7ff7fc1543e0 2023-11-28T21:56:41.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff7fb7fe640 1 -- 172.21.15.12:0/4028170675 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00a8ea0 con 0x7ff7fc1543e0 2023-11-28T21:56:41.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 -- 172.21.15.12:0/4028170675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7fc1543e0 msgr2=0x7ff7fc071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 --2- 172.21.15.12:0/4028170675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7fc1543e0 0x7ff7fc071e30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc08d880 tx=0x7ff7f001ecf0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 -- 172.21.15.12:0/4028170675 shutdown_connections 2023-11-28T21:56:41.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 --2- 172.21.15.12:0/4028170675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7fc072370 0x7ff7fc0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 --2- 172.21.15.12:0/4028170675 >> v2:172.21.15.144:3300/0 conn(0x7ff7fc0ccdb0 0x7ff7fc0cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 --2- 172.21.15.12:0/4028170675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7fc1543e0 0x7ff7fc071e30 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 -- 172.21.15.12:0/4028170675 >> 172.21.15.12:0/4028170675 conn(0x7ff7fc066360 msgr2=0x7ff7fc066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:41.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 -- 172.21.15.12:0/4028170675 shutdown_connections 2023-11-28T21:56:41.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 -- 172.21.15.12:0/4028170675 wait complete. 2023-11-28T21:56:41.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 Processor -- start 2023-11-28T21:56:41.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.132+0000 7ff801993640 1 -- start start 2023-11-28T21:56:41.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801993640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7fc072370 0x7ff7fc0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801993640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7fc0ccdb0 0x7ff7fc14f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801993640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7fc1543e0 0x7ff7fc14ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801993640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc14e8a0 con 0x7ff7fc1543e0 2023-11-28T21:56:41.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801993640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1f34f0 con 0x7ff7fc072370 2023-11-28T21:56:41.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801993640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc14eaf0 con 0x7ff7fc0ccdb0 2023-11-28T21:56:41.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801192640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7fc1543e0 0x7ff7fc14ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801192640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7fc1543e0 0x7ff7fc14ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48840/0 (socket says 172.21.15.12:48840) 2023-11-28T21:56:41.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801192640 1 -- 172.21.15.12:0/2905447413 learned_addr learned my addr 172.21.15.12:0/2905447413 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:41.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff800991640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7fc072370 0x7ff7fc0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff7fbfff640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7fc0ccdb0 0x7ff7fc14f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801192640 1 -- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7fc0ccdb0 msgr2=0x7ff7fc14f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801192640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7fc0ccdb0 0x7ff7fc14f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801192640 1 -- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7fc072370 msgr2=0x7ff7fc0cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801192640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7fc072370 0x7ff7fc0cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801192640 1 -- 172.21.15.12:0/2905447413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f00094d0 con 0x7ff7fc1543e0 2023-11-28T21:56:41.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801192640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7fc1543e0 0x7ff7fc14ee10 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4007e30 tx=0x7ff7f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff7f97fa640 1 -- 172.21.15.12:0/2905447413 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4014420 con 0x7ff7fc1543e0 2023-11-28T21:56:41.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff7f97fa640 1 -- 172.21.15.12:0/2905447413 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f4014580 con 0x7ff7fc1543e0 2023-11-28T21:56:41.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff7f97fa640 1 -- 172.21.15.12:0/2905447413 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4014930 con 0x7ff7fc1543e0 2023-11-28T21:56:41.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7fc1f37b0 con 0x7ff7fc1543e0 2023-11-28T21:56:41.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc1f3d70 con 0x7ff7fc1543e0 2023-11-28T21:56:41.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.136+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7fc15aa60 con 0x7ff7fc1543e0 2023-11-28T21:56:41.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.140+0000 7ff7f97fa640 1 -- 172.21.15.12:0/2905447413 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff7f4014a90 con 0x7ff7fc1543e0 2023-11-28T21:56:41.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.140+0000 7ff7f97fa640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 0x7ff7e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.140+0000 7ff7f97fa640 1 -- 172.21.15.12:0/2905447413 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7ff7f4085920 con 0x7ff7fc1543e0 2023-11-28T21:56:41.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.140+0000 7ff800991640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 0x7ff7e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.140+0000 7ff7f97fa640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] conn(0x7ff7e00704f0 0x7ff7e0072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.140+0000 7ff7f97fa640 1 -- 172.21.15.12:0/2905447413 --> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7e0073060 con 0x7ff7e00704f0 2023-11-28T21:56:41.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.140+0000 7ff7f97fa640 1 -- 172.21.15.12:0/2905447413 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f4085c90 con 0x7ff7fc1543e0 2023-11-28T21:56:41.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.140+0000 7ff7fbfff640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] conn(0x7ff7e00704f0 0x7ff7e0072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.140+0000 7ff800991640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 0x7ff7e0068930 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00b2000 tx=0x7ff7f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.140+0000 7ff7fbfff640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] conn(0x7ff7e00704f0 0x7ff7e0072990 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.144+0000 7ff7f97fa640 1 -- 172.21.15.12:0/2905447413 <== osd.4 v2:172.21.15.144:6800/3927315436 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff7e0073060 con 0x7ff7e00704f0 2023-11-28T21:56:41.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 --> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff7fc15a780 con 0x7ff7e00704f0 2023-11-28T21:56:41.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff7f97fa640 1 -- 172.21.15.12:0/2905447413 <== osd.4 v2:172.21.15.144:6800/3927315436 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff7e800bb00 con 0x7ff7e00704f0 2023-11-28T21:56:41.160 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:56:41.160 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-28T21:56:41.160 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:56:41.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] conn(0x7ff7e00704f0 msgr2=0x7ff7e0072990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.144:6800/3927315436,v1:172.21.15.144:6801/3927315436] conn(0x7ff7e00704f0 0x7ff7e0072990 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 msgr2=0x7ff7e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7ff7e0066470 0x7ff7e0068930 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00b2000 tx=0x7ff7f0009040 comp rx=0 tx=0).stop 2023-11-28T21:56:41.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7fc1543e0 msgr2=0x7ff7fc14ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 --2- 172.21.15.12:0/2905447413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7fc1543e0 0x7ff7fc14ee10 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4007e30 tx=0x7ff7f400e4b0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801192640 1 -- 172.21.15.12:0/2905447413 reap_dead start 2023-11-28T21:56:41.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 shutdown_connections 2023-11-28T21:56:41.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 >> 172.21.15.12:0/2905447413 conn(0x7ff7fc066360 msgr2=0x7ff7fc0b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:41.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 shutdown_connections 2023-11-28T21:56:41.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.156+0000 7ff801993640 1 -- 172.21.15.12:0/2905447413 wait complete. 2023-11-28T21:56:41.179 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T21:56:41.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f474a9b5640 1 Processor -- start 2023-11-28T21:56:41.324 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f474a9b5640 1 -- start start 2023-11-28T21:56:41.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f474a9b5640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4744153a40 0x7f47440ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f474a9b5640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f47440ca7c0 0x7f47440c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f474a9b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47440c5bd0 0x7f474415d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f474a9b5640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47440c5270 con 0x7f4744153a40 2023-11-28T21:56:41.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f474a9b5640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474415df30 con 0x7f47440ca7c0 2023-11-28T21:56:41.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f474a9b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47440c45f0 con 0x7f47440c5bd0 2023-11-28T21:56:41.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f47499b3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4744153a40 0x7f47440ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f47499b3640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4744153a40 0x7f47440ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48850/0 (socket says 172.21.15.12:48850) 2023-11-28T21:56:41.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f47499b3640 1 -- 172.21.15.12:0/1720821048 learned_addr learned my addr 172.21.15.12:0/1720821048 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:41.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f47491b2640 1 --2- 172.21.15.12:0/1720821048 >> v2:172.21.15.144:3300/0 conn(0x7f47440ca7c0 0x7f47440c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f474a1b4640 1 --2- 172.21.15.12:0/1720821048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47440c5bd0 0x7f474415d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f47499b3640 1 -- 172.21.15.12:0/1720821048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47440c5bd0 msgr2=0x7f474415d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:41.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f47499b3640 1 --2- 172.21.15.12:0/1720821048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47440c5bd0 0x7f474415d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f47499b3640 1 -- 172.21.15.12:0/1720821048 >> v2:172.21.15.144:3300/0 conn(0x7f47440ca7c0 msgr2=0x7f47440c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:41.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f47499b3640 1 --2- 172.21.15.12:0/1720821048 >> v2:172.21.15.144:3300/0 conn(0x7f47440ca7c0 0x7f47440c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.320+0000 7f47499b3640 1 -- 172.21.15.12:0/1720821048 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47440c48d0 con 0x7f4744153a40 2023-11-28T21:56:41.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f47499b3640 1 --2- 172.21.15.12:0/1720821048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4744153a40 0x7f47440ca280 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f474406ba20 tx=0x7f474001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6bd43eaecb0ad3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f47489b1640 1 -- 172.21.15.12:0/1720821048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400a89b0 con 0x7f4744153a40 2023-11-28T21:56:41.328 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f47489b1640 1 -- 172.21.15.12:0/1720821048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47400a8b10 con 0x7f4744153a40 2023-11-28T21:56:41.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f47489b1640 1 -- 172.21.15.12:0/1720821048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400a8ea0 con 0x7f4744153a40 2023-11-28T21:56:41.329 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f474a9b5640 1 -- 172.21.15.12:0/1720821048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4744153a40 msgr2=0x7f47440ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f474a9b5640 1 --2- 172.21.15.12:0/1720821048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4744153a40 0x7f47440ca280 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f474406ba20 tx=0x7f474001ecf0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f474a9b5640 1 -- 172.21.15.12:0/1720821048 shutdown_connections 2023-11-28T21:56:41.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f474a9b5640 1 --2- 172.21.15.12:0/1720821048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47440c5bd0 0x7f474415d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f474a9b5640 1 --2- 172.21.15.12:0/1720821048 >> v2:172.21.15.144:3300/0 conn(0x7f47440ca7c0 0x7f47440c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f474a9b5640 1 --2- 172.21.15.12:0/1720821048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4744153a40 0x7f47440ca280 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f474a9b5640 1 -- 172.21.15.12:0/1720821048 >> 172.21.15.12:0/1720821048 conn(0x7f474406bda0 msgr2=0x7f474406c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:41.330 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f474a9b5640 1 -- 172.21.15.12:0/1720821048 shutdown_connections 2023-11-28T21:56:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.324+0000 7f474a9b5640 1 -- 172.21.15.12:0/1720821048 wait complete. 2023-11-28T21:56:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a9b5640 1 Processor -- start 2023-11-28T21:56:41.331 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a9b5640 1 -- start start 2023-11-28T21:56:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a9b5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47440c5bd0 0x7f4744150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a9b5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47440ca7c0 0x7f47441f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a9b5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4744153a40 0x7f47441f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a9b5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47441f1850 con 0x7f47440ca7c0 2023-11-28T21:56:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a9b5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47441e3210 con 0x7f4744153a40 2023-11-28T21:56:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a9b5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47441e35c0 con 0x7f47440c5bd0 2023-11-28T21:56:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f47491b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47440ca7c0 0x7f47441f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.333 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f47491b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47440ca7c0 0x7f47441f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48860/0 (socket says 172.21.15.12:48860) 2023-11-28T21:56:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f47491b2640 1 -- 172.21.15.12:0/2319185502 learned_addr learned my addr 172.21.15.12:0/2319185502 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f47499b3640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47440c5bd0 0x7f4744150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a1b4640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4744153a40 0x7f47441f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f47491b2640 1 -- 172.21.15.12:0/2319185502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47440c5bd0 msgr2=0x7f4744150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f47491b2640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47440c5bd0 0x7f4744150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f47491b2640 1 -- 172.21.15.12:0/2319185502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4744153a40 msgr2=0x7f47441f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f47491b2640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4744153a40 0x7f47441f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.334 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f47491b2640 1 -- 172.21.15.12:0/2319185502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47400094d0 con 0x7f47440ca7c0 2023-11-28T21:56:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f47491b2640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47440ca7c0 0x7f47441f3370 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f473400d800 tx=0x7f473400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.335 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f473a7fc640 1 -- 172.21.15.12:0/2319185502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473400b660 con 0x7f47440ca7c0 2023-11-28T21:56:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f473a7fc640 1 -- 172.21.15.12:0/2319185502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473400b7c0 con 0x7f47440ca7c0 2023-11-28T21:56:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f473a7fc640 1 -- 172.21.15.12:0/2319185502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473400bb30 con 0x7f47440ca7c0 2023-11-28T21:56:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a9b5640 1 -- 172.21.15.12:0/2319185502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47441506b0 con 0x7f47440ca7c0 2023-11-28T21:56:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.328+0000 7f474a9b5640 1 -- 172.21.15.12:0/2319185502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4744150980 con 0x7f47440ca7c0 2023-11-28T21:56:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.332+0000 7f474a9b5640 1 -- 172.21.15.12:0/2319185502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47440c5670 con 0x7f47440ca7c0 2023-11-28T21:56:41.336 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.332+0000 7f473a7fc640 1 -- 172.21.15.12:0/2319185502 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f473400bc90 con 0x7f47440ca7c0 2023-11-28T21:56:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.332+0000 7f473a7fc640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4728066470 0x7f4728068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.332+0000 7f473a7fc640 1 -- 172.21.15.12:0/2319185502 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f4734084a70 con 0x7f47440ca7c0 2023-11-28T21:56:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.332+0000 7f47499b3640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4728066470 0x7f4728068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.337 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.332+0000 7f47499b3640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4728066470 0x7f4728068930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f47400b2000 tx=0x7f4740009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.340 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.336+0000 7f473a7fc640 1 -- 172.21.15.12:0/2319185502 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4734051750 con 0x7f47440ca7c0 2023-11-28T21:56:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.496+0000 7f474a9b5640 1 -- 172.21.15.12:0/2319185502 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f47441535b0 con 0x7f4728066470 2023-11-28T21:56:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.496+0000 7f473a7fc640 1 -- 172.21.15.12:0/2319185502 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f47441535b0 con 0x7f4728066470 2023-11-28T21:56:41.500 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T21:56:41.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 -- 172.21.15.12:0/2319185502 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4728066470 msgr2=0x7f4728068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.502 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4728066470 0x7f4728068930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f47400b2000 tx=0x7f4740009040 comp rx=0 tx=0).stop 2023-11-28T21:56:41.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 -- 172.21.15.12:0/2319185502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47440ca7c0 msgr2=0x7f47441f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47440ca7c0 0x7f47441f3370 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f473400d800 tx=0x7f473400dcc0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 -- 172.21.15.12:0/2319185502 shutdown_connections 2023-11-28T21:56:41.503 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4728066470 0x7f4728068930 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4744153a40 0x7f47441f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47440ca7c0 0x7f47441f3370 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 --2- 172.21.15.12:0/2319185502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47440c5bd0 0x7f4744150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 -- 172.21.15.12:0/2319185502 >> 172.21.15.12:0/2319185502 conn(0x7f474406bda0 msgr2=0x7f47440ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:41.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 -- 172.21.15.12:0/2319185502 shutdown_connections 2023-11-28T21:56:41.504 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.500+0000 7f474a9b5640 1 -- 172.21.15.12:0/2319185502 wait complete. 2023-11-28T21:56:41.521 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T21:56:41.521 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T21:56:41.671 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a3ce3640 1 Processor -- start 2023-11-28T21:56:41.672 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a3ce3640 1 -- start start 2023-11-28T21:56:41.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a3ce3640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd39c150c00 0x7fd39c1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a3ce3640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39c06e590 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a3ce3640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39c0c3570 con 0x7fd39c06e100 2023-11-28T21:56:41.673 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a3ce3640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39c0c4820 con 0x7fd39c150c00 2023-11-28T21:56:41.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a2280640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd39c0c6dd0 0x7fd39c161340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58162/0 (socket says 172.21.15.12:58162) 2023-11-28T21:56:41.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a2280640 1 -- 172.21.15.12:0/3112578474 learned_addr learned my addr 172.21.15.12:0/3112578474 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:41.674 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a1a7f640 1 --2- 172.21.15.12:0/3112578474 >> v2:172.21.15.144:3300/0 conn(0x7fd39c150c00 0x7fd39c1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a0a7d640 1 -- 172.21.15.12:0/3112578474 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656569034 0 0) 0x7fd39c06e590 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.668+0000 7fd3a0a7d640 1 -- 172.21.15.12:0/3112578474 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd388003530 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.672+0000 7fd3a1a7f640 1 -- 172.21.15.12:0/3112578474 >> v1:172.21.15.144:6789/0 conn(0x7fd39c06e100 legacy=0x7fd39c0c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.675 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.672+0000 7fd3a1a7f640 1 -- 172.21.15.12:0/3112578474 >> v1:172.21.15.12:6789/0 conn(0x7fd39c0c6dd0 legacy=0x7fd39c161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.676 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.672+0000 7fd3a1a7f640 1 -- 172.21.15.12:0/3112578474 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c0c3830 con 0x7fd39c150c00 2023-11-28T21:56:41.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.672+0000 7fd3a1a7f640 1 --2- 172.21.15.12:0/3112578474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c150c00 0x7fd39c1569c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd3980097c0 tx=0x7fd39801c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f689208f6f5230f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.672+0000 7fd3a0a7d640 1 -- 172.21.15.12:0/3112578474 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980ab600 con 0x7fd39c150c00 2023-11-28T21:56:41.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.672+0000 7fd3a0a7d640 1 -- 172.21.15.12:0/3112578474 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3980a7050 con 0x7fd39c150c00 2023-11-28T21:56:41.677 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.672+0000 7fd3a0a7d640 1 -- 172.21.15.12:0/3112578474 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980ab960 con 0x7fd39c150c00 2023-11-28T21:56:41.678 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.672+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/3112578474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c150c00 msgr2=0x7fd39c1569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.672+0000 7fd3a3ce3640 1 --2- 172.21.15.12:0/3112578474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c150c00 0x7fd39c1569c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd3980097c0 tx=0x7fd39801c370 comp rx=0 tx=0).stop 2023-11-28T21:56:41.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/3112578474 shutdown_connections 2023-11-28T21:56:41.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 --2- 172.21.15.12:0/3112578474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c150c00 0x7fd39c1569c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/3112578474 >> 172.21.15.12:0/3112578474 conn(0x7fd39c06d5e0 msgr2=0x7fd39c06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:41.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/3112578474 shutdown_connections 2023-11-28T21:56:41.679 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/3112578474 wait complete. 2023-11-28T21:56:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 Processor -- start 2023-11-28T21:56:41.680 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 -- start start 2023-11-28T21:56:41.682 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd39c06e100 0x7fd39c1f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd39c0c6dd0 0x7fd39c141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c150c00 0x7fd39c142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c071930 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c071d50 con 0x7fd39c150c00 2023-11-28T21:56:41.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a3ce3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c071020 con 0x7fd39c06e100 2023-11-28T21:56:41.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a127e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd39c0c6dd0 0x7fd39c141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.683 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a127e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd39c0c6dd0 0x7fd39c141b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48874/0 (socket says 172.21.15.12:48874) 2023-11-28T21:56:41.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a127e640 1 -- 172.21.15.12:0/2513549982 learned_addr learned my addr 172.21.15.12:0/2513549982 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:41.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a2280640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c150c00 0x7fd39c142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a1a7f640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd39c06e100 0x7fd39c1f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a127e640 1 -- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd39c06e100 msgr2=0x7fd39c1f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a127e640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd39c06e100 0x7fd39c1f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.684 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a127e640 1 -- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c150c00 msgr2=0x7fd39c142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a127e640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c150c00 0x7fd39c142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a127e640 1 -- 172.21.15.12:0/2513549982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398009420 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.676+0000 7fd3a127e640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd39c0c6dd0 0x7fd39c141b10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00c370 tx=0x7fd38c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3927fc640 1 -- 172.21.15.12:0/2513549982 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c0052d0 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3927fc640 1 -- 172.21.15.12:0/2513549982 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38c005430 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.685 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3927fc640 1 -- 172.21.15.12:0/2513549982 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c007260 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39c153dc0 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39c071270 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.686 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd39c0c3260 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3927fc640 1 -- 172.21.15.12:0/2513549982 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd38c0073c0 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3927fc640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3800664a0 0x7fd380068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.687 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3927fc640 1 -- 172.21.15.12:0/2513549982 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fd38c0844c0 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3a1a7f640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3800664a0 0x7fd380068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3927fc640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] conn(0x7fd380070520 0x7fd3800729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3927fc640 1 -- 172.21.15.12:0/2513549982 --> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd380073090 con 0x7fd380070520 2023-11-28T21:56:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3927fc640 1 -- 172.21.15.12:0/2513549982 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd38c084830 con 0x7fd39c0c6dd0 2023-11-28T21:56:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3a2280640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] conn(0x7fd380070520 0x7fd3800729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.688 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.680+0000 7fd3a1a7f640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3800664a0 0x7fd380068960 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd3980ad000 tx=0x7fd3980a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.684+0000 7fd3a2280640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] conn(0x7fd380070520 0x7fd3800729c0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.689 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.684+0000 7fd3927fc640 1 -- 172.21.15.12:0/2513549982 <== osd.5 v2:172.21.15.144:6816/2546756774 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd380073090 con 0x7fd380070520 2023-11-28T21:56:41.701 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.696+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 --> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd39c0c4440 con 0x7fd380070520 2023-11-28T21:56:41.702 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3927fc640 1 -- 172.21.15.12:0/2513549982 <== osd.5 v2:172.21.15.144:6816/2546756774 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd3940070b0 con 0x7fd380070520 2023-11-28T21:56:41.702 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:56:41.702 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-28T21:56:41.703 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:56:41.703 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] conn(0x7fd380070520 msgr2=0x7fd3800729c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a3ce3640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.144:6816/2546756774,v1:172.21.15.144:6817/2546756774] conn(0x7fd380070520 0x7fd3800729c0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3800664a0 msgr2=0x7fd380068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a3ce3640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd3800664a0 0x7fd380068960 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd3980ad000 tx=0x7fd3980a6000 comp rx=0 tx=0).stop 2023-11-28T21:56:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd39c0c6dd0 msgr2=0x7fd39c141b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a3ce3640 1 --2- 172.21.15.12:0/2513549982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd39c0c6dd0 0x7fd39c141b10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00c370 tx=0x7fd38c00c830 comp rx=0 tx=0).stop 2023-11-28T21:56:41.704 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a2280640 1 -- 172.21.15.12:0/2513549982 reap_dead start 2023-11-28T21:56:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 shutdown_connections 2023-11-28T21:56:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 >> 172.21.15.12:0/2513549982 conn(0x7fd39c06d5e0 msgr2=0x7fd39c146c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 shutdown_connections 2023-11-28T21:56:41.705 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.700+0000 7fd3a3ce3640 1 -- 172.21.15.12:0/2513549982 wait complete. 2023-11-28T21:56:41.723 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T21:56:41.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.864+0000 7f2f2e832640 1 Processor -- start 2023-11-28T21:56:41.868 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.864+0000 7f2f2e832640 1 -- start start 2023-11-28T21:56:41.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.864+0000 7f2f2e832640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2f28151af0 0x7f2f280c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.864+0000 7f2f2e832640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f28151ed0 0x7f2f280c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.864+0000 7f2f2e832640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f28157b70 con 0x7f2f2806e300 2023-11-28T21:56:41.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.864+0000 7f2f2e832640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f280707f0 con 0x7f2f28151ed0 2023-11-28T21:56:41.870 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.864+0000 7f2f2e832640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f280c7b90 con 0x7f2f28151af0 2023-11-28T21:56:41.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.864+0000 7f2f277fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f28151ed0 0x7f2f280c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.864+0000 7f2f277fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2f28151ed0 0x7f2f280c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48882/0 (socket says 172.21.15.12:48882) 2023-11-28T21:56:41.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.864+0000 7f2f277fe640 1 -- 172.21.15.12:0/3236312194 learned_addr learned my addr 172.21.15.12:0/3236312194 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:41.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.868+0000 7f2f27fff640 1 --2- 172.21.15.12:0/3236312194 >> v2:172.21.15.144:3300/0 conn(0x7f2f28151af0 0x7f2f280c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.868+0000 7f2f277fe640 1 -- 172.21.15.12:0/3236312194 >> v2:172.21.15.144:3300/0 conn(0x7f2f28151af0 msgr2=0x7f2f280c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:41.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.868+0000 7f2f277fe640 1 --2- 172.21.15.12:0/3236312194 >> v2:172.21.15.144:3300/0 conn(0x7f2f28151af0 0x7f2f280c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.868+0000 7f2f277fe640 1 -- 172.21.15.12:0/3236312194 >> v1:172.21.15.144:6789/0 conn(0x7f2f2806e300 legacy=0x7f2f28072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.871 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.868+0000 7f2f277fe640 1 -- 172.21.15.12:0/3236312194 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f28157e50 con 0x7f2f28151ed0 2023-11-28T21:56:41.872 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.868+0000 7f2f277fe640 1 --2- 172.21.15.12:0/3236312194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151ed0 0x7f2f280c49d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2f280c7d20 tx=0x7f2f1c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76da1653f4ab33f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.868+0000 7f2f26ffd640 1 -- 172.21.15.12:0/3236312194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0ab600 con 0x7f2f28151ed0 2023-11-28T21:56:41.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.868+0000 7f2f26ffd640 1 -- 172.21.15.12:0/3236312194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c0a7050 con 0x7f2f28151ed0 2023-11-28T21:56:41.873 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.868+0000 7f2f26ffd640 1 -- 172.21.15.12:0/3236312194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0ab980 con 0x7f2f28151ed0 2023-11-28T21:56:41.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 -- 172.21.15.12:0/3236312194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151ed0 msgr2=0x7f2f280c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.874 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 --2- 172.21.15.12:0/3236312194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151ed0 0x7f2f280c49d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2f280c7d20 tx=0x7f2f1c01c500 comp rx=0 tx=0).stop 2023-11-28T21:56:41.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 -- 172.21.15.12:0/3236312194 shutdown_connections 2023-11-28T21:56:41.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 --2- 172.21.15.12:0/3236312194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f28151ed0 0x7f2f280c49d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 --2- 172.21.15.12:0/3236312194 >> v2:172.21.15.144:3300/0 conn(0x7f2f28151af0 0x7f2f280c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 -- 172.21.15.12:0/3236312194 >> 172.21.15.12:0/3236312194 conn(0x7f2f28066360 msgr2=0x7f2f28066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:41.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 -- 172.21.15.12:0/3236312194 shutdown_connections 2023-11-28T21:56:41.875 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 -- 172.21.15.12:0/3236312194 wait complete. 2023-11-28T21:56:41.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 Processor -- start 2023-11-28T21:56:41.876 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 -- start start 2023-11-28T21:56:41.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 0x7f2f281f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f28151af0 0x7f2f280ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.878 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f280c93e0 0x7f2f2813ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f280c9f70 con 0x7f2f280c93e0 2023-11-28T21:56:41.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281578f0 con 0x7f2f28151af0 2023-11-28T21:56:41.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2e832640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281f3dc0 con 0x7f2f2806e300 2023-11-28T21:56:41.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2cdcf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f280c93e0 0x7f2f2813ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2cdcf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f280c93e0 0x7f2f2813ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48892/0 (socket says 172.21.15.12:48892) 2023-11-28T21:56:41.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2cdcf640 1 -- 172.21.15.12:0/883307975 learned_addr learned my addr 172.21.15.12:0/883307975 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:41.879 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f277fe640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f28151af0 0x7f2f280ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f27fff640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 0x7f2f281f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2cdcf640 1 -- 172.21.15.12:0/883307975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 msgr2=0x7f2f281f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2cdcf640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 0x7f2f281f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2cdcf640 1 -- 172.21.15.12:0/883307975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f28151af0 msgr2=0x7f2f280ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:41.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2cdcf640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f28151af0 0x7f2f280ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:41.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.872+0000 7f2f2cdcf640 1 -- 172.21.15.12:0/883307975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0094d0 con 0x7f2f280c93e0 2023-11-28T21:56:41.880 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f2cdcf640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f280c93e0 0x7f2f2813ed60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c7a0 tx=0x7f2f2000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f24ff9640 1 -- 172.21.15.12:0/883307975 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20003b80 con 0x7f2f280c93e0 2023-11-28T21:56:41.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f24ff9640 1 -- 172.21.15.12:0/883307975 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f20003ce0 con 0x7f2f280c93e0 2023-11-28T21:56:41.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f24ff9640 1 -- 172.21.15.12:0/883307975 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000aa30 con 0x7f2f280c93e0 2023-11-28T21:56:41.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f2e832640 1 -- 172.21.15.12:0/883307975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f280c99b0 con 0x7f2f280c93e0 2023-11-28T21:56:41.881 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f2e832640 1 -- 172.21.15.12:0/883307975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f28155af0 con 0x7f2f280c93e0 2023-11-28T21:56:41.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f2e832640 1 -- 172.21.15.12:0/883307975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f28155950 con 0x7f2f280c93e0 2023-11-28T21:56:41.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f24ff9640 1 -- 172.21.15.12:0/883307975 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2f2000ab90 con 0x7f2f280c93e0 2023-11-28T21:56:41.882 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f24ff9640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f140664a0 0x7f2f14068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:41.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f24ff9640 1 -- 172.21.15.12:0/883307975 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f2f200842e0 con 0x7f2f280c93e0 2023-11-28T21:56:41.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.876+0000 7f2f27fff640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f140664a0 0x7f2f14068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:41.883 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.880+0000 7f2f27fff640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f140664a0 0x7f2f14068960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009b60 tx=0x7f2f18009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:41.885 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:41.880+0000 7f2f24ff9640 1 -- 172.21.15.12:0/883307975 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2f28155950 con 0x7f2f280c93e0 2023-11-28T21:56:42.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 -- 172.21.15.12:0/883307975 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f28155950 con 0x7f2f140664a0 2023-11-28T21:56:42.043 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f24ff9640 1 -- 172.21.15.12:0/883307975 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2f28155950 con 0x7f2f140664a0 2023-11-28T21:56:42.044 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T21:56:42.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 -- 172.21.15.12:0/883307975 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f140664a0 msgr2=0x7f2f14068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.045 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f140664a0 0x7f2f14068960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009b60 tx=0x7f2f18009190 comp rx=0 tx=0).stop 2023-11-28T21:56:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 -- 172.21.15.12:0/883307975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f280c93e0 msgr2=0x7f2f2813ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f280c93e0 0x7f2f2813ed60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c7a0 tx=0x7f2f2000cc60 comp rx=0 tx=0).stop 2023-11-28T21:56:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 -- 172.21.15.12:0/883307975 shutdown_connections 2023-11-28T21:56:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2f140664a0 0x7f2f14068960 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f280c93e0 0x7f2f2813ed60 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f28151af0 0x7f2f280ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 --2- 172.21.15.12:0/883307975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2806e300 0x7f2f281f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.046 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 -- 172.21.15.12:0/883307975 >> 172.21.15.12:0/883307975 conn(0x7f2f28066360 msgr2=0x7f2f28073a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:42.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 -- 172.21.15.12:0/883307975 shutdown_connections 2023-11-28T21:56:42.047 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.040+0000 7f2f2e832640 1 -- 172.21.15.12:0/883307975 wait complete. 2023-11-28T21:56:42.065 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T21:56:42.065 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T21:56:42.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.208+0000 7f9de759e640 1 Processor -- start 2023-11-28T21:56:42.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.208+0000 7f9de759e640 1 -- start start 2023-11-28T21:56:42.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.208+0000 7f9de759e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9de80ca220 0x7f9de80c4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.208+0000 7f9de759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9de80cbee0 0x7f9de80cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.208+0000 7f9de759e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de815db90 con 0x7f9de8153e10 2023-11-28T21:56:42.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.208+0000 7f9de759e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de80cb920 con 0x7f9de80ca220 2023-11-28T21:56:42.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.208+0000 7f9de759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80cbc00 con 0x7f9de80cbee0 2023-11-28T21:56:42.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.208+0000 7f9de659c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9de8153e10 0x7f9de815a6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58178/0 (socket says 172.21.15.12:58178) 2023-11-28T21:56:42.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.208+0000 7f9de659c640 1 -- 172.21.15.12:0/497796084 learned_addr learned my addr 172.21.15.12:0/497796084 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:42.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.212+0000 7f9de6d9d640 1 --2- 172.21.15.12:0/497796084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9de80cbee0 0x7f9de80cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.212+0000 7f9de5d9b640 1 --2- 172.21.15.12:0/497796084 >> v2:172.21.15.144:3300/0 conn(0x7f9de80ca220 0x7f9de80c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.212+0000 7f9de559a640 1 -- 172.21.15.12:0/497796084 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393930555 0 0) 0x7f9de815db90 con 0x7f9de8153e10 2023-11-28T21:56:42.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.212+0000 7f9de559a640 1 -- 172.21.15.12:0/497796084 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd0003530 con 0x7f9de8153e10 2023-11-28T21:56:42.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.212+0000 7f9de6d9d640 1 -- 172.21.15.12:0/497796084 >> v2:172.21.15.144:3300/0 conn(0x7f9de80ca220 msgr2=0x7f9de80c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:42.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.212+0000 7f9de6d9d640 1 --2- 172.21.15.12:0/497796084 >> v2:172.21.15.144:3300/0 conn(0x7f9de80ca220 0x7f9de80c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.212+0000 7f9de6d9d640 1 -- 172.21.15.12:0/497796084 >> v1:172.21.15.12:6789/0 conn(0x7f9de8153e10 legacy=0x7f9de815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.212+0000 7f9de6d9d640 1 -- 172.21.15.12:0/497796084 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de815d260 con 0x7f9de80cbee0 2023-11-28T21:56:42.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.212+0000 7f9de6d9d640 1 --2- 172.21.15.12:0/497796084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9de80cbee0 0x7f9de80cc350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009d90 tx=0x7f9de001f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfae2f9aaab937ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de559a640 1 -- 172.21.15.12:0/497796084 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00af600 con 0x7f9de80cbee0 2023-11-28T21:56:42.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de559a640 1 -- 172.21.15.12:0/497796084 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de00ab050 con 0x7f9de80cbee0 2023-11-28T21:56:42.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de559a640 1 -- 172.21.15.12:0/497796084 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00af980 con 0x7f9de80cbee0 2023-11-28T21:56:42.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 -- 172.21.15.12:0/497796084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9de80cbee0 msgr2=0x7f9de80cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.219 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 --2- 172.21.15.12:0/497796084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9de80cbee0 0x7f9de80cc350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009d90 tx=0x7f9de001f130 comp rx=0 tx=0).stop 2023-11-28T21:56:42.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 -- 172.21.15.12:0/497796084 shutdown_connections 2023-11-28T21:56:42.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 --2- 172.21.15.12:0/497796084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9de80cbee0 0x7f9de80cc350 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 --2- 172.21.15.12:0/497796084 >> v2:172.21.15.144:3300/0 conn(0x7f9de80ca220 0x7f9de80c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 -- 172.21.15.12:0/497796084 >> 172.21.15.12:0/497796084 conn(0x7f9de8071ce0 msgr2=0x7f9de80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:42.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 -- 172.21.15.12:0/497796084 shutdown_connections 2023-11-28T21:56:42.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 -- 172.21.15.12:0/497796084 wait complete. 2023-11-28T21:56:42.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 Processor -- start 2023-11-28T21:56:42.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 -- start start 2023-11-28T21:56:42.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de80ca220 0x7f9de8155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.216+0000 7f9de759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9de80cbee0 0x7f9de80c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9de8153e10 0x7f9de80b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de759e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80c70e0 con 0x7f9de80ca220 2023-11-28T21:56:42.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de81515b0 con 0x7f9de80cbee0 2023-11-28T21:56:42.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de80ca720 con 0x7f9de8153e10 2023-11-28T21:56:42.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de80ca220 0x7f9de8155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de80ca220 0x7f9de8155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48900/0 (socket says 172.21.15.12:48900) 2023-11-28T21:56:42.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de659c640 1 -- 172.21.15.12:0/1465959331 learned_addr learned my addr 172.21.15.12:0/1465959331 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:42.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de6d9d640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9de8153e10 0x7f9de80b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de5d9b640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9de80cbee0 0x7f9de80c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de659c640 1 -- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9de8153e10 msgr2=0x7f9de80b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de659c640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9de8153e10 0x7f9de80b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de659c640 1 -- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9de80cbee0 msgr2=0x7f9de80c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de659c640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9de80cbee0 0x7f9de80c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de659c640 1 -- 172.21.15.12:0/1465959331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de0009a70 con 0x7f9de80ca220 2023-11-28T21:56:42.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de659c640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de80ca220 0x7f9de8155840 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c420 tx=0x7f9dd800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9dd6ffd640 1 -- 172.21.15.12:0/1465959331 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8005cf0 con 0x7f9de80ca220 2023-11-28T21:56:42.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9dd6ffd640 1 -- 172.21.15.12:0/1465959331 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd8005e50 con 0x7f9de80ca220 2023-11-28T21:56:42.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de815e1a0 con 0x7f9de80ca220 2023-11-28T21:56:42.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9dd6ffd640 1 -- 172.21.15.12:0/1465959331 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd80072d0 con 0x7f9de80ca220 2023-11-28T21:56:42.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de80b44a0 con 0x7f9de80ca220 2023-11-28T21:56:42.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9de815e020 con 0x7f9de80ca220 2023-11-28T21:56:42.226 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.220+0000 7f9dd6ffd640 1 -- 172.21.15.12:0/1465959331 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9dd8007430 con 0x7f9de80ca220 2023-11-28T21:56:42.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.224+0000 7f9dd6ffd640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9dc8066470 0x7f9dc8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.224+0000 7f9dd6ffd640 1 -- 172.21.15.12:0/1465959331 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f9dd8084230 con 0x7f9de80ca220 2023-11-28T21:56:42.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.224+0000 7f9de5d9b640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9dc8066470 0x7f9dc8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.224+0000 7f9dd6ffd640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] conn(0x7f9dc80704f0 0x7f9dc8072990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.224+0000 7f9dd6ffd640 1 -- 172.21.15.12:0/1465959331 --> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9dc8073060 con 0x7f9dc80704f0 2023-11-28T21:56:42.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.224+0000 7f9dd6ffd640 1 -- 172.21.15.12:0/1465959331 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dd8084680 con 0x7f9de80ca220 2023-11-28T21:56:42.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.224+0000 7f9de5d9b640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9dc8066470 0x7f9dc8068930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9de80b2e00 tx=0x7f9ddc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.224+0000 7f9de6d9d640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] conn(0x7f9dc80704f0 0x7f9dc8072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.224+0000 7f9de6d9d640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] conn(0x7f9dc80704f0 0x7f9dc8072990 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.224+0000 7f9dd6ffd640 1 -- 172.21.15.12:0/1465959331 <== osd.6 v2:172.21.15.144:6824/2995192634 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9dc8073060 con 0x7f9dc80704f0 2023-11-28T21:56:42.242 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 --> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9de815d820 con 0x7f9dc80704f0 2023-11-28T21:56:42.243 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9dd6ffd640 1 -- 172.21.15.12:0/1465959331 <== osd.6 v2:172.21.15.144:6824/2995192634 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9de0009450 con 0x7f9dc80704f0 2023-11-28T21:56:42.243 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:56:42.244 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-28T21:56:42.244 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:56:42.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] conn(0x7f9dc80704f0 msgr2=0x7f9dc8072990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.144:6824/2995192634,v1:172.21.15.144:6825/2995192634] conn(0x7f9dc80704f0 0x7f9dc8072990 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9dc8066470 msgr2=0x7f9dc8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f9dc8066470 0x7f9dc8068930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9de80b2e00 tx=0x7f9ddc008040 comp rx=0 tx=0).stop 2023-11-28T21:56:42.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de80ca220 msgr2=0x7f9de8155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.245 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 --2- 172.21.15.12:0/1465959331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9de80ca220 0x7f9de8155840 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800c420 tx=0x7f9dd800c8e0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de6d9d640 1 -- 172.21.15.12:0/1465959331 reap_dead start 2023-11-28T21:56:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 shutdown_connections 2023-11-28T21:56:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 >> 172.21.15.12:0/1465959331 conn(0x7f9de8071ce0 msgr2=0x7f9de81517c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 shutdown_connections 2023-11-28T21:56:42.246 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.240+0000 7f9de759e640 1 -- 172.21.15.12:0/1465959331 wait complete. 2023-11-28T21:56:42.262 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T21:56:42.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf313b640 1 Processor -- start 2023-11-28T21:56:42.407 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf313b640 1 -- start start 2023-11-28T21:56:42.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf313b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec0c5bb0 0x7f4cec1518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf313b640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cec157f50 con 0x7f4cec070110 2023-11-28T21:56:42.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf313b640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cec158210 con 0x7f4cec06fd30 2023-11-28T21:56:42.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf313b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec158ab0 con 0x7f4cec0c5bb0 2023-11-28T21:56:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf293a640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4cec070110 0x7f4cec1624b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58188/0 (socket says 172.21.15.12:58188) 2023-11-28T21:56:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf293a640 1 -- 172.21.15.12:0/3705956061 learned_addr learned my addr 172.21.15.12:0/3705956061 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:42.409 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf1938640 1 --2- 172.21.15.12:0/3705956061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec0c5bb0 0x7f4cec1518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf1137640 1 -- 172.21.15.12:0/3705956061 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601597918 0 0) 0x7f4cec157f50 con 0x7f4cec070110 2023-11-28T21:56:42.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf1137640 1 -- 172.21.15.12:0/3705956061 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cd8003530 con 0x7f4cec070110 2023-11-28T21:56:42.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf1938640 1 -- 172.21.15.12:0/3705956061 >> v1:172.21.15.144:6789/0 conn(0x7f4cec06fd30 legacy=0x7f4cec0722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf1137640 1 -- 172.21.15.12:0/3705956061 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255222721 0 0) 0x7f4cec158210 con 0x7f4cec06fd30 2023-11-28T21:56:42.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf1938640 1 -- 172.21.15.12:0/3705956061 >> v1:172.21.15.12:6789/0 conn(0x7f4cec070110 legacy=0x7f4cec1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.410 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.404+0000 7f4cf1938640 1 -- 172.21.15.12:0/3705956061 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec1584f0 con 0x7f4cec0c5bb0 2023-11-28T21:56:42.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf1938640 1 --2- 172.21.15.12:0/3705956061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec0c5bb0 0x7f4cec1518e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0097f0 tx=0x7f4cdc01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c1d22976920daf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.412 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf1137640 1 -- 172.21.15.12:0/3705956061 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0aa5f0 con 0x7f4cec0c5bb0 2023-11-28T21:56:42.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf1137640 1 -- 172.21.15.12:0/3705956061 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc0aa750 con 0x7f4cec0c5bb0 2023-11-28T21:56:42.413 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf1137640 1 -- 172.21.15.12:0/3705956061 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0aaae0 con 0x7f4cec0c5bb0 2023-11-28T21:56:42.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf313b640 1 -- 172.21.15.12:0/3705956061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec0c5bb0 msgr2=0x7f4cec1518e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf313b640 1 --2- 172.21.15.12:0/3705956061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec0c5bb0 0x7f4cec1518e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0097f0 tx=0x7f4cdc01c490 comp rx=0 tx=0).stop 2023-11-28T21:56:42.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf313b640 1 -- 172.21.15.12:0/3705956061 shutdown_connections 2023-11-28T21:56:42.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf313b640 1 --2- 172.21.15.12:0/3705956061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec0c5bb0 0x7f4cec1518e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf313b640 1 -- 172.21.15.12:0/3705956061 >> 172.21.15.12:0/3705956061 conn(0x7f4cec066360 msgr2=0x7f4cec066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:42.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf313b640 1 -- 172.21.15.12:0/3705956061 shutdown_connections 2023-11-28T21:56:42.414 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.408+0000 7f4cf313b640 1 -- 172.21.15.12:0/3705956061 wait complete. 2023-11-28T21:56:42.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf313b640 1 Processor -- start 2023-11-28T21:56:42.415 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf313b640 1 -- start start 2023-11-28T21:56:42.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf313b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec06fd30 0x7f4cec1f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf313b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec070110 0x7f4cec141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf313b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cec0c5bb0 0x7f4cec141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf313b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec14d0b0 con 0x7f4cec070110 2023-11-28T21:56:42.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf313b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec140cb0 con 0x7f4cec0c5bb0 2023-11-28T21:56:42.417 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf313b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec0c9020 con 0x7f4cec06fd30 2023-11-28T21:56:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf1938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec070110 0x7f4cec141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf1938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec070110 0x7f4cec141420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48904/0 (socket says 172.21.15.12:48904) 2023-11-28T21:56:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf1938640 1 -- 172.21.15.12:0/3151858824 learned_addr learned my addr 172.21.15.12:0/3151858824 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf2139640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec06fd30 0x7f4cec1f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf293a640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cec0c5bb0 0x7f4cec141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf1938640 1 -- 172.21.15.12:0/3151858824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec06fd30 msgr2=0x7f4cec1f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf1938640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec06fd30 0x7f4cec1f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf1938640 1 -- 172.21.15.12:0/3151858824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cec0c5bb0 msgr2=0x7f4cec141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf1938640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cec0c5bb0 0x7f4cec141960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.419 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf1938640 1 -- 172.21.15.12:0/3151858824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc009420 con 0x7f4cec070110 2023-11-28T21:56:42.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4cf1938640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec070110 0x7f4cec141420 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009740 tx=0x7f4cdc019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4ce2ffd640 1 -- 172.21.15.12:0/3151858824 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc004220 con 0x7f4cec070110 2023-11-28T21:56:42.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4ce2ffd640 1 -- 172.21.15.12:0/3151858824 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc004380 con 0x7f4cec070110 2023-11-28T21:56:42.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.412+0000 7f4ce2ffd640 1 -- 172.21.15.12:0/3151858824 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc0aaba0 con 0x7f4cec070110 2023-11-28T21:56:42.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.416+0000 7f4cf313b640 1 -- 172.21.15.12:0/3151858824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec1f6990 con 0x7f4cec070110 2023-11-28T21:56:42.420 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.416+0000 7f4cf313b640 1 -- 172.21.15.12:0/3151858824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cec14cdd0 con 0x7f4cec070110 2023-11-28T21:56:42.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.416+0000 7f4cf313b640 1 -- 172.21.15.12:0/3151858824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cec067c70 con 0x7f4cec070110 2023-11-28T21:56:42.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.416+0000 7f4ce2ffd640 1 -- 172.21.15.12:0/3151858824 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4cdc0bb070 con 0x7f4cec070110 2023-11-28T21:56:42.421 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.416+0000 7f4ce2ffd640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4cd0066470 0x7f4cd0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.416+0000 7f4ce2ffd640 1 -- 172.21.15.12:0/3151858824 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f4cdc0a7070 con 0x7f4cec070110 2023-11-28T21:56:42.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.416+0000 7f4cf2139640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4cd0066470 0x7f4cd0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.422 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.416+0000 7f4cf2139640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4cd0066470 0x7f4cd0068930 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8007fd0 tx=0x7f4ce8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.424 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.420+0000 7f4ce2ffd640 1 -- 172.21.15.12:0/3151858824 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4cdc0eb660 con 0x7f4cec070110 2023-11-28T21:56:42.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.576+0000 7f4cf313b640 1 -- 172.21.15.12:0/3151858824 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cec071710 con 0x7f4cd0066470 2023-11-28T21:56:42.580 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.576+0000 7f4ce2ffd640 1 -- 172.21.15.12:0/3151858824 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4cec071710 con 0x7f4cd0066470 2023-11-28T21:56:42.581 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T21:56:42.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 -- 172.21.15.12:0/3151858824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4cd0066470 msgr2=0x7f4cd0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.582 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4cd0066470 0x7f4cd0068930 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8007fd0 tx=0x7f4ce8009040 comp rx=0 tx=0).stop 2023-11-28T21:56:42.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 -- 172.21.15.12:0/3151858824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec070110 msgr2=0x7f4cec141420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec070110 0x7f4cec141420 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009740 tx=0x7f4cdc019230 comp rx=0 tx=0).stop 2023-11-28T21:56:42.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 -- 172.21.15.12:0/3151858824 shutdown_connections 2023-11-28T21:56:42.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4cd0066470 0x7f4cd0068930 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cec0c5bb0 0x7f4cec141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cec070110 0x7f4cec141420 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 --2- 172.21.15.12:0/3151858824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cec06fd30 0x7f4cec1f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.583 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 -- 172.21.15.12:0/3151858824 >> 172.21.15.12:0/3151858824 conn(0x7f4cec066360 msgr2=0x7f4cec06de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:42.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 -- 172.21.15.12:0/3151858824 shutdown_connections 2023-11-28T21:56:42.584 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.580+0000 7f4cf313b640 1 -- 172.21.15.12:0/3151858824 wait complete. 2023-11-28T21:56:42.601 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T21:56:42.602 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T21:56:42.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.744+0000 7f2a8f067640 1 Processor -- start 2023-11-28T21:56:42.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.744+0000 7f2a8f067640 1 -- start start 2023-11-28T21:56:42.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8f067640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a880c8c40 0x7f2a8815d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8f067640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2a880c7de0 0x7f2a88072380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8f067640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a88070120 con 0x7f2a88153e70 2023-11-28T21:56:42.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8f067640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a880c8210 con 0x7f2a880c7de0 2023-11-28T21:56:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8f067640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8806fdb0 con 0x7f2a880c8c40 2023-11-28T21:56:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8e065640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2a88153e70 0x7f2a88071c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:58198/0 (socket says 172.21.15.12:58198) 2023-11-28T21:56:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8e065640 1 -- 172.21.15.12:0/2282935740 learned_addr learned my addr 172.21.15.12:0/2282935740 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:56:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8e866640 1 --2- 172.21.15.12:0/2282935740 >> v2:172.21.15.144:3300/0 conn(0x7f2a880c7de0 0x7f2a88072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d864640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a880c8c40 0x7f2a8815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144179972 0 0) 0x7f2a88070120 con 0x7f2a88153e70 2023-11-28T21:56:42.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a74003530 con 0x7f2a88153e70 2023-11-28T21:56:42.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4260207751 0 0) 0x7f2a74003530 con 0x7f2a88153e70 2023-11-28T21:56:42.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a880c8c40 msgr2=0x7f2a8815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:42.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a880c8c40 0x7f2a8815d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 >> v2:172.21.15.144:3300/0 conn(0x7f2a880c7de0 msgr2=0x7f2a88072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:42.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 --2- 172.21.15.12:0/2282935740 >> v2:172.21.15.144:3300/0 conn(0x7f2a880c7de0 0x7f2a88072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a880c8660 con 0x7f2a88153e70 2023-11-28T21:56:42.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f2a84001680 con 0x7f2a88153e70 2023-11-28T21:56:42.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 >> v1:172.21.15.12:6789/0 conn(0x7f2a88153e70 legacy=0x7f2a88071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a880c8c40 0x7f2a74003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 0x7f2a740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a74004d10 0x7f2a74005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880c8210 con 0x7f2a74003f00 2023-11-28T21:56:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8806fdb0 con 0x7f2a74004d10 2023-11-28T21:56:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7400b990 con 0x7f2a880c8c40 2023-11-28T21:56:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d864640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 0x7f2a740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8e065640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a880c8c40 0x7f2a74003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8e866640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a74004d10 0x7f2a74005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.758 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d864640 1 -- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a880c8c40 msgr2=0x7f2a74003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d864640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a880c8c40 0x7f2a74003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d864640 1 -- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a74004d10 msgr2=0x7f2a74005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d864640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a74004d10 0x7f2a74005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.748+0000 7f2a8d864640 1 -- 172.21.15.12:0/2282935740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a74003530 con 0x7f2a74003f00 2023-11-28T21:56:42.764 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8d864640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 0x7f2a740047d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009ae0 tx=0x7f2a7801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2f6cee4873ea15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a780a5530 con 0x7f2a74003f00 2023-11-28T21:56:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a780a5690 con 0x7f2a74003f00 2023-11-28T21:56:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8d063640 1 -- 172.21.15.12:0/2282935740 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a780a5a40 con 0x7f2a74003f00 2023-11-28T21:56:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8f067640 1 -- 172.21.15.12:0/2282935740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 msgr2=0x7f2a740047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8f067640 1 --2- 172.21.15.12:0/2282935740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 0x7f2a740047d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009ae0 tx=0x7f2a7801c9f0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8e866640 1 -- 172.21.15.12:0/2282935740 reap_dead start 2023-11-28T21:56:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8f067640 1 -- 172.21.15.12:0/2282935740 shutdown_connections 2023-11-28T21:56:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8f067640 1 -- 172.21.15.12:0/2282935740 >> 172.21.15.12:0/2282935740 conn(0x7f2a880cd3e0 msgr2=0x7f2a880cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8f067640 1 -- 172.21.15.12:0/2282935740 shutdown_connections 2023-11-28T21:56:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.752+0000 7f2a8f067640 1 -- 172.21.15.12:0/2282935740 wait complete. 2023-11-28T21:56:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8f067640 1 Processor -- start 2023-11-28T21:56:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8f067640 1 -- start start 2023-11-28T21:56:42.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8f067640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 0x7f2a8815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8f067640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a8806bd10 0x7f2a880c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8f067640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a88157cf0 0x7f2a880c3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8f067640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815d340 con 0x7f2a74003f00 2023-11-28T21:56:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8f067640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a881efe10 con 0x7f2a8806bd10 2023-11-28T21:56:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8f067640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8815d560 con 0x7f2a88157cf0 2023-11-28T21:56:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8e065640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 0x7f2a8815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8e065640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 0x7f2a8815ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48924/0 (socket says 172.21.15.12:48924) 2023-11-28T21:56:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8e065640 1 -- 172.21.15.12:0/2279764485 learned_addr learned my addr 172.21.15.12:0/2279764485 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8d864640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a8806bd10 0x7f2a880c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8e866640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a88157cf0 0x7f2a880c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8e065640 1 -- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a88157cf0 msgr2=0x7f2a880c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8e065640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a88157cf0 0x7f2a880c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8e065640 1 -- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a8806bd10 msgr2=0x7f2a880c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8e065640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a8806bd10 0x7f2a880c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8e065640 1 -- 172.21.15.12:0/2279764485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a881ef4e0 con 0x7f2a74003f00 2023-11-28T21:56:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8e065640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 0x7f2a8815ce00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2a84011910 tx=0x7f2a84011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a7effd640 1 -- 172.21.15.12:0/2279764485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400fc20 con 0x7f2a74003f00 2023-11-28T21:56:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a7effd640 1 -- 172.21.15.12:0/2279764485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8400fd80 con 0x7f2a74003f00 2023-11-28T21:56:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a7effd640 1 -- 172.21.15.12:0/2279764485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84015650 con 0x7f2a74003f00 2023-11-28T21:56:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a881ef7c0 con 0x7f2a74003f00 2023-11-28T21:56:42.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.756+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a881ef1b0 con 0x7f2a74003f00 2023-11-28T21:56:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a880cc970 con 0x7f2a74003f00 2023-11-28T21:56:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a7effd640 1 -- 172.21.15.12:0/2279764485 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2a840157b0 con 0x7f2a74003f00 2023-11-28T21:56:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a7effd640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a6c0664f0 0x7f2a6c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a7effd640 1 -- 172.21.15.12:0/2279764485 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f2a84087940 con 0x7f2a74003f00 2023-11-28T21:56:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a8d864640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a6c0664f0 0x7f2a6c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a7effd640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] conn(0x7f2a6c0705f0 0x7f2a6c072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a7effd640 1 -- 172.21.15.12:0/2279764485 --> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a6c073160 con 0x7f2a6c0705f0 2023-11-28T21:56:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a7effd640 1 -- 172.21.15.12:0/2279764485 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a8408c2a0 con 0x7f2a74003f00 2023-11-28T21:56:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a8e866640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] conn(0x7f2a6c0705f0 0x7f2a6c072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a8d864640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a6c0664f0 0x7f2a6c0689b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2a780af000 tx=0x7f2a780046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.771 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.760+0000 7f2a8e866640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] conn(0x7f2a6c0705f0 0x7f2a6c072a90 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.764+0000 7f2a7effd640 1 -- 172.21.15.12:0/2279764485 <== osd.7 v2:172.21.15.144:6808/3855981118 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2a6c073160 con 0x7f2a6c0705f0 2023-11-28T21:56:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.776+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 --> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2a881efc70 con 0x7f2a6c0705f0 2023-11-28T21:56:42.781 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a7effd640 1 -- 172.21.15.12:0/2279764485 <== osd.7 v2:172.21.15.144:6808/3855981118 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2a8000f040 con 0x7f2a6c0705f0 2023-11-28T21:56:42.782 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2023-11-28T21:56:42.782 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-28T21:56:42.782 INFO:teuthology.orchestra.run.smithi012.stdout:} 2023-11-28T21:56:42.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] conn(0x7f2a6c0705f0 msgr2=0x7f2a6c072a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8f067640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.144:6808/3855981118,v1:172.21.15.144:6809/3855981118] conn(0x7f2a6c0705f0 0x7f2a6c072a90 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a6c0664f0 msgr2=0x7f2a6c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.783 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8f067640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2a6c0664f0 0x7f2a6c0689b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2a780af000 tx=0x7f2a780046d0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 msgr2=0x7f2a8815ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8f067640 1 --2- 172.21.15.12:0/2279764485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a74003f00 0x7f2a8815ce00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2a84011910 tx=0x7f2a84011dd0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8e866640 1 -- 172.21.15.12:0/2279764485 reap_dead start 2023-11-28T21:56:42.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 shutdown_connections 2023-11-28T21:56:42.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 >> 172.21.15.12:0/2279764485 conn(0x7f2a880cd3e0 msgr2=0x7f2a8806ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:42.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 shutdown_connections 2023-11-28T21:56:42.784 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.780+0000 7f2a8f067640 1 -- 172.21.15.12:0/2279764485 wait complete. 2023-11-28T21:56:42.801 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T21:56:42.959 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cc489c640 1 Processor -- start 2023-11-28T21:56:42.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cc489c640 1 -- start start 2023-11-28T21:56:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cc489c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 0x7f2cc0156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cc489c640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2cc006e300 0x7f2cc00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cc489c640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc0067410 con 0x7f2cc006cf40 2023-11-28T21:56:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cc489c640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc006e090 con 0x7f2cc006e300 2023-11-28T21:56:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cc489c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc0157f30 con 0x7f2cc0150c20 2023-11-28T21:56:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cb7fff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2cc006e300 0x7f2cc00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cbe59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 0x7f2cc0156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cb7fff640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2cc006e300 0x7f2cc00c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.12:48334/0 (socket says 172.21.15.12:48334) 2023-11-28T21:56:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cbed9d640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f2cc006cf40 0x7f2cc0158410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:41978/0 (socket says 172.21.15.12:41978) 2023-11-28T21:56:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.956+0000 7f2cb7fff640 1 -- 172.21.15.12:0/3658688018 learned_addr learned my addr 172.21.15.12:0/3658688018 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.960+0000 7f2cbe59c640 1 -- 172.21.15.12:0/3658688018 >> v2:172.21.15.144:3300/0 conn(0x7f2cc006e300 msgr2=0x7f2cc00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.960+0000 7f2cbe59c640 1 --2- 172.21.15.12:0/3658688018 >> v2:172.21.15.144:3300/0 conn(0x7f2cc006e300 0x7f2cc00c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.960+0000 7f2cbe59c640 1 -- 172.21.15.12:0/3658688018 >> v1:172.21.15.144:6789/0 conn(0x7f2cc006cf40 legacy=0x7f2cc0158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.960+0000 7f2cbe59c640 1 -- 172.21.15.12:0/3658688018 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc0151000 con 0x7f2cc0150c20 2023-11-28T21:56:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.960+0000 7f2cb7fff640 1 --2- 172.21.15.12:0/3658688018 >> v2:172.21.15.144:3300/0 conn(0x7f2cc006e300 0x7f2cc00c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:56:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.960+0000 7f2cbe59c640 1 --2- 172.21.15.12:0/3658688018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 0x7f2cc0156910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00097c0 tx=0x7f2cb001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1309eac49400bf7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.960+0000 7f2cbdd9b640 1 -- 172.21.15.12:0/3658688018 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00ad640 con 0x7f2cc0150c20 2023-11-28T21:56:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.960+0000 7f2cbdd9b640 1 -- 172.21.15.12:0/3658688018 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb00ad7a0 con 0x7f2cc0150c20 2023-11-28T21:56:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.960+0000 7f2cbdd9b640 1 -- 172.21.15.12:0/3658688018 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00adb30 con 0x7f2cc0150c20 2023-11-28T21:56:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 -- 172.21.15.12:0/3658688018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 msgr2=0x7f2cc0156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 --2- 172.21.15.12:0/3658688018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 0x7f2cc0156910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00097c0 tx=0x7f2cb001c480 comp rx=0 tx=0).stop 2023-11-28T21:56:42.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 -- 172.21.15.12:0/3658688018 shutdown_connections 2023-11-28T21:56:42.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 --2- 172.21.15.12:0/3658688018 >> v2:172.21.15.144:3300/0 conn(0x7f2cc006e300 0x7f2cc00c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 --2- 172.21.15.12:0/3658688018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 0x7f2cc0156910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 -- 172.21.15.12:0/3658688018 >> 172.21.15.12:0/3658688018 conn(0x7f2cc0066360 msgr2=0x7f2cc0066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:42.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 -- 172.21.15.12:0/3658688018 shutdown_connections 2023-11-28T21:56:42.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 -- 172.21.15.12:0/3658688018 wait complete. 2023-11-28T21:56:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 Processor -- start 2023-11-28T21:56:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 -- start start 2023-11-28T21:56:42.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc006cf40 0x7f2cc0146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cc006e300 0x7f2cc0142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 0x7f2cc01431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.969 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc0152b20 con 0x7f2cc006e300 2023-11-28T21:56:42.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc01f1110 con 0x7f2cc006cf40 2023-11-28T21:56:42.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cc489c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc0145950 con 0x7f2cc0150c20 2023-11-28T21:56:42.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cb7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cc006e300 0x7f2cc0142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cb7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cc006e300 0x7f2cc0142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48934/0 (socket says 172.21.15.12:48934) 2023-11-28T21:56:42.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cb7fff640 1 -- 172.21.15.12:0/3499488334 learned_addr learned my addr 172.21.15.12:0/3499488334 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:42.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cbe59c640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc006cf40 0x7f2cc0146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.970 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cbed9d640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 0x7f2cc01431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.971 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cb7fff640 1 -- 172.21.15.12:0/3499488334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 msgr2=0x7f2cc01431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cb7fff640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 0x7f2cc01431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cb7fff640 1 -- 172.21.15.12:0/3499488334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc006cf40 msgr2=0x7f2cc0146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:42.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cb7fff640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc006cf40 0x7f2cc0146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:42.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cb7fff640 1 -- 172.21.15.12:0/3499488334 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80094d0 con 0x7f2cc006e300 2023-11-28T21:56:42.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.964+0000 7f2cbe59c640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc006cf40 0x7f2cc0146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:56:42.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cb7fff640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cc006e300 0x7f2cc0142c80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e9d0 tx=0x7f2ca800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.972 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cb6ffd640 1 -- 172.21.15.12:0/3499488334 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800ca60 con 0x7f2cc006e300 2023-11-28T21:56:42.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cb6ffd640 1 -- 172.21.15.12:0/3499488334 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca800cbc0 con 0x7f2cc006e300 2023-11-28T21:56:42.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cb6ffd640 1 -- 172.21.15.12:0/3499488334 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8013470 con 0x7f2cc006e300 2023-11-28T21:56:42.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cc489c640 1 -- 172.21.15.12:0/3499488334 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb0009420 con 0x7f2cc006e300 2023-11-28T21:56:42.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cc489c640 1 -- 172.21.15.12:0/3499488334 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc01f5130 con 0x7f2cc006e300 2023-11-28T21:56:42.973 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cc489c640 1 -- 172.21.15.12:0/3499488334 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc0074f60 con 0x7f2cc006e300 2023-11-28T21:56:42.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cb6ffd640 1 -- 172.21.15.12:0/3499488334 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2ca80135d0 con 0x7f2cc006e300 2023-11-28T21:56:42.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cb6ffd640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ca0066470 0x7f2ca0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:42.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cb6ffd640 1 -- 172.21.15.12:0/3499488334 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f2ca80848c0 con 0x7f2cc006e300 2023-11-28T21:56:42.974 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.968+0000 7f2cbe59c640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ca0066470 0x7f2ca0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:42.975 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.972+0000 7f2cbe59c640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ca0066470 0x7f2ca0068930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00af000 tx=0x7f2cb00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:42.977 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:42.972+0000 7f2cb6ffd640 1 -- 172.21.15.12:0/3499488334 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2ca8051620 con 0x7f2cc006e300 2023-11-28T21:56:43.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.136+0000 7f2cc489c640 1 -- 172.21.15.12:0/3499488334 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cc0066f30 con 0x7f2ca0066470 2023-11-28T21:56:43.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.136+0000 7f2cb6ffd640 1 -- 172.21.15.12:0/3499488334 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2cc0066f30 con 0x7f2ca0066470 2023-11-28T21:56:43.141 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T21:56:43.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 -- 172.21.15.12:0/3499488334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ca0066470 msgr2=0x7f2ca0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:43.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ca0066470 0x7f2ca0068930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00af000 tx=0x7f2cb00a7070 comp rx=0 tx=0).stop 2023-11-28T21:56:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 -- 172.21.15.12:0/3499488334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cc006e300 msgr2=0x7f2cc0142c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cc006e300 0x7f2cc0142c80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e9d0 tx=0x7f2ca800ee90 comp rx=0 tx=0).stop 2023-11-28T21:56:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 -- 172.21.15.12:0/3499488334 shutdown_connections 2023-11-28T21:56:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f2ca0066470 0x7f2ca0068930 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cc0150c20 0x7f2cc01431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2cc006e300 0x7f2cc0142c80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 --2- 172.21.15.12:0/3499488334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cc006cf40 0x7f2cc0146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 -- 172.21.15.12:0/3499488334 >> 172.21.15.12:0/3499488334 conn(0x7f2cc0066360 msgr2=0x7f2cc006af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:43.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 -- 172.21.15.12:0/3499488334 shutdown_connections 2023-11-28T21:56:43.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.140+0000 7f2cc489c640 1 -- 172.21.15.12:0/3499488334 wait complete. 2023-11-28T21:56:43.162 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:56:43.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f6162640 1 Processor -- start 2023-11-28T21:56:43.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f6162640 1 -- start start 2023-11-28T21:56:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f6162640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f42f0153fb0 0x7f42f00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f6162640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42f015a980 0x7f42f00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f6162640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f015d390 con 0x7f42f00b3ad0 2023-11-28T21:56:43.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f6162640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f00b2b60 con 0x7f42f0153fb0 2023-11-28T21:56:43.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f6162640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f015e1b0 con 0x7f42f015a980 2023-11-28T21:56:43.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f5160640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f42f0153fb0 0x7f42f00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:43.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f5160640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f42f0153fb0 0x7f42f00ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48942/0 (socket says 172.21.15.12:48942) 2023-11-28T21:56:43.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f5160640 1 -- 172.21.15.12:0/3282967716 learned_addr learned my addr 172.21.15.12:0/3282967716 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:43.309 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f495f640 1 --2- 172.21.15.12:0/3282967716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42f015a980 0x7f42f00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:43.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f5160640 1 -- 172.21.15.12:0/3282967716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42f015a980 msgr2=0x7f42f00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:43.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f5160640 1 --2- 172.21.15.12:0/3282967716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42f015a980 0x7f42f00caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f5160640 1 -- 172.21.15.12:0/3282967716 >> v1:172.21.15.144:6789/0 conn(0x7f42f00b3ad0 legacy=0x7f42f00b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:56:43.310 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.304+0000 7f42f5160640 1 -- 172.21.15.12:0/3282967716 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f015d630 con 0x7f42f0153fb0 2023-11-28T21:56:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42f5160640 1 --2- 172.21.15.12:0/3282967716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f0153fb0 0x7f42f00ca4f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009870 tx=0x7f42ec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d611a1fe67017b23 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:43.311 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42e7fff640 1 -- 172.21.15.12:0/3282967716 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0aa5f0 con 0x7f42f0153fb0 2023-11-28T21:56:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42e7fff640 1 -- 172.21.15.12:0/3282967716 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec0aa750 con 0x7f42f0153fb0 2023-11-28T21:56:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42e7fff640 1 -- 172.21.15.12:0/3282967716 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0aaae0 con 0x7f42f0153fb0 2023-11-28T21:56:43.312 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42f6162640 1 -- 172.21.15.12:0/3282967716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f0153fb0 msgr2=0x7f42f00ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42f6162640 1 --2- 172.21.15.12:0/3282967716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f0153fb0 0x7f42f00ca4f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009870 tx=0x7f42ec01c4d0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42f6162640 1 -- 172.21.15.12:0/3282967716 shutdown_connections 2023-11-28T21:56:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42f6162640 1 --2- 172.21.15.12:0/3282967716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42f015a980 0x7f42f00caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42f6162640 1 --2- 172.21.15.12:0/3282967716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f0153fb0 0x7f42f00ca4f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42f6162640 1 -- 172.21.15.12:0/3282967716 >> 172.21.15.12:0/3282967716 conn(0x7f42f0071ce0 msgr2=0x7f42f00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42f6162640 1 -- 172.21.15.12:0/3282967716 shutdown_connections 2023-11-28T21:56:43.313 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.308+0000 7f42f6162640 1 -- 172.21.15.12:0/3282967716 wait complete. 2023-11-28T21:56:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f6162640 1 Processor -- start 2023-11-28T21:56:43.314 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f6162640 1 -- start start 2023-11-28T21:56:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f6162640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f00b3ad0 0x7f42f0143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f6162640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42f0153fb0 0x7f42f01f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f6162640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42f015a980 0x7f42f0149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f6162640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f01451c0 con 0x7f42f00b3ad0 2023-11-28T21:56:43.316 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f6162640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f0147080 con 0x7f42f0153fb0 2023-11-28T21:56:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f6162640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f01f6ed0 con 0x7f42f015a980 2023-11-28T21:56:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f5160640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f00b3ad0 0x7f42f0143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f5160640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f00b3ad0 0x7f42f0143f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48954/0 (socket says 172.21.15.12:48954) 2023-11-28T21:56:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f5160640 1 -- 172.21.15.12:0/3681618457 learned_addr learned my addr 172.21.15.12:0/3681618457 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:56:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f495f640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42f0153fb0 0x7f42f01f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f5961640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42f015a980 0x7f42f0149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f5160640 1 -- 172.21.15.12:0/3681618457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42f015a980 msgr2=0x7f42f0149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:43.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f5160640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42f015a980 0x7f42f0149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f5160640 1 -- 172.21.15.12:0/3681618457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42f0153fb0 msgr2=0x7f42f01f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:43.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f5160640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42f0153fb0 0x7f42f01f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f5160640 1 -- 172.21.15.12:0/3681618457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0094d0 con 0x7f42f00b3ad0 2023-11-28T21:56:43.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f5160640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f00b3ad0 0x7f42f0143f20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0ac040 tx=0x7f42ec01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42e5ffb640 1 -- 172.21.15.12:0/3681618457 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0bd3f0 con 0x7f42f00b3ad0 2023-11-28T21:56:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42e5ffb640 1 -- 172.21.15.12:0/3681618457 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec0bd570 con 0x7f42f00b3ad0 2023-11-28T21:56:43.319 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42e5ffb640 1 -- 172.21.15.12:0/3681618457 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0bd920 con 0x7f42f00b3ad0 2023-11-28T21:56:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f6162640 1 -- 172.21.15.12:0/3681618457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f01453f0 con 0x7f42f00b3ad0 2023-11-28T21:56:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.312+0000 7f42f6162640 1 -- 172.21.15.12:0/3681618457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f01f63f0 con 0x7f42f00b3ad0 2023-11-28T21:56:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.316+0000 7f42f6162640 1 -- 172.21.15.12:0/3681618457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f0074f60 con 0x7f42f00b3ad0 2023-11-28T21:56:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.316+0000 7f42e5ffb640 1 -- 172.21.15.12:0/3681618457 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f42ec0b33f0 con 0x7f42f00b3ad0 2023-11-28T21:56:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.316+0000 7f42e5ffb640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42d4066470 0x7f42d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:56:43.320 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.316+0000 7f42e5ffb640 1 -- 172.21.15.12:0/3681618457 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f42ec0a7070 con 0x7f42f00b3ad0 2023-11-28T21:56:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.316+0000 7f42f495f640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42d4066470 0x7f42d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:56:43.321 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.316+0000 7f42f495f640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42d4066470 0x7f42d4068930 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f42f015a780 tx=0x7f42e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:56:43.323 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.320+0000 7f42e5ffb640 1 -- 172.21.15.12:0/3681618457 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f42ec0eb5b0 con 0x7f42f00b3ad0 2023-11-28T21:56:43.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.476+0000 7f42f6162640 1 -- 172.21.15.12:0/3681618457 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42f0154390 con 0x7f42d4066470 2023-11-28T21:56:43.488 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.484+0000 7f42e5ffb640 1 -- 172.21.15.12:0/3681618457 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+372835 (secure 0 0 0) 0x7f42e000b5c0 con 0x7f42d4066470 2023-11-28T21:56:43.492 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:56:43.494 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:56:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 -- 172.21.15.12:0/3681618457 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42d4066470 msgr2=0x7f42d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42d4066470 0x7f42d4068930 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f42f015a780 tx=0x7f42e0008040 comp rx=0 tx=0).stop 2023-11-28T21:56:43.495 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 -- 172.21.15.12:0/3681618457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f00b3ad0 msgr2=0x7f42f0143f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:56:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f00b3ad0 0x7f42f0143f20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0ac040 tx=0x7f42ec01ad10 comp rx=0 tx=0).stop 2023-11-28T21:56:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 -- 172.21.15.12:0/3681618457 shutdown_connections 2023-11-28T21:56:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f42d4066470 0x7f42d4068930 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42f015a980 0x7f42f0149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42f0153fb0 0x7f42f01f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 --2- 172.21.15.12:0/3681618457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42f00b3ad0 0x7f42f0143f20 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:56:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 -- 172.21.15.12:0/3681618457 >> 172.21.15.12:0/3681618457 conn(0x7f42f0071ce0 msgr2=0x7f42f00b4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:56:43.496 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 -- 172.21.15.12:0/3681618457 shutdown_connections 2023-11-28T21:56:43.497 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:56:43.488+0000 7f42f6162640 1 -- 172.21.15.12:0/3681618457 wait complete. 2023-11-28T21:56:43.512 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":711,"stamp":"2023-11-28T21:56:42.212375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1231183380,"num_objects":10537,"num_object_clones":0,"num_object_copies":41754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10537,"num_whiteouts":0,"num_read":9144,"num_read_kb":2407205,"num_write":56573,"num_write_kb":2359612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77065,"ondisk_log_size":77065,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3011732,"kb_used_data":1952980,"kb_used_omap":12,"kb_used_meta":1058675,"kb_avail":751962988,"statfs":{"total":773094113280,"available":770010099712,"internally_reserved":0,"allocated":1999851520,"data_stored":2583485923,"data_compressed":206825574,"data_compressed_allocated":245854208,"data_compressed_original":831926166,"omap_allocated":12751,"internal_metadata":1084083761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,12,22,47,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":102,"apply_latency_ms":102,"commit_latency_ns":102000000,"apply_latency_ns":102000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26740033,"num_objects":1426,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1528,"num_write_kb":27988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004343"},"pg_stats":[{"pgid":"2.1f","version":"51'341","reported_seq":428,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.277257+0000","last_change":"2023-11-28T21:34:36.012038+0000","last_active":"2023-11-28T21:55:39.277257+0000","last_peered":"2023-11-28T21:55:39.277257+0000","last_clean":"2023-11-28T21:55:39.277257+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:55:39.277257+0000","last_undegraded":"2023-11-28T21:55:39.277257+0000","last_fullsized":"2023-11-28T21:55:39.277257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":61536,"num_write":310,"num_write_kb":8499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"52'277","reported_seq":2360,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.756226+0000","last_change":"2023-11-28T21:40:42.800615+0000","last_active":"2023-11-28T21:56:35.756226+0000","last_peered":"2023-11-28T21:56:35.756226+0000","last_clean":"2023-11-28T21:56:35.756226+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:56:35.756226+0000","last_undegraded":"2023-11-28T21:56:35.756226+0000","last_fullsized":"2023-11-28T21:56:35.756226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:37.956537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28950874199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"52'1318","reported_seq":3628,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.030721+0000","last_change":"2023-11-28T21:40:42.803511+0000","last_active":"2023-11-28T21:56:35.030721+0000","last_peered":"2023-11-28T21:56:35.030721+0000","last_clean":"2023-11-28T21:56:35.030721+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:56:35.030721+0000","last_undegraded":"2023-11-28T21:56:35.030721+0000","last_fullsized":"2023-11-28T21:56:35.030721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:31:55.934555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29329539100000002,"stat_sum":{"num_bytes":30193112,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":229,"num_read_kb":12319,"num_write":1291,"num_write_kb":57664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"52'490","reported_seq":564,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:37.981820+0000","last_change":"2023-11-28T21:34:35.995569+0000","last_active":"2023-11-28T21:56:37.981820+0000","last_peered":"2023-11-28T21:56:37.981820+0000","last_clean":"2023-11-28T21:56:37.981820+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:56:37.981820+0000","last_undegraded":"2023-11-28T21:56:37.981820+0000","last_fullsized":"2023-11-28T21:56:37.981820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:26.400176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20809090,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":81920,"num_write":445,"num_write_kb":24723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"52'269","reported_seq":2353,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.758097+0000","last_change":"2023-11-28T21:40:43.257406+0000","last_active":"2023-11-28T21:56:35.758097+0000","last_peered":"2023-11-28T21:56:35.758097+0000","last_clean":"2023-11-28T21:56:35.758097+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:56:35.758097+0000","last_undegraded":"2023-11-28T21:56:35.758097+0000","last_fullsized":"2023-11-28T21:56:35.758097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23094383900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"52'1401","reported_seq":3733,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.720695+0000","last_change":"2023-11-28T21:40:43.072665+0000","last_active":"2023-11-28T21:56:34.720695+0000","last_peered":"2023-11-28T21:56:34.720695+0000","last_clean":"2023-11-28T21:56:34.720695+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:56:34.720695+0000","last_undegraded":"2023-11-28T21:56:34.720695+0000","last_fullsized":"2023-11-28T21:56:34.720695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:41:20.225443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21613474099999999,"stat_sum":{"num_bytes":11729902,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":251,"num_read_kb":24615,"num_write":1375,"num_write_kb":48688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"52'762","reported_seq":855,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.844871+0000","last_change":"2023-11-28T21:34:36.034767+0000","last_active":"2023-11-28T21:56:34.844871+0000","last_peered":"2023-11-28T21:56:34.844871+0000","last_clean":"2023-11-28T21:56:34.844871+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:56:34.844871+0000","last_undegraded":"2023-11-28T21:56:34.844871+0000","last_fullsized":"2023-11-28T21:56:34.844871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:15:07.020555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":59,"num_read_kb":131099,"num_write":715,"num_write_kb":20835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"52'294","reported_seq":2376,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.752014+0000","last_change":"2023-11-28T21:40:42.830278+0000","last_active":"2023-11-28T21:56:35.752014+0000","last_peered":"2023-11-28T21:56:35.752014+0000","last_clean":"2023-11-28T21:56:35.752014+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:56:35.752014+0000","last_undegraded":"2023-11-28T21:56:35.752014+0000","last_fullsized":"2023-11-28T21:56:35.752014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:24.137854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30458173900000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"52'1324","reported_seq":3630,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.861331+0000","last_change":"2023-11-28T21:40:43.070433+0000","last_active":"2023-11-28T21:56:34.861331+0000","last_peered":"2023-11-28T21:56:34.861331+0000","last_clean":"2023-11-28T21:56:34.861331+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:56:34.861331+0000","last_undegraded":"2023-11-28T21:56:34.861331+0000","last_fullsized":"2023-11-28T21:56:34.861331+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:00.120053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23712909900000001,"stat_sum":{"num_bytes":32661416,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":226,"num_read_kb":7690,"num_write":1300,"num_write_kb":56471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"52'96","reported_seq":166,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.898738+0000","last_change":"2023-11-28T21:34:36.020471+0000","last_active":"2023-11-28T21:56:27.898738+0000","last_peered":"2023-11-28T21:56:27.898738+0000","last_clean":"2023-11-28T21:56:27.898738+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:56:27.898738+0000","last_undegraded":"2023-11-28T21:56:27.898738+0000","last_fullsized":"2023-11-28T21:56:27.898738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:55:50.562246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36891,"num_write":51,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"52'283","reported_seq":2365,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.735472+0000","last_change":"2023-11-28T21:40:42.827917+0000","last_active":"2023-11-28T21:56:35.735472+0000","last_peered":"2023-11-28T21:56:35.735472+0000","last_clean":"2023-11-28T21:56:35.735472+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:56:35.735472+0000","last_undegraded":"2023-11-28T21:56:35.735472+0000","last_fullsized":"2023-11-28T21:56:35.735472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:32:57.982365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30416134,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"52'1294","reported_seq":3610,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.796080+0000","last_change":"2023-11-28T21:40:43.055466+0000","last_active":"2023-11-28T21:56:34.796080+0000","last_peered":"2023-11-28T21:56:34.796080+0000","last_clean":"2023-11-28T21:56:34.796080+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:56:34.796080+0000","last_undegraded":"2023-11-28T21:56:34.796080+0000","last_fullsized":"2023-11-28T21:56:34.796080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:33:55.527250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26667665699999998,"stat_sum":{"num_bytes":37281632,"num_objects":312,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":0,"num_read":236,"num_read_kb":13169,"num_write":1275,"num_write_kb":61269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"52'303","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.754198+0000","last_change":"2023-11-28T21:40:42.917342+0000","last_active":"2023-11-28T21:56:35.754198+0000","last_peered":"2023-11-28T21:56:35.754198+0000","last_clean":"2023-11-28T21:56:35.754198+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:56:35.754198+0000","last_undegraded":"2023-11-28T21:56:35.754198+0000","last_fullsized":"2023-11-28T21:56:35.754198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:52:39.167984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.200606007,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"52'617","reported_seq":711,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.513022+0000","last_change":"2023-11-28T21:34:36.027364+0000","last_active":"2023-11-28T21:56:18.513022+0000","last_peered":"2023-11-28T21:56:18.513022+0000","last_clean":"2023-11-28T21:56:18.513022+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:56:18.513022+0000","last_undegraded":"2023-11-28T21:56:18.513022+0000","last_fullsized":"2023-11-28T21:56:18.513022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:58:07.425045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":114718,"num_write":571,"num_write_kb":20791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"52'1331","reported_seq":3650,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.683935+0000","last_change":"2023-11-28T21:40:43.071642+0000","last_active":"2023-11-28T21:56:34.683935+0000","last_peered":"2023-11-28T21:56:34.683935+0000","last_clean":"2023-11-28T21:56:34.683935+0000","last_became_active":"2023-11-28T21:35:25.616978+0000","last_became_peered":"2023-11-28T21:35:25.616978+0000","last_unstale":"2023-11-28T21:56:34.683935+0000","last_undegraded":"2023-11-28T21:56:34.683935+0000","last_fullsized":"2023-11-28T21:56:34.683935+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1331,"log_dups_size":0,"ondisk_log_size":1331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:05:37.470115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23129675,"stat_sum":{"num_bytes":32707084,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":229,"num_read_kb":4127,"num_write":1308,"num_write_kb":57872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"52'305","reported_seq":2387,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.765069+0000","last_change":"2023-11-28T21:40:42.798788+0000","last_active":"2023-11-28T21:56:35.765069+0000","last_peered":"2023-11-28T21:56:35.765069+0000","last_clean":"2023-11-28T21:56:35.765069+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:56:35.765069+0000","last_undegraded":"2023-11-28T21:56:35.765069+0000","last_fullsized":"2023-11-28T21:56:35.765069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:10:37.662225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28777871399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"52'956","reported_seq":1035,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:38.923278+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:56:38.923278+0000","last_peered":"2023-11-28T21:56:38.923278+0000","last_clean":"2023-11-28T21:56:38.923278+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:56:38.923278+0000","last_undegraded":"2023-11-28T21:56:38.923278+0000","last_fullsized":"2023-11-28T21:56:38.923278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":956,"log_dups_size":0,"ondisk_log_size":956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":75782,"num_write":935,"num_write_kb":13441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"52'1416","reported_seq":3740,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.956993+0000","last_change":"2023-11-28T21:40:43.070729+0000","last_active":"2023-11-28T21:56:34.956993+0000","last_peered":"2023-11-28T21:56:34.956993+0000","last_clean":"2023-11-28T21:56:34.956993+0000","last_became_active":"2023-11-28T21:35:25.790674+0000","last_became_peered":"2023-11-28T21:35:25.790674+0000","last_unstale":"2023-11-28T21:56:34.956993+0000","last_undegraded":"2023-11-28T21:56:34.956993+0000","last_fullsized":"2023-11-28T21:56:34.956993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1416,"log_dups_size":0,"ondisk_log_size":1416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:12:03.263147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.229623311,"stat_sum":{"num_bytes":17731185,"num_objects":354,"num_object_clones":0,"num_object_copies":1416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":234,"num_read_kb":20007,"num_write":1382,"num_write_kb":71863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"52'306","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.764010+0000","last_change":"2023-11-28T21:40:42.800893+0000","last_active":"2023-11-28T21:56:35.764010+0000","last_peered":"2023-11-28T21:56:35.764010+0000","last_clean":"2023-11-28T21:56:35.764010+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:56:35.764010+0000","last_undegraded":"2023-11-28T21:56:35.764010+0000","last_fullsized":"2023-11-28T21:56:35.764010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:35:30.683746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29027087200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"52'729","reported_seq":798,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:38.924171+0000","last_change":"2023-11-28T21:34:36.013394+0000","last_active":"2023-11-28T21:56:38.924171+0000","last_peered":"2023-11-28T21:56:38.924171+0000","last_clean":"2023-11-28T21:56:38.924171+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:56:38.924171+0000","last_undegraded":"2023-11-28T21:56:38.924171+0000","last_fullsized":"2023-11-28T21:56:38.924171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11371684,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":32782,"num_write":684,"num_write_kb":11456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"52'1341","reported_seq":3685,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.931061+0000","last_change":"2023-11-28T21:40:42.890432+0000","last_active":"2023-11-28T21:56:34.931061+0000","last_peered":"2023-11-28T21:56:34.931061+0000","last_clean":"2023-11-28T21:56:34.931061+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:56:34.931061+0000","last_undegraded":"2023-11-28T21:56:34.931061+0000","last_fullsized":"2023-11-28T21:56:34.931061+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1341,"log_dups_size":0,"ondisk_log_size":1341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24643899999999999,"stat_sum":{"num_bytes":14380393,"num_objects":336,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":262,"num_read_kb":16758,"num_write":1318,"num_write_kb":45879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"52'294","reported_seq":2377,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.755142+0000","last_change":"2023-11-28T21:40:43.071973+0000","last_active":"2023-11-28T21:56:35.755142+0000","last_peered":"2023-11-28T21:56:35.755142+0000","last_clean":"2023-11-28T21:56:35.755142+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:56:35.755142+0000","last_undegraded":"2023-11-28T21:56:35.755142+0000","last_fullsized":"2023-11-28T21:56:35.755142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:34:28.182888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21610210399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"52'342","reported_seq":412,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:20.134314+0000","last_change":"2023-11-28T21:34:36.016016+0000","last_active":"2023-11-28T21:56:20.134314+0000","last_peered":"2023-11-28T21:56:20.134314+0000","last_clean":"2023-11-28T21:56:20.134314+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:56:20.134314+0000","last_undegraded":"2023-11-28T21:56:20.134314+0000","last_fullsized":"2023-11-28T21:56:20.134314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:50:12.725975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":49164,"num_write":266,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"52'1262","reported_seq":3549,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.730763+0000","last_change":"2023-11-28T21:40:43.060232+0000","last_active":"2023-11-28T21:56:34.730763+0000","last_peered":"2023-11-28T21:56:34.730763+0000","last_clean":"2023-11-28T21:56:34.730763+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:56:34.730763+0000","last_undegraded":"2023-11-28T21:56:34.730763+0000","last_fullsized":"2023-11-28T21:56:34.730763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:54:07.778886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.244586779,"stat_sum":{"num_bytes":50267474,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":206,"num_read_kb":15432,"num_write":1240,"num_write_kb":82194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"52'302","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.761940+0000","last_change":"2023-11-28T21:40:43.249048+0000","last_active":"2023-11-28T21:56:35.761940+0000","last_peered":"2023-11-28T21:56:35.761940+0000","last_clean":"2023-11-28T21:56:35.761940+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:56:35.761940+0000","last_undegraded":"2023-11-28T21:56:35.761940+0000","last_fullsized":"2023-11-28T21:56:35.761940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26728515899999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"52'169","reported_seq":241,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:32.829644+0000","last_change":"2023-11-28T21:34:36.030365+0000","last_active":"2023-11-28T21:56:32.829644+0000","last_peered":"2023-11-28T21:56:32.829644+0000","last_clean":"2023-11-28T21:56:32.829644+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:56:32.829644+0000","last_undegraded":"2023-11-28T21:56:32.829644+0000","last_fullsized":"2023-11-28T21:56:32.829644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:57.024943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":45,"num_read_kb":24615,"num_write":148,"num_write_kb":4216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"52'1368","reported_seq":3700,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.654211+0000","last_change":"2023-11-28T21:40:43.072395+0000","last_active":"2023-11-28T21:56:34.654211+0000","last_peered":"2023-11-28T21:56:34.654211+0000","last_clean":"2023-11-28T21:56:34.654211+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:56:34.654211+0000","last_undegraded":"2023-11-28T21:56:34.654211+0000","last_fullsized":"2023-11-28T21:56:34.654211+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:25:44.577393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21672229300000001,"stat_sum":{"num_bytes":16299977,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":250,"num_read_kb":17253,"num_write":1343,"num_write_kb":69509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"52'268","reported_seq":2351,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762733+0000","last_change":"2023-11-28T21:40:42.801394+0000","last_active":"2023-11-28T21:56:35.762733+0000","last_peered":"2023-11-28T21:56:35.762733+0000","last_clean":"2023-11-28T21:56:35.762733+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:56:35.762733+0000","last_undegraded":"2023-11-28T21:56:35.762733+0000","last_fullsized":"2023-11-28T21:56:35.762733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:59:35.647639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29009278799999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"52'246","reported_seq":316,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:37.909005+0000","last_change":"2023-11-28T21:34:36.028560+0000","last_active":"2023-11-28T21:56:37.909005+0000","last_peered":"2023-11-28T21:56:37.909005+0000","last_clean":"2023-11-28T21:56:37.909005+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:56:37.909005+0000","last_undegraded":"2023-11-28T21:56:37.909005+0000","last_fullsized":"2023-11-28T21:56:37.909005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":607,"num_write":447,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"52'1386","reported_seq":3700,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.993814+0000","last_change":"2023-11-28T21:40:42.797955+0000","last_active":"2023-11-28T21:56:34.993814+0000","last_peered":"2023-11-28T21:56:34.993814+0000","last_clean":"2023-11-28T21:56:34.993814+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:56:34.993814+0000","last_undegraded":"2023-11-28T21:56:34.993814+0000","last_fullsized":"2023-11-28T21:56:34.993814+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1386,"log_dups_size":0,"ondisk_log_size":1386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:27:40.256597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.287791554,"stat_sum":{"num_bytes":19942162,"num_objects":313,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":232,"num_read_kb":16373,"num_write":1352,"num_write_kb":52080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"52'300","reported_seq":2385,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.755837+0000","last_change":"2023-11-28T21:40:42.884411+0000","last_active":"2023-11-28T21:56:35.755837+0000","last_peered":"2023-11-28T21:56:35.755837+0000","last_clean":"2023-11-28T21:56:35.755837+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:56:35.755837+0000","last_undegraded":"2023-11-28T21:56:35.755837+0000","last_fullsized":"2023-11-28T21:56:35.755837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24779062600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"52'587","reported_seq":665,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.695016+0000","last_change":"2023-11-28T21:34:36.013297+0000","last_active":"2023-11-28T21:55:43.695016+0000","last_peered":"2023-11-28T21:55:43.695016+0000","last_clean":"2023-11-28T21:55:43.695016+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:55:43.695016+0000","last_undegraded":"2023-11-28T21:55:43.695016+0000","last_fullsized":"2023-11-28T21:55:43.695016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:40.570393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":81538,"num_write":523,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"52'1410","reported_seq":3741,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.900338+0000","last_change":"2023-11-28T21:40:43.255749+0000","last_active":"2023-11-28T21:56:34.900338+0000","last_peered":"2023-11-28T21:56:34.900338+0000","last_clean":"2023-11-28T21:56:34.900338+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:56:34.900338+0000","last_undegraded":"2023-11-28T21:56:34.900338+0000","last_fullsized":"2023-11-28T21:56:34.900338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27155133199999998,"stat_sum":{"num_bytes":49110889,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":248,"num_read_kb":24602,"num_write":1394,"num_write_kb":89036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"52'284","reported_seq":2368,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762834+0000","last_change":"2023-11-28T21:40:42.828660+0000","last_active":"2023-11-28T21:56:35.762834+0000","last_peered":"2023-11-28T21:56:35.762834+0000","last_clean":"2023-11-28T21:56:35.762834+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:56:35.762834+0000","last_undegraded":"2023-11-28T21:56:35.762834+0000","last_fullsized":"2023-11-28T21:56:35.762834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:46:55.786023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30430776599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"52'164","reported_seq":226,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.504188+0000","last_change":"2023-11-28T21:34:35.998179+0000","last_active":"2023-11-28T21:56:18.504188+0000","last_peered":"2023-11-28T21:56:18.504188+0000","last_clean":"2023-11-28T21:56:18.504188+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:56:18.504188+0000","last_undegraded":"2023-11-28T21:56:18.504188+0000","last_fullsized":"2023-11-28T21:56:18.504188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24576,"num_write":146,"num_write_kb":4262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"52'1318","reported_seq":3623,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.858027+0000","last_change":"2023-11-28T21:40:42.887654+0000","last_active":"2023-11-28T21:56:34.858027+0000","last_peered":"2023-11-28T21:56:34.858027+0000","last_clean":"2023-11-28T21:56:34.858027+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:56:34.858027+0000","last_undegraded":"2023-11-28T21:56:34.858027+0000","last_fullsized":"2023-11-28T21:56:34.858027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.25659980999999998,"stat_sum":{"num_bytes":32626940,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":222,"num_read_kb":4128,"num_write":1285,"num_write_kb":60020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"52'300","reported_seq":2383,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.754991+0000","last_change":"2023-11-28T21:40:43.068901+0000","last_active":"2023-11-28T21:56:35.754991+0000","last_peered":"2023-11-28T21:56:35.754991+0000","last_clean":"2023-11-28T21:56:35.754991+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:56:35.754991+0000","last_undegraded":"2023-11-28T21:56:35.754991+0000","last_fullsized":"2023-11-28T21:56:35.754991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:05:04.092055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23142385100000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"52'367","reported_seq":441,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:37.941212+0000","last_change":"2023-11-28T21:34:35.999142+0000","last_active":"2023-11-28T21:56:37.941212+0000","last_peered":"2023-11-28T21:56:37.941212+0000","last_clean":"2023-11-28T21:56:37.941212+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:56:37.941212+0000","last_undegraded":"2023-11-28T21:56:37.941212+0000","last_fullsized":"2023-11-28T21:56:37.941212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:27:58.762824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":81920,"num_write":339,"num_write_kb":16655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"52'1263","reported_seq":3556,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.958982+0000","last_change":"2023-11-28T21:40:42.883278+0000","last_active":"2023-11-28T21:56:34.958982+0000","last_peered":"2023-11-28T21:56:34.958982+0000","last_clean":"2023-11-28T21:56:34.958982+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:56:34.958982+0000","last_undegraded":"2023-11-28T21:56:34.958982+0000","last_fullsized":"2023-11-28T21:56:34.958982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24685721299999999,"stat_sum":{"num_bytes":16718463,"num_objects":288,"num_object_clones":0,"num_object_copies":1152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":210,"num_read_kb":3302,"num_write":1235,"num_write_kb":35539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"52'299","reported_seq":2384,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.748203+0000","last_change":"2023-11-28T21:40:42.890540+0000","last_active":"2023-11-28T21:56:35.748203+0000","last_peered":"2023-11-28T21:56:35.748203+0000","last_clean":"2023-11-28T21:56:35.748203+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:56:35.748203+0000","last_undegraded":"2023-11-28T21:56:35.748203+0000","last_fullsized":"2023-11-28T21:56:35.748203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24023003500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"51'934","reported_seq":1405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.724754+0000","last_change":"2023-11-28T21:34:35.990064+0000","last_active":"2023-11-28T21:55:43.724754+0000","last_peered":"2023-11-28T21:55:43.724754+0000","last_clean":"2023-11-28T21:55:43.724754+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:55:43.724754+0000","last_undegraded":"2023-11-28T21:55:43.724754+0000","last_fullsized":"2023-11-28T21:55:43.724754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:18:58.468599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18884916,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":417,"num_read_kb":34819,"num_write":869,"num_write_kb":18975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"52'1231","reported_seq":3558,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.911673+0000","last_change":"2023-11-28T21:40:42.888191+0000","last_active":"2023-11-28T21:56:34.911673+0000","last_peered":"2023-11-28T21:56:34.911673+0000","last_clean":"2023-11-28T21:56:34.911673+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:56:34.911673+0000","last_undegraded":"2023-11-28T21:56:34.911673+0000","last_fullsized":"2023-11-28T21:56:34.911673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24393632200000001,"stat_sum":{"num_bytes":26023878,"num_objects":293,"num_object_clones":0,"num_object_copies":1172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":293,"num_whiteouts":0,"num_read":244,"num_read_kb":12314,"num_write":1202,"num_write_kb":62723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"52'277","reported_seq":2361,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.758790+0000","last_change":"2023-11-28T21:40:43.256849+0000","last_active":"2023-11-28T21:56:35.758790+0000","last_peered":"2023-11-28T21:56:35.758790+0000","last_clean":"2023-11-28T21:56:35.758790+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:56:35.758790+0000","last_undegraded":"2023-11-28T21:56:35.758790+0000","last_fullsized":"2023-11-28T21:56:35.758790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.25105937499999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"52'314","reported_seq":397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:37.943417+0000","last_change":"2023-11-28T21:34:36.017541+0000","last_active":"2023-11-28T21:56:37.943417+0000","last_peered":"2023-11-28T21:56:37.943417+0000","last_clean":"2023-11-28T21:56:37.943417+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:56:37.943417+0000","last_undegraded":"2023-11-28T21:56:37.943417+0000","last_fullsized":"2023-11-28T21:56:37.943417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:59:58.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":28,"num_read_kb":114688,"num_write":276,"num_write_kb":16551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"52'1365","reported_seq":3712,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.882394+0000","last_change":"2023-11-28T21:40:43.252159+0000","last_active":"2023-11-28T21:56:34.882394+0000","last_peered":"2023-11-28T21:56:34.882394+0000","last_clean":"2023-11-28T21:56:34.882394+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:56:34.882394+0000","last_undegraded":"2023-11-28T21:56:34.882394+0000","last_fullsized":"2023-11-28T21:56:34.882394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.270607394,"stat_sum":{"num_bytes":19916106,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":264,"num_read_kb":13370,"num_write":1336,"num_write_kb":59202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"52'283","reported_seq":2366,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762414+0000","last_change":"2023-11-28T21:40:42.802535+0000","last_active":"2023-11-28T21:56:35.762414+0000","last_peered":"2023-11-28T21:56:35.762414+0000","last_clean":"2023-11-28T21:56:35.762414+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:56:35.762414+0000","last_undegraded":"2023-11-28T21:56:35.762414+0000","last_fullsized":"2023-11-28T21:56:35.762414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:02:13.185718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29152715000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"52'311","reported_seq":375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:07.951419+0000","last_change":"2023-11-28T21:34:35.996159+0000","last_active":"2023-11-28T21:56:07.951419+0000","last_peered":"2023-11-28T21:56:07.951419+0000","last_clean":"2023-11-28T21:56:07.951419+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:56:07.951419+0000","last_undegraded":"2023-11-28T21:56:07.951419+0000","last_fullsized":"2023-11-28T21:56:07.951419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:16:09.211621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":24594,"num_write":297,"num_write_kb":8480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:38.725972+0000","last_change":"2023-11-28T21:34:31.336256+0000","last_active":"2023-11-28T21:55:38.725972+0000","last_peered":"2023-11-28T21:55:38.725972+0000","last_clean":"2023-11-28T21:55:38.725972+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:55:38.725972+0000","last_undegraded":"2023-11-28T21:55:38.725972+0000","last_fullsized":"2023-11-28T21:55:38.725972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:30.025809+0000","last_clean_scrub_stamp":"2023-11-28T21:34:30.025809+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:01:55.288399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"52'1342","reported_seq":3676,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.779903+0000","last_change":"2023-11-28T21:40:42.914002+0000","last_active":"2023-11-28T21:56:34.779903+0000","last_peered":"2023-11-28T21:56:34.779903+0000","last_clean":"2023-11-28T21:56:34.779903+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:56:34.779903+0000","last_undegraded":"2023-11-28T21:56:34.779903+0000","last_fullsized":"2023-11-28T21:56:34.779903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:47.036263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19732549099999999,"stat_sum":{"num_bytes":29762487,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":252,"num_read_kb":20508,"num_write":1310,"num_write_kb":73242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"52'280","reported_seq":2363,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.750254+0000","last_change":"2023-11-28T21:40:42.748386+0000","last_active":"2023-11-28T21:56:35.750254+0000","last_peered":"2023-11-28T21:56:35.750254+0000","last_clean":"2023-11-28T21:56:35.750254+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:56:35.750254+0000","last_undegraded":"2023-11-28T21:56:35.750254+0000","last_fullsized":"2023-11-28T21:56:35.750254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:16:59.734407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.219020891,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"52'475","reported_seq":534,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273615+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:55:39.273615+0000","last_peered":"2023-11-28T21:55:39.273615+0000","last_clean":"2023-11-28T21:55:39.273615+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:55:39.273615+0000","last_undegraded":"2023-11-28T21:55:39.273615+0000","last_fullsized":"2023-11-28T21:55:39.273615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:55.727267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":20485,"num_write":447,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"52'1372","reported_seq":3704,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.807900+0000","last_change":"2023-11-28T21:40:42.915914+0000","last_active":"2023-11-28T21:56:34.807900+0000","last_peered":"2023-11-28T21:56:34.807900+0000","last_clean":"2023-11-28T21:56:34.807900+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:56:34.807900+0000","last_undegraded":"2023-11-28T21:56:34.807900+0000","last_fullsized":"2023-11-28T21:56:34.807900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1372,"log_dups_size":0,"ondisk_log_size":1372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:52:07.090824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19929065100000001,"stat_sum":{"num_bytes":26470227,"num_objects":335,"num_object_clones":0,"num_object_copies":1340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":251,"num_read_kb":4134,"num_write":1344,"num_write_kb":47412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"52'330","reported_seq":2413,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.763413+0000","last_change":"2023-11-28T21:40:42.830692+0000","last_active":"2023-11-28T21:56:35.763413+0000","last_peered":"2023-11-28T21:56:35.763413+0000","last_clean":"2023-11-28T21:56:35.763413+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:56:35.763413+0000","last_undegraded":"2023-11-28T21:56:35.763413+0000","last_fullsized":"2023-11-28T21:56:35.763413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:33:58.325515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30458413400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"52'965","reported_seq":1046,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:37.923673+0000","last_change":"2023-11-28T21:34:36.019243+0000","last_active":"2023-11-28T21:56:37.923673+0000","last_peered":"2023-11-28T21:56:37.923673+0000","last_clean":"2023-11-28T21:56:37.923673+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:56:37.923673+0000","last_undegraded":"2023-11-28T21:56:37.923673+0000","last_fullsized":"2023-11-28T21:56:37.923673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":27,"num_read_kb":68346,"num_write":931,"num_write_kb":16947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"52'1319","reported_seq":3666,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.898961+0000","last_change":"2023-11-28T21:40:43.256533+0000","last_active":"2023-11-28T21:56:34.898961+0000","last_peered":"2023-11-28T21:56:34.898961+0000","last_clean":"2023-11-28T21:56:34.898961+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:56:34.898961+0000","last_undegraded":"2023-11-28T21:56:34.898961+0000","last_fullsized":"2023-11-28T21:56:34.898961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27089875299999999,"stat_sum":{"num_bytes":28004524,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":265,"num_read_kb":16422,"num_write":1294,"num_write_kb":50614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"52'274","reported_seq":2350,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.746982+0000","last_change":"2023-11-28T21:40:43.090122+0000","last_active":"2023-11-28T21:56:35.746982+0000","last_peered":"2023-11-28T21:56:35.746982+0000","last_clean":"2023-11-28T21:56:35.746982+0000","last_became_active":"2023-11-28T21:35:25.594533+0000","last_became_peered":"2023-11-28T21:35:25.594533+0000","last_unstale":"2023-11-28T21:56:35.746982+0000","last_undegraded":"2023-11-28T21:56:35.746982+0000","last_fullsized":"2023-11-28T21:56:35.746982+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:40:03.418589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14880083199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"52'304","reported_seq":379,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:27.892315+0000","last_change":"2023-11-28T21:34:35.989804+0000","last_active":"2023-11-28T21:56:27.892315+0000","last_peered":"2023-11-28T21:56:27.892315+0000","last_clean":"2023-11-28T21:56:27.892315+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:56:27.892315+0000","last_undegraded":"2023-11-28T21:56:27.892315+0000","last_fullsized":"2023-11-28T21:56:27.892315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:41:41.541039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":53275,"num_write":252,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"52'1307","reported_seq":3591,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.520158+0000","last_change":"2023-11-28T21:40:42.823686+0000","last_active":"2023-11-28T21:56:34.520158+0000","last_peered":"2023-11-28T21:56:34.520158+0000","last_clean":"2023-11-28T21:56:34.520158+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:56:34.520158+0000","last_undegraded":"2023-11-28T21:56:34.520158+0000","last_fullsized":"2023-11-28T21:56:34.520158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1307,"log_dups_size":0,"ondisk_log_size":1307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:34:35.818463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30648515900000001,"stat_sum":{"num_bytes":28110180,"num_objects":295,"num_object_clones":0,"num_object_copies":1180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":0,"num_read":202,"num_read_kb":8227,"num_write":1283,"num_write_kb":51039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"52'277","reported_seq":2360,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.749718+0000","last_change":"2023-11-28T21:40:42.749319+0000","last_active":"2023-11-28T21:56:35.749718+0000","last_peered":"2023-11-28T21:56:35.749718+0000","last_clean":"2023-11-28T21:56:35.749718+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:56:35.749718+0000","last_undegraded":"2023-11-28T21:56:35.749718+0000","last_fullsized":"2023-11-28T21:56:35.749718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:18:31.031890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21877398200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"52'194","reported_seq":267,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:28.522954+0000","last_change":"2023-11-28T21:34:36.018946+0000","last_active":"2023-11-28T21:56:28.522954+0000","last_peered":"2023-11-28T21:56:28.522954+0000","last_clean":"2023-11-28T21:56:28.522954+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:56:28.522954+0000","last_undegraded":"2023-11-28T21:56:28.522954+0000","last_fullsized":"2023-11-28T21:56:28.522954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:25:43.674632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":49161,"num_write":147,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"52'1273","reported_seq":3574,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.716263+0000","last_change":"2023-11-28T21:40:43.072161+0000","last_active":"2023-11-28T21:56:34.716263+0000","last_peered":"2023-11-28T21:56:34.716263+0000","last_clean":"2023-11-28T21:56:34.716263+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:56:34.716263+0000","last_undegraded":"2023-11-28T21:56:34.716263+0000","last_fullsized":"2023-11-28T21:56:34.716263+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:59:39.182750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.223266511,"stat_sum":{"num_bytes":11523503,"num_objects":308,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":219,"num_read_kb":12316,"num_write":1248,"num_write_kb":38807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"52'323","reported_seq":2406,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.755907+0000","last_change":"2023-11-28T21:40:42.747053+0000","last_active":"2023-11-28T21:56:35.755907+0000","last_peered":"2023-11-28T21:56:35.755907+0000","last_clean":"2023-11-28T21:56:35.755907+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:56:35.755907+0000","last_undegraded":"2023-11-28T21:56:35.755907+0000","last_fullsized":"2023-11-28T21:56:35.755907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:54:52.864875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21978382299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"52'854","reported_seq":964,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:18.035796+0000","last_change":"2023-11-28T21:34:35.998788+0000","last_active":"2023-11-28T21:56:18.035796+0000","last_peered":"2023-11-28T21:56:18.035796+0000","last_clean":"2023-11-28T21:56:18.035796+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:56:18.035796+0000","last_undegraded":"2023-11-28T21:56:18.035796+0000","last_fullsized":"2023-11-28T21:56:18.035796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:17.574725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":137,"num_read_kb":58709,"num_write":838,"num_write_kb":8642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"52'1274","reported_seq":3607,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.000372+0000","last_change":"2023-11-28T21:40:43.252839+0000","last_active":"2023-11-28T21:56:35.000372+0000","last_peered":"2023-11-28T21:56:35.000372+0000","last_clean":"2023-11-28T21:56:35.000372+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:56:35.000372+0000","last_undegraded":"2023-11-28T21:56:35.000372+0000","last_fullsized":"2023-11-28T21:56:35.000372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27152726799999999,"stat_sum":{"num_bytes":36307339,"num_objects":283,"num_object_clones":0,"num_object_copies":1132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":283,"num_whiteouts":0,"num_read":251,"num_read_kb":8046,"num_write":1253,"num_write_kb":60298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"52'284","reported_seq":2366,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762426+0000","last_change":"2023-11-28T21:40:42.918241+0000","last_active":"2023-11-28T21:56:35.762426+0000","last_peered":"2023-11-28T21:56:35.762426+0000","last_clean":"2023-11-28T21:56:35.762426+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:56:35.762426+0000","last_undegraded":"2023-11-28T21:56:35.762426+0000","last_fullsized":"2023-11-28T21:56:35.762426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:57.732993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18244292200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"52'239","reported_seq":293,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.710890+0000","last_change":"2023-11-28T21:34:36.025980+0000","last_active":"2023-11-28T21:55:43.710890+0000","last_peered":"2023-11-28T21:55:43.710890+0000","last_clean":"2023-11-28T21:55:43.710890+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:55:43.710890+0000","last_undegraded":"2023-11-28T21:55:43.710890+0000","last_fullsized":"2023-11-28T21:55:43.710890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:31:36.901507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195875,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":194,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"52'1346","reported_seq":3662,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.682315+0000","last_change":"2023-11-28T21:40:42.805322+0000","last_active":"2023-11-28T21:56:34.682315+0000","last_peered":"2023-11-28T21:56:34.682315+0000","last_clean":"2023-11-28T21:56:34.682315+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:56:34.682315+0000","last_undegraded":"2023-11-28T21:56:34.682315+0000","last_fullsized":"2023-11-28T21:56:34.682315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:22:10.703578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29439885100000002,"stat_sum":{"num_bytes":34458204,"num_objects":324,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":324,"num_whiteouts":0,"num_read":235,"num_read_kb":27797,"num_write":1320,"num_write_kb":70332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"52'313","reported_seq":2396,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.755443+0000","last_change":"2023-11-28T21:40:42.740620+0000","last_active":"2023-11-28T21:56:35.755443+0000","last_peered":"2023-11-28T21:56:35.755443+0000","last_clean":"2023-11-28T21:56:35.755443+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:56:35.755443+0000","last_undegraded":"2023-11-28T21:56:35.755443+0000","last_fullsized":"2023-11-28T21:56:35.755443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:41:24.855115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.218364582,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"51'155","reported_seq":213,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.226609+0000","last_change":"2023-11-28T21:34:35.994159+0000","last_active":"2023-11-28T21:56:08.226609+0000","last_peered":"2023-11-28T21:56:08.226609+0000","last_clean":"2023-11-28T21:56:08.226609+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:56:08.226609+0000","last_undegraded":"2023-11-28T21:56:08.226609+0000","last_fullsized":"2023-11-28T21:56:08.226609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:47:34.646643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":107,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"52'1271","reported_seq":3570,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.041533+0000","last_change":"2023-11-28T21:40:42.806545+0000","last_active":"2023-11-28T21:56:35.041533+0000","last_peered":"2023-11-28T21:56:35.041533+0000","last_clean":"2023-11-28T21:56:35.041533+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:56:35.041533+0000","last_undegraded":"2023-11-28T21:56:35.041533+0000","last_fullsized":"2023-11-28T21:56:35.041533+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:19:01.956213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29571303700000001,"stat_sum":{"num_bytes":33678171,"num_objects":350,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":350,"num_whiteouts":0,"num_read":218,"num_read_kb":4129,"num_write":1246,"num_write_kb":52865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"52'279","reported_seq":2362,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757090+0000","last_change":"2023-11-28T21:40:43.257612+0000","last_active":"2023-11-28T21:56:35.757090+0000","last_peered":"2023-11-28T21:56:35.757090+0000","last_clean":"2023-11-28T21:56:35.757090+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:56:35.757090+0000","last_undegraded":"2023-11-28T21:56:35.757090+0000","last_fullsized":"2023-11-28T21:56:35.757090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21362616300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"52'305","reported_seq":399,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:37.922875+0000","last_change":"2023-11-28T21:34:36.019206+0000","last_active":"2023-11-28T21:56:37.922875+0000","last_peered":"2023-11-28T21:56:37.922875+0000","last_clean":"2023-11-28T21:56:37.922875+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:56:37.922875+0000","last_undegraded":"2023-11-28T21:56:37.922875+0000","last_fullsized":"2023-11-28T21:56:37.922875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":67,"num_read_kb":77908,"num_write":272,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"52'1265","reported_seq":3536,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.764756+0000","last_change":"2023-11-28T21:40:42.831006+0000","last_active":"2023-11-28T21:56:34.764756+0000","last_peered":"2023-11-28T21:56:34.764756+0000","last_clean":"2023-11-28T21:56:34.764756+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:56:34.764756+0000","last_undegraded":"2023-11-28T21:56:34.764756+0000","last_fullsized":"2023-11-28T21:56:34.764756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1265,"log_dups_size":0,"ondisk_log_size":1265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:03:39.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30778537500000003,"stat_sum":{"num_bytes":30037804,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":190,"num_read_kb":33,"num_write":1241,"num_write_kb":36631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"52'297","reported_seq":2380,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.764009+0000","last_change":"2023-11-28T21:40:42.748885+0000","last_active":"2023-11-28T21:56:35.764009+0000","last_peered":"2023-11-28T21:56:35.764009+0000","last_clean":"2023-11-28T21:56:35.764009+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:56:35.764009+0000","last_undegraded":"2023-11-28T21:56:35.764009+0000","last_fullsized":"2023-11-28T21:56:35.764009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:04:42.978237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21990301600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"52'748","reported_seq":862,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:32.830789+0000","last_change":"2023-11-28T21:34:36.028812+0000","last_active":"2023-11-28T21:56:32.830789+0000","last_peered":"2023-11-28T21:56:32.830789+0000","last_clean":"2023-11-28T21:56:32.830789+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:56:32.830789+0000","last_undegraded":"2023-11-28T21:56:32.830789+0000","last_fullsized":"2023-11-28T21:56:32.830789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:30:43.351524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":90,"num_read_kb":122940,"num_write":682,"num_write_kb":20913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"52'1318","reported_seq":3604,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.710137+0000","last_change":"2023-11-28T21:40:42.828740+0000","last_active":"2023-11-28T21:56:34.710137+0000","last_peered":"2023-11-28T21:56:34.710137+0000","last_clean":"2023-11-28T21:56:34.710137+0000","last_became_active":"2023-11-28T21:35:25.790293+0000","last_became_peered":"2023-11-28T21:35:25.790293+0000","last_unstale":"2023-11-28T21:56:34.710137+0000","last_undegraded":"2023-11-28T21:56:34.710137+0000","last_fullsized":"2023-11-28T21:56:34.710137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:24:35.555501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30978138799999999,"stat_sum":{"num_bytes":36258609,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":208,"num_read_kb":8227,"num_write":1285,"num_write_kb":56637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"52'283","reported_seq":2365,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.761534+0000","last_change":"2023-11-28T21:40:43.068808+0000","last_active":"2023-11-28T21:56:35.761534+0000","last_peered":"2023-11-28T21:56:35.761534+0000","last_clean":"2023-11-28T21:56:35.761534+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:56:35.761534+0000","last_undegraded":"2023-11-28T21:56:35.761534+0000","last_fullsized":"2023-11-28T21:56:35.761534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:14:19.957018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22435063599999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"52'516","reported_seq":608,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:25.006280+0000","last_change":"2023-11-28T21:34:35.998158+0000","last_active":"2023-11-28T21:56:25.006280+0000","last_peered":"2023-11-28T21:56:25.006280+0000","last_clean":"2023-11-28T21:56:25.006280+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:56:25.006280+0000","last_undegraded":"2023-11-28T21:56:25.006280+0000","last_fullsized":"2023-11-28T21:56:25.006280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":69670,"num_write":485,"num_write_kb":16676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"52'1329","reported_seq":3668,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.860899+0000","last_change":"2023-11-28T21:40:43.071209+0000","last_active":"2023-11-28T21:56:34.860899+0000","last_peered":"2023-11-28T21:56:34.860899+0000","last_clean":"2023-11-28T21:56:34.860899+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:56:34.860899+0000","last_undegraded":"2023-11-28T21:56:34.860899+0000","last_fullsized":"2023-11-28T21:56:34.860899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:12:44.679504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.2340457,"stat_sum":{"num_bytes":46614746,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":259,"num_read_kb":20536,"num_write":1294,"num_write_kb":83351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"52'265","reported_seq":2348,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.758280+0000","last_change":"2023-11-28T21:40:43.257554+0000","last_active":"2023-11-28T21:56:35.758280+0000","last_peered":"2023-11-28T21:56:35.758280+0000","last_clean":"2023-11-28T21:56:35.758280+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:56:35.758280+0000","last_undegraded":"2023-11-28T21:56:35.758280+0000","last_fullsized":"2023-11-28T21:56:35.758280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.228570471,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"52'990","reported_seq":1069,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:08.352661+0000","last_change":"2023-11-28T21:34:36.030680+0000","last_active":"2023-11-28T21:56:08.352661+0000","last_peered":"2023-11-28T21:56:08.352661+0000","last_clean":"2023-11-28T21:56:08.352661+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:56:08.352661+0000","last_undegraded":"2023-11-28T21:56:08.352661+0000","last_fullsized":"2023-11-28T21:56:08.352661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:57:29.308408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":90124,"num_write":955,"num_write_kb":20985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"52'1334","reported_seq":3673,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.993668+0000","last_change":"2023-11-28T21:40:42.805595+0000","last_active":"2023-11-28T21:56:34.993668+0000","last_peered":"2023-11-28T21:56:34.993668+0000","last_clean":"2023-11-28T21:56:34.993668+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:56:34.993668+0000","last_undegraded":"2023-11-28T21:56:34.993668+0000","last_fullsized":"2023-11-28T21:56:34.993668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:49:48.909117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29518589499999998,"stat_sum":{"num_bytes":25457761,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":258,"num_read_kb":22214,"num_write":1317,"num_write_kb":78783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"52'308","reported_seq":2383,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.747596+0000","last_change":"2023-11-28T21:40:43.091522+0000","last_active":"2023-11-28T21:56:35.747596+0000","last_peered":"2023-11-28T21:56:35.747596+0000","last_clean":"2023-11-28T21:56:35.747596+0000","last_became_active":"2023-11-28T21:35:25.595386+0000","last_became_peered":"2023-11-28T21:35:25.595386+0000","last_unstale":"2023-11-28T21:56:35.747596+0000","last_undegraded":"2023-11-28T21:56:35.747596+0000","last_fullsized":"2023-11-28T21:56:35.747596+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:35.597109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.148630489,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"52'109","reported_seq":167,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:25.127696+0000","last_change":"2023-11-28T21:34:36.030982+0000","last_active":"2023-11-28T21:56:25.127696+0000","last_peered":"2023-11-28T21:56:25.127696+0000","last_clean":"2023-11-28T21:56:25.127696+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:56:25.127696+0000","last_undegraded":"2023-11-28T21:56:25.127696+0000","last_fullsized":"2023-11-28T21:56:25.127696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:38:55.454007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":67,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"52'1427","reported_seq":3771,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.994844+0000","last_change":"2023-11-28T21:40:42.795304+0000","last_active":"2023-11-28T21:56:34.994844+0000","last_peered":"2023-11-28T21:56:34.994844+0000","last_clean":"2023-11-28T21:56:34.994844+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:56:34.994844+0000","last_undegraded":"2023-11-28T21:56:34.994844+0000","last_fullsized":"2023-11-28T21:56:34.994844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:17.021758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28498848100000002,"stat_sum":{"num_bytes":33245588,"num_objects":317,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":263,"num_read_kb":28708,"num_write":1396,"num_write_kb":77612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"52'1309","reported_seq":3640,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.945963+0000","last_change":"2023-11-28T21:40:43.251187+0000","last_active":"2023-11-28T21:56:34.945963+0000","last_peered":"2023-11-28T21:56:34.945963+0000","last_clean":"2023-11-28T21:56:34.945963+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:56:34.945963+0000","last_undegraded":"2023-11-28T21:56:34.945963+0000","last_fullsized":"2023-11-28T21:56:34.945963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1309,"log_dups_size":0,"ondisk_log_size":1309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27353859400000002,"stat_sum":{"num_bytes":25285382,"num_objects":331,"num_object_clones":0,"num_object_copies":1324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":248,"num_read_kb":4117,"num_write":1287,"num_write_kb":58337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"52'285","reported_seq":2376,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.748373+0000","last_change":"2023-11-28T21:40:42.876095+0000","last_active":"2023-11-28T21:56:35.748373+0000","last_peered":"2023-11-28T21:56:35.748373+0000","last_clean":"2023-11-28T21:56:35.748373+0000","last_became_active":"2023-11-28T21:35:25.589378+0000","last_became_peered":"2023-11-28T21:35:25.589378+0000","last_unstale":"2023-11-28T21:56:35.748373+0000","last_undegraded":"2023-11-28T21:56:35.748373+0000","last_fullsized":"2023-11-28T21:56:35.748373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246426856,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"52'946","reported_seq":1033,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:37.942391+0000","last_change":"2023-11-28T21:34:36.006717+0000","last_active":"2023-11-28T21:56:37.942391+0000","last_peered":"2023-11-28T21:56:37.942391+0000","last_clean":"2023-11-28T21:56:37.942391+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:56:37.942391+0000","last_undegraded":"2023-11-28T21:56:37.942391+0000","last_fullsized":"2023-11-28T21:56:37.942391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":946,"log_dups_size":0,"ondisk_log_size":946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:53:49.703161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":899,"num_write_kb":25075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"52'1266","reported_seq":3553,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.472923+0000","last_change":"2023-11-28T21:40:42.918369+0000","last_active":"2023-11-28T21:56:34.472923+0000","last_peered":"2023-11-28T21:56:34.472923+0000","last_clean":"2023-11-28T21:56:34.472923+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:56:34.472923+0000","last_undegraded":"2023-11-28T21:56:34.472923+0000","last_fullsized":"2023-11-28T21:56:34.472923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1266,"log_dups_size":0,"ondisk_log_size":1266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:08:31.226473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.16839808100000001,"stat_sum":{"num_bytes":18660789,"num_objects":319,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":205,"num_read_kb":12320,"num_write":1249,"num_write_kb":51464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"52'321","reported_seq":2403,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.759859+0000","last_change":"2023-11-28T21:40:42.800175+0000","last_active":"2023-11-28T21:56:35.759859+0000","last_peered":"2023-11-28T21:56:35.759859+0000","last_clean":"2023-11-28T21:56:35.759859+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:56:35.759859+0000","last_undegraded":"2023-11-28T21:56:35.759859+0000","last_fullsized":"2023-11-28T21:56:35.759859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:57:52.149789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28961016299999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"51'85","reported_seq":147,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:39.273808+0000","last_change":"2023-11-28T21:34:36.008250+0000","last_active":"2023-11-28T21:55:39.273808+0000","last_peered":"2023-11-28T21:55:39.273808+0000","last_clean":"2023-11-28T21:55:39.273808+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:55:39.273808+0000","last_undegraded":"2023-11-28T21:55:39.273808+0000","last_fullsized":"2023-11-28T21:55:39.273808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:33:57.700971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":63,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"52'1071","reported_seq":1162,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:13.995104+0000","last_change":"2023-11-28T21:34:36.026200+0000","last_active":"2023-11-28T21:56:13.995104+0000","last_peered":"2023-11-28T21:56:13.995104+0000","last_clean":"2023-11-28T21:56:13.995104+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:56:13.995104+0000","last_undegraded":"2023-11-28T21:56:13.995104+0000","last_fullsized":"2023-11-28T21:56:13.995104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:20:02.158093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":146592,"num_write":1020,"num_write_kb":29113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"52'284","reported_seq":2366,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.766876+0000","last_change":"2023-11-28T21:40:43.254737+0000","last_active":"2023-11-28T21:56:35.766876+0000","last_peered":"2023-11-28T21:56:35.766876+0000","last_clean":"2023-11-28T21:56:35.766876+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:56:35.766876+0000","last_undegraded":"2023-11-28T21:56:35.766876+0000","last_fullsized":"2023-11-28T21:56:35.766876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26886027800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"52'1325","reported_seq":3623,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.710734+0000","last_change":"2023-11-28T21:40:43.102291+0000","last_active":"2023-11-28T21:56:34.710734+0000","last_peered":"2023-11-28T21:56:34.710734+0000","last_clean":"2023-11-28T21:56:34.710734+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:56:34.710734+0000","last_undegraded":"2023-11-28T21:56:34.710734+0000","last_fullsized":"2023-11-28T21:56:34.710734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1325,"log_dups_size":0,"ondisk_log_size":1325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:49:46.549792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.13232148399999999,"stat_sum":{"num_bytes":46546650,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":218,"num_read_kb":9401,"num_write":1296,"num_write_kb":74280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"51'467","reported_seq":534,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:55:43.709564+0000","last_change":"2023-11-28T21:34:36.010772+0000","last_active":"2023-11-28T21:55:43.709564+0000","last_peered":"2023-11-28T21:55:43.709564+0000","last_clean":"2023-11-28T21:55:43.709564+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:55:43.709564+0000","last_undegraded":"2023-11-28T21:55:43.709564+0000","last_fullsized":"2023-11-28T21:55:43.709564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:20:05.713621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":48896,"num_write":437,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"52'279","reported_seq":2360,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.748909+0000","last_change":"2023-11-28T21:40:43.054453+0000","last_active":"2023-11-28T21:56:35.748909+0000","last_peered":"2023-11-28T21:56:35.748909+0000","last_clean":"2023-11-28T21:56:35.748909+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:56:35.748909+0000","last_undegraded":"2023-11-28T21:56:35.748909+0000","last_fullsized":"2023-11-28T21:56:35.748909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:04:59.006875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24793844200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"52'1320","reported_seq":3657,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.943074+0000","last_change":"2023-11-28T21:40:42.807134+0000","last_active":"2023-11-28T21:56:34.943074+0000","last_peered":"2023-11-28T21:56:34.943074+0000","last_clean":"2023-11-28T21:56:34.943074+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:56:34.943074+0000","last_undegraded":"2023-11-28T21:56:34.943074+0000","last_fullsized":"2023-11-28T21:56:34.943074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1320,"log_dups_size":0,"ondisk_log_size":1320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:55.847688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29562192300000001,"stat_sum":{"num_bytes":29852008,"num_objects":333,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":257,"num_read_kb":28714,"num_write":1297,"num_write_kb":83698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"52'321","reported_seq":2404,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762405+0000","last_change":"2023-11-28T21:40:42.917455+0000","last_active":"2023-11-28T21:56:35.762405+0000","last_peered":"2023-11-28T21:56:35.762405+0000","last_clean":"2023-11-28T21:56:35.762405+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:56:35.762405+0000","last_undegraded":"2023-11-28T21:56:35.762405+0000","last_fullsized":"2023-11-28T21:56:35.762405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:29.420871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18187520500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"52'328","reported_seq":2412,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.740057+0000","last_change":"2023-11-28T21:40:43.100474+0000","last_active":"2023-11-28T21:56:35.740057+0000","last_peered":"2023-11-28T21:56:35.740057+0000","last_clean":"2023-11-28T21:56:35.740057+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:56:35.740057+0000","last_undegraded":"2023-11-28T21:56:35.740057+0000","last_fullsized":"2023-11-28T21:56:35.740057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:44:47.346946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14391041299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"52'303","reported_seq":2387,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.763112+0000","last_change":"2023-11-28T21:40:42.831609+0000","last_active":"2023-11-28T21:56:35.763112+0000","last_peered":"2023-11-28T21:56:35.763112+0000","last_clean":"2023-11-28T21:56:35.763112+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:56:35.763112+0000","last_undegraded":"2023-11-28T21:56:35.763112+0000","last_fullsized":"2023-11-28T21:56:35.763112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:49:50.710370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30621145,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"52'294","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.767559+0000","last_change":"2023-11-28T21:40:43.255460+0000","last_active":"2023-11-28T21:56:35.767559+0000","last_peered":"2023-11-28T21:56:35.767559+0000","last_clean":"2023-11-28T21:56:35.767559+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:56:35.767559+0000","last_undegraded":"2023-11-28T21:56:35.767559+0000","last_fullsized":"2023-11-28T21:56:35.767559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26967866099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"52'275","reported_seq":2358,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.758859+0000","last_change":"2023-11-28T21:40:42.801894+0000","last_active":"2023-11-28T21:56:35.758859+0000","last_peered":"2023-11-28T21:56:35.758859+0000","last_clean":"2023-11-28T21:56:35.758859+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:56:35.758859+0000","last_undegraded":"2023-11-28T21:56:35.758859+0000","last_fullsized":"2023-11-28T21:56:35.758859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:46:20.949106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29058787400000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"52'293","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.760786+0000","last_change":"2023-11-28T21:40:42.886237+0000","last_active":"2023-11-28T21:56:35.760786+0000","last_peered":"2023-11-28T21:56:35.760786+0000","last_clean":"2023-11-28T21:56:35.760786+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:56:35.760786+0000","last_undegraded":"2023-11-28T21:56:35.760786+0000","last_fullsized":"2023-11-28T21:56:35.760786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246741036,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"52'316","reported_seq":2399,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.747193+0000","last_change":"2023-11-28T21:40:42.916517+0000","last_active":"2023-11-28T21:56:35.747193+0000","last_peered":"2023-11-28T21:56:35.747193+0000","last_clean":"2023-11-28T21:56:35.747193+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:56:35.747193+0000","last_undegraded":"2023-11-28T21:56:35.747193+0000","last_fullsized":"2023-11-28T21:56:35.747193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:26.983521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18459727300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"52'287","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762179+0000","last_change":"2023-11-28T21:40:42.803865+0000","last_active":"2023-11-28T21:56:35.762179+0000","last_peered":"2023-11-28T21:56:35.762179+0000","last_clean":"2023-11-28T21:56:35.762179+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:56:35.762179+0000","last_undegraded":"2023-11-28T21:56:35.762179+0000","last_fullsized":"2023-11-28T21:56:35.762179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:06:41.883126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29272911699999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"52'285","reported_seq":2369,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762929+0000","last_change":"2023-11-28T21:40:42.823937+0000","last_active":"2023-11-28T21:56:35.762929+0000","last_peered":"2023-11-28T21:56:35.762929+0000","last_clean":"2023-11-28T21:56:35.762929+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:56:35.762929+0000","last_undegraded":"2023-11-28T21:56:35.762929+0000","last_fullsized":"2023-11-28T21:56:35.762929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:18:46.016564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30579284099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"52'272","reported_seq":2354,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.759359+0000","last_change":"2023-11-28T21:40:42.799326+0000","last_active":"2023-11-28T21:56:35.759359+0000","last_peered":"2023-11-28T21:56:35.759359+0000","last_clean":"2023-11-28T21:56:35.759359+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:56:35.759359+0000","last_undegraded":"2023-11-28T21:56:35.759359+0000","last_fullsized":"2023-11-28T21:56:35.759359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:42:13.718530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28793565599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"52'285","reported_seq":2368,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757085+0000","last_change":"2023-11-28T21:40:42.831259+0000","last_active":"2023-11-28T21:56:35.757085+0000","last_peered":"2023-11-28T21:56:35.757085+0000","last_clean":"2023-11-28T21:56:35.757085+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:56:35.757085+0000","last_undegraded":"2023-11-28T21:56:35.757085+0000","last_fullsized":"2023-11-28T21:56:35.757085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:47:17.850510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30636868099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"52'297","reported_seq":2380,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:36.101069+0000","last_change":"2023-11-28T21:40:43.253941+0000","last_active":"2023-11-28T21:56:36.101069+0000","last_peered":"2023-11-28T21:56:36.101069+0000","last_clean":"2023-11-28T21:56:36.101069+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:56:36.101069+0000","last_undegraded":"2023-11-28T21:56:36.101069+0000","last_fullsized":"2023-11-28T21:56:36.101069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26835999100000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"52'297","reported_seq":2379,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.755271+0000","last_change":"2023-11-28T21:40:42.910839+0000","last_active":"2023-11-28T21:56:35.755271+0000","last_peered":"2023-11-28T21:56:35.755271+0000","last_clean":"2023-11-28T21:56:35.755271+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:56:35.755271+0000","last_undegraded":"2023-11-28T21:56:35.755271+0000","last_fullsized":"2023-11-28T21:56:35.755271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:09:31.418117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.194469013,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"52'293","reported_seq":2376,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.750187+0000","last_change":"2023-11-28T21:40:42.749349+0000","last_active":"2023-11-28T21:56:35.750187+0000","last_peered":"2023-11-28T21:56:35.750187+0000","last_clean":"2023-11-28T21:56:35.750187+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:56:35.750187+0000","last_undegraded":"2023-11-28T21:56:35.750187+0000","last_fullsized":"2023-11-28T21:56:35.750187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:25:20.196676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21943309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"52'268","reported_seq":2351,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.743727+0000","last_change":"2023-11-28T21:40:43.092423+0000","last_active":"2023-11-28T21:56:35.743727+0000","last_peered":"2023-11-28T21:56:35.743727+0000","last_clean":"2023-11-28T21:56:35.743727+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:56:35.743727+0000","last_undegraded":"2023-11-28T21:56:35.743727+0000","last_fullsized":"2023-11-28T21:56:35.743727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:33:34.736900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14941532399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"52'313","reported_seq":2396,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.748050+0000","last_change":"2023-11-28T21:40:42.827330+0000","last_active":"2023-11-28T21:56:35.748050+0000","last_peered":"2023-11-28T21:56:35.748050+0000","last_clean":"2023-11-28T21:56:35.748050+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:56:35.748050+0000","last_undegraded":"2023-11-28T21:56:35.748050+0000","last_fullsized":"2023-11-28T21:56:35.748050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:47:49.288912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30374592,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"52'301","reported_seq":2384,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762147+0000","last_change":"2023-11-28T21:40:42.820428+0000","last_active":"2023-11-28T21:56:35.762147+0000","last_peered":"2023-11-28T21:56:35.762147+0000","last_clean":"2023-11-28T21:56:35.762147+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:56:35.762147+0000","last_undegraded":"2023-11-28T21:56:35.762147+0000","last_fullsized":"2023-11-28T21:56:35.762147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:52:59.333901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30238551699999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"52'316","reported_seq":2400,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.756399+0000","last_change":"2023-11-28T21:40:42.876664+0000","last_active":"2023-11-28T21:56:35.756399+0000","last_peered":"2023-11-28T21:56:35.756399+0000","last_clean":"2023-11-28T21:56:35.756399+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:56:35.756399+0000","last_undegraded":"2023-11-28T21:56:35.756399+0000","last_fullsized":"2023-11-28T21:56:35.756399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246777987,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"52'301","reported_seq":2384,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.764229+0000","last_change":"2023-11-28T21:40:43.257791+0000","last_active":"2023-11-28T21:56:35.764229+0000","last_peered":"2023-11-28T21:56:35.764229+0000","last_clean":"2023-11-28T21:56:35.764229+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:56:35.764229+0000","last_undegraded":"2023-11-28T21:56:35.764229+0000","last_fullsized":"2023-11-28T21:56:35.764229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21399896700000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"52'274","reported_seq":2356,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.758017+0000","last_change":"2023-11-28T21:40:42.798449+0000","last_active":"2023-11-28T21:56:35.758017+0000","last_peered":"2023-11-28T21:56:35.758017+0000","last_clean":"2023-11-28T21:56:35.758017+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:56:35.758017+0000","last_undegraded":"2023-11-28T21:56:35.758017+0000","last_fullsized":"2023-11-28T21:56:35.758017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:01:12.073001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.287656776,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"52'261","reported_seq":2343,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.761596+0000","last_change":"2023-11-28T21:40:43.071081+0000","last_active":"2023-11-28T21:56:35.761596+0000","last_peered":"2023-11-28T21:56:35.761596+0000","last_clean":"2023-11-28T21:56:35.761596+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:56:35.761596+0000","last_undegraded":"2023-11-28T21:56:35.761596+0000","last_fullsized":"2023-11-28T21:56:35.761596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:01:10.753163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.222804104,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"52'261","reported_seq":2343,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.753765+0000","last_change":"2023-11-28T21:40:42.914448+0000","last_active":"2023-11-28T21:56:35.753765+0000","last_peered":"2023-11-28T21:56:35.753765+0000","last_clean":"2023-11-28T21:56:35.753765+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:56:35.753765+0000","last_undegraded":"2023-11-28T21:56:35.753765+0000","last_fullsized":"2023-11-28T21:56:35.753765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:33:37.008462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19768582900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"52'283","reported_seq":2365,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.971414+0000","last_change":"2023-11-28T21:40:42.916248+0000","last_active":"2023-11-28T21:56:35.971414+0000","last_peered":"2023-11-28T21:56:35.971414+0000","last_clean":"2023-11-28T21:56:35.971414+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:56:35.971414+0000","last_undegraded":"2023-11-28T21:56:35.971414+0000","last_fullsized":"2023-11-28T21:56:35.971414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:18:17.635492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18434383600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"52'314","reported_seq":2404,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757738+0000","last_change":"2023-11-28T21:40:42.754000+0000","last_active":"2023-11-28T21:56:35.757738+0000","last_peered":"2023-11-28T21:56:35.757738+0000","last_clean":"2023-11-28T21:56:35.757738+0000","last_became_active":"2023-11-28T21:35:25.598839+0000","last_became_peered":"2023-11-28T21:35:25.598839+0000","last_unstale":"2023-11-28T21:56:35.757738+0000","last_undegraded":"2023-11-28T21:56:35.757738+0000","last_fullsized":"2023-11-28T21:56:35.757738+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:15:51.550099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22244017299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"52'291","reported_seq":2372,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.764150+0000","last_change":"2023-11-28T21:40:42.796731+0000","last_active":"2023-11-28T21:56:35.764150+0000","last_peered":"2023-11-28T21:56:35.764150+0000","last_clean":"2023-11-28T21:56:35.764150+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:56:35.764150+0000","last_undegraded":"2023-11-28T21:56:35.764150+0000","last_fullsized":"2023-11-28T21:56:35.764150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:50.215245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28637254899999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"52'292","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.746097+0000","last_change":"2023-11-28T21:40:42.745933+0000","last_active":"2023-11-28T21:56:35.746097+0000","last_peered":"2023-11-28T21:56:35.746097+0000","last_clean":"2023-11-28T21:56:35.746097+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:56:35.746097+0000","last_undegraded":"2023-11-28T21:56:35.746097+0000","last_fullsized":"2023-11-28T21:56:35.746097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:46:52.720487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.219142752,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"52'282","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.761971+0000","last_change":"2023-11-28T21:40:42.831458+0000","last_active":"2023-11-28T21:56:35.761971+0000","last_peered":"2023-11-28T21:56:35.761971+0000","last_clean":"2023-11-28T21:56:35.761971+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:56:35.761971+0000","last_undegraded":"2023-11-28T21:56:35.761971+0000","last_fullsized":"2023-11-28T21:56:35.761971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:33:21.021313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.307113203,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"52'297","reported_seq":2379,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.745163+0000","last_change":"2023-11-28T21:40:42.736971+0000","last_active":"2023-11-28T21:56:35.745163+0000","last_peered":"2023-11-28T21:56:35.745163+0000","last_clean":"2023-11-28T21:56:35.745163+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:56:35.745163+0000","last_undegraded":"2023-11-28T21:56:35.745163+0000","last_fullsized":"2023-11-28T21:56:35.745163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:33:12.599313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21561430200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"52'287","reported_seq":2372,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.763805+0000","last_change":"2023-11-28T21:40:42.829941+0000","last_active":"2023-11-28T21:56:35.763805+0000","last_peered":"2023-11-28T21:56:35.763805+0000","last_clean":"2023-11-28T21:56:35.763805+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:56:35.763805+0000","last_undegraded":"2023-11-28T21:56:35.763805+0000","last_fullsized":"2023-11-28T21:56:35.763805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:26:40.360501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30486287499999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"52'281","reported_seq":2365,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757024+0000","last_change":"2023-11-28T21:40:43.239342+0000","last_active":"2023-11-28T21:56:35.757024+0000","last_peered":"2023-11-28T21:56:35.757024+0000","last_clean":"2023-11-28T21:56:35.757024+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:56:35.757024+0000","last_undegraded":"2023-11-28T21:56:35.757024+0000","last_fullsized":"2023-11-28T21:56:35.757024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26474323500000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"52'292","reported_seq":2377,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:36.084027+0000","last_change":"2023-11-28T21:40:42.885846+0000","last_active":"2023-11-28T21:56:36.084027+0000","last_peered":"2023-11-28T21:56:36.084027+0000","last_clean":"2023-11-28T21:56:36.084027+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:56:36.084027+0000","last_undegraded":"2023-11-28T21:56:36.084027+0000","last_fullsized":"2023-11-28T21:56:36.084027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24685295800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"52'289","reported_seq":2373,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757067+0000","last_change":"2023-11-28T21:40:42.753421+0000","last_active":"2023-11-28T21:56:35.757067+0000","last_peered":"2023-11-28T21:56:35.757067+0000","last_clean":"2023-11-28T21:56:35.757067+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:56:35.757067+0000","last_undegraded":"2023-11-28T21:56:35.757067+0000","last_fullsized":"2023-11-28T21:56:35.757067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:06:35.683601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22257470400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":927864585,"num_objects":10340,"num_object_clones":0,"num_object_copies":41360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10340,"num_whiteouts":0,"num_read":7542,"num_read_kb":441308,"num_write":41596,"num_write_kb":1955411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1822244864,"data_stored":1974304768,"data_compressed":124280088,"data_compressed_allocated":140013568,"data_compressed_original":292073472,"omap_allocated":0,"internal_metadata":0},"log_size":42425,"ondisk_log_size":42425,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18675,"ondisk_log_size":18675,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":302728427,"num_objects":195,"num_object_clones":0,"num_object_copies":390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":195,"num_whiteouts":0,"num_read":1486,"num_read_kb":1965797,"num_write":14818,"num_write_kb":401785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172695552,"data_stored":605456854,"data_compressed":82457192,"data_compressed_allocated":105717760,"data_compressed_original":538640278,"omap_allocated":0,"internal_metadata":0},"log_size":15848,"ondisk_log_size":15848,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575119,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345672,"kb_used_data":211776,"kb_used_omap":1,"kb_used_meta":133886,"kb_avail":94026168,"statfs":{"total":96636764160,"available":96282796032,"internally_reserved":0,"allocated":216858624,"data_stored":292324283,"data_compressed":24941039,"data_compressed_allocated":29831168,"data_compressed_original":105603072,"omap_allocated":1590,"internal_metadata":137099722},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.97799999999999998},"min":{"1min":0.51400000000000001,"5min":0.41399999999999998,"15min":0.35299999999999998},"max":{"1min":1.232,"5min":5.056,"15min":47.475000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.82299999999999995},"min":{"1min":0.54200000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.145,"5min":4.069,"15min":5.3310000000000004},"last":0.68600000000000005}]},{"osd":1,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.98999999999999999,"15min":1.099},"min":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":2.5630000000000002,"15min":47.529000000000003},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.083,"15min":1.008},"min":{"1min":0.64300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.171,"5min":17.460000000000001,"15min":17.460000000000001},"last":0.83999999999999997}]},{"osd":2,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.86599999999999999,"15min":0.91200000000000003},"min":{"1min":0.58399999999999996,"5min":0.504,"15min":0.47299999999999998},"max":{"1min":1.0660000000000001,"5min":2.9359999999999999,"15min":5.5140000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0229999999999999,"15min":1.022},"min":{"1min":0.70199999999999996,"5min":0.60499999999999998,"15min":0.498},"max":{"1min":1.248,"5min":2.972,"15min":5.5979999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1719999999999999,"15min":1.179},"min":{"1min":0.63800000000000001,"5min":0.57299999999999995,"15min":0.44600000000000001},"max":{"1min":1.196,"5min":16.113,"15min":47.57},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.171,"15min":1.0649999999999999},"min":{"1min":0.73599999999999999,"5min":0.65400000000000003,"15min":0.53800000000000003},"max":{"1min":1.2749999999999999,"5min":11.148999999999999,"15min":12.273},"last":0.81100000000000005}]},{"osd":4,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.1180000000000001,"15min":1.054},"min":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.35499999999999998},"max":{"1min":1.1990000000000001,"5min":17.542000000000002,"15min":17.542000000000002},"last":0.998},{"interface":"front","average":{"1min":1.075,"5min":1.0489999999999999,"15min":1.196},"min":{"1min":0.84699999999999998,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.363,"5min":1.5329999999999999,"15min":49.106000000000002},"last":1.0860000000000001}]},{"osd":5,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.1220000000000001,"15min":1.2909999999999999},"min":{"1min":0.80600000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.345,"5min":6.6420000000000003,"15min":64.855999999999995},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.1679999999999999,"15min":1.1970000000000001},"min":{"1min":0.73499999999999999,"5min":0.71399999999999997,"15min":0.56399999999999995},"max":{"1min":1.2689999999999999,"5min":11.327,"15min":16.753},"last":1.048}]},{"osd":6,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.194,"15min":1.1639999999999999},"min":{"1min":0.88600000000000001,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":1.1930000000000001,"5min":11.641999999999999,"15min":11.641999999999999},"last":1.22},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1399999999999999,"15min":1.153},"min":{"1min":0.97999999999999998,"5min":0.61499999999999999,"15min":0.502},"max":{"1min":1.4199999999999999,"5min":1.474,"15min":7.4909999999999997},"last":0.93400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575117,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":380176,"kb_used_data":254600,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":93991664,"statfs":{"total":96636764160,"available":96247463936,"internally_reserved":0,"allocated":260710400,"data_stored":335877014,"data_compressed":24633140,"data_compressed_allocated":29302784,"data_compressed_original":104771531,"omap_allocated":1594,"internal_metadata":128580038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,32,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":2.9159999999999999,"5min":1.248,"15min":1.0109999999999999},"min":{"1min":0.51500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":40.999000000000002,"5min":40.999000000000002,"15min":40.999000000000002},"last":2.9119999999999999},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.05,"15min":1.036},"min":{"1min":0.58299999999999996,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":14.132999999999999,"5min":14.132999999999999,"15min":24.399999999999999},"last":2.3370000000000002}]},{"osd":1,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.89500000000000002,"15min":0.88700000000000001},"min":{"1min":0.71999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2769999999999999,"5min":2.2400000000000002,"15min":11.196},"last":7.0709999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.90800000000000003,"15min":0.95899999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.35799999999999998},"max":{"1min":1.1759999999999999,"5min":6.1550000000000002,"15min":7.9130000000000003},"last":13.404}]},{"osd":2,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.89500000000000002,"15min":0.94799999999999995},"min":{"1min":0.66300000000000003,"5min":0.502,"15min":0.39100000000000001},"max":{"1min":3.855,"5min":3.855,"15min":21.986999999999998},"last":2.6829999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.92800000000000005,"15min":0.99099999999999999},"min":{"1min":0.621,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2829999999999999,"5min":1.419,"15min":8.6170000000000009},"last":8.4730000000000008}]},{"osd":3,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.94799999999999995,"15min":0.96299999999999997},"min":{"1min":0.57099999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.8439999999999999,"5min":3.2679999999999998,"15min":11.239000000000001},"last":15.964},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0069999999999999,"15min":1.087},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.7959999999999998,"5min":3.29,"15min":24.745000000000001},"last":16.602}]},{"osd":4,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91800000000000004,"15min":1.014},"min":{"1min":0.71599999999999997,"5min":0.32300000000000001,"15min":0.30299999999999999},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":16.015000000000001},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.044,"15min":1.079},"min":{"1min":0.69499999999999995,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":13.193},"last":2.4929999999999999}]},{"osd":5,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.98099999999999998,"15min":1.109},"min":{"1min":0.65000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":2.069,"5min":2.069,"15min":25.006},"last":1.661},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0629999999999999,"15min":1.1419999999999999},"min":{"1min":0.46700000000000003,"5min":0.33800000000000002,"15min":0.33000000000000002},"max":{"1min":1.405,"5min":1.4790000000000001,"15min":7.843},"last":3.2959999999999998}]},{"osd":7,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.139},"min":{"1min":0.79200000000000004,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":1.1950000000000001,"5min":1.9179999999999999,"15min":7.883},"last":5.6799999999999997},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.177,"15min":1.276},"min":{"1min":0.93300000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.3169999999999999,"5min":1.9610000000000001,"15min":27.942},"last":5.7199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575117,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363676,"kb_used_data":238100,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":94008164,"statfs":{"total":96636764160,"available":96264359936,"internally_reserved":0,"allocated":243814400,"data_stored":323612635,"data_compressed":28196796,"data_compressed_allocated":33439744,"data_compressed_original":113541120,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.99399999999999999,"15min":0.89800000000000002},"min":{"1min":0.47399999999999998,"5min":0.46500000000000002,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":15.005000000000001,"15min":15.005000000000001},"last":1.089},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88800000000000001},"min":{"1min":0.501,"5min":0.47799999999999998,"15min":0.46700000000000003},"max":{"1min":1.2290000000000001,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":1.2170000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81100000000000005,"15min":0.83999999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.48199999999999998},"max":{"1min":0.89800000000000002,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.88900000000000001,"15min":0.91300000000000003},"min":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.496},"max":{"1min":1.0609999999999999,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.135}]},{"osd":2,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.016,"15min":0.97399999999999998},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.105,"5min":14.775,"15min":14.775},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.1040000000000001,"15min":0.94599999999999995},"min":{"1min":0.72899999999999998,"5min":0.63800000000000001,"15min":0.57499999999999996},"max":{"1min":1.1919999999999999,"5min":14.901999999999999,"15min":14.901999999999999},"last":1.1799999999999999}]},{"osd":3,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.034,"15min":0.98899999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.54400000000000004},"max":{"1min":1.2529999999999999,"5min":15.154,"15min":15.154},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.98999999999999999,"15min":1.0109999999999999},"min":{"1min":0.63900000000000001,"5min":0.60799999999999998,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":3.4289999999999998,"15min":5.5170000000000003},"last":1.28}]},{"osd":4,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.526,"15min":1.181},"min":{"1min":0.64600000000000002,"5min":0.59299999999999997,"15min":0.51400000000000001},"max":{"1min":1.292,"5min":61.020000000000003,"15min":61.020000000000003},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.143,"15min":1.0920000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2490000000000001,"5min":15.085000000000001,"15min":15.085000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.139,"15min":1.123},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.56699999999999995},"max":{"1min":1.5469999999999999,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.194,"15min":1.1699999999999999},"min":{"1min":1.052,"5min":0.72499999999999998,"15min":0.48899999999999999},"max":{"1min":1.6799999999999999,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.379}]},{"osd":7,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1759999999999999,"15min":1.1639999999999999},"min":{"1min":0.94099999999999995,"5min":0.75600000000000001,"15min":0.56899999999999995},"max":{"1min":1.4219999999999999,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.331},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.054,"15min":1.0189999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.33200000000000002},"max":{"1min":1.085,"5min":6.2869999999999999,"15min":6.2869999999999999},"last":0.96599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575117,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357644,"kb_used_data":236228,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":94014196,"statfs":{"total":96636764160,"available":96270536704,"internally_reserved":0,"allocated":241897472,"data_stored":321511706,"data_compressed":25475129,"data_compressed_allocated":30593024,"data_compressed_original":110510027,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.155,"15min":0.97399999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0449999999999999,"5min":43.139000000000003,"15min":43.139000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84399999999999997,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.39900000000000002},"max":{"1min":1.3049999999999999,"5min":4.444,"15min":17.446000000000002},"last":3.9809999999999999}]},{"osd":1,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.98799999999999999,"15min":1.054},"min":{"1min":0.64900000000000002,"5min":0.48199999999999998,"15min":0.44},"max":{"1min":1.2849999999999999,"5min":11.557,"15min":21.763999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0229999999999999,"15min":1.4790000000000001},"min":{"1min":0.65700000000000003,"5min":0.53800000000000003,"15min":0.46500000000000002},"max":{"1min":1.452,"5min":5.3479999999999999,"15min":78.649000000000001},"last":1.1839999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":0.96499999999999997,"15min":1.0529999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.880000000000001,"5min":10.880000000000001,"15min":28.341000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.073,"15min":1.548},"min":{"1min":0.61699999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"max":{"1min":2.069,"5min":11.614000000000001,"15min":94.998999999999995},"last":0.77300000000000002}]},{"osd":3,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.208,"15min":1.103},"min":{"1min":0.86299999999999999,"5min":0.67500000000000004,"15min":0.51300000000000001},"max":{"1min":9.984,"5min":9.984,"15min":13.202999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.335,"5min":1.0609999999999999,"15min":1.056},"min":{"1min":0.69299999999999995,"5min":0.54800000000000004,"15min":0.45600000000000002},"max":{"1min":9.952,"5min":9.952,"15min":16.277999999999999},"last":0.80400000000000005}]},{"osd":5,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.0049999999999999,"15min":1.1259999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.28199999999999997},"max":{"1min":1.385,"5min":3.4409999999999998,"15min":17.033000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.3799999999999999,"15min":1.3},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.34000000000000002},"max":{"1min":1.1759999999999999,"5min":38.939,"15min":38.939},"last":1.3740000000000001}]},{"osd":6,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.046,"15min":1.1699999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.37},"max":{"1min":1.4710000000000001,"5min":4.476,"15min":17.085000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.026,"5min":1.4850000000000001,"15min":1.3380000000000001},"min":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.39000000000000001},"max":{"1min":1.631,"5min":43.420000000000002,"15min":43.420000000000002},"last":1.4259999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.121,"15min":1.6160000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.309},"max":{"1min":1.5269999999999999,"5min":3.2109999999999999,"15min":77.638999999999996},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.24,"15min":1.175},"min":{"1min":0.90200000000000002,"5min":0.75900000000000001,"15min":0.51500000000000001},"max":{"1min":1.798,"5min":3.613,"15min":5.6200000000000001},"last":1.2889999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575118,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460908,"kb_used_data":307556,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":93910932,"statfs":{"total":96636764160,"available":96164794368,"internally_reserved":0,"allocated":314937344,"data_stored":410561521,"data_compressed":36989356,"data_compressed_allocated":43745280,"data_compressed_original":139673600,"omap_allocated":1592,"internal_metadata":157022664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.89500000000000002,"15min":0.70999999999999996},"min":{"1min":0.246,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.67900000000000005,"5min":36.585000000000001,"15min":36.585000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.89500000000000002,"15min":0.67800000000000005},"min":{"1min":0.379,"5min":0.34300000000000003,"15min":0.33100000000000002},"max":{"1min":0.68400000000000005,"5min":30.065000000000001,"15min":30.065000000000001},"last":0.44600000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.59999999999999998},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.26300000000000001},"max":{"1min":0.876,"5min":4.8099999999999996,"15min":10.449},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.61799999999999999,"15min":0.67400000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.98599999999999999,"5min":4.8520000000000003,"15min":10.481999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59099999999999997,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.27900000000000003},"max":{"1min":0.70299999999999996,"5min":1.25,"15min":10.369},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.56799999999999995,"15min":0.69599999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78600000000000003,"5min":1.1240000000000001,"15min":10.406000000000001},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91700000000000004,"15min":1.361},"min":{"1min":0.71799999999999997,"5min":0.50700000000000001,"15min":0.376},"max":{"1min":1.0129999999999999,"5min":4.2119999999999997,"15min":110.39700000000001},"last":0.872},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.147,"5min":7.2030000000000003,"15min":108.777},"last":0.83999999999999997}]},{"osd":5,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.86199999999999999,"15min":0.92600000000000005},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.39900000000000002},"max":{"1min":0.98799999999999999,"5min":7.2679999999999998,"15min":17.513000000000002},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.873,"15min":0.92900000000000005},"min":{"1min":0.621,"5min":0.621,"15min":0.52300000000000002},"max":{"1min":0.998,"5min":1.7070000000000001,"15min":10.516999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82399999999999995,"15min":1.4019999999999999},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.365,"15min":138.91300000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.875,"15min":0.94499999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.48299999999999998},"max":{"1min":1.0980000000000001,"5min":1.766,"15min":10.302},"last":0.65000000000000002}]},{"osd":7,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.91900000000000004,"15min":0.95799999999999996},"min":{"1min":0.63100000000000001,"5min":0.60999999999999999,"15min":0.45200000000000001},"max":{"1min":1.1990000000000001,"5min":7.3040000000000003,"15min":17.263000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.97799999999999998,"15min":1.034},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.49299999999999999},"max":{"1min":1.0289999999999999,"5min":7.3339999999999996,"15min":28.613},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575116,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288308,"kb_used_data":176620,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94083532,"statfs":{"total":96636764160,"available":96341536768,"internally_reserved":0,"allocated":180858880,"data_stored":223905413,"data_compressed":16019215,"data_compressed_allocated":18952192,"data_compressed_original":62304256,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.623,"15min":0.57999999999999996},"min":{"1min":0.29199999999999998,"5min":0.26800000000000002,"15min":0.23599999999999999},"max":{"1min":0.58299999999999996,"5min":13.599,"15min":13.599},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.67300000000000004,"15min":0.60699999999999998},"min":{"1min":0.34100000000000003,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.73099999999999998,"5min":8.0630000000000006,"15min":8.0630000000000006},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.53300000000000003,"15min":0.57499999999999996},"min":{"1min":0.33400000000000002,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.61799999999999999,"5min":3.0760000000000001,"15min":9.3260000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.70699999999999996,"15min":0.67000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":5.6660000000000004,"15min":7.4779999999999998},"last":0.56399999999999995}]},{"osd":3,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.95199999999999996,"15min":0.79200000000000004},"min":{"1min":0.42099999999999999,"5min":0.32300000000000001,"15min":0.307},"max":{"1min":0.878,"5min":29.123999999999999,"15min":29.123999999999999},"last":0.495},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.63100000000000001,"15min":0.59699999999999998},"min":{"1min":0.40000000000000002,"5min":0.312,"15min":0.28000000000000003},"max":{"1min":0.754,"5min":5.5410000000000004,"15min":7.4390000000000001},"last":0.36199999999999999}]},{"osd":4,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.94099999999999995,"15min":0.92300000000000004},"min":{"1min":0.58799999999999997,"5min":0.58599999999999997,"15min":0.49099999999999999},"max":{"1min":1.0489999999999999,"5min":5.6289999999999996,"15min":7.3630000000000004},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.87,"15min":0.879},"min":{"1min":0.60599999999999998,"5min":0.51200000000000001,"15min":0.41399999999999998},"max":{"1min":0.98599999999999999,"5min":5.5490000000000004,"15min":9.2289999999999992},"last":0.745}]},{"osd":5,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.85499999999999998,"15min":0.84299999999999997},"min":{"1min":0.622,"5min":0.501,"15min":0.501},"max":{"1min":1.01,"5min":3.3610000000000002,"15min":3.3610000000000002},"last":0.875},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.0109999999999999,"15min":0.93700000000000006},"min":{"1min":0.67300000000000004,"5min":0.59399999999999997,"15min":0.48399999999999999},"max":{"1min":1.266,"5min":6.343,"15min":7.2649999999999997},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.0489999999999999,"15min":0.91800000000000004},"min":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":1.044,"5min":18.959,"15min":18.959},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.1339999999999999,"15min":0.97499999999999998},"min":{"1min":0.68300000000000005,"5min":0.622,"15min":0.52200000000000002},"max":{"1min":1.0700000000000001,"5min":21.957999999999998,"15min":21.957999999999998},"last":0.90300000000000002}]},{"osd":7,"last update":"Tue Nov 28 21:55:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.89700000000000002,"15min":0.92100000000000004},"min":{"1min":0.68400000000000005,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":0.999,"5min":3.5299999999999998,"15min":9.2799999999999994},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.93300000000000005,"15min":0.90900000000000003},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.57099999999999995},"max":{"1min":1.042,"5min":2.6379999999999999,"15min":3.4470000000000001},"last":0.85499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575116,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469208,"kb_used_data":311696,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93902632,"statfs":{"total":96636764160,"available":96156295168,"internally_reserved":0,"allocated":319176704,"data_stored":398176883,"data_compressed":30704819,"data_compressed_allocated":36466688,"data_compressed_original":115777536,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":0.75700000000000001,"15min":0.69599999999999995},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.246},"max":{"1min":10.096,"5min":12.311999999999999,"15min":14.538},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.623,"15min":0.65100000000000002},"min":{"1min":0.39000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.4119999999999999,"5min":8.2479999999999993,"15min":14.618},"last":0.52100000000000002}]},{"osd":2,"last update":"Tue Nov 28 21:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.66100000000000003,"15min":0.63100000000000001},"min":{"1min":0.40799999999999997,"5min":0.34899999999999998,"15min":0.31900000000000001},"max":{"1min":1.7150000000000001,"5min":5.1689999999999996,"15min":8.4749999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":4.7030000000000003,"15min":8.5150000000000006},"last":0.54900000000000004}]},{"osd":3,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.75900000000000001,"15min":0.84699999999999998},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.27600000000000002},"max":{"1min":3.028,"5min":13.385,"15min":43.536999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.84699999999999998},"min":{"1min":0.501,"5min":0.443,"15min":0.375},"max":{"1min":5.6050000000000004,"5min":8.2899999999999991,"15min":18.308},"last":0.59399999999999997}]},{"osd":4,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":0.997,"15min":0.95699999999999996},"min":{"1min":0.63500000000000001,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":9.9849999999999994,"5min":9.9849999999999994,"15min":9.9849999999999994},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":0.97899999999999998,"15min":1.079},"min":{"1min":0.61699999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":10.287000000000001,"5min":10.287000000000001,"15min":43.497999999999998},"last":0.85199999999999998}]},{"osd":5,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.93100000000000005,"15min":0.91600000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":5.5350000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.016,"15min":0.99399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.448,"5min":3.448,"15min":5.7409999999999997},"last":0.96899999999999997}]},{"osd":6,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.0449999999999999,"15min":1.0109999999999999},"min":{"1min":0.69899999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":10.023,"5min":10.023,"15min":10.385},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.139,"5min":1.05,"15min":1.2470000000000001},"min":{"1min":0.77700000000000002,"5min":0.65100000000000002,"15min":0.55200000000000005},"max":{"1min":3.7480000000000002,"5min":5.9349999999999996,"15min":43.448999999999998},"last":0.94899999999999995}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0469999999999999,"15min":1.024},"min":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":5.4109999999999996,"5min":5.9779999999999998,"15min":7.96},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.516,"5min":1.1899999999999999,"15min":1.1639999999999999},"min":{"1min":0.66500000000000004,"5min":0.55800000000000005,"15min":0.51500000000000001},"max":{"1min":10.063000000000001,"5min":15.672000000000001,"15min":42.473999999999997},"last":0.98499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575117,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":346140,"kb_used_data":216404,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94025700,"statfs":{"total":96636764160,"available":96282316800,"internally_reserved":0,"allocated":221597696,"data_stored":277516468,"data_compressed":19866080,"data_compressed_allocated":23523328,"data_compressed_original":79745024,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,22,15],"upper_bound":64},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47299999999999998},"min":{"1min":0.29499999999999998,"5min":0.27800000000000002,"15min":0.215},"max":{"1min":1.581,"5min":1.958,"15min":3.3929999999999998},"last":21.814},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.60199999999999998,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.27700000000000002,"15min":0.248},"max":{"1min":4.5739999999999998,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":21.925000000000001}]},{"osd":2,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.64900000000000002,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.4489999999999998,"5min":5.4489999999999998,"15min":5.8559999999999999},"last":21.852},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.60599999999999998,"15min":0.60199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":3.0840000000000001,"15min":5.4210000000000003},"last":18.213000000000001}]},{"osd":3,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.77700000000000002,"15min":0.623},"min":{"1min":0.30599999999999999,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":5.0019999999999998,"5min":16.187999999999999,"15min":16.187999999999999},"last":21.98},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81200000000000006,"15min":0.69199999999999995},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.5410000000000004,"5min":15.004,"15min":15.004},"last":20.931999999999999}]},{"osd":4,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":0.93999999999999995,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.56699999999999995,"15min":0.42899999999999999},"max":{"1min":4.9119999999999999,"5min":5.2069999999999999,"15min":5.734},"last":4.0810000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.873,"15min":0.90500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0940000000000001,"5min":3.2160000000000002,"15min":5.3730000000000002},"last":17.98}]},{"osd":5,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.91900000000000004,"15min":0.84299999999999997},"min":{"1min":0.67300000000000004,"5min":0.59899999999999998,"15min":0.40699999999999997},"max":{"1min":4.4939999999999998,"5min":4.4939999999999998,"15min":4.4939999999999998},"last":21.890999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.92700000000000005,"15min":0.88500000000000001},"min":{"1min":0.65000000000000002,"5min":0.60799999999999998,"15min":0.50900000000000001},"max":{"1min":1.7350000000000001,"5min":4.7910000000000004,"15min":4.7910000000000004},"last":18.181000000000001}]},{"osd":6,"last update":"Tue Nov 28 21:55:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.01,"15min":0.89900000000000002},"min":{"1min":0.65500000000000003,"5min":0.60999999999999999,"15min":0.48699999999999999},"max":{"1min":4.8630000000000004,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":4.0410000000000004},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.88100000000000001},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.55100000000000005},"max":{"1min":1.667,"5min":3.1339999999999999,"15min":3.4710000000000001},"last":18.030999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.90800000000000003},"min":{"1min":0.627,"5min":0.627,"15min":0.48999999999999999},"max":{"1min":1.6950000000000001,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":18.140999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.95299999999999996,"15min":0.89600000000000002},"min":{"1min":0.68799999999999994,"5min":0.63,"15min":0.55700000000000005},"max":{"1min":1.627,"5min":3.1819999999999999,"15min":3.806},"last":18.071000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14327808,"data_stored":54364182,"data_compressed":7758193,"data_compressed_allocated":9936896,"data_compressed_original":49991680,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24334336,"data_stored":78481402,"data_compressed":10446153,"data_compressed_allocated":13598720,"data_compressed_original":67772416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15634432,"data_stored":48245772,"data_compressed":6171423,"data_compressed_allocated":8048640,"data_compressed_original":40681472,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25858048,"data_stored":94220664,"data_compressed":13014704,"data_compressed_allocated":16723968,"data_compressed_original":85106688,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23392256,"data_stored":83887777,"data_compressed":11283626,"data_compressed_allocated":14446592,"data_compressed_original":74960843,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22241280,"data_stored":79693154,"data_compressed":11120127,"data_compressed_allocated":14139392,"data_compressed_original":71610368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":22917120,"data_stored":83888413,"data_compressed":11073880,"data_compressed_allocated":14143488,"data_compressed_original":75132875,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":23990272,"data_stored":82675490,"data_compressed":11589086,"data_compressed_allocated":14680064,"data_compressed_original":73383936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":206610432,"data_stored":222244864,"data_compressed":12071885,"data_compressed_allocated":13537280,"data_compressed_original":29171712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294244352,"data_stored":319377408,"data_compressed":20255951,"data_compressed_allocated":22863872,"data_compressed_original":47996928,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164626432,"data_stored":175341568,"data_compressed":9845077,"data_compressed_allocated":10899456,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":288481280,"data_stored":316022784,"data_compressed":23971937,"data_compressed_allocated":27017216,"data_compressed_original":54558720,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":217907200,"data_stored":237305856,"data_compressed":14188788,"data_compressed_allocated":16142336,"data_compressed_original":35540992,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":220975104,"data_stored":243601408,"data_compressed":17073954,"data_compressed_allocated":19296256,"data_compressed_original":41922560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":237195264,"data_stored":251670528,"data_compressed":13556545,"data_compressed_allocated":15155200,"data_compressed_original":29630464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":192204800,"data_stored":208740352,"data_compressed":13315951,"data_compressed_allocated":15101952,"data_compressed_original":31637504,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:56:43.563 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.563 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.564 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.564 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.564 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.564 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.565 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.565 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.565 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.566 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.566 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.566 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.567 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.567 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.567 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.568 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.568 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.568 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.569 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.569 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.569 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.569 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.570 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.570 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.570 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.570 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.571 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.571 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.571 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.572 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.572 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.572 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.573 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.573 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.573 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.574 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.574 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.574 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.574 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.575 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.575 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.575 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.576 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.576 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.576 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.577 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.577 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.577 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T21:34:30.025809+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=30, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.578 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.578 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.578 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.579 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.579 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.579 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.579 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.579 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.580 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.580 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.580 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.580 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.580 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.580 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.581 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.581 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.581 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.581 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.581 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.582 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.582 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.582 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.582 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.582 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.582 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.583 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.583 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.583 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.583 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.584 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.584 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.584 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.584 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.584 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.584 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.585 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.585 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.585 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.585 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.585 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.585 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.586 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.586 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.586 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.586 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.586 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.586 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.587 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.587 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.587 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.587 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.587 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.588 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.588 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.588 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.588 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.588 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.588 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.589 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.589 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.589 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.589 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.589 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.589 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.590 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.590 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.590 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.590 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.590 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.590 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.591 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.591 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.591 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.591 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.591 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.592 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.592 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.592 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.592 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.592 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.593 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:56:43.593 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T21:56:47.189 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:56:47.193 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:57:03.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:57:03.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90da254640 1 Processor -- start 2023-11-28T21:57:03.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90da254640 1 -- start start 2023-11-28T21:57:03.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90da254640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d4153e40 0x7f90d40ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:03.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90da254640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90d40cb180 0x7f90d40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:03.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90da254640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d415acc0 con 0x7f90d415dc80 2023-11-28T21:57:03.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90da254640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d4159cb0 con 0x7f90d40cb180 2023-11-28T21:57:03.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90da254640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d4159ea0 con 0x7f90d4153e40 2023-11-28T21:57:03.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90d3fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90d40cb180 0x7f90d40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:03.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90d3fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f90d40cb180 0x7f90d40cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60484/0 (socket says 172.21.15.12:60484) 2023-11-28T21:57:03.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90d3fff640 1 -- 172.21.15.12:0/4122836232 learned_addr learned my addr 172.21.15.12:0/4122836232 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:57:03.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90d37fe640 1 --2- 172.21.15.12:0/4122836232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d4153e40 0x7f90d40ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:03.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90d3fff640 1 -- 172.21.15.12:0/4122836232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d4153e40 msgr2=0x7f90d40ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:57:03.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90d3fff640 1 --2- 172.21.15.12:0/4122836232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d4153e40 0x7f90d40ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:03.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90d3fff640 1 -- 172.21.15.12:0/4122836232 >> v1:172.21.15.144:6789/0 conn(0x7f90d415dc80 legacy=0x7f90d40cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:03.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90d3fff640 1 -- 172.21.15.12:0/4122836232 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d40cbaf0 con 0x7f90d40cb180 2023-11-28T21:57:03.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.752+0000 7f90d37fe640 1 --2- 172.21.15.12:0/4122836232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d4153e40 0x7f90d40ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:57:03.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90d3fff640 1 --2- 172.21.15.12:0/4122836232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d40cb180 0x7f90d40cb5b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009e50 tx=0x7f90cc01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d58c1cc273f5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:57:03.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90d27fc640 1 -- 172.21.15.12:0/4122836232 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0a58a0 con 0x7f90d40cb180 2023-11-28T21:57:03.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90d27fc640 1 -- 172.21.15.12:0/4122836232 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc0a5a00 con 0x7f90d40cb180 2023-11-28T21:57:03.759 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90d27fc640 1 -- 172.21.15.12:0/4122836232 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0a5d90 con 0x7f90d40cb180 2023-11-28T21:57:03.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90da254640 1 -- 172.21.15.12:0/4122836232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d40cb180 msgr2=0x7f90d40cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:03.761 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90da254640 1 --2- 172.21.15.12:0/4122836232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d40cb180 0x7f90d40cb5b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009e50 tx=0x7f90cc01c9e0 comp rx=0 tx=0).stop 2023-11-28T21:57:03.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90da254640 1 -- 172.21.15.12:0/4122836232 shutdown_connections 2023-11-28T21:57:03.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90da254640 1 --2- 172.21.15.12:0/4122836232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d40cb180 0x7f90d40cb5b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:03.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90da254640 1 --2- 172.21.15.12:0/4122836232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d4153e40 0x7f90d40ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:03.762 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90da254640 1 -- 172.21.15.12:0/4122836232 >> 172.21.15.12:0/4122836232 conn(0x7f90d406b7b0 msgr2=0x7f90d4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:57:03.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90da254640 1 -- 172.21.15.12:0/4122836232 shutdown_connections 2023-11-28T21:57:03.763 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90da254640 1 -- 172.21.15.12:0/4122836232 wait complete. 2023-11-28T21:57:03.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.756+0000 7f90da254640 1 Processor -- start 2023-11-28T21:57:03.765 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90da254640 1 -- start start 2023-11-28T21:57:03.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90da254640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d40cb180 0x7f90d40c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:03.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90da254640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4153e40 0x7f90d40c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:03.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90da254640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d415dc80 0x7f90d415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:03.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90da254640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d41f2310 con 0x7f90d415dc80 2023-11-28T21:57:03.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90da254640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d40c9250 con 0x7f90d4153e40 2023-11-28T21:57:03.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90da254640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d41f2b70 con 0x7f90d40cb180 2023-11-28T21:57:03.766 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d415dc80 0x7f90d415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:03.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d415dc80 0x7f90d415ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60496/0 (socket says 172.21.15.12:60496) 2023-11-28T21:57:03.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d3fff640 1 -- 172.21.15.12:0/318857318 learned_addr learned my addr 172.21.15.12:0/318857318 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:57:03.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d37fe640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d40cb180 0x7f90d40c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:03.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d2ffd640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4153e40 0x7f90d40c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:03.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d3fff640 1 -- 172.21.15.12:0/318857318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d40cb180 msgr2=0x7f90d40c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:03.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d3fff640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d40cb180 0x7f90d40c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:03.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d3fff640 1 -- 172.21.15.12:0/318857318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4153e40 msgr2=0x7f90d40c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:03.767 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d3fff640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4153e40 0x7f90d40c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:03.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d3fff640 1 -- 172.21.15.12:0/318857318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc009b30 con 0x7f90d415dc80 2023-11-28T21:57:03.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d2ffd640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4153e40 0x7f90d40c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:57:03.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d37fe640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d40cb180 0x7f90d40c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:57:03.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90d3fff640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d415dc80 0x7f90d415ae70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f90cc01a610 tx=0x7f90cc019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:57:03.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90bbfff640 1 -- 172.21.15.12:0/318857318 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0b8670 con 0x7f90d415dc80 2023-11-28T21:57:03.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90bbfff640 1 -- 172.21.15.12:0/318857318 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc0b87f0 con 0x7f90d415dc80 2023-11-28T21:57:03.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90bbfff640 1 -- 172.21.15.12:0/318857318 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0b8ba0 con 0x7f90d415dc80 2023-11-28T21:57:03.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90da254640 1 -- 172.21.15.12:0/318857318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d40c8a20 con 0x7f90d415dc80 2023-11-28T21:57:03.768 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90da254640 1 -- 172.21.15.12:0/318857318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d40c8f70 con 0x7f90d415dc80 2023-11-28T21:57:03.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.760+0000 7f90da254640 1 -- 172.21.15.12:0/318857318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d41f2d20 con 0x7f90d415dc80 2023-11-28T21:57:03.769 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.764+0000 7f90bbfff640 1 -- 172.21.15.12:0/318857318 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f90cc0b8d00 con 0x7f90d415dc80 2023-11-28T21:57:03.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.764+0000 7f90bbfff640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90c00664a0 0x7f90c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:03.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.764+0000 7f90bbfff640 1 -- 172.21.15.12:0/318857318 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f90cc11fa30 con 0x7f90d415dc80 2023-11-28T21:57:03.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.764+0000 7f90d37fe640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90c00664a0 0x7f90c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:03.770 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.764+0000 7f90d37fe640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90c00664a0 0x7f90c0068960 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009970 tx=0x7f90c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:57:03.772 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.768+0000 7f90bbfff640 1 -- 172.21.15.12:0/318857318 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f90d41f2d20 con 0x7f90d415dc80 2023-11-28T21:57:03.926 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.924+0000 7f90da254640 1 -- 172.21.15.12:0/318857318 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90d4067ae0 con 0x7f90c00664a0 2023-11-28T21:57:03.936 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.932+0000 7f90bbfff640 1 -- 172.21.15.12:0/318857318 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+372197 (secure 0 0 0) 0x7f90d4067ae0 con 0x7f90c00664a0 2023-11-28T21:57:03.940 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:57:03.942 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:57:03.943 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 -- 172.21.15.12:0/318857318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90c00664a0 msgr2=0x7f90c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:03.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90c00664a0 0x7f90c0068960 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009970 tx=0x7f90c4008040 comp rx=0 tx=0).stop 2023-11-28T21:57:03.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 -- 172.21.15.12:0/318857318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d415dc80 msgr2=0x7f90d415ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:03.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d415dc80 0x7f90d415ae70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f90cc01a610 tx=0x7f90cc019780 comp rx=0 tx=0).stop 2023-11-28T21:57:03.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 -- 172.21.15.12:0/318857318 shutdown_connections 2023-11-28T21:57:03.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f90c00664a0 0x7f90c0068960 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:03.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d415dc80 0x7f90d415ae70 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:03.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d4153e40 0x7f90d40c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:03.944 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 --2- 172.21.15.12:0/318857318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90d40cb180 0x7f90d40c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:03.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 -- 172.21.15.12:0/318857318 >> 172.21.15.12:0/318857318 conn(0x7f90d406b7b0 msgr2=0x7f90d4157ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:57:03.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 -- 172.21.15.12:0/318857318 shutdown_connections 2023-11-28T21:57:03.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:03.936+0000 7f90da254640 1 -- 172.21.15.12:0/318857318 wait complete. 2023-11-28T21:57:03.962 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":721,"stamp":"2023-11-28T21:57:02.218289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1231571986,"num_objects":10538,"num_object_clones":0,"num_object_copies":41756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10538,"num_whiteouts":0,"num_read":9144,"num_read_kb":2407205,"num_write":56629,"num_write_kb":2360024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":628062,"num_omap_keys":1091,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77285,"ondisk_log_size":77285,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3012052,"kb_used_data":1953300,"kb_used_omap":12,"kb_used_meta":1058675,"kb_avail":751962668,"statfs":{"total":773094113280,"available":770009772032,"internally_reserved":0,"allocated":2000179200,"data_stored":2584263135,"data_compressed":206888094,"data_compressed_allocated":245936128,"data_compressed_original":832450454,"omap_allocated":12751,"internal_metadata":1084083761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":171942,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":301365,"num_omap_keys":494,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003875"},"pg_stats":[{"pgid":"2.1f","version":"52'347","reported_seq":442,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.479830+0000","last_change":"2023-11-28T21:56:50.479855+0000","last_active":"2023-11-28T21:56:50.479830+0000","last_peered":"2023-11-28T21:56:50.479830+0000","last_clean":"2023-11-28T21:56:50.479830+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:56:50.479830+0000","last_undegraded":"2023-11-28T21:56:50.479830+0000","last_fullsized":"2023-11-28T21:56:50.479830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'347","last_scrub_stamp":"2023-11-28T21:56:50.479765+0000","last_deep_scrub":"52'347","last_deep_scrub_stamp":"2023-11-28T21:56:50.479765+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.479765+0000","objects_scrubbed":6,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:00.234465+0000","scrub_duration":0.028234239000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":61536,"num_write":310,"num_write_kb":8499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":41082,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"52'277","reported_seq":2362,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:40.959567+0000","last_change":"2023-11-28T21:40:42.800615+0000","last_active":"2023-11-28T21:56:40.959567+0000","last_peered":"2023-11-28T21:56:40.959567+0000","last_clean":"2023-11-28T21:56:40.959567+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:56:40.959567+0000","last_undegraded":"2023-11-28T21:56:40.959567+0000","last_fullsized":"2023-11-28T21:56:40.959567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28950874199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"52'1318","reported_seq":3629,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.060425+0000","last_change":"2023-11-28T21:40:42.803511+0000","last_active":"2023-11-28T21:56:35.060425+0000","last_peered":"2023-11-28T21:56:35.060425+0000","last_clean":"2023-11-28T21:56:35.060425+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:56:35.060425+0000","last_undegraded":"2023-11-28T21:56:35.060425+0000","last_fullsized":"2023-11-28T21:56:35.060425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29329539100000002,"stat_sum":{"num_bytes":30193112,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":229,"num_read_kb":12319,"num_write":1291,"num_write_kb":57664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"52'534","reported_seq":618,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:54.494340+0000","last_change":"2023-11-28T21:56:54.494393+0000","last_active":"2023-11-28T21:56:54.494340+0000","last_peered":"2023-11-28T21:56:54.494340+0000","last_clean":"2023-11-28T21:56:54.494340+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:56:54.494340+0000","last_undegraded":"2023-11-28T21:56:54.494340+0000","last_fullsized":"2023-11-28T21:56:54.494340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'534","last_scrub_stamp":"2023-11-28T21:56:54.494241+0000","last_deep_scrub":"52'534","last_deep_scrub_stamp":"2023-11-28T21:56:54.494241+0000","last_clean_scrub_stamp":"2023-11-28T21:56:54.494241+0000","objects_scrubbed":7,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:51:32.158044+0000","scrub_duration":0.057898067999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":81920,"num_write":485,"num_write_kb":24906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8778,"num_omap_keys":17,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"52'269","reported_seq":2361,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:42.409919+0000","last_change":"2023-11-28T21:56:42.409939+0000","last_active":"2023-11-28T21:56:42.409919+0000","last_peered":"2023-11-28T21:56:42.409919+0000","last_clean":"2023-11-28T21:56:42.409919+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:56:42.409919+0000","last_undegraded":"2023-11-28T21:56:42.409919+0000","last_fullsized":"2023-11-28T21:56:42.409919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'269","last_scrub_stamp":"2023-11-28T21:56:42.409827+0000","last_deep_scrub":"52'269","last_deep_scrub_stamp":"2023-11-28T21:56:42.409827+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.409827+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:28:29.334975+0000","scrub_duration":0.0029008010000000002,"objects_trimmed":0,"snaptrim_duration":0.23094383900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"52'1401","reported_seq":3734,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.720864+0000","last_change":"2023-11-28T21:40:43.072665+0000","last_active":"2023-11-28T21:56:34.720864+0000","last_peered":"2023-11-28T21:56:34.720864+0000","last_clean":"2023-11-28T21:56:34.720864+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:56:34.720864+0000","last_undegraded":"2023-11-28T21:56:34.720864+0000","last_fullsized":"2023-11-28T21:56:34.720864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21613474099999999,"stat_sum":{"num_bytes":11729902,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":251,"num_read_kb":24615,"num_write":1375,"num_write_kb":48688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"52'764","reported_seq":866,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.611517+0000","last_change":"2023-11-28T21:56:50.611643+0000","last_active":"2023-11-28T21:56:50.611517+0000","last_peered":"2023-11-28T21:56:50.611517+0000","last_clean":"2023-11-28T21:56:50.611517+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:56:50.611517+0000","last_undegraded":"2023-11-28T21:56:50.611517+0000","last_fullsized":"2023-11-28T21:56:50.611517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'764","last_scrub_stamp":"2023-11-28T21:56:50.611223+0000","last_deep_scrub":"52'764","last_deep_scrub_stamp":"2023-11-28T21:56:50.611223+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.611223+0000","objects_scrubbed":5,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:28:25.261471+0000","scrub_duration":0.059433334999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":59,"num_read_kb":131099,"num_write":715,"num_write_kb":20835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13932,"num_omap_keys":27,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"52'294","reported_seq":2377,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.752133+0000","last_change":"2023-11-28T21:40:42.830278+0000","last_active":"2023-11-28T21:56:35.752133+0000","last_peered":"2023-11-28T21:56:35.752133+0000","last_clean":"2023-11-28T21:56:35.752133+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:56:35.752133+0000","last_undegraded":"2023-11-28T21:56:35.752133+0000","last_fullsized":"2023-11-28T21:56:35.752133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30458173900000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"52'1324","reported_seq":3651,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.623001+0000","last_change":"2023-11-28T21:56:57.623143+0000","last_active":"2023-11-28T21:56:57.623001+0000","last_peered":"2023-11-28T21:56:57.623001+0000","last_clean":"2023-11-28T21:56:57.623001+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:56:57.623001+0000","last_undegraded":"2023-11-28T21:56:57.623001+0000","last_fullsized":"2023-11-28T21:56:57.623001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1324","last_scrub_stamp":"2023-11-28T21:56:57.622774+0000","last_deep_scrub":"52'1324","last_deep_scrub_stamp":"2023-11-28T21:56:57.622774+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.622774+0000","objects_scrubbed":314,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:08:12.479556+0000","scrub_duration":0.25549122899999999,"objects_trimmed":0,"snaptrim_duration":0.23712909900000001,"stat_sum":{"num_bytes":32661416,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":226,"num_read_kb":7690,"num_write":1300,"num_write_kb":56471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"52'96","reported_seq":175,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:47.599844+0000","last_change":"2023-11-28T21:56:47.599947+0000","last_active":"2023-11-28T21:56:47.599844+0000","last_peered":"2023-11-28T21:56:47.599844+0000","last_clean":"2023-11-28T21:56:47.599844+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:56:47.599844+0000","last_undegraded":"2023-11-28T21:56:47.599844+0000","last_fullsized":"2023-11-28T21:56:47.599844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'96","last_scrub_stamp":"2023-11-28T21:56:47.599552+0000","last_deep_scrub":"52'96","last_deep_scrub_stamp":"2023-11-28T21:56:47.599552+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.599552+0000","objects_scrubbed":2,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:41:39.184546+0000","scrub_duration":0.012005902000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36891,"num_write":51,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10256,"num_omap_keys":19,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"52'283","reported_seq":2366,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.736219+0000","last_change":"2023-11-28T21:40:42.827917+0000","last_active":"2023-11-28T21:56:35.736219+0000","last_peered":"2023-11-28T21:56:35.736219+0000","last_clean":"2023-11-28T21:56:35.736219+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:56:35.736219+0000","last_undegraded":"2023-11-28T21:56:35.736219+0000","last_fullsized":"2023-11-28T21:56:35.736219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30416134,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"52'1294","reported_seq":3631,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:56.624580+0000","last_change":"2023-11-28T21:56:56.624661+0000","last_active":"2023-11-28T21:56:56.624580+0000","last_peered":"2023-11-28T21:56:56.624580+0000","last_clean":"2023-11-28T21:56:56.624580+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:56:56.624580+0000","last_undegraded":"2023-11-28T21:56:56.624580+0000","last_fullsized":"2023-11-28T21:56:56.624580+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1294","last_scrub_stamp":"2023-11-28T21:56:56.624359+0000","last_deep_scrub":"52'1294","last_deep_scrub_stamp":"2023-11-28T21:56:56.624359+0000","last_clean_scrub_stamp":"2023-11-28T21:56:56.624359+0000","objects_scrubbed":312,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:59:20.046078+0000","scrub_duration":0.25756709,"objects_trimmed":0,"snaptrim_duration":0.26667665699999998,"stat_sum":{"num_bytes":37281632,"num_objects":312,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":0,"num_read":236,"num_read_kb":13169,"num_write":1275,"num_write_kb":61269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"52'303","reported_seq":2387,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.944343+0000","last_change":"2023-11-28T21:40:42.917342+0000","last_active":"2023-11-28T21:56:35.944343+0000","last_peered":"2023-11-28T21:56:35.944343+0000","last_clean":"2023-11-28T21:56:35.944343+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:56:35.944343+0000","last_undegraded":"2023-11-28T21:56:35.944343+0000","last_fullsized":"2023-11-28T21:56:35.944343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.200606007,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"52'627","reported_seq":730,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.667166+0000","last_change":"2023-11-28T21:56:48.667245+0000","last_active":"2023-11-28T21:56:48.667166+0000","last_peered":"2023-11-28T21:56:48.667166+0000","last_clean":"2023-11-28T21:56:48.667166+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:56:48.667166+0000","last_undegraded":"2023-11-28T21:56:48.667166+0000","last_fullsized":"2023-11-28T21:56:48.667166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'627","last_scrub_stamp":"2023-11-28T21:56:48.666890+0000","last_deep_scrub":"52'627","last_deep_scrub_stamp":"2023-11-28T21:56:48.666890+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.666890+0000","objects_scrubbed":10,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:43:39.567306+0000","scrub_duration":0.095730051999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":114718,"num_write":571,"num_write_kb":20791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64058,"num_omap_keys":115,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"52'1331","reported_seq":3651,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.684039+0000","last_change":"2023-11-28T21:40:43.071642+0000","last_active":"2023-11-28T21:56:34.684039+0000","last_peered":"2023-11-28T21:56:34.684039+0000","last_clean":"2023-11-28T21:56:34.684039+0000","last_became_active":"2023-11-28T21:35:25.616978+0000","last_became_peered":"2023-11-28T21:35:25.616978+0000","last_unstale":"2023-11-28T21:56:34.684039+0000","last_undegraded":"2023-11-28T21:56:34.684039+0000","last_fullsized":"2023-11-28T21:56:34.684039+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1331,"log_dups_size":0,"ondisk_log_size":1331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.23129675,"stat_sum":{"num_bytes":32707084,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":229,"num_read_kb":4127,"num_write":1308,"num_write_kb":57872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"52'305","reported_seq":2388,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.765511+0000","last_change":"2023-11-28T21:40:42.798788+0000","last_active":"2023-11-28T21:56:35.765511+0000","last_peered":"2023-11-28T21:56:35.765511+0000","last_clean":"2023-11-28T21:56:35.765511+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:56:35.765511+0000","last_undegraded":"2023-11-28T21:56:35.765511+0000","last_fullsized":"2023-11-28T21:56:35.765511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28777871399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"52'956","reported_seq":1035,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:38.923278+0000","last_change":"2023-11-28T21:34:36.000727+0000","last_active":"2023-11-28T21:56:38.923278+0000","last_peered":"2023-11-28T21:56:38.923278+0000","last_clean":"2023-11-28T21:56:38.923278+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:56:38.923278+0000","last_undegraded":"2023-11-28T21:56:38.923278+0000","last_fullsized":"2023-11-28T21:56:38.923278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":956,"log_dups_size":0,"ondisk_log_size":956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":75782,"num_write":935,"num_write_kb":13441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"52'1416","reported_seq":3763,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:59.657520+0000","last_change":"2023-11-28T21:56:59.657597+0000","last_active":"2023-11-28T21:56:59.657520+0000","last_peered":"2023-11-28T21:56:59.657520+0000","last_clean":"2023-11-28T21:56:59.657520+0000","last_became_active":"2023-11-28T21:35:25.790674+0000","last_became_peered":"2023-11-28T21:35:25.790674+0000","last_unstale":"2023-11-28T21:56:59.657520+0000","last_undegraded":"2023-11-28T21:56:59.657520+0000","last_fullsized":"2023-11-28T21:56:59.657520+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1416","last_scrub_stamp":"2023-11-28T21:56:59.657277+0000","last_deep_scrub":"52'1416","last_deep_scrub_stamp":"2023-11-28T21:56:59.657277+0000","last_clean_scrub_stamp":"2023-11-28T21:56:59.657277+0000","objects_scrubbed":354,"log_size":1416,"log_dups_size":0,"ondisk_log_size":1416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:47.719457+0000","scrub_duration":0.27211498200000001,"objects_trimmed":0,"snaptrim_duration":0.229623311,"stat_sum":{"num_bytes":17731185,"num_objects":354,"num_object_clones":0,"num_object_copies":1416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":234,"num_read_kb":20007,"num_write":1382,"num_write_kb":71863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"52'306","reported_seq":2398,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.421624+0000","last_change":"2023-11-28T21:56:43.421641+0000","last_active":"2023-11-28T21:56:43.421624+0000","last_peered":"2023-11-28T21:56:43.421624+0000","last_clean":"2023-11-28T21:56:43.421624+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:56:43.421624+0000","last_undegraded":"2023-11-28T21:56:43.421624+0000","last_fullsized":"2023-11-28T21:56:43.421624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'306","last_scrub_stamp":"2023-11-28T21:56:43.421546+0000","last_deep_scrub":"52'306","last_deep_scrub_stamp":"2023-11-28T21:56:43.421546+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.421546+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:18:49.309763+0000","scrub_duration":0.001616658,"objects_trimmed":0,"snaptrim_duration":0.29027087200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"52'740","reported_seq":817,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.925621+0000","last_change":"2023-11-28T21:56:42.774090+0000","last_active":"2023-11-28T21:56:53.925621+0000","last_peered":"2023-11-28T21:56:53.925621+0000","last_clean":"2023-11-28T21:56:53.925621+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:56:53.925621+0000","last_undegraded":"2023-11-28T21:56:53.925621+0000","last_fullsized":"2023-11-28T21:56:53.925621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'734","last_scrub_stamp":"2023-11-28T21:56:42.773996+0000","last_deep_scrub":"52'734","last_deep_scrub_stamp":"2023-11-28T21:56:42.773996+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.773996+0000","objects_scrubbed":6,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:44:34.363577+0000","scrub_duration":0.055636247999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11382152,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":32782,"num_write":690,"num_write_kb":11469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8864,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"52'1341","reported_seq":3685,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.931061+0000","last_change":"2023-11-28T21:40:42.890432+0000","last_active":"2023-11-28T21:56:34.931061+0000","last_peered":"2023-11-28T21:56:34.931061+0000","last_clean":"2023-11-28T21:56:34.931061+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:56:34.931061+0000","last_undegraded":"2023-11-28T21:56:34.931061+0000","last_fullsized":"2023-11-28T21:56:34.931061+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1341,"log_dups_size":0,"ondisk_log_size":1341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24643899999999999,"stat_sum":{"num_bytes":14380393,"num_objects":336,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":262,"num_read_kb":16758,"num_write":1318,"num_write_kb":45879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"52'294","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.274947+0000","last_change":"2023-11-28T21:56:48.275049+0000","last_active":"2023-11-28T21:56:48.274947+0000","last_peered":"2023-11-28T21:56:48.274947+0000","last_clean":"2023-11-28T21:56:48.274947+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:56:48.274947+0000","last_undegraded":"2023-11-28T21:56:48.274947+0000","last_fullsized":"2023-11-28T21:56:48.274947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'294","last_scrub_stamp":"2023-11-28T21:56:48.274731+0000","last_deep_scrub":"52'294","last_deep_scrub_stamp":"2023-11-28T21:56:48.274731+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.274731+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:02:56.783147+0000","scrub_duration":0.0035010150000000001,"objects_trimmed":0,"snaptrim_duration":0.21610210399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"52'343","reported_seq":422,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.528829+0000","last_change":"2023-11-28T21:56:57.528897+0000","last_active":"2023-11-28T21:56:57.528829+0000","last_peered":"2023-11-28T21:56:57.528829+0000","last_clean":"2023-11-28T21:56:57.528829+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:56:57.528829+0000","last_undegraded":"2023-11-28T21:56:57.528829+0000","last_fullsized":"2023-11-28T21:56:57.528829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'343","last_scrub_stamp":"2023-11-28T21:56:57.528632+0000","last_deep_scrub":"52'343","last_deep_scrub_stamp":"2023-11-28T21:56:57.528632+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.528632+0000","objects_scrubbed":1,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:11:06.480768+0000","scrub_duration":0.0086859099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":49164,"num_write":266,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"52'1262","reported_seq":3550,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.743605+0000","last_change":"2023-11-28T21:40:43.060232+0000","last_active":"2023-11-28T21:56:34.743605+0000","last_peered":"2023-11-28T21:56:34.743605+0000","last_clean":"2023-11-28T21:56:34.743605+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:56:34.743605+0000","last_undegraded":"2023-11-28T21:56:34.743605+0000","last_fullsized":"2023-11-28T21:56:34.743605+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.244586779,"stat_sum":{"num_bytes":50267474,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":206,"num_read_kb":15432,"num_write":1240,"num_write_kb":82194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"52'302","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.761940+0000","last_change":"2023-11-28T21:40:43.249048+0000","last_active":"2023-11-28T21:56:35.761940+0000","last_peered":"2023-11-28T21:56:35.761940+0000","last_clean":"2023-11-28T21:56:35.761940+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:56:35.761940+0000","last_undegraded":"2023-11-28T21:56:35.761940+0000","last_fullsized":"2023-11-28T21:56:35.761940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26728515899999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"52'172","reported_seq":253,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.150811+0000","last_change":"2023-11-28T21:56:49.150879+0000","last_active":"2023-11-28T21:56:49.150811+0000","last_peered":"2023-11-28T21:56:49.150811+0000","last_clean":"2023-11-28T21:56:49.150811+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:56:49.150811+0000","last_undegraded":"2023-11-28T21:56:49.150811+0000","last_fullsized":"2023-11-28T21:56:49.150811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'172","last_scrub_stamp":"2023-11-28T21:56:49.150620+0000","last_deep_scrub":"52'172","last_deep_scrub_stamp":"2023-11-28T21:56:49.150620+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.150620+0000","objects_scrubbed":4,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:13:50.517958+0000","scrub_duration":0.035921929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":45,"num_read_kb":24615,"num_write":148,"num_write_kb":4216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6802,"num_omap_keys":13,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"52'1368","reported_seq":3701,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.663282+0000","last_change":"2023-11-28T21:40:43.072395+0000","last_active":"2023-11-28T21:56:34.663282+0000","last_peered":"2023-11-28T21:56:34.663282+0000","last_clean":"2023-11-28T21:56:34.663282+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:56:34.663282+0000","last_undegraded":"2023-11-28T21:56:34.663282+0000","last_fullsized":"2023-11-28T21:56:34.663282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21672229300000001,"stat_sum":{"num_bytes":16299977,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":250,"num_read_kb":17253,"num_write":1343,"num_write_kb":69509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"52'268","reported_seq":2360,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.444623+0000","last_change":"2023-11-28T21:56:50.444642+0000","last_active":"2023-11-28T21:56:50.444623+0000","last_peered":"2023-11-28T21:56:50.444623+0000","last_clean":"2023-11-28T21:56:50.444623+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:56:50.444623+0000","last_undegraded":"2023-11-28T21:56:50.444623+0000","last_fullsized":"2023-11-28T21:56:50.444623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'268","last_scrub_stamp":"2023-11-28T21:56:50.444546+0000","last_deep_scrub":"52'268","last_deep_scrub_stamp":"2023-11-28T21:56:50.444546+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.444546+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:58:47.104569+0000","scrub_duration":0.0027754860000000002,"objects_trimmed":0,"snaptrim_duration":0.29009278799999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"52'248","reported_seq":326,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:47.891843+0000","last_change":"2023-11-28T21:56:47.891843+0000","last_active":"2023-11-28T21:56:47.891843+0000","last_peered":"2023-11-28T21:56:47.891843+0000","last_clean":"2023-11-28T21:56:47.891843+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:56:47.891843+0000","last_undegraded":"2023-11-28T21:56:47.891843+0000","last_fullsized":"2023-11-28T21:56:47.891843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'247","last_scrub_stamp":"2023-11-28T21:56:47.891725+0000","last_deep_scrub":"52'247","last_deep_scrub_stamp":"2023-11-28T21:56:47.891725+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.891725+0000","objects_scrubbed":3,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:49:45.140707+0000","scrub_duration":0.119752109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":607,"num_write":448,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8486,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"52'1386","reported_seq":3702,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:40.959735+0000","last_change":"2023-11-28T21:40:42.797955+0000","last_active":"2023-11-28T21:56:40.959735+0000","last_peered":"2023-11-28T21:56:40.959735+0000","last_clean":"2023-11-28T21:56:40.959735+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:56:40.959735+0000","last_undegraded":"2023-11-28T21:56:40.959735+0000","last_fullsized":"2023-11-28T21:56:40.959735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1386,"log_dups_size":0,"ondisk_log_size":1386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.287791554,"stat_sum":{"num_bytes":19942162,"num_objects":313,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":232,"num_read_kb":16373,"num_write":1352,"num_write_kb":52080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"52'300","reported_seq":2385,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.755837+0000","last_change":"2023-11-28T21:40:42.884411+0000","last_active":"2023-11-28T21:56:35.755837+0000","last_peered":"2023-11-28T21:56:35.755837+0000","last_clean":"2023-11-28T21:56:35.755837+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:56:35.755837+0000","last_undegraded":"2023-11-28T21:56:35.755837+0000","last_fullsized":"2023-11-28T21:56:35.755837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24779062600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"52'593","reported_seq":680,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.579200+0000","last_change":"2023-11-28T21:56:55.579266+0000","last_active":"2023-11-28T21:56:55.579200+0000","last_peered":"2023-11-28T21:56:55.579200+0000","last_clean":"2023-11-28T21:56:55.579200+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:56:55.579200+0000","last_undegraded":"2023-11-28T21:56:55.579200+0000","last_fullsized":"2023-11-28T21:56:55.579200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'593","last_scrub_stamp":"2023-11-28T21:56:55.579027+0000","last_deep_scrub":"52'593","last_deep_scrub_stamp":"2023-11-28T21:56:55.579027+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.579027+0000","objects_scrubbed":6,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:56:30.723466+0000","scrub_duration":0.077952980000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":81538,"num_write":523,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17674,"num_omap_keys":32,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"52'1410","reported_seq":3741,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.900338+0000","last_change":"2023-11-28T21:40:43.255749+0000","last_active":"2023-11-28T21:56:34.900338+0000","last_peered":"2023-11-28T21:56:34.900338+0000","last_clean":"2023-11-28T21:56:34.900338+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:56:34.900338+0000","last_undegraded":"2023-11-28T21:56:34.900338+0000","last_fullsized":"2023-11-28T21:56:34.900338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27155133199999998,"stat_sum":{"num_bytes":49110889,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":248,"num_read_kb":24602,"num_write":1394,"num_write_kb":89036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"52'284","reported_seq":2369,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762958+0000","last_change":"2023-11-28T21:40:42.828660+0000","last_active":"2023-11-28T21:56:35.762958+0000","last_peered":"2023-11-28T21:56:35.762958+0000","last_clean":"2023-11-28T21:56:35.762958+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:56:35.762958+0000","last_undegraded":"2023-11-28T21:56:35.762958+0000","last_fullsized":"2023-11-28T21:56:35.762958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30430776599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"52'167","reported_seq":237,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.743243+0000","last_change":"2023-11-28T21:56:55.743275+0000","last_active":"2023-11-28T21:56:55.743243+0000","last_peered":"2023-11-28T21:56:55.743243+0000","last_clean":"2023-11-28T21:56:55.743243+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:56:55.743243+0000","last_undegraded":"2023-11-28T21:56:55.743243+0000","last_fullsized":"2023-11-28T21:56:55.743243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'167","last_scrub_stamp":"2023-11-28T21:56:55.743149+0000","last_deep_scrub":"52'167","last_deep_scrub_stamp":"2023-11-28T21:56:55.743149+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.743149+0000","objects_scrubbed":3,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:24.042531+0000","scrub_duration":0.015377216000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24576,"num_write":146,"num_write_kb":4262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19890,"num_omap_keys":36,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"52'1318","reported_seq":3623,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.858027+0000","last_change":"2023-11-28T21:40:42.887654+0000","last_active":"2023-11-28T21:56:34.858027+0000","last_peered":"2023-11-28T21:56:34.858027+0000","last_clean":"2023-11-28T21:56:34.858027+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:56:34.858027+0000","last_undegraded":"2023-11-28T21:56:34.858027+0000","last_fullsized":"2023-11-28T21:56:34.858027+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.25659980999999998,"stat_sum":{"num_bytes":32626940,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":222,"num_read_kb":4128,"num_write":1285,"num_write_kb":60020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"52'300","reported_seq":2392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.336206+0000","last_change":"2023-11-28T21:56:44.336260+0000","last_active":"2023-11-28T21:56:44.336206+0000","last_peered":"2023-11-28T21:56:44.336206+0000","last_clean":"2023-11-28T21:56:44.336206+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:56:44.336206+0000","last_undegraded":"2023-11-28T21:56:44.336206+0000","last_fullsized":"2023-11-28T21:56:44.336206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'300","last_scrub_stamp":"2023-11-28T21:56:44.335964+0000","last_deep_scrub":"52'300","last_deep_scrub_stamp":"2023-11-28T21:56:44.335964+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.335964+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:50:07.131774+0000","scrub_duration":0.0032352040000000002,"objects_trimmed":0,"snaptrim_duration":0.23142385100000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"52'370","reported_seq":453,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:51.659222+0000","last_change":"2023-11-28T21:56:51.659286+0000","last_active":"2023-11-28T21:56:51.659222+0000","last_peered":"2023-11-28T21:56:51.659222+0000","last_clean":"2023-11-28T21:56:51.659222+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:56:51.659222+0000","last_undegraded":"2023-11-28T21:56:51.659222+0000","last_fullsized":"2023-11-28T21:56:51.659222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'370","last_scrub_stamp":"2023-11-28T21:56:51.659033+0000","last_deep_scrub":"52'370","last_deep_scrub_stamp":"2023-11-28T21:56:51.659033+0000","last_clean_scrub_stamp":"2023-11-28T21:56:51.659033+0000","objects_scrubbed":3,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:45:32.821715+0000","scrub_duration":0.043796694999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":81920,"num_write":339,"num_write_kb":16655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"52'1263","reported_seq":3556,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.958982+0000","last_change":"2023-11-28T21:40:42.883278+0000","last_active":"2023-11-28T21:56:34.958982+0000","last_peered":"2023-11-28T21:56:34.958982+0000","last_clean":"2023-11-28T21:56:34.958982+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:56:34.958982+0000","last_undegraded":"2023-11-28T21:56:34.958982+0000","last_fullsized":"2023-11-28T21:56:34.958982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24685721299999999,"stat_sum":{"num_bytes":16718463,"num_objects":288,"num_object_clones":0,"num_object_copies":1152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":210,"num_read_kb":3302,"num_write":1235,"num_write_kb":35539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"52'299","reported_seq":2385,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:39.876983+0000","last_change":"2023-11-28T21:40:42.890540+0000","last_active":"2023-11-28T21:56:39.876983+0000","last_peered":"2023-11-28T21:56:39.876983+0000","last_clean":"2023-11-28T21:56:39.876983+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:56:39.876983+0000","last_undegraded":"2023-11-28T21:56:39.876983+0000","last_fullsized":"2023-11-28T21:56:39.876983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24023003500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"52'943","reported_seq":1423,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.244672+0000","last_change":"2023-11-28T21:56:45.244727+0000","last_active":"2023-11-28T21:56:45.244672+0000","last_peered":"2023-11-28T21:56:45.244672+0000","last_clean":"2023-11-28T21:56:45.244672+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:56:45.244672+0000","last_undegraded":"2023-11-28T21:56:45.244672+0000","last_fullsized":"2023-11-28T21:56:45.244672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'943","last_scrub_stamp":"2023-11-28T21:56:45.244580+0000","last_deep_scrub":"52'943","last_deep_scrub_stamp":"2023-11-28T21:56:45.244580+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.244580+0000","objects_scrubbed":9,"log_size":943,"log_dups_size":0,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:47:30.047518+0000","scrub_duration":0.055949021000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18884916,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":417,"num_read_kb":34819,"num_write":869,"num_write_kb":18975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17074,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"52'1231","reported_seq":3558,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.911673+0000","last_change":"2023-11-28T21:40:42.888191+0000","last_active":"2023-11-28T21:56:34.911673+0000","last_peered":"2023-11-28T21:56:34.911673+0000","last_clean":"2023-11-28T21:56:34.911673+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:56:34.911673+0000","last_undegraded":"2023-11-28T21:56:34.911673+0000","last_fullsized":"2023-11-28T21:56:34.911673+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24393632200000001,"stat_sum":{"num_bytes":26023878,"num_objects":293,"num_object_clones":0,"num_object_copies":1172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":293,"num_whiteouts":0,"num_read":244,"num_read_kb":12314,"num_write":1202,"num_write_kb":62723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"52'277","reported_seq":2361,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.758790+0000","last_change":"2023-11-28T21:40:43.256849+0000","last_active":"2023-11-28T21:56:35.758790+0000","last_peered":"2023-11-28T21:56:35.758790+0000","last_clean":"2023-11-28T21:56:35.758790+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:56:35.758790+0000","last_undegraded":"2023-11-28T21:56:35.758790+0000","last_fullsized":"2023-11-28T21:56:35.758790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.25105937499999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"52'325","reported_seq":417,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:59.491902+0000","last_change":"2023-11-28T21:56:59.491989+0000","last_active":"2023-11-28T21:56:59.491902+0000","last_peered":"2023-11-28T21:56:59.491902+0000","last_clean":"2023-11-28T21:56:59.491902+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:56:59.491902+0000","last_undegraded":"2023-11-28T21:56:59.491902+0000","last_fullsized":"2023-11-28T21:56:59.491902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'325","last_scrub_stamp":"2023-11-28T21:56:59.491653+0000","last_deep_scrub":"52'325","last_deep_scrub_stamp":"2023-11-28T21:56:59.491653+0000","last_clean_scrub_stamp":"2023-11-28T21:56:59.491653+0000","objects_scrubbed":5,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:19:40.745953+0000","scrub_duration":0.050149041999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8604316,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":114688,"num_write":283,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5654,"num_omap_keys":11,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"52'1365","reported_seq":3712,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.882394+0000","last_change":"2023-11-28T21:40:43.252159+0000","last_active":"2023-11-28T21:56:34.882394+0000","last_peered":"2023-11-28T21:56:34.882394+0000","last_clean":"2023-11-28T21:56:34.882394+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:56:34.882394+0000","last_undegraded":"2023-11-28T21:56:34.882394+0000","last_fullsized":"2023-11-28T21:56:34.882394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.270607394,"stat_sum":{"num_bytes":19916106,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":264,"num_read_kb":13370,"num_write":1336,"num_write_kb":59202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"52'283","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.428157+0000","last_change":"2023-11-28T21:56:57.428177+0000","last_active":"2023-11-28T21:56:57.428157+0000","last_peered":"2023-11-28T21:56:57.428157+0000","last_clean":"2023-11-28T21:56:57.428157+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:56:57.428157+0000","last_undegraded":"2023-11-28T21:56:57.428157+0000","last_fullsized":"2023-11-28T21:56:57.428157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:56:57.428075+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:56:57.428075+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.428075+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:09:37.862848+0000","scrub_duration":0.0026023819999999999,"objects_trimmed":0,"snaptrim_duration":0.29152715000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"52'318","reported_seq":391,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.764022+0000","last_change":"2023-11-28T21:56:57.764096+0000","last_active":"2023-11-28T21:56:57.764022+0000","last_peered":"2023-11-28T21:56:57.764022+0000","last_clean":"2023-11-28T21:56:57.764022+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:56:57.764022+0000","last_undegraded":"2023-11-28T21:56:57.764022+0000","last_fullsized":"2023-11-28T21:56:57.764022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'318","last_scrub_stamp":"2023-11-28T21:56:57.763819+0000","last_deep_scrub":"52'318","last_deep_scrub_stamp":"2023-11-28T21:56:57.763819+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.763819+0000","objects_scrubbed":7,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:20:01.143626+0000","scrub_duration":0.047864816999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":24594,"num_write":297,"num_write_kb":8480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":38186,"num_omap_keys":70,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"52'119","reported_seq":219,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.591456+0000","last_change":"2023-11-28T21:56:45.591517+0000","last_active":"2023-11-28T21:56:45.591456+0000","last_peered":"2023-11-28T21:56:45.591456+0000","last_clean":"2023-11-28T21:56:45.591456+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:56:45.591456+0000","last_undegraded":"2023-11-28T21:56:45.591456+0000","last_fullsized":"2023-11-28T21:56:45.591456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'119","last_scrub_stamp":"2023-11-28T21:56:45.591265+0000","last_deep_scrub":"52'119","last_deep_scrub_stamp":"2023-11-28T21:56:45.591265+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.591265+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:52:45.646774+0000","scrub_duration":0.0095868710000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"52'1342","reported_seq":3677,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.825311+0000","last_change":"2023-11-28T21:40:42.914002+0000","last_active":"2023-11-28T21:56:34.825311+0000","last_peered":"2023-11-28T21:56:34.825311+0000","last_clean":"2023-11-28T21:56:34.825311+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:56:34.825311+0000","last_undegraded":"2023-11-28T21:56:34.825311+0000","last_fullsized":"2023-11-28T21:56:34.825311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19732549099999999,"stat_sum":{"num_bytes":29762487,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":252,"num_read_kb":20508,"num_write":1310,"num_write_kb":73242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"52'280","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.750326+0000","last_change":"2023-11-28T21:40:42.748386+0000","last_active":"2023-11-28T21:56:35.750326+0000","last_peered":"2023-11-28T21:56:35.750326+0000","last_clean":"2023-11-28T21:56:35.750326+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:56:35.750326+0000","last_undegraded":"2023-11-28T21:56:35.750326+0000","last_fullsized":"2023-11-28T21:56:35.750326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.219020891,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"52'476","reported_seq":536,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:40.418742+0000","last_change":"2023-11-28T21:34:35.997449+0000","last_active":"2023-11-28T21:56:40.418742+0000","last_peered":"2023-11-28T21:56:40.418742+0000","last_clean":"2023-11-28T21:56:40.418742+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:56:40.418742+0000","last_undegraded":"2023-11-28T21:56:40.418742+0000","last_fullsized":"2023-11-28T21:56:40.418742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:34.961186+0000","last_clean_scrub_stamp":"2023-11-28T21:34:34.961186+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":20485,"num_write":447,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"52'1372","reported_seq":3705,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.808066+0000","last_change":"2023-11-28T21:40:42.915914+0000","last_active":"2023-11-28T21:56:34.808066+0000","last_peered":"2023-11-28T21:56:34.808066+0000","last_clean":"2023-11-28T21:56:34.808066+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:56:34.808066+0000","last_undegraded":"2023-11-28T21:56:34.808066+0000","last_fullsized":"2023-11-28T21:56:34.808066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1372,"log_dups_size":0,"ondisk_log_size":1372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19929065100000001,"stat_sum":{"num_bytes":26470227,"num_objects":335,"num_object_clones":0,"num_object_copies":1340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":251,"num_read_kb":4134,"num_write":1344,"num_write_kb":47412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"52'330","reported_seq":2414,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.763685+0000","last_change":"2023-11-28T21:40:42.830692+0000","last_active":"2023-11-28T21:56:35.763685+0000","last_peered":"2023-11-28T21:56:35.763685+0000","last_clean":"2023-11-28T21:56:35.763685+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:56:35.763685+0000","last_undegraded":"2023-11-28T21:56:35.763685+0000","last_fullsized":"2023-11-28T21:56:35.763685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30458413400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"52'971","reported_seq":1060,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.753647+0000","last_change":"2023-11-28T21:56:49.753678+0000","last_active":"2023-11-28T21:56:49.753647+0000","last_peered":"2023-11-28T21:56:49.753647+0000","last_clean":"2023-11-28T21:56:49.753647+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:56:49.753647+0000","last_undegraded":"2023-11-28T21:56:49.753647+0000","last_fullsized":"2023-11-28T21:56:49.753647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'971","last_scrub_stamp":"2023-11-28T21:56:49.753562+0000","last_deep_scrub":"52'971","last_deep_scrub_stamp":"2023-11-28T21:56:49.753562+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.753562+0000","objects_scrubbed":7,"log_size":971,"log_dups_size":0,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:05.338455+0000","scrub_duration":0.053294293999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":27,"num_read_kb":68346,"num_write":931,"num_write_kb":16947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":38524,"num_omap_keys":71,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"52'1319","reported_seq":3666,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.898961+0000","last_change":"2023-11-28T21:40:43.256533+0000","last_active":"2023-11-28T21:56:34.898961+0000","last_peered":"2023-11-28T21:56:34.898961+0000","last_clean":"2023-11-28T21:56:34.898961+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:56:34.898961+0000","last_undegraded":"2023-11-28T21:56:34.898961+0000","last_fullsized":"2023-11-28T21:56:34.898961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27089875299999999,"stat_sum":{"num_bytes":28004524,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":265,"num_read_kb":16422,"num_write":1294,"num_write_kb":50614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"52'274","reported_seq":2351,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.967653+0000","last_change":"2023-11-28T21:40:43.090122+0000","last_active":"2023-11-28T21:56:35.967653+0000","last_peered":"2023-11-28T21:56:35.967653+0000","last_clean":"2023-11-28T21:56:35.967653+0000","last_became_active":"2023-11-28T21:35:25.594533+0000","last_became_peered":"2023-11-28T21:35:25.594533+0000","last_unstale":"2023-11-28T21:56:35.967653+0000","last_undegraded":"2023-11-28T21:56:35.967653+0000","last_fullsized":"2023-11-28T21:56:35.967653+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14880083199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"52'307","reported_seq":391,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.178563+0000","last_change":"2023-11-28T21:56:48.178599+0000","last_active":"2023-11-28T21:56:48.178563+0000","last_peered":"2023-11-28T21:56:48.178563+0000","last_clean":"2023-11-28T21:56:48.178563+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:56:48.178563+0000","last_undegraded":"2023-11-28T21:56:48.178563+0000","last_fullsized":"2023-11-28T21:56:48.178563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'307","last_scrub_stamp":"2023-11-28T21:56:48.178460+0000","last_deep_scrub":"52'307","last_deep_scrub_stamp":"2023-11-28T21:56:48.178460+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.178460+0000","objects_scrubbed":5,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:26:15.702492+0000","scrub_duration":0.026539895000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":53275,"num_write":252,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21228,"num_omap_keys":39,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"52'1307","reported_seq":3592,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.520316+0000","last_change":"2023-11-28T21:40:42.823686+0000","last_active":"2023-11-28T21:56:34.520316+0000","last_peered":"2023-11-28T21:56:34.520316+0000","last_clean":"2023-11-28T21:56:34.520316+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:56:34.520316+0000","last_undegraded":"2023-11-28T21:56:34.520316+0000","last_fullsized":"2023-11-28T21:56:34.520316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1307,"log_dups_size":0,"ondisk_log_size":1307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30648515900000001,"stat_sum":{"num_bytes":28110180,"num_objects":295,"num_object_clones":0,"num_object_copies":1180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":0,"num_read":202,"num_read_kb":8227,"num_write":1283,"num_write_kb":51039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"52'277","reported_seq":2361,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.749837+0000","last_change":"2023-11-28T21:40:42.749319+0000","last_active":"2023-11-28T21:56:35.749837+0000","last_peered":"2023-11-28T21:56:35.749837+0000","last_clean":"2023-11-28T21:56:35.749837+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:56:35.749837+0000","last_undegraded":"2023-11-28T21:56:35.749837+0000","last_fullsized":"2023-11-28T21:56:35.749837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21877398200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"52'197","reported_seq":279,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.311286+0000","last_change":"2023-11-28T21:56:43.311351+0000","last_active":"2023-11-28T21:56:43.311286+0000","last_peered":"2023-11-28T21:56:43.311286+0000","last_clean":"2023-11-28T21:56:43.311286+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:56:43.311286+0000","last_undegraded":"2023-11-28T21:56:43.311286+0000","last_fullsized":"2023-11-28T21:56:43.311286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'197","last_scrub_stamp":"2023-11-28T21:56:43.311087+0000","last_deep_scrub":"52'197","last_deep_scrub_stamp":"2023-11-28T21:56:43.311087+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.311087+0000","objects_scrubbed":4,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:17:01.116728+0000","scrub_duration":0.027225414999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":49161,"num_write":147,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11661,"num_omap_keys":22,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"52'1273","reported_seq":3575,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.720268+0000","last_change":"2023-11-28T21:40:43.072161+0000","last_active":"2023-11-28T21:56:34.720268+0000","last_peered":"2023-11-28T21:56:34.720268+0000","last_clean":"2023-11-28T21:56:34.720268+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:56:34.720268+0000","last_undegraded":"2023-11-28T21:56:34.720268+0000","last_fullsized":"2023-11-28T21:56:34.720268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.223266511,"stat_sum":{"num_bytes":11523503,"num_objects":308,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":219,"num_read_kb":12316,"num_write":1248,"num_write_kb":38807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"52'323","reported_seq":2415,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.543891+0000","last_change":"2023-11-28T21:56:43.543999+0000","last_active":"2023-11-28T21:56:43.543891+0000","last_peered":"2023-11-28T21:56:43.543891+0000","last_clean":"2023-11-28T21:56:43.543891+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:56:43.543891+0000","last_undegraded":"2023-11-28T21:56:43.543891+0000","last_fullsized":"2023-11-28T21:56:43.543891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'323","last_scrub_stamp":"2023-11-28T21:56:43.543658+0000","last_deep_scrub":"52'323","last_deep_scrub_stamp":"2023-11-28T21:56:43.543658+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.543658+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:59.532405+0000","scrub_duration":0.003404327,"objects_trimmed":0,"snaptrim_duration":0.21978382299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"52'868","reported_seq":987,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.270821+0000","last_change":"2023-11-28T21:56:50.270874+0000","last_active":"2023-11-28T21:56:50.270821+0000","last_peered":"2023-11-28T21:56:50.270821+0000","last_clean":"2023-11-28T21:56:50.270821+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:56:50.270821+0000","last_undegraded":"2023-11-28T21:56:50.270821+0000","last_fullsized":"2023-11-28T21:56:50.270821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'868","last_scrub_stamp":"2023-11-28T21:56:50.270733+0000","last_deep_scrub":"52'868","last_deep_scrub_stamp":"2023-11-28T21:56:50.270733+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.270733+0000","objects_scrubbed":14,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:25.187061+0000","scrub_duration":0.051670183000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":137,"num_read_kb":58709,"num_write":838,"num_write_kb":8642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34555,"num_omap_keys":61,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"52'1274","reported_seq":3607,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.000372+0000","last_change":"2023-11-28T21:40:43.252839+0000","last_active":"2023-11-28T21:56:35.000372+0000","last_peered":"2023-11-28T21:56:35.000372+0000","last_clean":"2023-11-28T21:56:35.000372+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:56:35.000372+0000","last_undegraded":"2023-11-28T21:56:35.000372+0000","last_fullsized":"2023-11-28T21:56:35.000372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27152726799999999,"stat_sum":{"num_bytes":36307339,"num_objects":283,"num_object_clones":0,"num_object_copies":1132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":283,"num_whiteouts":0,"num_read":251,"num_read_kb":8046,"num_write":1253,"num_write_kb":60298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"52'284","reported_seq":2367,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762517+0000","last_change":"2023-11-28T21:40:42.918241+0000","last_active":"2023-11-28T21:56:35.762517+0000","last_peered":"2023-11-28T21:56:35.762517+0000","last_clean":"2023-11-28T21:56:35.762517+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:56:35.762517+0000","last_undegraded":"2023-11-28T21:56:35.762517+0000","last_fullsized":"2023-11-28T21:56:35.762517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18244292200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"52'243","reported_seq":306,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.132811+0000","last_change":"2023-11-28T21:56:48.132928+0000","last_active":"2023-11-28T21:56:48.132811+0000","last_peered":"2023-11-28T21:56:48.132811+0000","last_clean":"2023-11-28T21:56:48.132811+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:56:48.132811+0000","last_undegraded":"2023-11-28T21:56:48.132811+0000","last_fullsized":"2023-11-28T21:56:48.132811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'243","last_scrub_stamp":"2023-11-28T21:56:48.132597+0000","last_deep_scrub":"52'243","last_deep_scrub_stamp":"2023-11-28T21:56:48.132597+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.132597+0000","objects_scrubbed":4,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:38.258199+0000","scrub_duration":0.035274727999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195875,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":194,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12188,"num_omap_keys":22,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"52'1346","reported_seq":3663,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.687516+0000","last_change":"2023-11-28T21:40:42.805322+0000","last_active":"2023-11-28T21:56:34.687516+0000","last_peered":"2023-11-28T21:56:34.687516+0000","last_clean":"2023-11-28T21:56:34.687516+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:56:34.687516+0000","last_undegraded":"2023-11-28T21:56:34.687516+0000","last_fullsized":"2023-11-28T21:56:34.687516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29439885100000002,"stat_sum":{"num_bytes":34458204,"num_objects":324,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":324,"num_whiteouts":0,"num_read":235,"num_read_kb":27797,"num_write":1320,"num_write_kb":70332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"52'313","reported_seq":2397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.755624+0000","last_change":"2023-11-28T21:40:42.740620+0000","last_active":"2023-11-28T21:56:35.755624+0000","last_peered":"2023-11-28T21:56:35.755624+0000","last_clean":"2023-11-28T21:56:35.755624+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:56:35.755624+0000","last_undegraded":"2023-11-28T21:56:35.755624+0000","last_fullsized":"2023-11-28T21:56:35.755624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.218364582,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"52'160","reported_seq":227,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.715676+0000","last_change":"2023-11-28T21:56:55.715743+0000","last_active":"2023-11-28T21:56:55.715676+0000","last_peered":"2023-11-28T21:56:55.715676+0000","last_clean":"2023-11-28T21:56:55.715676+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:56:55.715676+0000","last_undegraded":"2023-11-28T21:56:55.715676+0000","last_fullsized":"2023-11-28T21:56:55.715676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'160","last_scrub_stamp":"2023-11-28T21:56:55.715502+0000","last_deep_scrub":"52'160","last_deep_scrub_stamp":"2023-11-28T21:56:55.715502+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.715502+0000","objects_scrubbed":5,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:08:17.898435+0000","scrub_duration":0.024710507999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":107,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23214,"num_omap_keys":42,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"52'1271","reported_seq":3571,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.041667+0000","last_change":"2023-11-28T21:40:42.806545+0000","last_active":"2023-11-28T21:56:35.041667+0000","last_peered":"2023-11-28T21:56:35.041667+0000","last_clean":"2023-11-28T21:56:35.041667+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:56:35.041667+0000","last_undegraded":"2023-11-28T21:56:35.041667+0000","last_fullsized":"2023-11-28T21:56:35.041667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29571303700000001,"stat_sum":{"num_bytes":33678171,"num_objects":350,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":350,"num_whiteouts":0,"num_read":218,"num_read_kb":4129,"num_write":1246,"num_write_kb":52865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"52'279","reported_seq":2362,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757090+0000","last_change":"2023-11-28T21:40:43.257612+0000","last_active":"2023-11-28T21:56:35.757090+0000","last_peered":"2023-11-28T21:56:35.757090+0000","last_clean":"2023-11-28T21:56:35.757090+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:56:35.757090+0000","last_undegraded":"2023-11-28T21:56:35.757090+0000","last_fullsized":"2023-11-28T21:56:35.757090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21362616300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"52'313","reported_seq":415,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.422213+0000","last_change":"2023-11-28T21:56:44.422244+0000","last_active":"2023-11-28T21:56:44.422213+0000","last_peered":"2023-11-28T21:56:44.422213+0000","last_clean":"2023-11-28T21:56:44.422213+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:56:44.422213+0000","last_undegraded":"2023-11-28T21:56:44.422213+0000","last_fullsized":"2023-11-28T21:56:44.422213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'313","last_scrub_stamp":"2023-11-28T21:56:44.422134+0000","last_deep_scrub":"52'313","last_deep_scrub_stamp":"2023-11-28T21:56:44.422134+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.422134+0000","objects_scrubbed":9,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:50.028108+0000","scrub_duration":0.030421210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":67,"num_read_kb":77908,"num_write":272,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57622,"num_omap_keys":105,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"52'1265","reported_seq":3537,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.764966+0000","last_change":"2023-11-28T21:40:42.831006+0000","last_active":"2023-11-28T21:56:34.764966+0000","last_peered":"2023-11-28T21:56:34.764966+0000","last_clean":"2023-11-28T21:56:34.764966+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:56:34.764966+0000","last_undegraded":"2023-11-28T21:56:34.764966+0000","last_fullsized":"2023-11-28T21:56:34.764966+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1265,"log_dups_size":0,"ondisk_log_size":1265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30778537500000003,"stat_sum":{"num_bytes":30037804,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":190,"num_read_kb":33,"num_write":1241,"num_write_kb":36631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"52'297","reported_seq":2381,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.764231+0000","last_change":"2023-11-28T21:40:42.748885+0000","last_active":"2023-11-28T21:56:35.764231+0000","last_peered":"2023-11-28T21:56:35.764231+0000","last_clean":"2023-11-28T21:56:35.764231+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:56:35.764231+0000","last_undegraded":"2023-11-28T21:56:35.764231+0000","last_fullsized":"2023-11-28T21:56:35.764231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21990301600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"52'758","reported_seq":881,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:47.144324+0000","last_change":"2023-11-28T21:56:47.144324+0000","last_active":"2023-11-28T21:56:47.144324+0000","last_peered":"2023-11-28T21:56:47.144324+0000","last_clean":"2023-11-28T21:56:47.144324+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:56:47.144324+0000","last_undegraded":"2023-11-28T21:56:47.144324+0000","last_fullsized":"2023-11-28T21:56:47.144324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'757","last_scrub_stamp":"2023-11-28T21:56:47.144054+0000","last_deep_scrub":"52'757","last_deep_scrub_stamp":"2023-11-28T21:56:47.144054+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.144054+0000","objects_scrubbed":12,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:36:20.172063+0000","scrub_duration":0.085472306999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":90,"num_read_kb":122940,"num_write":683,"num_write_kb":20914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":36218,"num_omap_keys":66,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"52'1318","reported_seq":3605,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.731622+0000","last_change":"2023-11-28T21:40:42.828740+0000","last_active":"2023-11-28T21:56:34.731622+0000","last_peered":"2023-11-28T21:56:34.731622+0000","last_clean":"2023-11-28T21:56:34.731622+0000","last_became_active":"2023-11-28T21:35:25.790293+0000","last_became_peered":"2023-11-28T21:35:25.790293+0000","last_unstale":"2023-11-28T21:56:34.731622+0000","last_undegraded":"2023-11-28T21:56:34.731622+0000","last_fullsized":"2023-11-28T21:56:34.731622+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30978138799999999,"stat_sum":{"num_bytes":36258609,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":208,"num_read_kb":8227,"num_write":1285,"num_write_kb":56637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"52'283","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.299355+0000","last_change":"2023-11-28T21:56:50.299408+0000","last_active":"2023-11-28T21:56:50.299355+0000","last_peered":"2023-11-28T21:56:50.299355+0000","last_clean":"2023-11-28T21:56:50.299355+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:56:50.299355+0000","last_undegraded":"2023-11-28T21:56:50.299355+0000","last_fullsized":"2023-11-28T21:56:50.299355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:56:50.299160+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:56:50.299160+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.299160+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:20.213618+0000","scrub_duration":0.003897317,"objects_trimmed":0,"snaptrim_duration":0.22435063599999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"52'525","reported_seq":625,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.790155+0000","last_change":"2023-11-28T21:56:53.790210+0000","last_active":"2023-11-28T21:56:53.790155+0000","last_peered":"2023-11-28T21:56:53.790155+0000","last_clean":"2023-11-28T21:56:53.790155+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:56:53.790155+0000","last_undegraded":"2023-11-28T21:56:53.790155+0000","last_fullsized":"2023-11-28T21:56:53.790155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'525","last_scrub_stamp":"2023-11-28T21:56:53.790048+0000","last_deep_scrub":"52'525","last_deep_scrub_stamp":"2023-11-28T21:56:53.790048+0000","last_clean_scrub_stamp":"2023-11-28T21:56:53.790048+0000","objects_scrubbed":9,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:47:27.057628+0000","scrub_duration":0.097465174000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":69670,"num_write":485,"num_write_kb":16676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24376,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"52'1329","reported_seq":3690,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:52.591960+0000","last_change":"2023-11-28T21:56:52.592017+0000","last_active":"2023-11-28T21:56:52.591960+0000","last_peered":"2023-11-28T21:56:52.591960+0000","last_clean":"2023-11-28T21:56:52.591960+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:56:52.591960+0000","last_undegraded":"2023-11-28T21:56:52.591960+0000","last_fullsized":"2023-11-28T21:56:52.591960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1329","last_scrub_stamp":"2023-11-28T21:56:52.591801+0000","last_deep_scrub":"52'1329","last_deep_scrub_stamp":"2023-11-28T21:56:52.591801+0000","last_clean_scrub_stamp":"2023-11-28T21:56:52.591801+0000","objects_scrubbed":342,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:47:53.831632+0000","scrub_duration":0.25412254099999998,"objects_trimmed":0,"snaptrim_duration":0.2340457,"stat_sum":{"num_bytes":46614746,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":259,"num_read_kb":20536,"num_write":1294,"num_write_kb":83351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"52'265","reported_seq":2348,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.758280+0000","last_change":"2023-11-28T21:40:43.257554+0000","last_active":"2023-11-28T21:56:35.758280+0000","last_peered":"2023-11-28T21:56:35.758280+0000","last_clean":"2023-11-28T21:56:35.758280+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:56:35.758280+0000","last_undegraded":"2023-11-28T21:56:35.758280+0000","last_fullsized":"2023-11-28T21:56:35.758280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.228570471,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"52'997","reported_seq":1085,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.686854+0000","last_change":"2023-11-28T21:56:49.686931+0000","last_active":"2023-11-28T21:56:49.686854+0000","last_peered":"2023-11-28T21:56:49.686854+0000","last_clean":"2023-11-28T21:56:49.686854+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:56:49.686854+0000","last_undegraded":"2023-11-28T21:56:49.686854+0000","last_fullsized":"2023-11-28T21:56:49.686854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'997","last_scrub_stamp":"2023-11-28T21:56:49.686647+0000","last_deep_scrub":"52'997","last_deep_scrub_stamp":"2023-11-28T21:56:49.686647+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.686647+0000","objects_scrubbed":7,"log_size":997,"log_dups_size":0,"ondisk_log_size":997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:07:13.091549+0000","scrub_duration":0.104808501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":90124,"num_write":955,"num_write_kb":20985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9418,"num_omap_keys":17,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"52'1334","reported_seq":3674,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.997981+0000","last_change":"2023-11-28T21:40:42.805595+0000","last_active":"2023-11-28T21:56:34.997981+0000","last_peered":"2023-11-28T21:56:34.997981+0000","last_clean":"2023-11-28T21:56:34.997981+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:56:34.997981+0000","last_undegraded":"2023-11-28T21:56:34.997981+0000","last_fullsized":"2023-11-28T21:56:34.997981+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29518589499999998,"stat_sum":{"num_bytes":25457761,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":258,"num_read_kb":22214,"num_write":1317,"num_write_kb":78783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"52'308","reported_seq":2384,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.749079+0000","last_change":"2023-11-28T21:40:43.091522+0000","last_active":"2023-11-28T21:56:35.749079+0000","last_peered":"2023-11-28T21:56:35.749079+0000","last_clean":"2023-11-28T21:56:35.749079+0000","last_became_active":"2023-11-28T21:35:25.595386+0000","last_became_peered":"2023-11-28T21:35:25.595386+0000","last_unstale":"2023-11-28T21:56:35.749079+0000","last_undegraded":"2023-11-28T21:56:35.749079+0000","last_fullsized":"2023-11-28T21:56:35.749079+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.148630489,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"52'112","reported_seq":179,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.072726+0000","last_change":"2023-11-28T21:56:55.072786+0000","last_active":"2023-11-28T21:56:55.072726+0000","last_peered":"2023-11-28T21:56:55.072726+0000","last_clean":"2023-11-28T21:56:55.072726+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:56:55.072726+0000","last_undegraded":"2023-11-28T21:56:55.072726+0000","last_fullsized":"2023-11-28T21:56:55.072726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'112","last_scrub_stamp":"2023-11-28T21:56:55.072507+0000","last_deep_scrub":"52'112","last_deep_scrub_stamp":"2023-11-28T21:56:55.072507+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.072507+0000","objects_scrubbed":3,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:04:39.066804+0000","scrub_duration":0.0092242439999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":68,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"52'1427","reported_seq":3772,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.027004+0000","last_change":"2023-11-28T21:40:42.795304+0000","last_active":"2023-11-28T21:56:35.027004+0000","last_peered":"2023-11-28T21:56:35.027004+0000","last_clean":"2023-11-28T21:56:35.027004+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:56:35.027004+0000","last_undegraded":"2023-11-28T21:56:35.027004+0000","last_fullsized":"2023-11-28T21:56:35.027004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28498848100000002,"stat_sum":{"num_bytes":33245588,"num_objects":317,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":263,"num_read_kb":28708,"num_write":1396,"num_write_kb":77612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"52'1309","reported_seq":3641,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:39.342045+0000","last_change":"2023-11-28T21:40:43.251187+0000","last_active":"2023-11-28T21:56:39.342045+0000","last_peered":"2023-11-28T21:56:39.342045+0000","last_clean":"2023-11-28T21:56:39.342045+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:56:39.342045+0000","last_undegraded":"2023-11-28T21:56:39.342045+0000","last_fullsized":"2023-11-28T21:56:39.342045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1309,"log_dups_size":0,"ondisk_log_size":1309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27353859400000002,"stat_sum":{"num_bytes":25285382,"num_objects":331,"num_object_clones":0,"num_object_copies":1324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":248,"num_read_kb":4117,"num_write":1287,"num_write_kb":58337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"52'285","reported_seq":2376,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.748373+0000","last_change":"2023-11-28T21:40:42.876095+0000","last_active":"2023-11-28T21:56:35.748373+0000","last_peered":"2023-11-28T21:56:35.748373+0000","last_clean":"2023-11-28T21:56:35.748373+0000","last_became_active":"2023-11-28T21:35:25.589378+0000","last_became_peered":"2023-11-28T21:35:25.589378+0000","last_unstale":"2023-11-28T21:56:35.748373+0000","last_undegraded":"2023-11-28T21:56:35.748373+0000","last_fullsized":"2023-11-28T21:56:35.748373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246426856,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"52'951","reported_seq":1047,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.694770+0000","last_change":"2023-11-28T21:56:50.694839+0000","last_active":"2023-11-28T21:56:50.694770+0000","last_peered":"2023-11-28T21:56:50.694770+0000","last_clean":"2023-11-28T21:56:50.694770+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:56:50.694770+0000","last_undegraded":"2023-11-28T21:56:50.694770+0000","last_fullsized":"2023-11-28T21:56:50.694770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'951","last_scrub_stamp":"2023-11-28T21:56:50.694579+0000","last_deep_scrub":"52'951","last_deep_scrub_stamp":"2023-11-28T21:56:50.694579+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.694579+0000","objects_scrubbed":5,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:57:02.007325+0000","scrub_duration":0.095599400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":899,"num_write_kb":25075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12742,"num_omap_keys":23,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"52'1266","reported_seq":3554,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.473089+0000","last_change":"2023-11-28T21:40:42.918369+0000","last_active":"2023-11-28T21:56:34.473089+0000","last_peered":"2023-11-28T21:56:34.473089+0000","last_clean":"2023-11-28T21:56:34.473089+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:56:34.473089+0000","last_undegraded":"2023-11-28T21:56:34.473089+0000","last_fullsized":"2023-11-28T21:56:34.473089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1266,"log_dups_size":0,"ondisk_log_size":1266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.16839808100000001,"stat_sum":{"num_bytes":18660789,"num_objects":319,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":205,"num_read_kb":12320,"num_write":1249,"num_write_kb":51464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"52'321","reported_seq":2412,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:58.470634+0000","last_change":"2023-11-28T21:56:58.470657+0000","last_active":"2023-11-28T21:56:58.470634+0000","last_peered":"2023-11-28T21:56:58.470634+0000","last_clean":"2023-11-28T21:56:58.470634+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:56:58.470634+0000","last_undegraded":"2023-11-28T21:56:58.470634+0000","last_fullsized":"2023-11-28T21:56:58.470634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'321","last_scrub_stamp":"2023-11-28T21:56:58.470530+0000","last_deep_scrub":"52'321","last_deep_scrub_stamp":"2023-11-28T21:56:58.470530+0000","last_clean_scrub_stamp":"2023-11-28T21:56:58.470530+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:47:39.793970+0000","scrub_duration":0.0019542520000000001,"objects_trimmed":0,"snaptrim_duration":0.28961016299999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"52'91","reported_seq":162,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:52.421023+0000","last_change":"2023-11-28T21:56:52.421071+0000","last_active":"2023-11-28T21:56:52.421023+0000","last_peered":"2023-11-28T21:56:52.421023+0000","last_clean":"2023-11-28T21:56:52.421023+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:56:52.421023+0000","last_undegraded":"2023-11-28T21:56:52.421023+0000","last_fullsized":"2023-11-28T21:56:52.421023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'91","last_scrub_stamp":"2023-11-28T21:56:52.420930+0000","last_deep_scrub":"52'91","last_deep_scrub_stamp":"2023-11-28T21:56:52.420930+0000","last_clean_scrub_stamp":"2023-11-28T21:56:52.420930+0000","objects_scrubbed":6,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:15:04.674788+0000","scrub_duration":0.030405108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":63,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":37270,"num_omap_keys":68,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"52'1081","reported_seq":1181,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.227814+0000","last_change":"2023-11-28T21:56:46.227886+0000","last_active":"2023-11-28T21:56:46.227814+0000","last_peered":"2023-11-28T21:56:46.227814+0000","last_clean":"2023-11-28T21:56:46.227814+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:56:46.227814+0000","last_undegraded":"2023-11-28T21:56:46.227814+0000","last_fullsized":"2023-11-28T21:56:46.227814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1081","last_scrub_stamp":"2023-11-28T21:56:46.227627+0000","last_deep_scrub":"52'1081","last_deep_scrub_stamp":"2023-11-28T21:56:46.227627+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.227627+0000","objects_scrubbed":10,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:49:00.930323+0000","scrub_duration":0.124157213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":146592,"num_write":1020,"num_write_kb":29113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":33716,"num_omap_keys":61,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"52'284","reported_seq":2366,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.766876+0000","last_change":"2023-11-28T21:40:43.254737+0000","last_active":"2023-11-28T21:56:35.766876+0000","last_peered":"2023-11-28T21:56:35.766876+0000","last_clean":"2023-11-28T21:56:35.766876+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:56:35.766876+0000","last_undegraded":"2023-11-28T21:56:35.766876+0000","last_fullsized":"2023-11-28T21:56:35.766876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26886027800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"52'1325","reported_seq":3624,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.070219+0000","last_change":"2023-11-28T21:40:43.102291+0000","last_active":"2023-11-28T21:56:35.070219+0000","last_peered":"2023-11-28T21:56:35.070219+0000","last_clean":"2023-11-28T21:56:35.070219+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:56:35.070219+0000","last_undegraded":"2023-11-28T21:56:35.070219+0000","last_fullsized":"2023-11-28T21:56:35.070219+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1325,"log_dups_size":0,"ondisk_log_size":1325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.13232148399999999,"stat_sum":{"num_bytes":46546650,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":218,"num_read_kb":9401,"num_write":1296,"num_write_kb":74280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"52'473","reported_seq":549,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.461672+0000","last_change":"2023-11-28T21:56:53.461705+0000","last_active":"2023-11-28T21:56:53.461672+0000","last_peered":"2023-11-28T21:56:53.461672+0000","last_clean":"2023-11-28T21:56:53.461672+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:56:53.461672+0000","last_undegraded":"2023-11-28T21:56:53.461672+0000","last_fullsized":"2023-11-28T21:56:53.461672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'473","last_scrub_stamp":"2023-11-28T21:56:53.461589+0000","last_deep_scrub":"52'473","last_deep_scrub_stamp":"2023-11-28T21:56:53.461589+0000","last_clean_scrub_stamp":"2023-11-28T21:56:53.461589+0000","objects_scrubbed":6,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:21:39.580720+0000","scrub_duration":0.057920754999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":48896,"num_write":437,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":14594,"num_omap_keys":28,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"52'279","reported_seq":2369,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.249077+0000","last_change":"2023-11-28T21:56:49.249131+0000","last_active":"2023-11-28T21:56:49.249077+0000","last_peered":"2023-11-28T21:56:49.249077+0000","last_clean":"2023-11-28T21:56:49.249077+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:56:49.249077+0000","last_undegraded":"2023-11-28T21:56:49.249077+0000","last_fullsized":"2023-11-28T21:56:49.249077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'279","last_scrub_stamp":"2023-11-28T21:56:49.248890+0000","last_deep_scrub":"52'279","last_deep_scrub_stamp":"2023-11-28T21:56:49.248890+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.248890+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:20:21.740665+0000","scrub_duration":0.0035163680000000002,"objects_trimmed":0,"snaptrim_duration":0.24793844200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"52'1320","reported_seq":3658,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.945641+0000","last_change":"2023-11-28T21:40:42.807134+0000","last_active":"2023-11-28T21:56:34.945641+0000","last_peered":"2023-11-28T21:56:34.945641+0000","last_clean":"2023-11-28T21:56:34.945641+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:56:34.945641+0000","last_undegraded":"2023-11-28T21:56:34.945641+0000","last_fullsized":"2023-11-28T21:56:34.945641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1320,"log_dups_size":0,"ondisk_log_size":1320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29562192300000001,"stat_sum":{"num_bytes":29852008,"num_objects":333,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":257,"num_read_kb":28714,"num_write":1297,"num_write_kb":83698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"52'321","reported_seq":2405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762513+0000","last_change":"2023-11-28T21:40:42.917455+0000","last_active":"2023-11-28T21:56:35.762513+0000","last_peered":"2023-11-28T21:56:35.762513+0000","last_clean":"2023-11-28T21:56:35.762513+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:56:35.762513+0000","last_undegraded":"2023-11-28T21:56:35.762513+0000","last_fullsized":"2023-11-28T21:56:35.762513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18187520500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"52'328","reported_seq":2413,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.745025+0000","last_change":"2023-11-28T21:40:43.100474+0000","last_active":"2023-11-28T21:56:35.745025+0000","last_peered":"2023-11-28T21:56:35.745025+0000","last_clean":"2023-11-28T21:56:35.745025+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:56:35.745025+0000","last_undegraded":"2023-11-28T21:56:35.745025+0000","last_fullsized":"2023-11-28T21:56:35.745025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14391041299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"52'303","reported_seq":2396,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.533484+0000","last_change":"2023-11-28T21:56:44.533536+0000","last_active":"2023-11-28T21:56:44.533484+0000","last_peered":"2023-11-28T21:56:44.533484+0000","last_clean":"2023-11-28T21:56:44.533484+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:56:44.533484+0000","last_undegraded":"2023-11-28T21:56:44.533484+0000","last_fullsized":"2023-11-28T21:56:44.533484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'303","last_scrub_stamp":"2023-11-28T21:56:44.533290+0000","last_deep_scrub":"52'303","last_deep_scrub_stamp":"2023-11-28T21:56:44.533290+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.533290+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:11:39.585022+0000","scrub_duration":0.0035272839999999999,"objects_trimmed":0,"snaptrim_duration":0.30621145,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"52'294","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.767559+0000","last_change":"2023-11-28T21:40:43.255460+0000","last_active":"2023-11-28T21:56:35.767559+0000","last_peered":"2023-11-28T21:56:35.767559+0000","last_clean":"2023-11-28T21:56:35.767559+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:56:35.767559+0000","last_undegraded":"2023-11-28T21:56:35.767559+0000","last_fullsized":"2023-11-28T21:56:35.767559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26967866099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"52'275","reported_seq":2367,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.387085+0000","last_change":"2023-11-28T21:56:46.387109+0000","last_active":"2023-11-28T21:56:46.387085+0000","last_peered":"2023-11-28T21:56:46.387085+0000","last_clean":"2023-11-28T21:56:46.387085+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:56:46.387085+0000","last_undegraded":"2023-11-28T21:56:46.387085+0000","last_fullsized":"2023-11-28T21:56:46.387085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'275","last_scrub_stamp":"2023-11-28T21:56:46.386968+0000","last_deep_scrub":"52'275","last_deep_scrub_stamp":"2023-11-28T21:56:46.386968+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.386968+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:09:10.256531+0000","scrub_duration":0.0028373539999999998,"objects_trimmed":0,"snaptrim_duration":0.29058787400000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"52'293","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.760786+0000","last_change":"2023-11-28T21:40:42.886237+0000","last_active":"2023-11-28T21:56:35.760786+0000","last_peered":"2023-11-28T21:56:35.760786+0000","last_clean":"2023-11-28T21:56:35.760786+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:56:35.760786+0000","last_undegraded":"2023-11-28T21:56:35.760786+0000","last_fullsized":"2023-11-28T21:56:35.760786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246741036,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"52'316","reported_seq":2400,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.756488+0000","last_change":"2023-11-28T21:40:42.916517+0000","last_active":"2023-11-28T21:56:35.756488+0000","last_peered":"2023-11-28T21:56:35.756488+0000","last_clean":"2023-11-28T21:56:35.756488+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:56:35.756488+0000","last_undegraded":"2023-11-28T21:56:35.756488+0000","last_fullsized":"2023-11-28T21:56:35.756488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18459727300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"52'287","reported_seq":2371,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762288+0000","last_change":"2023-11-28T21:40:42.803865+0000","last_active":"2023-11-28T21:56:35.762288+0000","last_peered":"2023-11-28T21:56:35.762288+0000","last_clean":"2023-11-28T21:56:35.762288+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:56:35.762288+0000","last_undegraded":"2023-11-28T21:56:35.762288+0000","last_fullsized":"2023-11-28T21:56:35.762288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29272911699999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"52'285","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.763038+0000","last_change":"2023-11-28T21:40:42.823937+0000","last_active":"2023-11-28T21:56:35.763038+0000","last_peered":"2023-11-28T21:56:35.763038+0000","last_clean":"2023-11-28T21:56:35.763038+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:56:35.763038+0000","last_undegraded":"2023-11-28T21:56:35.763038+0000","last_fullsized":"2023-11-28T21:56:35.763038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30579284099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"52'272","reported_seq":2363,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:51.436161+0000","last_change":"2023-11-28T21:56:51.436185+0000","last_active":"2023-11-28T21:56:51.436161+0000","last_peered":"2023-11-28T21:56:51.436161+0000","last_clean":"2023-11-28T21:56:51.436161+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:56:51.436161+0000","last_undegraded":"2023-11-28T21:56:51.436161+0000","last_fullsized":"2023-11-28T21:56:51.436161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'272","last_scrub_stamp":"2023-11-28T21:56:51.436052+0000","last_deep_scrub":"52'272","last_deep_scrub_stamp":"2023-11-28T21:56:51.436052+0000","last_clean_scrub_stamp":"2023-11-28T21:56:51.436052+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:27.893115+0000","scrub_duration":0.0032975470000000001,"objects_trimmed":0,"snaptrim_duration":0.28793565599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"52'285","reported_seq":2369,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757220+0000","last_change":"2023-11-28T21:40:42.831259+0000","last_active":"2023-11-28T21:56:35.757220+0000","last_peered":"2023-11-28T21:56:35.757220+0000","last_clean":"2023-11-28T21:56:35.757220+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:56:35.757220+0000","last_undegraded":"2023-11-28T21:56:35.757220+0000","last_fullsized":"2023-11-28T21:56:35.757220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30636868099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"52'297","reported_seq":2380,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:36.101069+0000","last_change":"2023-11-28T21:40:43.253941+0000","last_active":"2023-11-28T21:56:36.101069+0000","last_peered":"2023-11-28T21:56:36.101069+0000","last_clean":"2023-11-28T21:56:36.101069+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:56:36.101069+0000","last_undegraded":"2023-11-28T21:56:36.101069+0000","last_fullsized":"2023-11-28T21:56:36.101069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26835999100000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"52'297","reported_seq":2380,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.755460+0000","last_change":"2023-11-28T21:40:42.910839+0000","last_active":"2023-11-28T21:56:35.755460+0000","last_peered":"2023-11-28T21:56:35.755460+0000","last_clean":"2023-11-28T21:56:35.755460+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:56:35.755460+0000","last_undegraded":"2023-11-28T21:56:35.755460+0000","last_fullsized":"2023-11-28T21:56:35.755460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.194469013,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"52'293","reported_seq":2377,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.750393+0000","last_change":"2023-11-28T21:40:42.749349+0000","last_active":"2023-11-28T21:56:35.750393+0000","last_peered":"2023-11-28T21:56:35.750393+0000","last_clean":"2023-11-28T21:56:35.750393+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:56:35.750393+0000","last_undegraded":"2023-11-28T21:56:35.750393+0000","last_fullsized":"2023-11-28T21:56:35.750393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21943309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"52'268","reported_seq":2352,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.743829+0000","last_change":"2023-11-28T21:40:43.092423+0000","last_active":"2023-11-28T21:56:35.743829+0000","last_peered":"2023-11-28T21:56:35.743829+0000","last_clean":"2023-11-28T21:56:35.743829+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:56:35.743829+0000","last_undegraded":"2023-11-28T21:56:35.743829+0000","last_fullsized":"2023-11-28T21:56:35.743829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.14941532399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"52'313","reported_seq":2405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:42.478620+0000","last_change":"2023-11-28T21:56:42.478673+0000","last_active":"2023-11-28T21:56:42.478620+0000","last_peered":"2023-11-28T21:56:42.478620+0000","last_clean":"2023-11-28T21:56:42.478620+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:56:42.478620+0000","last_undegraded":"2023-11-28T21:56:42.478620+0000","last_fullsized":"2023-11-28T21:56:42.478620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'313","last_scrub_stamp":"2023-11-28T21:56:42.478435+0000","last_deep_scrub":"52'313","last_deep_scrub_stamp":"2023-11-28T21:56:42.478435+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.478435+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:59.037978+0000","scrub_duration":0.0037738749999999999,"objects_trimmed":0,"snaptrim_duration":0.30374592,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"52'301","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.566855+0000","last_change":"2023-11-28T21:56:45.566906+0000","last_active":"2023-11-28T21:56:45.566855+0000","last_peered":"2023-11-28T21:56:45.566855+0000","last_clean":"2023-11-28T21:56:45.566855+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:56:45.566855+0000","last_undegraded":"2023-11-28T21:56:45.566855+0000","last_fullsized":"2023-11-28T21:56:45.566855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'301","last_scrub_stamp":"2023-11-28T21:56:45.566667+0000","last_deep_scrub":"52'301","last_deep_scrub_stamp":"2023-11-28T21:56:45.566667+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.566667+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:37:12.783651+0000","scrub_duration":0.0036235360000000001,"objects_trimmed":0,"snaptrim_duration":0.30238551699999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"52'316","reported_seq":2400,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.756399+0000","last_change":"2023-11-28T21:40:42.876664+0000","last_active":"2023-11-28T21:56:35.756399+0000","last_peered":"2023-11-28T21:56:35.756399+0000","last_clean":"2023-11-28T21:56:35.756399+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:56:35.756399+0000","last_undegraded":"2023-11-28T21:56:35.756399+0000","last_fullsized":"2023-11-28T21:56:35.756399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246777987,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"52'301","reported_seq":2392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.425032+0000","last_change":"2023-11-28T21:56:55.425054+0000","last_active":"2023-11-28T21:56:55.425032+0000","last_peered":"2023-11-28T21:56:55.425032+0000","last_clean":"2023-11-28T21:56:55.425032+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:56:55.425032+0000","last_undegraded":"2023-11-28T21:56:55.425032+0000","last_fullsized":"2023-11-28T21:56:55.425032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'301","last_scrub_stamp":"2023-11-28T21:56:55.424936+0000","last_deep_scrub":"52'301","last_deep_scrub_stamp":"2023-11-28T21:56:55.424936+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.424936+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:58:23.742113+0000","scrub_duration":0.0019665469999999999,"objects_trimmed":0,"snaptrim_duration":0.21399896700000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"52'274","reported_seq":2365,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.388835+0000","last_change":"2023-11-28T21:56:48.388857+0000","last_active":"2023-11-28T21:56:48.388835+0000","last_peered":"2023-11-28T21:56:48.388835+0000","last_clean":"2023-11-28T21:56:48.388835+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:56:48.388835+0000","last_undegraded":"2023-11-28T21:56:48.388835+0000","last_fullsized":"2023-11-28T21:56:48.388835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'274","last_scrub_stamp":"2023-11-28T21:56:48.388743+0000","last_deep_scrub":"52'274","last_deep_scrub_stamp":"2023-11-28T21:56:48.388743+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.388743+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:55.401441+0000","scrub_duration":0.0030358709999999999,"objects_trimmed":0,"snaptrim_duration":0.287656776,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"52'261","reported_seq":2352,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.326425+0000","last_change":"2023-11-28T21:56:46.326533+0000","last_active":"2023-11-28T21:56:46.326425+0000","last_peered":"2023-11-28T21:56:46.326425+0000","last_clean":"2023-11-28T21:56:46.326425+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:56:46.326425+0000","last_undegraded":"2023-11-28T21:56:46.326425+0000","last_fullsized":"2023-11-28T21:56:46.326425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'261","last_scrub_stamp":"2023-11-28T21:56:46.326201+0000","last_deep_scrub":"52'261","last_deep_scrub_stamp":"2023-11-28T21:56:46.326201+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.326201+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:57.520054+0000","scrub_duration":0.0037943849999999999,"objects_trimmed":0,"snaptrim_duration":0.222804104,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"52'261","reported_seq":2344,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757320+0000","last_change":"2023-11-28T21:40:42.914448+0000","last_active":"2023-11-28T21:56:35.757320+0000","last_peered":"2023-11-28T21:56:35.757320+0000","last_clean":"2023-11-28T21:56:35.757320+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:56:35.757320+0000","last_undegraded":"2023-11-28T21:56:35.757320+0000","last_fullsized":"2023-11-28T21:56:35.757320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19768582900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"52'283","reported_seq":2366,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:36.537584+0000","last_change":"2023-11-28T21:40:42.916248+0000","last_active":"2023-11-28T21:56:36.537584+0000","last_peered":"2023-11-28T21:56:36.537584+0000","last_clean":"2023-11-28T21:56:36.537584+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:56:36.537584+0000","last_undegraded":"2023-11-28T21:56:36.537584+0000","last_fullsized":"2023-11-28T21:56:36.537584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18434383600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"52'314","reported_seq":2405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757872+0000","last_change":"2023-11-28T21:40:42.754000+0000","last_active":"2023-11-28T21:56:35.757872+0000","last_peered":"2023-11-28T21:56:35.757872+0000","last_clean":"2023-11-28T21:56:35.757872+0000","last_became_active":"2023-11-28T21:35:25.598839+0000","last_became_peered":"2023-11-28T21:35:25.598839+0000","last_unstale":"2023-11-28T21:56:35.757872+0000","last_undegraded":"2023-11-28T21:56:35.757872+0000","last_fullsized":"2023-11-28T21:56:35.757872+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22244017299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"52'291","reported_seq":2373,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.764269+0000","last_change":"2023-11-28T21:40:42.796731+0000","last_active":"2023-11-28T21:56:35.764269+0000","last_peered":"2023-11-28T21:56:35.764269+0000","last_clean":"2023-11-28T21:56:35.764269+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:56:35.764269+0000","last_undegraded":"2023-11-28T21:56:35.764269+0000","last_fullsized":"2023-11-28T21:56:35.764269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.28637254899999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"52'292","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.746244+0000","last_change":"2023-11-28T21:40:42.745933+0000","last_active":"2023-11-28T21:56:35.746244+0000","last_peered":"2023-11-28T21:56:35.746244+0000","last_clean":"2023-11-28T21:56:35.746244+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:56:35.746244+0000","last_undegraded":"2023-11-28T21:56:35.746244+0000","last_fullsized":"2023-11-28T21:56:35.746244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.219142752,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"52'282","reported_seq":2365,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.762100+0000","last_change":"2023-11-28T21:40:42.831458+0000","last_active":"2023-11-28T21:56:35.762100+0000","last_peered":"2023-11-28T21:56:35.762100+0000","last_clean":"2023-11-28T21:56:35.762100+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:56:35.762100+0000","last_undegraded":"2023-11-28T21:56:35.762100+0000","last_fullsized":"2023-11-28T21:56:35.762100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.307113203,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"52'297","reported_seq":2380,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.745275+0000","last_change":"2023-11-28T21:40:42.736971+0000","last_active":"2023-11-28T21:56:35.745275+0000","last_peered":"2023-11-28T21:56:35.745275+0000","last_clean":"2023-11-28T21:56:35.745275+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:56:35.745275+0000","last_undegraded":"2023-11-28T21:56:35.745275+0000","last_fullsized":"2023-11-28T21:56:35.745275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21561430200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"52'287","reported_seq":2373,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:40.216243+0000","last_change":"2023-11-28T21:40:42.829941+0000","last_active":"2023-11-28T21:56:40.216243+0000","last_peered":"2023-11-28T21:56:40.216243+0000","last_clean":"2023-11-28T21:56:40.216243+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:56:40.216243+0000","last_undegraded":"2023-11-28T21:56:40.216243+0000","last_fullsized":"2023-11-28T21:56:40.216243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30486287499999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"52'281","reported_seq":2365,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757024+0000","last_change":"2023-11-28T21:40:43.239342+0000","last_active":"2023-11-28T21:56:35.757024+0000","last_peered":"2023-11-28T21:56:35.757024+0000","last_clean":"2023-11-28T21:56:35.757024+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:56:35.757024+0000","last_undegraded":"2023-11-28T21:56:35.757024+0000","last_fullsized":"2023-11-28T21:56:35.757024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.26474323500000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"52'292","reported_seq":2377,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:36.084027+0000","last_change":"2023-11-28T21:40:42.885846+0000","last_active":"2023-11-28T21:56:36.084027+0000","last_peered":"2023-11-28T21:56:36.084027+0000","last_clean":"2023-11-28T21:56:36.084027+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:56:36.084027+0000","last_undegraded":"2023-11-28T21:56:36.084027+0000","last_fullsized":"2023-11-28T21:56:36.084027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24685295800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"52'289","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757430+0000","last_change":"2023-11-28T21:40:42.753421+0000","last_active":"2023-11-28T21:56:35.757430+0000","last_peered":"2023-11-28T21:56:35.757430+0000","last_clean":"2023-11-28T21:56:35.757430+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:56:35.757430+0000","last_undegraded":"2023-11-28T21:56:35.757430+0000","last_fullsized":"2023-11-28T21:56:35.757430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22257470400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":927864585,"num_objects":10340,"num_object_clones":0,"num_object_copies":41360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10340,"num_whiteouts":0,"num_read":7542,"num_read_kb":441308,"num_write":41596,"num_write_kb":1955411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1822244864,"data_stored":1974304768,"data_compressed":124280088,"data_compressed_allocated":140013568,"data_compressed_original":292073472,"omap_allocated":0,"internal_metadata":0},"log_size":42425,"ondisk_log_size":42425,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18675,"ondisk_log_size":18675,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":303117033,"num_objects":196,"num_object_clones":0,"num_object_copies":392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1486,"num_read_kb":1965797,"num_write":14874,"num_write_kb":402197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":628062,"num_omap_keys":1091,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173023232,"data_stored":606234066,"data_compressed":82519712,"data_compressed_allocated":105799680,"data_compressed_original":539164566,"omap_allocated":0,"internal_metadata":0},"log_size":16066,"ondisk_log_size":16066,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575123,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":345736,"kb_used_data":211840,"kb_used_omap":1,"kb_used_meta":133886,"kb_avail":94026104,"statfs":{"total":96636764160,"available":96282730496,"internally_reserved":0,"allocated":216924160,"data_stored":292550459,"data_compressed":24963242,"data_compressed_allocated":29859840,"data_compressed_original":105791488,"omap_allocated":1590,"internal_metadata":137099722},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.97799999999999998},"min":{"1min":0.51400000000000001,"5min":0.41399999999999998,"15min":0.35299999999999998},"max":{"1min":1.232,"5min":5.056,"15min":47.475000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.84199999999999997,"15min":0.82299999999999995},"min":{"1min":0.54200000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.145,"5min":4.069,"15min":5.3310000000000004},"last":0.93500000000000005}]},{"osd":1,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.98999999999999999,"15min":1.099},"min":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.39200000000000002},"max":{"1min":1.278,"5min":2.5630000000000002,"15min":47.529000000000003},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":1.083,"15min":1.008},"min":{"1min":0.64300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.171,"5min":17.460000000000001,"15min":17.460000000000001},"last":1.016}]},{"osd":2,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.86599999999999999,"15min":0.91200000000000003},"min":{"1min":0.58399999999999996,"5min":0.504,"15min":0.47299999999999998},"max":{"1min":1.0660000000000001,"5min":2.9359999999999999,"15min":5.5140000000000002},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0229999999999999,"15min":1.022},"min":{"1min":0.70199999999999996,"5min":0.60499999999999998,"15min":0.498},"max":{"1min":1.248,"5min":2.972,"15min":5.5979999999999999},"last":1.212}]},{"osd":3,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.1719999999999999,"15min":1.179},"min":{"1min":0.63800000000000001,"5min":0.57299999999999995,"15min":0.44600000000000001},"max":{"1min":1.196,"5min":16.113,"15min":47.57},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.171,"15min":1.0649999999999999},"min":{"1min":0.73599999999999999,"5min":0.65400000000000003,"15min":0.53800000000000003},"max":{"1min":1.2749999999999999,"5min":11.148999999999999,"15min":12.273},"last":1.123}]},{"osd":4,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.1180000000000001,"15min":1.054},"min":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.35499999999999998},"max":{"1min":1.1990000000000001,"5min":17.542000000000002,"15min":17.542000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.075,"5min":1.0489999999999999,"15min":1.196},"min":{"1min":0.84699999999999998,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.363,"5min":1.5329999999999999,"15min":49.106000000000002},"last":1.0649999999999999}]},{"osd":5,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.1220000000000001,"15min":1.2909999999999999},"min":{"1min":0.80600000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.345,"5min":6.6420000000000003,"15min":64.855999999999995},"last":1.101},{"interface":"front","average":{"1min":1.056,"5min":1.1679999999999999,"15min":1.1970000000000001},"min":{"1min":0.73499999999999999,"5min":0.71399999999999997,"15min":0.56399999999999995},"max":{"1min":1.2689999999999999,"5min":11.327,"15min":16.753},"last":0.89600000000000002}]},{"osd":6,"last update":"Tue Nov 28 21:56:02 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.194,"15min":1.1639999999999999},"min":{"1min":0.88600000000000001,"5min":0.42699999999999999,"15min":0.37},"max":{"1min":1.1930000000000001,"5min":11.641999999999999,"15min":11.641999999999999},"last":1.145},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1399999999999999,"15min":1.153},"min":{"1min":0.97999999999999998,"5min":0.61499999999999999,"15min":0.502},"max":{"1min":1.4199999999999999,"5min":1.474,"15min":7.4909999999999997},"last":0.86899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575121,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":380176,"kb_used_data":254600,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":93991664,"statfs":{"total":96636764160,"available":96247463936,"internally_reserved":0,"allocated":260710400,"data_stored":335877014,"data_compressed":24633140,"data_compressed_allocated":29302784,"data_compressed_original":104771531,"omap_allocated":1594,"internal_metadata":128580038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":2.9159999999999999,"5min":1.248,"15min":1.0109999999999999},"min":{"1min":0.51500000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":40.999000000000002,"5min":40.999000000000002,"15min":40.999000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.05,"15min":1.036},"min":{"1min":0.58299999999999996,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":14.132999999999999,"5min":14.132999999999999,"15min":24.399999999999999},"last":0.70699999999999996}]},{"osd":1,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.89500000000000002,"15min":0.88700000000000001},"min":{"1min":0.71999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2769999999999999,"5min":2.2400000000000002,"15min":11.196},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.90800000000000003,"15min":0.95899999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.35799999999999998},"max":{"1min":1.1759999999999999,"5min":6.1550000000000002,"15min":7.9130000000000003},"last":0.76000000000000001}]},{"osd":2,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":0.89500000000000002,"15min":0.94799999999999995},"min":{"1min":0.66300000000000003,"5min":0.502,"15min":0.39100000000000001},"max":{"1min":3.855,"5min":3.855,"15min":21.986999999999998},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.92800000000000005,"15min":0.99099999999999999},"min":{"1min":0.621,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2829999999999999,"5min":1.419,"15min":8.6170000000000009},"last":0.90000000000000002}]},{"osd":3,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.94799999999999995,"15min":0.96299999999999997},"min":{"1min":0.57099999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.8439999999999999,"5min":3.2679999999999998,"15min":11.239000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0069999999999999,"15min":1.087},"min":{"1min":0.73399999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.7959999999999998,"5min":3.29,"15min":24.745000000000001},"last":0.97799999999999998}]},{"osd":4,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91800000000000004,"15min":1.014},"min":{"1min":0.71599999999999997,"5min":0.32300000000000001,"15min":0.30299999999999999},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":16.015000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.044,"15min":1.079},"min":{"1min":0.69499999999999995,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":13.193},"last":1.181}]},{"osd":5,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":0.98099999999999998,"15min":1.109},"min":{"1min":0.65000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":2.069,"5min":2.069,"15min":25.006},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0629999999999999,"15min":1.1419999999999999},"min":{"1min":0.46700000000000003,"5min":0.33800000000000002,"15min":0.33000000000000002},"max":{"1min":1.405,"5min":1.4790000000000001,"15min":7.843},"last":1.3360000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:56:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0369999999999999,"15min":1.139},"min":{"1min":0.79200000000000004,"5min":0.38100000000000001,"15min":0.36099999999999999},"max":{"1min":1.1950000000000001,"5min":1.9179999999999999,"15min":7.883},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.177,"15min":1.276},"min":{"1min":0.93300000000000005,"5min":0.432,"15min":0.432},"max":{"1min":1.3169999999999999,"5min":1.9610000000000001,"15min":27.942},"last":1.5129999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575121,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363676,"kb_used_data":238100,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":94008164,"statfs":{"total":96636764160,"available":96264359936,"internally_reserved":0,"allocated":243814400,"data_stored":323612635,"data_compressed":28196796,"data_compressed_allocated":33439744,"data_compressed_original":113541120,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.99399999999999999,"15min":0.89800000000000002},"min":{"1min":0.47399999999999998,"5min":0.46500000000000002,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":15.005000000000001,"15min":15.005000000000001},"last":1.113},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88800000000000001},"min":{"1min":0.501,"5min":0.47799999999999998,"15min":0.46700000000000003},"max":{"1min":1.2290000000000001,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":1.1479999999999999}]},{"osd":1,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.81100000000000005,"15min":0.83999999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.48199999999999998},"max":{"1min":0.89800000000000002,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.88900000000000001,"15min":0.91300000000000003},"min":{"1min":0.57299999999999995,"5min":0.55100000000000005,"15min":0.496},"max":{"1min":1.0609999999999999,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.1419999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.016,"15min":0.97399999999999998},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.105,"5min":14.775,"15min":14.775},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.1040000000000001,"15min":0.94599999999999995},"min":{"1min":0.72899999999999998,"5min":0.63800000000000001,"15min":0.57499999999999996},"max":{"1min":1.1919999999999999,"5min":14.901999999999999,"15min":14.901999999999999},"last":1.1919999999999999}]},{"osd":3,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.034,"15min":0.98899999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.54400000000000004},"max":{"1min":1.2529999999999999,"5min":15.154,"15min":15.154},"last":1.032},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.98999999999999999,"15min":1.0109999999999999},"min":{"1min":0.63900000000000001,"5min":0.60799999999999998,"15min":0.53500000000000003},"max":{"1min":1.286,"5min":3.4289999999999998,"15min":5.5170000000000003},"last":0.76100000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.526,"15min":1.181},"min":{"1min":0.64600000000000002,"5min":0.59299999999999997,"15min":0.51400000000000001},"max":{"1min":1.292,"5min":61.020000000000003,"15min":61.020000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.143,"15min":1.0920000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.2490000000000001,"5min":15.085000000000001,"15min":15.085000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.139,"15min":1.123},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.56699999999999995},"max":{"1min":1.5469999999999999,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.194,"15min":1.1699999999999999},"min":{"1min":1.052,"5min":0.72499999999999998,"15min":0.48899999999999999},"max":{"1min":1.6799999999999999,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.4970000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:56:11 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.1759999999999999,"15min":1.1639999999999999},"min":{"1min":0.94099999999999995,"5min":0.75600000000000001,"15min":0.56899999999999995},"max":{"1min":1.4219999999999999,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.054,"15min":1.0189999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.33200000000000002},"max":{"1min":1.085,"5min":6.2869999999999999,"15min":6.2869999999999999},"last":1.0780000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575121,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357644,"kb_used_data":236228,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":94014196,"statfs":{"total":96636764160,"available":96270536704,"internally_reserved":0,"allocated":241897472,"data_stored":321511706,"data_compressed":25475129,"data_compressed_allocated":30593024,"data_compressed_original":110510027,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":1.155,"15min":0.97399999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0449999999999999,"5min":43.139000000000003,"15min":43.139000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84399999999999997,"15min":0.89600000000000002},"min":{"1min":0.53400000000000003,"5min":0.499,"15min":0.39900000000000002},"max":{"1min":1.3049999999999999,"5min":4.444,"15min":17.446000000000002},"last":0.80600000000000005}]},{"osd":1,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.98799999999999999,"15min":1.054},"min":{"1min":0.64900000000000002,"5min":0.48199999999999998,"15min":0.44},"max":{"1min":1.2849999999999999,"5min":11.557,"15min":21.763999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0229999999999999,"15min":1.4790000000000001},"min":{"1min":0.65700000000000003,"5min":0.53800000000000003,"15min":0.46500000000000002},"max":{"1min":1.452,"5min":5.3479999999999999,"15min":78.649000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":0.96499999999999997,"15min":1.0529999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":10.880000000000001,"5min":10.880000000000001,"15min":28.341000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.073,"15min":1.548},"min":{"1min":0.61699999999999999,"5min":0.58299999999999996,"15min":0.57899999999999996},"max":{"1min":2.069,"5min":11.614000000000001,"15min":94.998999999999995},"last":0.86099999999999999}]},{"osd":3,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.208,"15min":1.103},"min":{"1min":0.86299999999999999,"5min":0.67500000000000004,"15min":0.51300000000000001},"max":{"1min":9.984,"5min":9.984,"15min":13.202999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.0609999999999999,"15min":1.056},"min":{"1min":0.69299999999999995,"5min":0.54800000000000004,"15min":0.45600000000000002},"max":{"1min":9.952,"5min":9.952,"15min":16.277999999999999},"last":1.0049999999999999}]},{"osd":5,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.0049999999999999,"15min":1.1259999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.28199999999999997},"max":{"1min":1.385,"5min":3.4409999999999998,"15min":17.033000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.3799999999999999,"15min":1.3},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.34000000000000002},"max":{"1min":1.1759999999999999,"5min":38.939,"15min":38.939},"last":1.1279999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.046,"15min":1.1699999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.37},"max":{"1min":1.4710000000000001,"5min":4.476,"15min":17.085000000000001},"last":1.036},{"interface":"front","average":{"1min":1.026,"5min":1.4850000000000001,"15min":1.3380000000000001},"min":{"1min":0.61399999999999999,"5min":0.58199999999999996,"15min":0.39000000000000001},"max":{"1min":1.631,"5min":43.420000000000002,"15min":43.420000000000002},"last":1.224}]},{"osd":7,"last update":"Tue Nov 28 21:56:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.121,"15min":1.6160000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.309},"max":{"1min":1.5269999999999999,"5min":3.2109999999999999,"15min":77.638999999999996},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.24,"15min":1.175},"min":{"1min":0.90200000000000002,"5min":0.75900000000000001,"15min":0.51500000000000001},"max":{"1min":1.798,"5min":3.613,"15min":5.6200000000000001},"last":1.2989999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575122,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461004,"kb_used_data":307652,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":93910836,"statfs":{"total":96636764160,"available":96164696064,"internally_reserved":0,"allocated":315035648,"data_stored":410723951,"data_compressed":36998413,"data_compressed_allocated":43757568,"data_compressed_original":139747328,"omap_allocated":1592,"internal_metadata":157022664},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.89500000000000002,"15min":0.70999999999999996},"min":{"1min":0.246,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.67900000000000005,"5min":36.585000000000001,"15min":36.585000000000001},"last":0.373},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.89500000000000002,"15min":0.67800000000000005},"min":{"1min":0.379,"5min":0.34300000000000003,"15min":0.33100000000000002},"max":{"1min":0.68400000000000005,"5min":30.065000000000001,"15min":30.065000000000001},"last":0.61899999999999999}]},{"osd":1,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60699999999999998,"15min":0.59999999999999998},"min":{"1min":0.37,"5min":0.33200000000000002,"15min":0.26300000000000001},"max":{"1min":0.876,"5min":4.8099999999999996,"15min":10.449},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.61799999999999999,"15min":0.67400000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.98599999999999999,"5min":4.8520000000000003,"15min":10.481999999999999},"last":0.40400000000000003}]},{"osd":2,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.59099999999999997,"15min":0.65300000000000002},"min":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.27900000000000003},"max":{"1min":0.70299999999999996,"5min":1.25,"15min":10.369},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.56799999999999995,"15min":0.69599999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78600000000000003,"5min":1.1240000000000001,"15min":10.406000000000001},"last":1.2170000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91700000000000004,"15min":1.361},"min":{"1min":0.71799999999999997,"5min":0.50700000000000001,"15min":0.376},"max":{"1min":1.0129999999999999,"5min":4.2119999999999997,"15min":110.39700000000001},"last":1.036},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":1.3919999999999999},"min":{"1min":0.71499999999999997,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.147,"5min":7.2030000000000003,"15min":108.777},"last":1.0549999999999999}]},{"osd":5,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.86199999999999999,"15min":0.92600000000000005},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.39900000000000002},"max":{"1min":0.98799999999999999,"5min":7.2679999999999998,"15min":17.513000000000002},"last":1.196},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.873,"15min":0.92900000000000005},"min":{"1min":0.621,"5min":0.621,"15min":0.52300000000000002},"max":{"1min":0.998,"5min":1.7070000000000001,"15min":10.516999999999999},"last":1.165}]},{"osd":6,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82399999999999995,"15min":1.4019999999999999},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.365,"15min":138.91300000000001},"last":1.097},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.875,"15min":0.94499999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.48299999999999998},"max":{"1min":1.0980000000000001,"5min":1.766,"15min":10.302},"last":1.1120000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.91900000000000004,"15min":0.95799999999999996},"min":{"1min":0.63100000000000001,"5min":0.60999999999999999,"15min":0.45200000000000001},"max":{"1min":1.1990000000000001,"5min":7.3040000000000003,"15min":17.263000000000002},"last":1.079},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.97799999999999998,"15min":1.034},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.49299999999999999},"max":{"1min":1.0289999999999999,"5min":7.3339999999999996,"15min":28.613},"last":1.1379999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575120,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288308,"kb_used_data":176620,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94083532,"statfs":{"total":96636764160,"available":96341536768,"internally_reserved":0,"allocated":180858880,"data_stored":223905413,"data_compressed":16019215,"data_compressed_allocated":18952192,"data_compressed_original":62304256,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.63500000000000001,"15min":0.58699999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.23599999999999999},"max":{"1min":1.6679999999999999,"5min":13.599,"15min":13.599},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.74299999999999999,"15min":0.63700000000000001},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.318},"max":{"1min":7.1159999999999997,"5min":8.0630000000000006,"15min":8.0630000000000006},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.32000000000000001,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":7.8970000000000002,"5min":7.8970000000000002,"15min":9.3260000000000005},"last":0.38},{"interface":"front","average":{"1min":1.173,"5min":0.81200000000000006,"15min":0.71299999999999997},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.34200000000000003},"max":{"1min":10.657,"5min":10.657,"15min":10.657},"last":0.41699999999999998}]},{"osd":3,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.046,"15min":0.81899999999999995},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.307},"max":{"1min":7.1639999999999997,"5min":29.123999999999999,"15min":29.123999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":1.498,"5min":0.80600000000000005,"15min":0.66500000000000004},"min":{"1min":0.36199999999999999,"5min":0.312,"15min":0.28000000000000003},"max":{"1min":10.18,"5min":10.18,"15min":10.18},"last":0.443}]},{"osd":4,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97299999999999998,"15min":0.93500000000000005},"min":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.49099999999999999},"max":{"1min":2.9769999999999999,"5min":5.6289999999999996,"15min":7.3630000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.002,"15min":0.92600000000000005},"min":{"1min":0.59299999999999997,"5min":0.51200000000000001,"15min":0.41399999999999998},"max":{"1min":12.723000000000001,"5min":12.723000000000001,"15min":12.723000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":0.94099999999999995,"15min":0.87},"min":{"1min":0.64900000000000002,"5min":0.501,"15min":0.501},"max":{"1min":6.875,"5min":6.875,"15min":6.875},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.2070000000000001,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.59399999999999997,"15min":0.48399999999999999},"max":{"1min":9.4809999999999999,"5min":9.4809999999999999,"15min":9.4809999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.093,"15min":0.92900000000000005},"min":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":3.5489999999999999,"5min":18.959,"15min":18.959},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.897,"5min":1.325,"15min":1.0369999999999999},"min":{"1min":0.77700000000000002,"5min":0.622,"15min":0.52200000000000002},"max":{"1min":9.4399999999999995,"5min":21.957999999999998,"15min":21.957999999999998},"last":0.77700000000000002}]},{"osd":7,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":1.0800000000000001,"15min":0.98099999999999998},"min":{"1min":0.64200000000000002,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":9.2249999999999996,"5min":9.2249999999999996,"15min":9.2799999999999994},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.7989999999999999,"5min":1.109,"15min":0.97199999999999998},"min":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":9.1679999999999993,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":0.61799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575120,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469272,"kb_used_data":311760,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93902568,"statfs":{"total":96636764160,"available":96156229632,"internally_reserved":0,"allocated":319242240,"data_stored":398403059,"data_compressed":30727022,"data_compressed_allocated":36495360,"data_compressed_original":115965952,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":0.75700000000000001,"15min":0.69599999999999995},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.246},"max":{"1min":10.096,"5min":12.311999999999999,"15min":14.538},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.623,"15min":0.65100000000000002},"min":{"1min":0.39000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.4119999999999999,"5min":8.2479999999999993,"15min":14.618},"last":0.53000000000000003}]},{"osd":2,"last update":"Tue Nov 28 21:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.66100000000000003,"15min":0.63100000000000001},"min":{"1min":0.40799999999999997,"5min":0.34899999999999998,"15min":0.31900000000000001},"max":{"1min":1.7150000000000001,"5min":5.1689999999999996,"15min":8.4749999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":4.7030000000000003,"15min":8.5150000000000006},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.75900000000000001,"15min":0.84699999999999998},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.27600000000000002},"max":{"1min":3.028,"5min":13.385,"15min":43.536999999999999},"last":0.438},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.84699999999999998},"min":{"1min":0.501,"5min":0.443,"15min":0.375},"max":{"1min":5.6050000000000004,"5min":8.2899999999999991,"15min":18.308},"last":0.69199999999999995}]},{"osd":4,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":0.997,"15min":0.95699999999999996},"min":{"1min":0.63500000000000001,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":9.9849999999999994,"5min":9.9849999999999994,"15min":9.9849999999999994},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.4670000000000001,"5min":0.97899999999999998,"15min":1.079},"min":{"1min":0.61699999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":10.287000000000001,"5min":10.287000000000001,"15min":43.497999999999998},"last":0.64600000000000002}]},{"osd":5,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.93100000000000005,"15min":0.91600000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":3.3740000000000001,"5min":3.3740000000000001,"15min":5.5350000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.016,"15min":0.99399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.448,"5min":3.448,"15min":5.7409999999999997},"last":0.89100000000000001}]},{"osd":6,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.0449999999999999,"15min":1.0109999999999999},"min":{"1min":0.69899999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":10.023,"5min":10.023,"15min":10.385},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.05,"15min":1.2470000000000001},"min":{"1min":0.77700000000000002,"5min":0.65100000000000002,"15min":0.55200000000000005},"max":{"1min":3.7480000000000002,"5min":5.9349999999999996,"15min":43.448999999999998},"last":0.86899999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0469999999999999,"15min":1.024},"min":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.56899999999999995},"max":{"1min":5.4109999999999996,"5min":5.9779999999999998,"15min":7.96},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.516,"5min":1.1899999999999999,"15min":1.1639999999999999},"min":{"1min":0.66500000000000004,"5min":0.55800000000000005,"15min":0.51500000000000001},"max":{"1min":10.063000000000001,"5min":15.672000000000001,"15min":42.473999999999997},"last":0.71699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575121,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":346236,"kb_used_data":216500,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94025604,"statfs":{"total":96636764160,"available":96282218496,"internally_reserved":0,"allocated":221696000,"data_stored":277678898,"data_compressed":19875137,"data_compressed_allocated":23535616,"data_compressed_original":79818752,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.7909999999999999,"5min":0.75,"15min":0.56399999999999995},"min":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.215},"max":{"1min":21.814,"5min":21.814,"15min":21.814},"last":0.35499999999999998},{"interface":"front","average":{"1min":1.8500000000000001,"5min":0.88300000000000001,"15min":0.624},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.248},"max":{"1min":21.925000000000001,"5min":21.925000000000001,"15min":21.925000000000001},"last":0.40899999999999997}]},{"osd":2,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.9399999999999999,"5min":0.93700000000000006,"15min":0.70199999999999996},"min":{"1min":0.47499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":21.852,"5min":21.852,"15min":21.852},"last":0.55000000000000004},{"interface":"front","average":{"1min":1.617,"5min":0.82099999999999995,"15min":0.67600000000000005},"min":{"1min":0.36399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":18.213000000000001,"5min":18.213000000000001,"15min":18.213000000000001},"last":0.53200000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.083,"15min":0.72199999999999998},"min":{"1min":0.495,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":21.98,"5min":21.98,"15min":21.98},"last":0.51500000000000001},{"interface":"front","average":{"1min":1.857,"5min":1.071,"15min":0.77600000000000002},"min":{"1min":0.437,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":20.931999999999999,"5min":20.931999999999999,"15min":20.931999999999999},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.98599999999999999,"15min":0.92500000000000004},"min":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.42899999999999999},"max":{"1min":4.0810000000000004,"5min":5.2069999999999999,"15min":5.734},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.9379999999999999,"5min":1.1000000000000001,"15min":0.97399999999999998},"min":{"1min":0.65100000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":17.98,"5min":17.98,"15min":17.98},"last":0.81999999999999995}]},{"osd":5,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":2.1960000000000002,"5min":1.1919999999999999,"15min":0.93400000000000005},"min":{"1min":0.65800000000000003,"5min":0.61799999999999999,"15min":0.40699999999999997},"max":{"1min":21.890999999999998,"5min":21.890999999999998,"15min":21.890999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.931,"5min":1.1519999999999999,"15min":0.95799999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":18.181000000000001,"5min":18.181000000000001,"15min":18.181000000000001},"last":1.0409999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.056,"15min":0.91500000000000004},"min":{"1min":0.63,"5min":0.61799999999999999,"15min":0.48699999999999999},"max":{"1min":4.0410000000000004,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.872},{"interface":"front","average":{"1min":1.9670000000000001,"5min":1.1399999999999999,"15min":0.95799999999999996},"min":{"1min":0.68600000000000005,"5min":0.60499999999999998,"15min":0.55100000000000005},"max":{"1min":18.030999999999999,"5min":18.030999999999999,"15min":18.030999999999999},"last":0.71099999999999997}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.90800000000000003},"min":{"1min":0.627,"5min":0.627,"15min":0.48999999999999999},"max":{"1min":1.6950000000000001,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":0.75},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.95299999999999996,"15min":0.89600000000000002},"min":{"1min":0.68799999999999994,"5min":0.63,"15min":0.55700000000000005},"max":{"1min":1.627,"5min":3.1819999999999999,"15min":3.806},"last":0.91300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14426112,"data_stored":54526612,"data_compressed":7767250,"data_compressed_allocated":9949184,"data_compressed_original":50065408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24399872,"data_stored":78707578,"data_compressed":10468356,"data_compressed_allocated":13627392,"data_compressed_original":67960832,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15634432,"data_stored":48245772,"data_compressed":6171423,"data_compressed_allocated":8048640,"data_compressed_original":40681472,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25956352,"data_stored":94383094,"data_compressed":13023761,"data_compressed_allocated":16736256,"data_compressed_original":85180416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23392256,"data_stored":83887777,"data_compressed":11283626,"data_compressed_allocated":14446592,"data_compressed_original":74960843,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22241280,"data_stored":79693154,"data_compressed":11120127,"data_compressed_allocated":14139392,"data_compressed_original":71610368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":22917120,"data_stored":83888413,"data_compressed":11073880,"data_compressed_allocated":14143488,"data_compressed_original":75132875,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24055808,"data_stored":82901666,"data_compressed":11611289,"data_compressed_allocated":14708736,"data_compressed_original":73572352,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":206610432,"data_stored":222244864,"data_compressed":12071885,"data_compressed_allocated":13537280,"data_compressed_original":29171712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294244352,"data_stored":319377408,"data_compressed":20255951,"data_compressed_allocated":22863872,"data_compressed_original":47996928,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164626432,"data_stored":175341568,"data_compressed":9845077,"data_compressed_allocated":10899456,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":288481280,"data_stored":316022784,"data_compressed":23971937,"data_compressed_allocated":27017216,"data_compressed_original":54558720,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":217907200,"data_stored":237305856,"data_compressed":14188788,"data_compressed_allocated":16142336,"data_compressed_original":35540992,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":220975104,"data_stored":243601408,"data_compressed":17073954,"data_compressed_allocated":19296256,"data_compressed_original":41922560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":237195264,"data_stored":251670528,"data_compressed":13556545,"data_compressed_allocated":15155200,"data_compressed_original":29630464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":192204800,"data_stored":208740352,"data_compressed":13315951,"data_compressed_allocated":15101952,"data_compressed_original":31637504,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:57:03.969 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.969 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.969 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.969 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.969 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.970 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.970 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.970 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.970 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.970 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.971 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.971 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.971 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.971 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.971 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.972 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.972 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.972 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.972 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.972 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.972 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.973 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.973 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.973 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.973 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.973 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2023-11-28T21:34:34.961186+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=34, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.974 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.974 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.974 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.974 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.974 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.974 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.975 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.975 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.975 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.975 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.975 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.975 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.976 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.976 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.976 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.976 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.976 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.977 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.977 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.977 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.977 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.977 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.977 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.978 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.978 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.978 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.978 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.978 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.979 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.979 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.979 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.979 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.979 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.979 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.980 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.980 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.980 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.980 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.980 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.981 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.981 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.981 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.981 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.981 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.981 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.982 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.982 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.982 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.982 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.982 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:03.982 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T21:57:17.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:57:17.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:57:23.983 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:57:24.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.124+0000 7f4e12b9d640 1 Processor -- start 2023-11-28T21:57:24.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.124+0000 7f4e12b9d640 1 -- start start 2023-11-28T21:57:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e12b9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0c151af0 0x7f4e0c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e12b9d640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e0c06e300 0x7f4e0c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e12b9d640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0c157b70 con 0x7f4e0c151ed0 2023-11-28T21:57:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e12b9d640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0c0707f0 con 0x7f4e0c06e300 2023-11-28T21:57:24.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e12b9d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0c7b90 con 0x7f4e0c151af0 2023-11-28T21:57:24.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e1113a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e0c06e300 0x7f4e0c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e0bfff640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f4e0c151ed0 0x7f4e0c0c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.12:43730/0 (socket says 172.21.15.12:43730) 2023-11-28T21:57:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e0bfff640 1 -- 172.21.15.12:0/187190080 learned_addr learned my addr 172.21.15.12:0/187190080 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:57:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e10939640 1 --2- 172.21.15.12:0/187190080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0c151af0 0x7f4e0c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e1113a640 1 -- 172.21.15.12:0/187190080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0c151af0 msgr2=0x7f4e0c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:57:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e1113a640 1 --2- 172.21.15.12:0/187190080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0c151af0 0x7f4e0c0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e1113a640 1 -- 172.21.15.12:0/187190080 >> v1:172.21.15.144:6789/0 conn(0x7f4e0c151ed0 legacy=0x7f4e0c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:24.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.128+0000 7f4e1113a640 1 -- 172.21.15.12:0/187190080 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c157e50 con 0x7f4e0c06e300 2023-11-28T21:57:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e1113a640 1 --2- 172.21.15.12:0/187190080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e0c06e300 0x7f4e0c072070 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009d90 tx=0x7f4e0401c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e14dc485ec126f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:57:24.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e0b7fe640 1 -- 172.21.15.12:0/187190080 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e040a5810 con 0x7f4e0c06e300 2023-11-28T21:57:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e0b7fe640 1 -- 172.21.15.12:0/187190080 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e040a5970 con 0x7f4e0c06e300 2023-11-28T21:57:24.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e0b7fe640 1 -- 172.21.15.12:0/187190080 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e040a5d00 con 0x7f4e0c06e300 2023-11-28T21:57:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e12b9d640 1 -- 172.21.15.12:0/187190080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e0c06e300 msgr2=0x7f4e0c072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e12b9d640 1 --2- 172.21.15.12:0/187190080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e0c06e300 0x7f4e0c072070 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009d90 tx=0x7f4e0401c950 comp rx=0 tx=0).stop 2023-11-28T21:57:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e12b9d640 1 -- 172.21.15.12:0/187190080 shutdown_connections 2023-11-28T21:57:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e12b9d640 1 --2- 172.21.15.12:0/187190080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e0c06e300 0x7f4e0c072070 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e12b9d640 1 --2- 172.21.15.12:0/187190080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0c151af0 0x7f4e0c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:24.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e12b9d640 1 -- 172.21.15.12:0/187190080 >> 172.21.15.12:0/187190080 conn(0x7f4e0c066360 msgr2=0x7f4e0c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:57:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e12b9d640 1 -- 172.21.15.12:0/187190080 shutdown_connections 2023-11-28T21:57:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.132+0000 7f4e12b9d640 1 -- 172.21.15.12:0/187190080 wait complete. 2023-11-28T21:57:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 Processor -- start 2023-11-28T21:57:24.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 -- start start 2023-11-28T21:57:24.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e0c06e300 0x7f4e0c0c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e0c151af0 0x7f4e0c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0c157560 0x7f4e0c14d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c156c90 con 0x7f4e0c151af0 2023-11-28T21:57:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c13cac0 con 0x7f4e0c06e300 2023-11-28T21:57:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c14b910 con 0x7f4e0c157560 2023-11-28T21:57:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e0bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e0c151af0 0x7f4e0c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e0bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e0c151af0 0x7f4e0c155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49346/0 (socket says 172.21.15.12:49346) 2023-11-28T21:57:24.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e0bfff640 1 -- 172.21.15.12:0/2001642530 learned_addr learned my addr 172.21.15.12:0/2001642530 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:57:24.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e10939640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e0c06e300 0x7f4e0c0c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:24.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e1113a640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0c157560 0x7f4e0c14d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:24.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e0bfff640 1 -- 172.21.15.12:0/2001642530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0c157560 msgr2=0x7f4e0c14d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:24.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e0bfff640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0c157560 0x7f4e0c14d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:24.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e0bfff640 1 -- 172.21.15.12:0/2001642530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e0c06e300 msgr2=0x7f4e0c0c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:24.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e0bfff640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e0c06e300 0x7f4e0c0c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:24.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e0bfff640 1 -- 172.21.15.12:0/2001642530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04009a70 con 0x7f4e0c151af0 2023-11-28T21:57:24.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e0bfff640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e0c151af0 0x7f4e0c155870 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4df8001ec0 tx=0x7f4df800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:57:24.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e097fa640 1 -- 172.21.15.12:0/2001642530 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8005cf0 con 0x7f4e0c151af0 2023-11-28T21:57:24.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e097fa640 1 -- 172.21.15.12:0/2001642530 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df8005e50 con 0x7f4e0c151af0 2023-11-28T21:57:24.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e097fa640 1 -- 172.21.15.12:0/2001642530 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df80072d0 con 0x7f4e0c151af0 2023-11-28T21:57:24.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 -- 172.21.15.12:0/2001642530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0c1f3ae0 con 0x7f4e0c151af0 2023-11-28T21:57:24.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 -- 172.21.15.12:0/2001642530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c13c470 con 0x7f4e0c151af0 2023-11-28T21:57:24.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.136+0000 7f4e12b9d640 1 -- 172.21.15.12:0/2001642530 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0c13cc70 con 0x7f4e0c151af0 2023-11-28T21:57:24.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.140+0000 7f4e097fa640 1 -- 172.21.15.12:0/2001642530 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4df8007430 con 0x7f4e0c151af0 2023-11-28T21:57:24.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.140+0000 7f4e097fa640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4df00664a0 0x7f4df0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:24.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.140+0000 7f4e097fa640 1 -- 172.21.15.12:0/2001642530 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f4df8083a30 con 0x7f4e0c151af0 2023-11-28T21:57:24.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.140+0000 7f4e10939640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4df00664a0 0x7f4df0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:24.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.140+0000 7f4e10939640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4df00664a0 0x7f4df0068960 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009b60 tx=0x7f4e00009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:57:24.151 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.148+0000 7f4e097fa640 1 -- 172.21.15.12:0/2001642530 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e0c13cc70 con 0x7f4e0c151af0 2023-11-28T21:57:24.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.304+0000 7f4e12b9d640 1 -- 172.21.15.12:0/2001642530 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e0c13c2f0 con 0x7f4df00664a0 2023-11-28T21:57:24.317 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.312+0000 7f4e097fa640 1 -- 172.21.15.12:0/2001642530 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+375324 (secure 0 0 0) 0x7f4e00011ac0 con 0x7f4df00664a0 2023-11-28T21:57:24.322 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:57:24.324 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:57:24.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 -- 172.21.15.12:0/2001642530 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4df00664a0 msgr2=0x7f4df0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:24.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4df00664a0 0x7f4df0068960 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009b60 tx=0x7f4e00009190 comp rx=0 tx=0).stop 2023-11-28T21:57:24.325 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 -- 172.21.15.12:0/2001642530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e0c151af0 msgr2=0x7f4e0c155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:24.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e0c151af0 0x7f4e0c155870 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4df8001ec0 tx=0x7f4df800c890 comp rx=0 tx=0).stop 2023-11-28T21:57:24.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 -- 172.21.15.12:0/2001642530 shutdown_connections 2023-11-28T21:57:24.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f4df00664a0 0x7f4df0068960 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:24.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e0c157560 0x7f4e0c14d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:24.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4e0c151af0 0x7f4e0c155870 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:24.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 --2- 172.21.15.12:0/2001642530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e0c06e300 0x7f4e0c0c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:24.326 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 -- 172.21.15.12:0/2001642530 >> 172.21.15.12:0/2001642530 conn(0x7f4e0c066360 msgr2=0x7f4e0c0c7420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:57:24.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 -- 172.21.15.12:0/2001642530 shutdown_connections 2023-11-28T21:57:24.327 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:24.320+0000 7f4e12b9d640 1 -- 172.21.15.12:0/2001642530 wait complete. 2023-11-28T21:57:24.344 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":732,"stamp":"2023-11-28T21:57:24.224348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1231729201,"num_objects":10538,"num_object_clones":0,"num_object_copies":41756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10538,"num_whiteouts":0,"num_read":9144,"num_read_kb":2407205,"num_write":56636,"num_write_kb":2360182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":687608,"num_omap_keys":1199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77304,"ondisk_log_size":77304,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2999660,"kb_used_data":1953388,"kb_used_omap":12,"kb_used_meta":1046195,"kb_avail":751975060,"statfs":{"total":773094113280,"available":770022461440,"internally_reserved":0,"allocated":2000269312,"data_stored":2584577565,"data_compressed":206919120,"data_compressed_allocated":245977088,"data_compressed_original":832720790,"omap_allocated":12751,"internal_metadata":1071304241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":91931,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002921"},"pg_stats":[{"pgid":"2.1f","version":"52'347","reported_seq":442,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.479830+0000","last_change":"2023-11-28T21:56:50.479855+0000","last_active":"2023-11-28T21:56:50.479830+0000","last_peered":"2023-11-28T21:56:50.479830+0000","last_clean":"2023-11-28T21:56:50.479830+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:56:50.479830+0000","last_undegraded":"2023-11-28T21:56:50.479830+0000","last_fullsized":"2023-11-28T21:56:50.479830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'347","last_scrub_stamp":"2023-11-28T21:56:50.479765+0000","last_deep_scrub":"52'347","last_deep_scrub_stamp":"2023-11-28T21:56:50.479765+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.479765+0000","objects_scrubbed":6,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:00.234465+0000","scrub_duration":0.028234239000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":61536,"num_write":310,"num_write_kb":8499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":41082,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"52'277","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.442324+0000","last_change":"2023-11-28T21:57:04.442342+0000","last_active":"2023-11-28T21:57:04.442324+0000","last_peered":"2023-11-28T21:57:04.442324+0000","last_clean":"2023-11-28T21:57:04.442324+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:57:04.442324+0000","last_undegraded":"2023-11-28T21:57:04.442324+0000","last_fullsized":"2023-11-28T21:57:04.442324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'277","last_scrub_stamp":"2023-11-28T21:57:04.442253+0000","last_deep_scrub":"52'277","last_deep_scrub_stamp":"2023-11-28T21:57:04.442253+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.442253+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:47:27.933208+0000","scrub_duration":0.0025024589999999998,"objects_trimmed":0,"snaptrim_duration":0.28950874199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"52'1318","reported_seq":3629,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.060425+0000","last_change":"2023-11-28T21:40:42.803511+0000","last_active":"2023-11-28T21:56:35.060425+0000","last_peered":"2023-11-28T21:56:35.060425+0000","last_clean":"2023-11-28T21:56:35.060425+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:56:35.060425+0000","last_undegraded":"2023-11-28T21:56:35.060425+0000","last_fullsized":"2023-11-28T21:56:35.060425+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29329539100000002,"stat_sum":{"num_bytes":30193112,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":229,"num_read_kb":12319,"num_write":1291,"num_write_kb":57664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"52'534","reported_seq":618,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:54.494340+0000","last_change":"2023-11-28T21:56:54.494393+0000","last_active":"2023-11-28T21:56:54.494340+0000","last_peered":"2023-11-28T21:56:54.494340+0000","last_clean":"2023-11-28T21:56:54.494340+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:56:54.494340+0000","last_undegraded":"2023-11-28T21:56:54.494340+0000","last_fullsized":"2023-11-28T21:56:54.494340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'534","last_scrub_stamp":"2023-11-28T21:56:54.494241+0000","last_deep_scrub":"52'534","last_deep_scrub_stamp":"2023-11-28T21:56:54.494241+0000","last_clean_scrub_stamp":"2023-11-28T21:56:54.494241+0000","objects_scrubbed":7,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:51:32.158044+0000","scrub_duration":0.057898067999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":81920,"num_write":485,"num_write_kb":24906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8778,"num_omap_keys":17,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"52'269","reported_seq":2361,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:42.409919+0000","last_change":"2023-11-28T21:56:42.409939+0000","last_active":"2023-11-28T21:56:42.409919+0000","last_peered":"2023-11-28T21:56:42.409919+0000","last_clean":"2023-11-28T21:56:42.409919+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:56:42.409919+0000","last_undegraded":"2023-11-28T21:56:42.409919+0000","last_fullsized":"2023-11-28T21:56:42.409919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'269","last_scrub_stamp":"2023-11-28T21:56:42.409827+0000","last_deep_scrub":"52'269","last_deep_scrub_stamp":"2023-11-28T21:56:42.409827+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.409827+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:28:29.334975+0000","scrub_duration":0.0029008010000000002,"objects_trimmed":0,"snaptrim_duration":0.23094383900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"52'1401","reported_seq":3755,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.530886+0000","last_change":"2023-11-28T21:57:17.530942+0000","last_active":"2023-11-28T21:57:17.530886+0000","last_peered":"2023-11-28T21:57:17.530886+0000","last_clean":"2023-11-28T21:57:17.530886+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:57:17.530886+0000","last_undegraded":"2023-11-28T21:57:17.530886+0000","last_fullsized":"2023-11-28T21:57:17.530886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1401","last_scrub_stamp":"2023-11-28T21:57:17.530763+0000","last_deep_scrub":"52'1401","last_deep_scrub_stamp":"2023-11-28T21:57:17.530763+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.530763+0000","objects_scrubbed":330,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:05:20.452581+0000","scrub_duration":0.19727914399999999,"objects_trimmed":0,"snaptrim_duration":0.21613474099999999,"stat_sum":{"num_bytes":11729902,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":251,"num_read_kb":24615,"num_write":1375,"num_write_kb":48688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"52'764","reported_seq":867,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.611643+0000","last_change":"2023-11-28T21:56:50.611643+0000","last_active":"2023-11-28T21:56:50.611643+0000","last_peered":"2023-11-28T21:56:50.611643+0000","last_clean":"2023-11-28T21:56:50.611643+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:56:50.611643+0000","last_undegraded":"2023-11-28T21:56:50.611643+0000","last_fullsized":"2023-11-28T21:56:50.611643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'764","last_scrub_stamp":"2023-11-28T21:56:50.611223+0000","last_deep_scrub":"52'764","last_deep_scrub_stamp":"2023-11-28T21:56:50.611223+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.611223+0000","objects_scrubbed":5,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:28:25.261471+0000","scrub_duration":0.059433334999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":59,"num_read_kb":131099,"num_write":715,"num_write_kb":20835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13932,"num_omap_keys":27,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"52'294","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.715849+0000","last_change":"2023-11-28T21:57:17.715901+0000","last_active":"2023-11-28T21:57:17.715849+0000","last_peered":"2023-11-28T21:57:17.715849+0000","last_clean":"2023-11-28T21:57:17.715849+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:57:17.715849+0000","last_undegraded":"2023-11-28T21:57:17.715849+0000","last_fullsized":"2023-11-28T21:57:17.715849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'294","last_scrub_stamp":"2023-11-28T21:57:17.715645+0000","last_deep_scrub":"52'294","last_deep_scrub_stamp":"2023-11-28T21:57:17.715645+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.715645+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:16:17.284423+0000","scrub_duration":0.0037761639999999998,"objects_trimmed":0,"snaptrim_duration":0.30458173900000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"52'1324","reported_seq":3652,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.623143+0000","last_change":"2023-11-28T21:56:57.623143+0000","last_active":"2023-11-28T21:56:57.623143+0000","last_peered":"2023-11-28T21:56:57.623143+0000","last_clean":"2023-11-28T21:56:57.623143+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:56:57.623143+0000","last_undegraded":"2023-11-28T21:56:57.623143+0000","last_fullsized":"2023-11-28T21:56:57.623143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1324","last_scrub_stamp":"2023-11-28T21:56:57.622774+0000","last_deep_scrub":"52'1324","last_deep_scrub_stamp":"2023-11-28T21:56:57.622774+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.622774+0000","objects_scrubbed":314,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:08:12.479556+0000","scrub_duration":0.25549122899999999,"objects_trimmed":0,"snaptrim_duration":0.23712909900000001,"stat_sum":{"num_bytes":32661416,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":226,"num_read_kb":7690,"num_write":1300,"num_write_kb":56471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"52'96","reported_seq":176,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:47.599947+0000","last_change":"2023-11-28T21:56:47.599947+0000","last_active":"2023-11-28T21:56:47.599947+0000","last_peered":"2023-11-28T21:56:47.599947+0000","last_clean":"2023-11-28T21:56:47.599947+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:56:47.599947+0000","last_undegraded":"2023-11-28T21:56:47.599947+0000","last_fullsized":"2023-11-28T21:56:47.599947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'96","last_scrub_stamp":"2023-11-28T21:56:47.599552+0000","last_deep_scrub":"52'96","last_deep_scrub_stamp":"2023-11-28T21:56:47.599552+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.599552+0000","objects_scrubbed":2,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:41:39.184546+0000","scrub_duration":0.012005902000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36891,"num_write":51,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10256,"num_omap_keys":19,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"52'283","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.791963+0000","last_change":"2023-11-28T21:57:09.791963+0000","last_active":"2023-11-28T21:57:09.791963+0000","last_peered":"2023-11-28T21:57:09.791963+0000","last_clean":"2023-11-28T21:57:09.791963+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:57:09.791963+0000","last_undegraded":"2023-11-28T21:57:09.791963+0000","last_fullsized":"2023-11-28T21:57:09.791963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:57:09.791711+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:57:09.791711+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.791711+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:05.807760+0000","scrub_duration":0.0033624179999999998,"objects_trimmed":0,"snaptrim_duration":0.30416134,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"52'1294","reported_seq":3632,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:56.624661+0000","last_change":"2023-11-28T21:56:56.624661+0000","last_active":"2023-11-28T21:56:56.624661+0000","last_peered":"2023-11-28T21:56:56.624661+0000","last_clean":"2023-11-28T21:56:56.624661+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:56:56.624661+0000","last_undegraded":"2023-11-28T21:56:56.624661+0000","last_fullsized":"2023-11-28T21:56:56.624661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1294","last_scrub_stamp":"2023-11-28T21:56:56.624359+0000","last_deep_scrub":"52'1294","last_deep_scrub_stamp":"2023-11-28T21:56:56.624359+0000","last_clean_scrub_stamp":"2023-11-28T21:56:56.624359+0000","objects_scrubbed":312,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:59:20.046078+0000","scrub_duration":0.25756709,"objects_trimmed":0,"snaptrim_duration":0.26667665699999998,"stat_sum":{"num_bytes":37281632,"num_objects":312,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":0,"num_read":236,"num_read_kb":13169,"num_write":1275,"num_write_kb":61269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"52'303","reported_seq":2387,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.944343+0000","last_change":"2023-11-28T21:40:42.917342+0000","last_active":"2023-11-28T21:56:35.944343+0000","last_peered":"2023-11-28T21:56:35.944343+0000","last_clean":"2023-11-28T21:56:35.944343+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:56:35.944343+0000","last_undegraded":"2023-11-28T21:56:35.944343+0000","last_fullsized":"2023-11-28T21:56:35.944343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.200606007,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"52'627","reported_seq":731,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.667245+0000","last_change":"2023-11-28T21:56:48.667245+0000","last_active":"2023-11-28T21:56:48.667245+0000","last_peered":"2023-11-28T21:56:48.667245+0000","last_clean":"2023-11-28T21:56:48.667245+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:56:48.667245+0000","last_undegraded":"2023-11-28T21:56:48.667245+0000","last_fullsized":"2023-11-28T21:56:48.667245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'627","last_scrub_stamp":"2023-11-28T21:56:48.666890+0000","last_deep_scrub":"52'627","last_deep_scrub_stamp":"2023-11-28T21:56:48.666890+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.666890+0000","objects_scrubbed":10,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:43:39.567306+0000","scrub_duration":0.095730051999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":114718,"num_write":571,"num_write_kb":20791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64058,"num_omap_keys":115,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"52'1331","reported_seq":3673,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.617376+0000","last_change":"2023-11-28T21:57:01.617376+0000","last_active":"2023-11-28T21:57:01.617376+0000","last_peered":"2023-11-28T21:57:01.617376+0000","last_clean":"2023-11-28T21:57:01.617376+0000","last_became_active":"2023-11-28T21:35:25.616978+0000","last_became_peered":"2023-11-28T21:35:25.616978+0000","last_unstale":"2023-11-28T21:57:01.617376+0000","last_undegraded":"2023-11-28T21:57:01.617376+0000","last_fullsized":"2023-11-28T21:57:01.617376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1331","last_scrub_stamp":"2023-11-28T21:57:01.617097+0000","last_deep_scrub":"52'1331","last_deep_scrub_stamp":"2023-11-28T21:57:01.617097+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.617097+0000","objects_scrubbed":332,"log_size":1331,"log_dups_size":0,"ondisk_log_size":1331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:25.432186+0000","scrub_duration":0.26810302699999999,"objects_trimmed":0,"snaptrim_duration":0.23129675,"stat_sum":{"num_bytes":32707084,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":229,"num_read_kb":4127,"num_write":1308,"num_write_kb":57872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"52'305","reported_seq":2397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:00.459939+0000","last_change":"2023-11-28T21:57:00.459939+0000","last_active":"2023-11-28T21:57:00.459939+0000","last_peered":"2023-11-28T21:57:00.459939+0000","last_clean":"2023-11-28T21:57:00.459939+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:57:00.459939+0000","last_undegraded":"2023-11-28T21:57:00.459939+0000","last_fullsized":"2023-11-28T21:57:00.459939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'305","last_scrub_stamp":"2023-11-28T21:57:00.459826+0000","last_deep_scrub":"52'305","last_deep_scrub_stamp":"2023-11-28T21:57:00.459826+0000","last_clean_scrub_stamp":"2023-11-28T21:57:00.459826+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:07.320593+0000","scrub_duration":0.0022599199999999999,"objects_trimmed":0,"snaptrim_duration":0.28777871399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"52'962","reported_seq":1049,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.736149+0000","last_change":"2023-11-28T21:57:01.736178+0000","last_active":"2023-11-28T21:57:01.736149+0000","last_peered":"2023-11-28T21:57:01.736149+0000","last_clean":"2023-11-28T21:57:01.736149+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:57:01.736149+0000","last_undegraded":"2023-11-28T21:57:01.736149+0000","last_fullsized":"2023-11-28T21:57:01.736149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'962","last_scrub_stamp":"2023-11-28T21:57:01.736083+0000","last_deep_scrub":"52'962","last_deep_scrub_stamp":"2023-11-28T21:57:01.736083+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.736083+0000","objects_scrubbed":7,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:06:38.504425+0000","scrub_duration":0.040000139999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":75782,"num_write":935,"num_write_kb":13441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31916,"num_omap_keys":58,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"52'1416","reported_seq":3764,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:59.657597+0000","last_change":"2023-11-28T21:56:59.657597+0000","last_active":"2023-11-28T21:56:59.657597+0000","last_peered":"2023-11-28T21:56:59.657597+0000","last_clean":"2023-11-28T21:56:59.657597+0000","last_became_active":"2023-11-28T21:35:25.790674+0000","last_became_peered":"2023-11-28T21:35:25.790674+0000","last_unstale":"2023-11-28T21:56:59.657597+0000","last_undegraded":"2023-11-28T21:56:59.657597+0000","last_fullsized":"2023-11-28T21:56:59.657597+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1416","last_scrub_stamp":"2023-11-28T21:56:59.657277+0000","last_deep_scrub":"52'1416","last_deep_scrub_stamp":"2023-11-28T21:56:59.657277+0000","last_clean_scrub_stamp":"2023-11-28T21:56:59.657277+0000","objects_scrubbed":354,"log_size":1416,"log_dups_size":0,"ondisk_log_size":1416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:47.719457+0000","scrub_duration":0.27211498200000001,"objects_trimmed":0,"snaptrim_duration":0.229623311,"stat_sum":{"num_bytes":17731185,"num_objects":354,"num_object_clones":0,"num_object_copies":1416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":234,"num_read_kb":20007,"num_write":1382,"num_write_kb":71863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"52'306","reported_seq":2398,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.421624+0000","last_change":"2023-11-28T21:56:43.421641+0000","last_active":"2023-11-28T21:56:43.421624+0000","last_peered":"2023-11-28T21:56:43.421624+0000","last_clean":"2023-11-28T21:56:43.421624+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:56:43.421624+0000","last_undegraded":"2023-11-28T21:56:43.421624+0000","last_fullsized":"2023-11-28T21:56:43.421624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'306","last_scrub_stamp":"2023-11-28T21:56:43.421546+0000","last_deep_scrub":"52'306","last_deep_scrub_stamp":"2023-11-28T21:56:43.421546+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.421546+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:18:49.309763+0000","scrub_duration":0.001616658,"objects_trimmed":0,"snaptrim_duration":0.29027087200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"52'741","reported_seq":818,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:58.924936+0000","last_change":"2023-11-28T21:56:42.774090+0000","last_active":"2023-11-28T21:56:58.924936+0000","last_peered":"2023-11-28T21:56:58.924936+0000","last_clean":"2023-11-28T21:56:58.924936+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:56:58.924936+0000","last_undegraded":"2023-11-28T21:56:58.924936+0000","last_fullsized":"2023-11-28T21:56:58.924936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'734","last_scrub_stamp":"2023-11-28T21:56:42.773996+0000","last_deep_scrub":"52'734","last_deep_scrub_stamp":"2023-11-28T21:56:42.773996+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.773996+0000","objects_scrubbed":6,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:44:34.363577+0000","scrub_duration":0.055636247999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11383300,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":32782,"num_write":691,"num_write_kb":11471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8864,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"52'1341","reported_seq":3686,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:39.877156+0000","last_change":"2023-11-28T21:40:42.890432+0000","last_active":"2023-11-28T21:56:39.877156+0000","last_peered":"2023-11-28T21:56:39.877156+0000","last_clean":"2023-11-28T21:56:39.877156+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:56:39.877156+0000","last_undegraded":"2023-11-28T21:56:39.877156+0000","last_fullsized":"2023-11-28T21:56:39.877156+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1341,"log_dups_size":0,"ondisk_log_size":1341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.24643899999999999,"stat_sum":{"num_bytes":14380393,"num_objects":336,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":262,"num_read_kb":16758,"num_write":1318,"num_write_kb":45879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"52'294","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.274947+0000","last_change":"2023-11-28T21:56:48.275049+0000","last_active":"2023-11-28T21:56:48.274947+0000","last_peered":"2023-11-28T21:56:48.274947+0000","last_clean":"2023-11-28T21:56:48.274947+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:56:48.274947+0000","last_undegraded":"2023-11-28T21:56:48.274947+0000","last_fullsized":"2023-11-28T21:56:48.274947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'294","last_scrub_stamp":"2023-11-28T21:56:48.274731+0000","last_deep_scrub":"52'294","last_deep_scrub_stamp":"2023-11-28T21:56:48.274731+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.274731+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:02:56.783147+0000","scrub_duration":0.0035010150000000001,"objects_trimmed":0,"snaptrim_duration":0.21610210399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"52'343","reported_seq":422,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.528829+0000","last_change":"2023-11-28T21:56:57.528897+0000","last_active":"2023-11-28T21:56:57.528829+0000","last_peered":"2023-11-28T21:56:57.528829+0000","last_clean":"2023-11-28T21:56:57.528829+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:56:57.528829+0000","last_undegraded":"2023-11-28T21:56:57.528829+0000","last_fullsized":"2023-11-28T21:56:57.528829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'343","last_scrub_stamp":"2023-11-28T21:56:57.528632+0000","last_deep_scrub":"52'343","last_deep_scrub_stamp":"2023-11-28T21:56:57.528632+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.528632+0000","objects_scrubbed":1,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:11:06.480768+0000","scrub_duration":0.0086859099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":49164,"num_write":266,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"52'1262","reported_seq":3571,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.616628+0000","last_change":"2023-11-28T21:57:16.616828+0000","last_active":"2023-11-28T21:57:16.616628+0000","last_peered":"2023-11-28T21:57:16.616628+0000","last_clean":"2023-11-28T21:57:16.616628+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:57:16.616628+0000","last_undegraded":"2023-11-28T21:57:16.616628+0000","last_fullsized":"2023-11-28T21:57:16.616628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1262","last_scrub_stamp":"2023-11-28T21:57:16.616299+0000","last_deep_scrub":"52'1262","last_deep_scrub_stamp":"2023-11-28T21:57:16.616299+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.616299+0000","objects_scrubbed":318,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:23:10.854185+0000","scrub_duration":0.284423806,"objects_trimmed":0,"snaptrim_duration":0.244586779,"stat_sum":{"num_bytes":50267474,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":206,"num_read_kb":15432,"num_write":1240,"num_write_kb":82194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"52'302","reported_seq":2394,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:05.285530+0000","last_change":"2023-11-28T21:57:05.285554+0000","last_active":"2023-11-28T21:57:05.285530+0000","last_peered":"2023-11-28T21:57:05.285530+0000","last_clean":"2023-11-28T21:57:05.285530+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:57:05.285530+0000","last_undegraded":"2023-11-28T21:57:05.285530+0000","last_fullsized":"2023-11-28T21:57:05.285530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'302","last_scrub_stamp":"2023-11-28T21:57:05.285424+0000","last_deep_scrub":"52'302","last_deep_scrub_stamp":"2023-11-28T21:57:05.285424+0000","last_clean_scrub_stamp":"2023-11-28T21:57:05.285424+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:27.928951+0000","scrub_duration":0.0020171199999999999,"objects_trimmed":0,"snaptrim_duration":0.26728515899999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"52'172","reported_seq":253,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.150811+0000","last_change":"2023-11-28T21:56:49.150879+0000","last_active":"2023-11-28T21:56:49.150811+0000","last_peered":"2023-11-28T21:56:49.150811+0000","last_clean":"2023-11-28T21:56:49.150811+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:56:49.150811+0000","last_undegraded":"2023-11-28T21:56:49.150811+0000","last_fullsized":"2023-11-28T21:56:49.150811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'172","last_scrub_stamp":"2023-11-28T21:56:49.150620+0000","last_deep_scrub":"52'172","last_deep_scrub_stamp":"2023-11-28T21:56:49.150620+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.150620+0000","objects_scrubbed":4,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:13:50.517958+0000","scrub_duration":0.035921929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":45,"num_read_kb":24615,"num_write":148,"num_write_kb":4216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6802,"num_omap_keys":13,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"52'1368","reported_seq":3721,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.407631+0000","last_change":"2023-11-28T21:57:11.407745+0000","last_active":"2023-11-28T21:57:11.407631+0000","last_peered":"2023-11-28T21:57:11.407631+0000","last_clean":"2023-11-28T21:57:11.407631+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:57:11.407631+0000","last_undegraded":"2023-11-28T21:57:11.407631+0000","last_fullsized":"2023-11-28T21:57:11.407631+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1368","last_scrub_stamp":"2023-11-28T21:57:11.407483+0000","last_deep_scrub":"52'1368","last_deep_scrub_stamp":"2023-11-28T21:57:11.407483+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.407483+0000","objects_scrubbed":316,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:58:46.833003+0000","scrub_duration":0.207722514,"objects_trimmed":0,"snaptrim_duration":0.21672229300000001,"stat_sum":{"num_bytes":16299977,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":250,"num_read_kb":17253,"num_write":1343,"num_write_kb":69509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"52'268","reported_seq":2360,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.444623+0000","last_change":"2023-11-28T21:56:50.444642+0000","last_active":"2023-11-28T21:56:50.444623+0000","last_peered":"2023-11-28T21:56:50.444623+0000","last_clean":"2023-11-28T21:56:50.444623+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:56:50.444623+0000","last_undegraded":"2023-11-28T21:56:50.444623+0000","last_fullsized":"2023-11-28T21:56:50.444623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'268","last_scrub_stamp":"2023-11-28T21:56:50.444546+0000","last_deep_scrub":"52'268","last_deep_scrub_stamp":"2023-11-28T21:56:50.444546+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.444546+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:58:47.104569+0000","scrub_duration":0.0027754860000000002,"objects_trimmed":0,"snaptrim_duration":0.29009278799999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"52'248","reported_seq":326,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:47.891843+0000","last_change":"2023-11-28T21:56:47.891843+0000","last_active":"2023-11-28T21:56:47.891843+0000","last_peered":"2023-11-28T21:56:47.891843+0000","last_clean":"2023-11-28T21:56:47.891843+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:56:47.891843+0000","last_undegraded":"2023-11-28T21:56:47.891843+0000","last_fullsized":"2023-11-28T21:56:47.891843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'247","last_scrub_stamp":"2023-11-28T21:56:47.891725+0000","last_deep_scrub":"52'247","last_deep_scrub_stamp":"2023-11-28T21:56:47.891725+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.891725+0000","objects_scrubbed":3,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:49:45.140707+0000","scrub_duration":0.119752109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":607,"num_write":448,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8486,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"52'1386","reported_seq":3722,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.550973+0000","last_change":"2023-11-28T21:57:11.551001+0000","last_active":"2023-11-28T21:57:11.550973+0000","last_peered":"2023-11-28T21:57:11.550973+0000","last_clean":"2023-11-28T21:57:11.550973+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:57:11.550973+0000","last_undegraded":"2023-11-28T21:57:11.550973+0000","last_fullsized":"2023-11-28T21:57:11.550973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1386","last_scrub_stamp":"2023-11-28T21:57:11.550878+0000","last_deep_scrub":"52'1386","last_deep_scrub_stamp":"2023-11-28T21:57:11.550878+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.550878+0000","objects_scrubbed":313,"log_size":1386,"log_dups_size":0,"ondisk_log_size":1386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:03:55.485268+0000","scrub_duration":0.14493951099999999,"objects_trimmed":0,"snaptrim_duration":0.287791554,"stat_sum":{"num_bytes":19942162,"num_objects":313,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":232,"num_read_kb":16373,"num_write":1352,"num_write_kb":52080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"52'300","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:12.806201+0000","last_change":"2023-11-28T21:57:12.806224+0000","last_active":"2023-11-28T21:57:12.806201+0000","last_peered":"2023-11-28T21:57:12.806201+0000","last_clean":"2023-11-28T21:57:12.806201+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:57:12.806201+0000","last_undegraded":"2023-11-28T21:57:12.806201+0000","last_fullsized":"2023-11-28T21:57:12.806201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'300","last_scrub_stamp":"2023-11-28T21:57:12.806104+0000","last_deep_scrub":"52'300","last_deep_scrub_stamp":"2023-11-28T21:57:12.806104+0000","last_clean_scrub_stamp":"2023-11-28T21:57:12.806104+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:07:35.352399+0000","scrub_duration":0.0019360530000000001,"objects_trimmed":0,"snaptrim_duration":0.24779062600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"52'593","reported_seq":680,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.579200+0000","last_change":"2023-11-28T21:56:55.579266+0000","last_active":"2023-11-28T21:56:55.579200+0000","last_peered":"2023-11-28T21:56:55.579200+0000","last_clean":"2023-11-28T21:56:55.579200+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:56:55.579200+0000","last_undegraded":"2023-11-28T21:56:55.579200+0000","last_fullsized":"2023-11-28T21:56:55.579200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'593","last_scrub_stamp":"2023-11-28T21:56:55.579027+0000","last_deep_scrub":"52'593","last_deep_scrub_stamp":"2023-11-28T21:56:55.579027+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.579027+0000","objects_scrubbed":6,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:56:30.723466+0000","scrub_duration":0.077952980000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":81538,"num_write":523,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17674,"num_omap_keys":32,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"52'1410","reported_seq":3741,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.900338+0000","last_change":"2023-11-28T21:40:43.255749+0000","last_active":"2023-11-28T21:56:34.900338+0000","last_peered":"2023-11-28T21:56:34.900338+0000","last_clean":"2023-11-28T21:56:34.900338+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:56:34.900338+0000","last_undegraded":"2023-11-28T21:56:34.900338+0000","last_fullsized":"2023-11-28T21:56:34.900338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27155133199999998,"stat_sum":{"num_bytes":49110889,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":248,"num_read_kb":24602,"num_write":1394,"num_write_kb":89036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"52'284","reported_seq":2377,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:00.790251+0000","last_change":"2023-11-28T21:57:00.790304+0000","last_active":"2023-11-28T21:57:00.790251+0000","last_peered":"2023-11-28T21:57:00.790251+0000","last_clean":"2023-11-28T21:57:00.790251+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:57:00.790251+0000","last_undegraded":"2023-11-28T21:57:00.790251+0000","last_fullsized":"2023-11-28T21:57:00.790251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'284","last_scrub_stamp":"2023-11-28T21:57:00.790051+0000","last_deep_scrub":"52'284","last_deep_scrub_stamp":"2023-11-28T21:57:00.790051+0000","last_clean_scrub_stamp":"2023-11-28T21:57:00.790051+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:01:01.704329+0000","scrub_duration":0.0035905590000000001,"objects_trimmed":0,"snaptrim_duration":0.30430776599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"52'167","reported_seq":237,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.743243+0000","last_change":"2023-11-28T21:56:55.743275+0000","last_active":"2023-11-28T21:56:55.743243+0000","last_peered":"2023-11-28T21:56:55.743243+0000","last_clean":"2023-11-28T21:56:55.743243+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:56:55.743243+0000","last_undegraded":"2023-11-28T21:56:55.743243+0000","last_fullsized":"2023-11-28T21:56:55.743243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'167","last_scrub_stamp":"2023-11-28T21:56:55.743149+0000","last_deep_scrub":"52'167","last_deep_scrub_stamp":"2023-11-28T21:56:55.743149+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.743149+0000","objects_scrubbed":3,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:24.042531+0000","scrub_duration":0.015377216000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24576,"num_write":146,"num_write_kb":4262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19890,"num_omap_keys":36,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"52'1318","reported_seq":3643,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:14.990214+0000","last_change":"2023-11-28T21:57:14.990245+0000","last_active":"2023-11-28T21:57:14.990214+0000","last_peered":"2023-11-28T21:57:14.990214+0000","last_clean":"2023-11-28T21:57:14.990214+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:57:14.990214+0000","last_undegraded":"2023-11-28T21:57:14.990214+0000","last_fullsized":"2023-11-28T21:57:14.990214+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1318","last_scrub_stamp":"2023-11-28T21:57:14.990114+0000","last_deep_scrub":"52'1318","last_deep_scrub_stamp":"2023-11-28T21:57:14.990114+0000","last_clean_scrub_stamp":"2023-11-28T21:57:14.990114+0000","objects_scrubbed":314,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:16:53.276374+0000","scrub_duration":0.197652458,"objects_trimmed":0,"snaptrim_duration":0.25659980999999998,"stat_sum":{"num_bytes":32626940,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":222,"num_read_kb":4128,"num_write":1285,"num_write_kb":60020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"52'300","reported_seq":2392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.336206+0000","last_change":"2023-11-28T21:56:44.336260+0000","last_active":"2023-11-28T21:56:44.336206+0000","last_peered":"2023-11-28T21:56:44.336206+0000","last_clean":"2023-11-28T21:56:44.336206+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:56:44.336206+0000","last_undegraded":"2023-11-28T21:56:44.336206+0000","last_fullsized":"2023-11-28T21:56:44.336206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'300","last_scrub_stamp":"2023-11-28T21:56:44.335964+0000","last_deep_scrub":"52'300","last_deep_scrub_stamp":"2023-11-28T21:56:44.335964+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.335964+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:50:07.131774+0000","scrub_duration":0.0032352040000000002,"objects_trimmed":0,"snaptrim_duration":0.23142385100000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"52'370","reported_seq":453,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:51.659222+0000","last_change":"2023-11-28T21:56:51.659286+0000","last_active":"2023-11-28T21:56:51.659222+0000","last_peered":"2023-11-28T21:56:51.659222+0000","last_clean":"2023-11-28T21:56:51.659222+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:56:51.659222+0000","last_undegraded":"2023-11-28T21:56:51.659222+0000","last_fullsized":"2023-11-28T21:56:51.659222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'370","last_scrub_stamp":"2023-11-28T21:56:51.659033+0000","last_deep_scrub":"52'370","last_deep_scrub_stamp":"2023-11-28T21:56:51.659033+0000","last_clean_scrub_stamp":"2023-11-28T21:56:51.659033+0000","objects_scrubbed":3,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:45:32.821715+0000","scrub_duration":0.043796694999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":81920,"num_write":339,"num_write_kb":16655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"52'1263","reported_seq":3575,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.935320+0000","last_change":"2023-11-28T21:57:13.935360+0000","last_active":"2023-11-28T21:57:13.935320+0000","last_peered":"2023-11-28T21:57:13.935320+0000","last_clean":"2023-11-28T21:57:13.935320+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:57:13.935320+0000","last_undegraded":"2023-11-28T21:57:13.935320+0000","last_fullsized":"2023-11-28T21:57:13.935320+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1263","last_scrub_stamp":"2023-11-28T21:57:13.935220+0000","last_deep_scrub":"52'1263","last_deep_scrub_stamp":"2023-11-28T21:57:13.935220+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.935220+0000","objects_scrubbed":288,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:20.055177+0000","scrub_duration":0.14496025300000001,"objects_trimmed":0,"snaptrim_duration":0.24685721299999999,"stat_sum":{"num_bytes":16718463,"num_objects":288,"num_object_clones":0,"num_object_copies":1152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":210,"num_read_kb":3302,"num_write":1235,"num_write_kb":35539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"52'299","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:07.792410+0000","last_change":"2023-11-28T21:57:07.792430+0000","last_active":"2023-11-28T21:57:07.792410+0000","last_peered":"2023-11-28T21:57:07.792410+0000","last_clean":"2023-11-28T21:57:07.792410+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:57:07.792410+0000","last_undegraded":"2023-11-28T21:57:07.792410+0000","last_fullsized":"2023-11-28T21:57:07.792410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'299","last_scrub_stamp":"2023-11-28T21:57:07.792311+0000","last_deep_scrub":"52'299","last_deep_scrub_stamp":"2023-11-28T21:57:07.792311+0000","last_clean_scrub_stamp":"2023-11-28T21:57:07.792311+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:44.781100+0000","scrub_duration":0.0021660130000000001,"objects_trimmed":0,"snaptrim_duration":0.24023003500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"52'943","reported_seq":1423,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.244672+0000","last_change":"2023-11-28T21:56:45.244727+0000","last_active":"2023-11-28T21:56:45.244672+0000","last_peered":"2023-11-28T21:56:45.244672+0000","last_clean":"2023-11-28T21:56:45.244672+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:56:45.244672+0000","last_undegraded":"2023-11-28T21:56:45.244672+0000","last_fullsized":"2023-11-28T21:56:45.244672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'943","last_scrub_stamp":"2023-11-28T21:56:45.244580+0000","last_deep_scrub":"52'943","last_deep_scrub_stamp":"2023-11-28T21:56:45.244580+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.244580+0000","objects_scrubbed":9,"log_size":943,"log_dups_size":0,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:47:30.047518+0000","scrub_duration":0.055949021000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18884916,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":417,"num_read_kb":34819,"num_write":869,"num_write_kb":18975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17074,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"52'1231","reported_seq":3577,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:19.911197+0000","last_change":"2023-11-28T21:57:19.911246+0000","last_active":"2023-11-28T21:57:19.911197+0000","last_peered":"2023-11-28T21:57:19.911197+0000","last_clean":"2023-11-28T21:57:19.911197+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:57:19.911197+0000","last_undegraded":"2023-11-28T21:57:19.911197+0000","last_fullsized":"2023-11-28T21:57:19.911197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1231","last_scrub_stamp":"2023-11-28T21:57:19.911063+0000","last_deep_scrub":"52'1231","last_deep_scrub_stamp":"2023-11-28T21:57:19.911063+0000","last_clean_scrub_stamp":"2023-11-28T21:57:19.911063+0000","objects_scrubbed":293,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:45:27.145851+0000","scrub_duration":0.164741481,"objects_trimmed":0,"snaptrim_duration":0.24393632200000001,"stat_sum":{"num_bytes":26023878,"num_objects":293,"num_object_clones":0,"num_object_copies":1172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":293,"num_whiteouts":0,"num_read":244,"num_read_kb":12314,"num_write":1202,"num_write_kb":62723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"52'277","reported_seq":2369,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.370434+0000","last_change":"2023-11-28T21:57:01.370466+0000","last_active":"2023-11-28T21:57:01.370434+0000","last_peered":"2023-11-28T21:57:01.370434+0000","last_clean":"2023-11-28T21:57:01.370434+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:57:01.370434+0000","last_undegraded":"2023-11-28T21:57:01.370434+0000","last_fullsized":"2023-11-28T21:57:01.370434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'277","last_scrub_stamp":"2023-11-28T21:57:01.370337+0000","last_deep_scrub":"52'277","last_deep_scrub_stamp":"2023-11-28T21:57:01.370337+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.370337+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:30:28.221635+0000","scrub_duration":0.002672757,"objects_trimmed":0,"snaptrim_duration":0.25105937499999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"52'330","reported_seq":422,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:12.831554+0000","last_change":"2023-11-28T21:56:59.491989+0000","last_active":"2023-11-28T21:57:12.831554+0000","last_peered":"2023-11-28T21:57:12.831554+0000","last_clean":"2023-11-28T21:57:12.831554+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:57:12.831554+0000","last_undegraded":"2023-11-28T21:57:12.831554+0000","last_fullsized":"2023-11-28T21:57:12.831554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'325","last_scrub_stamp":"2023-11-28T21:56:59.491653+0000","last_deep_scrub":"52'325","last_deep_scrub_stamp":"2023-11-28T21:56:59.491653+0000","last_clean_scrub_stamp":"2023-11-28T21:56:59.491653+0000","objects_scrubbed":5,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:19:40.745953+0000","scrub_duration":0.050149041999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8760383,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":114688,"num_write":288,"num_write_kb":16920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5654,"num_omap_keys":11,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"52'1365","reported_seq":3712,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.882394+0000","last_change":"2023-11-28T21:40:43.252159+0000","last_active":"2023-11-28T21:56:34.882394+0000","last_peered":"2023-11-28T21:56:34.882394+0000","last_clean":"2023-11-28T21:56:34.882394+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:56:34.882394+0000","last_undegraded":"2023-11-28T21:56:34.882394+0000","last_fullsized":"2023-11-28T21:56:34.882394+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.270607394,"stat_sum":{"num_bytes":19916106,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":264,"num_read_kb":13370,"num_write":1336,"num_write_kb":59202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"52'283","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.428157+0000","last_change":"2023-11-28T21:56:57.428177+0000","last_active":"2023-11-28T21:56:57.428157+0000","last_peered":"2023-11-28T21:56:57.428157+0000","last_clean":"2023-11-28T21:56:57.428157+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:56:57.428157+0000","last_undegraded":"2023-11-28T21:56:57.428157+0000","last_fullsized":"2023-11-28T21:56:57.428157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:56:57.428075+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:56:57.428075+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.428075+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:09:37.862848+0000","scrub_duration":0.0026023819999999999,"objects_trimmed":0,"snaptrim_duration":0.29152715000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"52'318","reported_seq":391,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.764022+0000","last_change":"2023-11-28T21:56:57.764096+0000","last_active":"2023-11-28T21:56:57.764022+0000","last_peered":"2023-11-28T21:56:57.764022+0000","last_clean":"2023-11-28T21:56:57.764022+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:56:57.764022+0000","last_undegraded":"2023-11-28T21:56:57.764022+0000","last_fullsized":"2023-11-28T21:56:57.764022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'318","last_scrub_stamp":"2023-11-28T21:56:57.763819+0000","last_deep_scrub":"52'318","last_deep_scrub_stamp":"2023-11-28T21:56:57.763819+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.763819+0000","objects_scrubbed":7,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:20:01.143626+0000","scrub_duration":0.047864816999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":24594,"num_write":297,"num_write_kb":8480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":38186,"num_omap_keys":70,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"52'119","reported_seq":219,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.591456+0000","last_change":"2023-11-28T21:56:45.591517+0000","last_active":"2023-11-28T21:56:45.591456+0000","last_peered":"2023-11-28T21:56:45.591456+0000","last_clean":"2023-11-28T21:56:45.591456+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:56:45.591456+0000","last_undegraded":"2023-11-28T21:56:45.591456+0000","last_fullsized":"2023-11-28T21:56:45.591456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'119","last_scrub_stamp":"2023-11-28T21:56:45.591265+0000","last_deep_scrub":"52'119","last_deep_scrub_stamp":"2023-11-28T21:56:45.591265+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.591265+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:52:45.646774+0000","scrub_duration":0.0095868710000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"52'1342","reported_seq":3677,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.825311+0000","last_change":"2023-11-28T21:40:42.914002+0000","last_active":"2023-11-28T21:56:34.825311+0000","last_peered":"2023-11-28T21:56:34.825311+0000","last_clean":"2023-11-28T21:56:34.825311+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:56:34.825311+0000","last_undegraded":"2023-11-28T21:56:34.825311+0000","last_fullsized":"2023-11-28T21:56:34.825311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19732549099999999,"stat_sum":{"num_bytes":29762487,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":252,"num_read_kb":20508,"num_write":1310,"num_write_kb":73242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"52'280","reported_seq":2373,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.333148+0000","last_change":"2023-11-28T21:57:09.333148+0000","last_active":"2023-11-28T21:57:09.333148+0000","last_peered":"2023-11-28T21:57:09.333148+0000","last_clean":"2023-11-28T21:57:09.333148+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:57:09.333148+0000","last_undegraded":"2023-11-28T21:57:09.333148+0000","last_fullsized":"2023-11-28T21:57:09.333148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'280","last_scrub_stamp":"2023-11-28T21:57:09.332900+0000","last_deep_scrub":"52'280","last_deep_scrub_stamp":"2023-11-28T21:57:09.332900+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.332900+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:46:05.592588+0000","scrub_duration":0.00363196,"objects_trimmed":0,"snaptrim_duration":0.219020891,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"52'482","reported_seq":551,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.334575+0000","last_change":"2023-11-28T21:57:01.334575+0000","last_active":"2023-11-28T21:57:01.334575+0000","last_peered":"2023-11-28T21:57:01.334575+0000","last_clean":"2023-11-28T21:57:01.334575+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:57:01.334575+0000","last_undegraded":"2023-11-28T21:57:01.334575+0000","last_fullsized":"2023-11-28T21:57:01.334575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'482","last_scrub_stamp":"2023-11-28T21:57:01.334419+0000","last_deep_scrub":"52'482","last_deep_scrub_stamp":"2023-11-28T21:57:01.334419+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.334419+0000","objects_scrubbed":6,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:10:08.252390+0000","scrub_duration":0.032982576999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":20485,"num_write":447,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":27630,"num_omap_keys":50,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"52'1372","reported_seq":3705,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.808066+0000","last_change":"2023-11-28T21:40:42.915914+0000","last_active":"2023-11-28T21:56:34.808066+0000","last_peered":"2023-11-28T21:56:34.808066+0000","last_clean":"2023-11-28T21:56:34.808066+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:56:34.808066+0000","last_undegraded":"2023-11-28T21:56:34.808066+0000","last_fullsized":"2023-11-28T21:56:34.808066+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1372,"log_dups_size":0,"ondisk_log_size":1372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19929065100000001,"stat_sum":{"num_bytes":26470227,"num_objects":335,"num_object_clones":0,"num_object_copies":1340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":251,"num_read_kb":4134,"num_write":1344,"num_write_kb":47412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"52'330","reported_seq":2423,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:14.726189+0000","last_change":"2023-11-28T21:57:14.726238+0000","last_active":"2023-11-28T21:57:14.726189+0000","last_peered":"2023-11-28T21:57:14.726189+0000","last_clean":"2023-11-28T21:57:14.726189+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:57:14.726189+0000","last_undegraded":"2023-11-28T21:57:14.726189+0000","last_fullsized":"2023-11-28T21:57:14.726189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'330","last_scrub_stamp":"2023-11-28T21:57:14.726054+0000","last_deep_scrub":"52'330","last_deep_scrub_stamp":"2023-11-28T21:57:14.726054+0000","last_clean_scrub_stamp":"2023-11-28T21:57:14.726054+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:53:00.065479+0000","scrub_duration":0.0023321930000000002,"objects_trimmed":0,"snaptrim_duration":0.30458413400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"52'971","reported_seq":1061,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.753678+0000","last_change":"2023-11-28T21:56:49.753678+0000","last_active":"2023-11-28T21:56:49.753678+0000","last_peered":"2023-11-28T21:56:49.753678+0000","last_clean":"2023-11-28T21:56:49.753678+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:56:49.753678+0000","last_undegraded":"2023-11-28T21:56:49.753678+0000","last_fullsized":"2023-11-28T21:56:49.753678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'971","last_scrub_stamp":"2023-11-28T21:56:49.753562+0000","last_deep_scrub":"52'971","last_deep_scrub_stamp":"2023-11-28T21:56:49.753562+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.753562+0000","objects_scrubbed":7,"log_size":971,"log_dups_size":0,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:05.338455+0000","scrub_duration":0.053294293999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":27,"num_read_kb":68346,"num_write":931,"num_write_kb":16947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":38524,"num_omap_keys":71,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"52'1319","reported_seq":3667,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:39.342107+0000","last_change":"2023-11-28T21:40:43.256533+0000","last_active":"2023-11-28T21:56:39.342107+0000","last_peered":"2023-11-28T21:56:39.342107+0000","last_clean":"2023-11-28T21:56:39.342107+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:56:39.342107+0000","last_undegraded":"2023-11-28T21:56:39.342107+0000","last_fullsized":"2023-11-28T21:56:39.342107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27089875299999999,"stat_sum":{"num_bytes":28004524,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":265,"num_read_kb":16422,"num_write":1294,"num_write_kb":50614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"52'274","reported_seq":2359,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.276188+0000","last_change":"2023-11-28T21:57:16.276211+0000","last_active":"2023-11-28T21:57:16.276188+0000","last_peered":"2023-11-28T21:57:16.276188+0000","last_clean":"2023-11-28T21:57:16.276188+0000","last_became_active":"2023-11-28T21:35:25.594533+0000","last_became_peered":"2023-11-28T21:35:25.594533+0000","last_unstale":"2023-11-28T21:57:16.276188+0000","last_undegraded":"2023-11-28T21:57:16.276188+0000","last_fullsized":"2023-11-28T21:57:16.276188+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'274","last_scrub_stamp":"2023-11-28T21:57:16.276089+0000","last_deep_scrub":"52'274","last_deep_scrub_stamp":"2023-11-28T21:57:16.276089+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.276089+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:46:27.069531+0000","scrub_duration":0.0027233159999999999,"objects_trimmed":0,"snaptrim_duration":0.14880083199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"52'307","reported_seq":392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.178599+0000","last_change":"2023-11-28T21:56:48.178599+0000","last_active":"2023-11-28T21:56:48.178599+0000","last_peered":"2023-11-28T21:56:48.178599+0000","last_clean":"2023-11-28T21:56:48.178599+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:56:48.178599+0000","last_undegraded":"2023-11-28T21:56:48.178599+0000","last_fullsized":"2023-11-28T21:56:48.178599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'307","last_scrub_stamp":"2023-11-28T21:56:48.178460+0000","last_deep_scrub":"52'307","last_deep_scrub_stamp":"2023-11-28T21:56:48.178460+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.178460+0000","objects_scrubbed":5,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:26:15.702492+0000","scrub_duration":0.026539895000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":53275,"num_write":252,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21228,"num_omap_keys":39,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"52'1307","reported_seq":3592,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.520316+0000","last_change":"2023-11-28T21:40:42.823686+0000","last_active":"2023-11-28T21:56:34.520316+0000","last_peered":"2023-11-28T21:56:34.520316+0000","last_clean":"2023-11-28T21:56:34.520316+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:56:34.520316+0000","last_undegraded":"2023-11-28T21:56:34.520316+0000","last_fullsized":"2023-11-28T21:56:34.520316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1307,"log_dups_size":0,"ondisk_log_size":1307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30648515900000001,"stat_sum":{"num_bytes":28110180,"num_objects":295,"num_object_clones":0,"num_object_copies":1180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":0,"num_read":202,"num_read_kb":8227,"num_write":1283,"num_write_kb":51039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"52'277","reported_seq":2362,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.144651+0000","last_change":"2023-11-28T21:40:42.749319+0000","last_active":"2023-11-28T21:56:43.144651+0000","last_peered":"2023-11-28T21:56:43.144651+0000","last_clean":"2023-11-28T21:56:43.144651+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:56:43.144651+0000","last_undegraded":"2023-11-28T21:56:43.144651+0000","last_fullsized":"2023-11-28T21:56:43.144651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21877398200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"52'197","reported_seq":280,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.311351+0000","last_change":"2023-11-28T21:56:43.311351+0000","last_active":"2023-11-28T21:56:43.311351+0000","last_peered":"2023-11-28T21:56:43.311351+0000","last_clean":"2023-11-28T21:56:43.311351+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:56:43.311351+0000","last_undegraded":"2023-11-28T21:56:43.311351+0000","last_fullsized":"2023-11-28T21:56:43.311351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'197","last_scrub_stamp":"2023-11-28T21:56:43.311087+0000","last_deep_scrub":"52'197","last_deep_scrub_stamp":"2023-11-28T21:56:43.311087+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.311087+0000","objects_scrubbed":4,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:17:01.116728+0000","scrub_duration":0.027225414999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":49161,"num_write":147,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11661,"num_omap_keys":22,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"52'1273","reported_seq":3595,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.508129+0000","last_change":"2023-11-28T21:57:13.508211+0000","last_active":"2023-11-28T21:57:13.508129+0000","last_peered":"2023-11-28T21:57:13.508129+0000","last_clean":"2023-11-28T21:57:13.508129+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:57:13.508129+0000","last_undegraded":"2023-11-28T21:57:13.508129+0000","last_fullsized":"2023-11-28T21:57:13.508129+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1273","last_scrub_stamp":"2023-11-28T21:57:13.507890+0000","last_deep_scrub":"52'1273","last_deep_scrub_stamp":"2023-11-28T21:57:13.507890+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.507890+0000","objects_scrubbed":308,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:10:17.277733+0000","scrub_duration":0.25249365600000001,"objects_trimmed":0,"snaptrim_duration":0.223266511,"stat_sum":{"num_bytes":11523503,"num_objects":308,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":219,"num_read_kb":12316,"num_write":1248,"num_write_kb":38807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"52'323","reported_seq":2415,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.543891+0000","last_change":"2023-11-28T21:56:43.543999+0000","last_active":"2023-11-28T21:56:43.543891+0000","last_peered":"2023-11-28T21:56:43.543891+0000","last_clean":"2023-11-28T21:56:43.543891+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:56:43.543891+0000","last_undegraded":"2023-11-28T21:56:43.543891+0000","last_fullsized":"2023-11-28T21:56:43.543891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'323","last_scrub_stamp":"2023-11-28T21:56:43.543658+0000","last_deep_scrub":"52'323","last_deep_scrub_stamp":"2023-11-28T21:56:43.543658+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.543658+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:59.532405+0000","scrub_duration":0.003404327,"objects_trimmed":0,"snaptrim_duration":0.21978382299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"52'868","reported_seq":987,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.270821+0000","last_change":"2023-11-28T21:56:50.270874+0000","last_active":"2023-11-28T21:56:50.270821+0000","last_peered":"2023-11-28T21:56:50.270821+0000","last_clean":"2023-11-28T21:56:50.270821+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:56:50.270821+0000","last_undegraded":"2023-11-28T21:56:50.270821+0000","last_fullsized":"2023-11-28T21:56:50.270821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'868","last_scrub_stamp":"2023-11-28T21:56:50.270733+0000","last_deep_scrub":"52'868","last_deep_scrub_stamp":"2023-11-28T21:56:50.270733+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.270733+0000","objects_scrubbed":14,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:25.187061+0000","scrub_duration":0.051670183000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":137,"num_read_kb":58709,"num_write":838,"num_write_kb":8642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34555,"num_omap_keys":61,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"52'1274","reported_seq":3607,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.000372+0000","last_change":"2023-11-28T21:40:43.252839+0000","last_active":"2023-11-28T21:56:35.000372+0000","last_peered":"2023-11-28T21:56:35.000372+0000","last_clean":"2023-11-28T21:56:35.000372+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:56:35.000372+0000","last_undegraded":"2023-11-28T21:56:35.000372+0000","last_fullsized":"2023-11-28T21:56:35.000372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27152726799999999,"stat_sum":{"num_bytes":36307339,"num_objects":283,"num_object_clones":0,"num_object_copies":1132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":283,"num_whiteouts":0,"num_read":251,"num_read_kb":8046,"num_write":1253,"num_write_kb":60298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"52'284","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.131960+0000","last_change":"2023-11-28T21:57:01.132013+0000","last_active":"2023-11-28T21:57:01.131960+0000","last_peered":"2023-11-28T21:57:01.131960+0000","last_clean":"2023-11-28T21:57:01.131960+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:57:01.131960+0000","last_undegraded":"2023-11-28T21:57:01.131960+0000","last_fullsized":"2023-11-28T21:57:01.131960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'284","last_scrub_stamp":"2023-11-28T21:57:01.131764+0000","last_deep_scrub":"52'284","last_deep_scrub_stamp":"2023-11-28T21:57:01.131764+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.131764+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:44:58.129846+0000","scrub_duration":0.0035951529999999998,"objects_trimmed":0,"snaptrim_duration":0.18244292200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"52'243","reported_seq":306,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.132811+0000","last_change":"2023-11-28T21:56:48.132928+0000","last_active":"2023-11-28T21:56:48.132811+0000","last_peered":"2023-11-28T21:56:48.132811+0000","last_clean":"2023-11-28T21:56:48.132811+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:56:48.132811+0000","last_undegraded":"2023-11-28T21:56:48.132811+0000","last_fullsized":"2023-11-28T21:56:48.132811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'243","last_scrub_stamp":"2023-11-28T21:56:48.132597+0000","last_deep_scrub":"52'243","last_deep_scrub_stamp":"2023-11-28T21:56:48.132597+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.132597+0000","objects_scrubbed":4,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:38.258199+0000","scrub_duration":0.035274727999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195875,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":194,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12188,"num_omap_keys":22,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"52'1346","reported_seq":3663,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.687516+0000","last_change":"2023-11-28T21:40:42.805322+0000","last_active":"2023-11-28T21:56:34.687516+0000","last_peered":"2023-11-28T21:56:34.687516+0000","last_clean":"2023-11-28T21:56:34.687516+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:56:34.687516+0000","last_undegraded":"2023-11-28T21:56:34.687516+0000","last_fullsized":"2023-11-28T21:56:34.687516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29439885100000002,"stat_sum":{"num_bytes":34458204,"num_objects":324,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":324,"num_whiteouts":0,"num_read":235,"num_read_kb":27797,"num_write":1320,"num_write_kb":70332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"52'313","reported_seq":2397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.755624+0000","last_change":"2023-11-28T21:40:42.740620+0000","last_active":"2023-11-28T21:56:35.755624+0000","last_peered":"2023-11-28T21:56:35.755624+0000","last_clean":"2023-11-28T21:56:35.755624+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:56:35.755624+0000","last_undegraded":"2023-11-28T21:56:35.755624+0000","last_fullsized":"2023-11-28T21:56:35.755624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.218364582,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"52'160","reported_seq":227,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.715676+0000","last_change":"2023-11-28T21:56:55.715743+0000","last_active":"2023-11-28T21:56:55.715676+0000","last_peered":"2023-11-28T21:56:55.715676+0000","last_clean":"2023-11-28T21:56:55.715676+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:56:55.715676+0000","last_undegraded":"2023-11-28T21:56:55.715676+0000","last_fullsized":"2023-11-28T21:56:55.715676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'160","last_scrub_stamp":"2023-11-28T21:56:55.715502+0000","last_deep_scrub":"52'160","last_deep_scrub_stamp":"2023-11-28T21:56:55.715502+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.715502+0000","objects_scrubbed":5,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:08:17.898435+0000","scrub_duration":0.024710507999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":107,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23214,"num_omap_keys":42,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"52'1271","reported_seq":3593,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.743280+0000","last_change":"2023-11-28T21:57:16.743301+0000","last_active":"2023-11-28T21:57:16.743280+0000","last_peered":"2023-11-28T21:57:16.743280+0000","last_clean":"2023-11-28T21:57:16.743280+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:57:16.743280+0000","last_undegraded":"2023-11-28T21:57:16.743280+0000","last_fullsized":"2023-11-28T21:57:16.743280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1271","last_scrub_stamp":"2023-11-28T21:57:16.743187+0000","last_deep_scrub":"52'1271","last_deep_scrub_stamp":"2023-11-28T21:57:16.743187+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.743187+0000","objects_scrubbed":350,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:32.276987+0000","scrub_duration":0.193506183,"objects_trimmed":0,"snaptrim_duration":0.29571303700000001,"stat_sum":{"num_bytes":33678171,"num_objects":350,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":350,"num_whiteouts":0,"num_read":218,"num_read_kb":4129,"num_write":1246,"num_write_kb":52865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"52'279","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.192213+0000","last_change":"2023-11-28T21:57:13.192234+0000","last_active":"2023-11-28T21:57:13.192213+0000","last_peered":"2023-11-28T21:57:13.192213+0000","last_clean":"2023-11-28T21:57:13.192213+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:57:13.192213+0000","last_undegraded":"2023-11-28T21:57:13.192213+0000","last_fullsized":"2023-11-28T21:57:13.192213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'279","last_scrub_stamp":"2023-11-28T21:57:13.192121+0000","last_deep_scrub":"52'279","last_deep_scrub_stamp":"2023-11-28T21:57:13.192121+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.192121+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:37:10.470818+0000","scrub_duration":0.003054668,"objects_trimmed":0,"snaptrim_duration":0.21362616300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"52'313","reported_seq":415,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.422213+0000","last_change":"2023-11-28T21:56:44.422244+0000","last_active":"2023-11-28T21:56:44.422213+0000","last_peered":"2023-11-28T21:56:44.422213+0000","last_clean":"2023-11-28T21:56:44.422213+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:56:44.422213+0000","last_undegraded":"2023-11-28T21:56:44.422213+0000","last_fullsized":"2023-11-28T21:56:44.422213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'313","last_scrub_stamp":"2023-11-28T21:56:44.422134+0000","last_deep_scrub":"52'313","last_deep_scrub_stamp":"2023-11-28T21:56:44.422134+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.422134+0000","objects_scrubbed":9,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:50.028108+0000","scrub_duration":0.030421210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":67,"num_read_kb":77908,"num_write":272,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57622,"num_omap_keys":105,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"52'1265","reported_seq":3537,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.764966+0000","last_change":"2023-11-28T21:40:42.831006+0000","last_active":"2023-11-28T21:56:34.764966+0000","last_peered":"2023-11-28T21:56:34.764966+0000","last_clean":"2023-11-28T21:56:34.764966+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:56:34.764966+0000","last_undegraded":"2023-11-28T21:56:34.764966+0000","last_fullsized":"2023-11-28T21:56:34.764966+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1265,"log_dups_size":0,"ondisk_log_size":1265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30778537500000003,"stat_sum":{"num_bytes":30037804,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":190,"num_read_kb":33,"num_write":1241,"num_write_kb":36631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.264901+0000","last_change":"2023-11-28T21:57:17.264953+0000","last_active":"2023-11-28T21:57:17.264901+0000","last_peered":"2023-11-28T21:57:17.264901+0000","last_clean":"2023-11-28T21:57:17.264901+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:57:17.264901+0000","last_undegraded":"2023-11-28T21:57:17.264901+0000","last_fullsized":"2023-11-28T21:57:17.264901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:17.264706+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:17.264706+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.264706+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:11:48.499371+0000","scrub_duration":0.0034211150000000002,"objects_trimmed":0,"snaptrim_duration":0.21990301600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"52'759","reported_seq":882,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.931156+0000","last_change":"2023-11-28T21:56:47.144324+0000","last_active":"2023-11-28T21:56:53.931156+0000","last_peered":"2023-11-28T21:56:53.931156+0000","last_clean":"2023-11-28T21:56:53.931156+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:56:53.931156+0000","last_undegraded":"2023-11-28T21:56:53.931156+0000","last_fullsized":"2023-11-28T21:56:53.931156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'757","last_scrub_stamp":"2023-11-28T21:56:47.144054+0000","last_deep_scrub":"52'757","last_deep_scrub_stamp":"2023-11-28T21:56:47.144054+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.144054+0000","objects_scrubbed":12,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:36:20.172063+0000","scrub_duration":0.085472306999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":90,"num_read_kb":122940,"num_write":684,"num_write_kb":20915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":36218,"num_omap_keys":66,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"52'1318","reported_seq":3605,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.731622+0000","last_change":"2023-11-28T21:40:42.828740+0000","last_active":"2023-11-28T21:56:34.731622+0000","last_peered":"2023-11-28T21:56:34.731622+0000","last_clean":"2023-11-28T21:56:34.731622+0000","last_became_active":"2023-11-28T21:35:25.790293+0000","last_became_peered":"2023-11-28T21:35:25.790293+0000","last_unstale":"2023-11-28T21:56:34.731622+0000","last_undegraded":"2023-11-28T21:56:34.731622+0000","last_fullsized":"2023-11-28T21:56:34.731622+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30978138799999999,"stat_sum":{"num_bytes":36258609,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":208,"num_read_kb":8227,"num_write":1285,"num_write_kb":56637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"52'283","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.299355+0000","last_change":"2023-11-28T21:56:50.299408+0000","last_active":"2023-11-28T21:56:50.299355+0000","last_peered":"2023-11-28T21:56:50.299355+0000","last_clean":"2023-11-28T21:56:50.299355+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:56:50.299355+0000","last_undegraded":"2023-11-28T21:56:50.299355+0000","last_fullsized":"2023-11-28T21:56:50.299355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:56:50.299160+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:56:50.299160+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.299160+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:20.213618+0000","scrub_duration":0.003897317,"objects_trimmed":0,"snaptrim_duration":0.22435063599999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"52'525","reported_seq":625,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.790155+0000","last_change":"2023-11-28T21:56:53.790210+0000","last_active":"2023-11-28T21:56:53.790155+0000","last_peered":"2023-11-28T21:56:53.790155+0000","last_clean":"2023-11-28T21:56:53.790155+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:56:53.790155+0000","last_undegraded":"2023-11-28T21:56:53.790155+0000","last_fullsized":"2023-11-28T21:56:53.790155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'525","last_scrub_stamp":"2023-11-28T21:56:53.790048+0000","last_deep_scrub":"52'525","last_deep_scrub_stamp":"2023-11-28T21:56:53.790048+0000","last_clean_scrub_stamp":"2023-11-28T21:56:53.790048+0000","objects_scrubbed":9,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:47:27.057628+0000","scrub_duration":0.097465174000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":69670,"num_write":485,"num_write_kb":16676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24376,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"52'1329","reported_seq":3690,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:52.591960+0000","last_change":"2023-11-28T21:56:52.592017+0000","last_active":"2023-11-28T21:56:52.591960+0000","last_peered":"2023-11-28T21:56:52.591960+0000","last_clean":"2023-11-28T21:56:52.591960+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:56:52.591960+0000","last_undegraded":"2023-11-28T21:56:52.591960+0000","last_fullsized":"2023-11-28T21:56:52.591960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1329","last_scrub_stamp":"2023-11-28T21:56:52.591801+0000","last_deep_scrub":"52'1329","last_deep_scrub_stamp":"2023-11-28T21:56:52.591801+0000","last_clean_scrub_stamp":"2023-11-28T21:56:52.591801+0000","objects_scrubbed":342,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:47:53.831632+0000","scrub_duration":0.25412254099999998,"objects_trimmed":0,"snaptrim_duration":0.2340457,"stat_sum":{"num_bytes":46614746,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":259,"num_read_kb":20536,"num_write":1294,"num_write_kb":83351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"52'265","reported_seq":2356,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.218640+0000","last_change":"2023-11-28T21:57:11.218663+0000","last_active":"2023-11-28T21:57:11.218640+0000","last_peered":"2023-11-28T21:57:11.218640+0000","last_clean":"2023-11-28T21:57:11.218640+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:57:11.218640+0000","last_undegraded":"2023-11-28T21:57:11.218640+0000","last_fullsized":"2023-11-28T21:57:11.218640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'265","last_scrub_stamp":"2023-11-28T21:57:11.218547+0000","last_deep_scrub":"52'265","last_deep_scrub_stamp":"2023-11-28T21:57:11.218547+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.218547+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:14.792134+0000","scrub_duration":0.0033770470000000002,"objects_trimmed":0,"snaptrim_duration":0.228570471,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"52'997","reported_seq":1085,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.686854+0000","last_change":"2023-11-28T21:56:49.686931+0000","last_active":"2023-11-28T21:56:49.686854+0000","last_peered":"2023-11-28T21:56:49.686854+0000","last_clean":"2023-11-28T21:56:49.686854+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:56:49.686854+0000","last_undegraded":"2023-11-28T21:56:49.686854+0000","last_fullsized":"2023-11-28T21:56:49.686854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'997","last_scrub_stamp":"2023-11-28T21:56:49.686647+0000","last_deep_scrub":"52'997","last_deep_scrub_stamp":"2023-11-28T21:56:49.686647+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.686647+0000","objects_scrubbed":7,"log_size":997,"log_dups_size":0,"ondisk_log_size":997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:07:13.091549+0000","scrub_duration":0.104808501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":90124,"num_write":955,"num_write_kb":20985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9418,"num_omap_keys":17,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"52'1334","reported_seq":3694,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.715355+0000","last_change":"2023-11-28T21:57:17.715381+0000","last_active":"2023-11-28T21:57:17.715355+0000","last_peered":"2023-11-28T21:57:17.715355+0000","last_clean":"2023-11-28T21:57:17.715355+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:57:17.715355+0000","last_undegraded":"2023-11-28T21:57:17.715355+0000","last_fullsized":"2023-11-28T21:57:17.715355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1334","last_scrub_stamp":"2023-11-28T21:57:17.715260+0000","last_deep_scrub":"52'1334","last_deep_scrub_stamp":"2023-11-28T21:57:17.715260+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.715260+0000","objects_scrubbed":314,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:34:07.662604+0000","scrub_duration":0.20735551499999999,"objects_trimmed":0,"snaptrim_duration":0.29518589499999998,"stat_sum":{"num_bytes":25457761,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":258,"num_read_kb":22214,"num_write":1317,"num_write_kb":78783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"52'308","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:15.248158+0000","last_change":"2023-11-28T21:57:15.248196+0000","last_active":"2023-11-28T21:57:15.248158+0000","last_peered":"2023-11-28T21:57:15.248158+0000","last_clean":"2023-11-28T21:57:15.248158+0000","last_became_active":"2023-11-28T21:35:25.595386+0000","last_became_peered":"2023-11-28T21:35:25.595386+0000","last_unstale":"2023-11-28T21:57:15.248158+0000","last_undegraded":"2023-11-28T21:57:15.248158+0000","last_fullsized":"2023-11-28T21:57:15.248158+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'308","last_scrub_stamp":"2023-11-28T21:57:15.248071+0000","last_deep_scrub":"52'308","last_deep_scrub_stamp":"2023-11-28T21:57:15.248071+0000","last_clean_scrub_stamp":"2023-11-28T21:57:15.248071+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:34:19.589282+0000","scrub_duration":0.0030527179999999998,"objects_trimmed":0,"snaptrim_duration":0.148630489,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"52'112","reported_seq":179,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.072726+0000","last_change":"2023-11-28T21:56:55.072786+0000","last_active":"2023-11-28T21:56:55.072726+0000","last_peered":"2023-11-28T21:56:55.072726+0000","last_clean":"2023-11-28T21:56:55.072726+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:56:55.072726+0000","last_undegraded":"2023-11-28T21:56:55.072726+0000","last_fullsized":"2023-11-28T21:56:55.072726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'112","last_scrub_stamp":"2023-11-28T21:56:55.072507+0000","last_deep_scrub":"52'112","last_deep_scrub_stamp":"2023-11-28T21:56:55.072507+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.072507+0000","objects_scrubbed":3,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:04:39.066804+0000","scrub_duration":0.0092242439999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":68,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"52'1427","reported_seq":3792,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:14.697621+0000","last_change":"2023-11-28T21:57:14.697652+0000","last_active":"2023-11-28T21:57:14.697621+0000","last_peered":"2023-11-28T21:57:14.697621+0000","last_clean":"2023-11-28T21:57:14.697621+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:57:14.697621+0000","last_undegraded":"2023-11-28T21:57:14.697621+0000","last_fullsized":"2023-11-28T21:57:14.697621+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1427","last_scrub_stamp":"2023-11-28T21:57:14.697528+0000","last_deep_scrub":"52'1427","last_deep_scrub_stamp":"2023-11-28T21:57:14.697528+0000","last_clean_scrub_stamp":"2023-11-28T21:57:14.697528+0000","objects_scrubbed":317,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:15:02.264993+0000","scrub_duration":0.20879229699999999,"objects_trimmed":0,"snaptrim_duration":0.28498848100000002,"stat_sum":{"num_bytes":33245588,"num_objects":317,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":263,"num_read_kb":28708,"num_write":1396,"num_write_kb":77612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"52'1309","reported_seq":3641,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:39.342045+0000","last_change":"2023-11-28T21:40:43.251187+0000","last_active":"2023-11-28T21:56:39.342045+0000","last_peered":"2023-11-28T21:56:39.342045+0000","last_clean":"2023-11-28T21:56:39.342045+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:56:39.342045+0000","last_undegraded":"2023-11-28T21:56:39.342045+0000","last_fullsized":"2023-11-28T21:56:39.342045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1309,"log_dups_size":0,"ondisk_log_size":1309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27353859400000002,"stat_sum":{"num_bytes":25285382,"num_objects":331,"num_object_clones":0,"num_object_copies":1324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":248,"num_read_kb":4117,"num_write":1287,"num_write_kb":58337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"52'285","reported_seq":2376,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.748373+0000","last_change":"2023-11-28T21:40:42.876095+0000","last_active":"2023-11-28T21:56:35.748373+0000","last_peered":"2023-11-28T21:56:35.748373+0000","last_clean":"2023-11-28T21:56:35.748373+0000","last_became_active":"2023-11-28T21:35:25.589378+0000","last_became_peered":"2023-11-28T21:35:25.589378+0000","last_unstale":"2023-11-28T21:56:35.748373+0000","last_undegraded":"2023-11-28T21:56:35.748373+0000","last_fullsized":"2023-11-28T21:56:35.748373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.246426856,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"52'951","reported_seq":1047,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.694770+0000","last_change":"2023-11-28T21:56:50.694839+0000","last_active":"2023-11-28T21:56:50.694770+0000","last_peered":"2023-11-28T21:56:50.694770+0000","last_clean":"2023-11-28T21:56:50.694770+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:56:50.694770+0000","last_undegraded":"2023-11-28T21:56:50.694770+0000","last_fullsized":"2023-11-28T21:56:50.694770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'951","last_scrub_stamp":"2023-11-28T21:56:50.694579+0000","last_deep_scrub":"52'951","last_deep_scrub_stamp":"2023-11-28T21:56:50.694579+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.694579+0000","objects_scrubbed":5,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:57:02.007325+0000","scrub_duration":0.095599400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":899,"num_write_kb":25075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12742,"num_omap_keys":23,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"52'1266","reported_seq":3554,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:34.473089+0000","last_change":"2023-11-28T21:40:42.918369+0000","last_active":"2023-11-28T21:56:34.473089+0000","last_peered":"2023-11-28T21:56:34.473089+0000","last_clean":"2023-11-28T21:56:34.473089+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:56:34.473089+0000","last_undegraded":"2023-11-28T21:56:34.473089+0000","last_fullsized":"2023-11-28T21:56:34.473089+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1266,"log_dups_size":0,"ondisk_log_size":1266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.16839808100000001,"stat_sum":{"num_bytes":18660789,"num_objects":319,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":205,"num_read_kb":12320,"num_write":1249,"num_write_kb":51464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"52'321","reported_seq":2412,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:58.470634+0000","last_change":"2023-11-28T21:56:58.470657+0000","last_active":"2023-11-28T21:56:58.470634+0000","last_peered":"2023-11-28T21:56:58.470634+0000","last_clean":"2023-11-28T21:56:58.470634+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:56:58.470634+0000","last_undegraded":"2023-11-28T21:56:58.470634+0000","last_fullsized":"2023-11-28T21:56:58.470634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'321","last_scrub_stamp":"2023-11-28T21:56:58.470530+0000","last_deep_scrub":"52'321","last_deep_scrub_stamp":"2023-11-28T21:56:58.470530+0000","last_clean_scrub_stamp":"2023-11-28T21:56:58.470530+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:47:39.793970+0000","scrub_duration":0.0019542520000000001,"objects_trimmed":0,"snaptrim_duration":0.28961016299999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"52'91","reported_seq":162,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:52.421023+0000","last_change":"2023-11-28T21:56:52.421071+0000","last_active":"2023-11-28T21:56:52.421023+0000","last_peered":"2023-11-28T21:56:52.421023+0000","last_clean":"2023-11-28T21:56:52.421023+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:56:52.421023+0000","last_undegraded":"2023-11-28T21:56:52.421023+0000","last_fullsized":"2023-11-28T21:56:52.421023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'91","last_scrub_stamp":"2023-11-28T21:56:52.420930+0000","last_deep_scrub":"52'91","last_deep_scrub_stamp":"2023-11-28T21:56:52.420930+0000","last_clean_scrub_stamp":"2023-11-28T21:56:52.420930+0000","objects_scrubbed":6,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:15:04.674788+0000","scrub_duration":0.030405108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":63,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":37270,"num_omap_keys":68,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"52'1081","reported_seq":1182,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.227886+0000","last_change":"2023-11-28T21:56:46.227886+0000","last_active":"2023-11-28T21:56:46.227886+0000","last_peered":"2023-11-28T21:56:46.227886+0000","last_clean":"2023-11-28T21:56:46.227886+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:56:46.227886+0000","last_undegraded":"2023-11-28T21:56:46.227886+0000","last_fullsized":"2023-11-28T21:56:46.227886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1081","last_scrub_stamp":"2023-11-28T21:56:46.227627+0000","last_deep_scrub":"52'1081","last_deep_scrub_stamp":"2023-11-28T21:56:46.227627+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.227627+0000","objects_scrubbed":10,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:49:00.930323+0000","scrub_duration":0.124157213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":146592,"num_write":1020,"num_write_kb":29113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":33716,"num_omap_keys":61,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"52'284","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:19.161483+0000","last_change":"2023-11-28T21:57:19.161506+0000","last_active":"2023-11-28T21:57:19.161483+0000","last_peered":"2023-11-28T21:57:19.161483+0000","last_clean":"2023-11-28T21:57:19.161483+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:57:19.161483+0000","last_undegraded":"2023-11-28T21:57:19.161483+0000","last_fullsized":"2023-11-28T21:57:19.161483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'284","last_scrub_stamp":"2023-11-28T21:57:19.161376+0000","last_deep_scrub":"52'284","last_deep_scrub_stamp":"2023-11-28T21:57:19.161376+0000","last_clean_scrub_stamp":"2023-11-28T21:57:19.161376+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:45.359179+0000","scrub_duration":0.0032904639999999999,"objects_trimmed":0,"snaptrim_duration":0.26886027800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"52'1325","reported_seq":3646,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.515073+0000","last_change":"2023-11-28T21:57:09.515073+0000","last_active":"2023-11-28T21:57:09.515073+0000","last_peered":"2023-11-28T21:57:09.515073+0000","last_clean":"2023-11-28T21:57:09.515073+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:57:09.515073+0000","last_undegraded":"2023-11-28T21:57:09.515073+0000","last_fullsized":"2023-11-28T21:57:09.515073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1325","last_scrub_stamp":"2023-11-28T21:57:09.514950+0000","last_deep_scrub":"52'1325","last_deep_scrub_stamp":"2023-11-28T21:57:09.514950+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.514950+0000","objects_scrubbed":342,"log_size":1325,"log_dups_size":0,"ondisk_log_size":1325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:43:36.262680+0000","scrub_duration":0.21339559899999999,"objects_trimmed":0,"snaptrim_duration":0.13232148399999999,"stat_sum":{"num_bytes":46546650,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":218,"num_read_kb":9401,"num_write":1296,"num_write_kb":74280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"52'473","reported_seq":550,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.461705+0000","last_change":"2023-11-28T21:56:53.461705+0000","last_active":"2023-11-28T21:56:53.461705+0000","last_peered":"2023-11-28T21:56:53.461705+0000","last_clean":"2023-11-28T21:56:53.461705+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:56:53.461705+0000","last_undegraded":"2023-11-28T21:56:53.461705+0000","last_fullsized":"2023-11-28T21:56:53.461705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'473","last_scrub_stamp":"2023-11-28T21:56:53.461589+0000","last_deep_scrub":"52'473","last_deep_scrub_stamp":"2023-11-28T21:56:53.461589+0000","last_clean_scrub_stamp":"2023-11-28T21:56:53.461589+0000","objects_scrubbed":6,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:21:39.580720+0000","scrub_duration":0.057920754999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":48896,"num_write":437,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":14594,"num_omap_keys":28,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"52'279","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.249131+0000","last_change":"2023-11-28T21:56:49.249131+0000","last_active":"2023-11-28T21:56:49.249131+0000","last_peered":"2023-11-28T21:56:49.249131+0000","last_clean":"2023-11-28T21:56:49.249131+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:56:49.249131+0000","last_undegraded":"2023-11-28T21:56:49.249131+0000","last_fullsized":"2023-11-28T21:56:49.249131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'279","last_scrub_stamp":"2023-11-28T21:56:49.248890+0000","last_deep_scrub":"52'279","last_deep_scrub_stamp":"2023-11-28T21:56:49.248890+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.248890+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:20:21.740665+0000","scrub_duration":0.0035163680000000002,"objects_trimmed":0,"snaptrim_duration":0.24793844200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"52'1320","reported_seq":3659,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:40.959835+0000","last_change":"2023-11-28T21:40:42.807134+0000","last_active":"2023-11-28T21:56:40.959835+0000","last_peered":"2023-11-28T21:56:40.959835+0000","last_clean":"2023-11-28T21:56:40.959835+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:56:40.959835+0000","last_undegraded":"2023-11-28T21:56:40.959835+0000","last_fullsized":"2023-11-28T21:56:40.959835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1320,"log_dups_size":0,"ondisk_log_size":1320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.29562192300000001,"stat_sum":{"num_bytes":29852008,"num_objects":333,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":257,"num_read_kb":28714,"num_write":1297,"num_write_kb":83698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"52'321","reported_seq":2413,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:10.252751+0000","last_change":"2023-11-28T21:57:10.252806+0000","last_active":"2023-11-28T21:57:10.252751+0000","last_peered":"2023-11-28T21:57:10.252751+0000","last_clean":"2023-11-28T21:57:10.252751+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:57:10.252751+0000","last_undegraded":"2023-11-28T21:57:10.252751+0000","last_fullsized":"2023-11-28T21:57:10.252751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'321","last_scrub_stamp":"2023-11-28T21:57:10.252559+0000","last_deep_scrub":"52'321","last_deep_scrub_stamp":"2023-11-28T21:57:10.252559+0000","last_clean_scrub_stamp":"2023-11-28T21:57:10.252559+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:04.185942+0000","scrub_duration":0.0035793169999999998,"objects_trimmed":0,"snaptrim_duration":0.18187520500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"52'328","reported_seq":2421,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:03.303092+0000","last_change":"2023-11-28T21:57:03.303115+0000","last_active":"2023-11-28T21:57:03.303092+0000","last_peered":"2023-11-28T21:57:03.303092+0000","last_clean":"2023-11-28T21:57:03.303092+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:57:03.303092+0000","last_undegraded":"2023-11-28T21:57:03.303092+0000","last_fullsized":"2023-11-28T21:57:03.303092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'328","last_scrub_stamp":"2023-11-28T21:57:03.302989+0000","last_deep_scrub":"52'328","last_deep_scrub_stamp":"2023-11-28T21:57:03.302989+0000","last_clean_scrub_stamp":"2023-11-28T21:57:03.302989+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:52:13.280846+0000","scrub_duration":0.0030730839999999998,"objects_trimmed":0,"snaptrim_duration":0.14391041299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"52'303","reported_seq":2396,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.533484+0000","last_change":"2023-11-28T21:56:44.533536+0000","last_active":"2023-11-28T21:56:44.533484+0000","last_peered":"2023-11-28T21:56:44.533484+0000","last_clean":"2023-11-28T21:56:44.533484+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:56:44.533484+0000","last_undegraded":"2023-11-28T21:56:44.533484+0000","last_fullsized":"2023-11-28T21:56:44.533484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'303","last_scrub_stamp":"2023-11-28T21:56:44.533290+0000","last_deep_scrub":"52'303","last_deep_scrub_stamp":"2023-11-28T21:56:44.533290+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.533290+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:11:39.585022+0000","scrub_duration":0.0035272839999999999,"objects_trimmed":0,"snaptrim_duration":0.30621145,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"52'294","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.242623+0000","last_change":"2023-11-28T21:57:09.242644+0000","last_active":"2023-11-28T21:57:09.242623+0000","last_peered":"2023-11-28T21:57:09.242623+0000","last_clean":"2023-11-28T21:57:09.242623+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:57:09.242623+0000","last_undegraded":"2023-11-28T21:57:09.242623+0000","last_fullsized":"2023-11-28T21:57:09.242623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'294","last_scrub_stamp":"2023-11-28T21:57:09.242522+0000","last_deep_scrub":"52'294","last_deep_scrub_stamp":"2023-11-28T21:57:09.242522+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.242522+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:12:54.800329+0000","scrub_duration":0.001978142,"objects_trimmed":0,"snaptrim_duration":0.26967866099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"52'275","reported_seq":2367,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.387085+0000","last_change":"2023-11-28T21:56:46.387109+0000","last_active":"2023-11-28T21:56:46.387085+0000","last_peered":"2023-11-28T21:56:46.387085+0000","last_clean":"2023-11-28T21:56:46.387085+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:56:46.387085+0000","last_undegraded":"2023-11-28T21:56:46.387085+0000","last_fullsized":"2023-11-28T21:56:46.387085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'275","last_scrub_stamp":"2023-11-28T21:56:46.386968+0000","last_deep_scrub":"52'275","last_deep_scrub_stamp":"2023-11-28T21:56:46.386968+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.386968+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:09:10.256531+0000","scrub_duration":0.0028373539999999998,"objects_trimmed":0,"snaptrim_duration":0.29058787400000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"52'293","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.787300+0000","last_change":"2023-11-28T21:57:11.787318+0000","last_active":"2023-11-28T21:57:11.787300+0000","last_peered":"2023-11-28T21:57:11.787300+0000","last_clean":"2023-11-28T21:57:11.787300+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:57:11.787300+0000","last_undegraded":"2023-11-28T21:57:11.787300+0000","last_fullsized":"2023-11-28T21:57:11.787300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'293","last_scrub_stamp":"2023-11-28T21:57:11.787231+0000","last_deep_scrub":"52'293","last_deep_scrub_stamp":"2023-11-28T21:57:11.787231+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.787231+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:35:53.843661+0000","scrub_duration":0.0015063699999999999,"objects_trimmed":0,"snaptrim_duration":0.246741036,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"52'316","reported_seq":2408,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.330062+0000","last_change":"2023-11-28T21:57:13.330127+0000","last_active":"2023-11-28T21:57:13.330062+0000","last_peered":"2023-11-28T21:57:13.330062+0000","last_clean":"2023-11-28T21:57:13.330062+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:57:13.330062+0000","last_undegraded":"2023-11-28T21:57:13.330062+0000","last_fullsized":"2023-11-28T21:57:13.330062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'316","last_scrub_stamp":"2023-11-28T21:57:13.329847+0000","last_deep_scrub":"52'316","last_deep_scrub_stamp":"2023-11-28T21:57:13.329847+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.329847+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:32:35.225710+0000","scrub_duration":0.0039367100000000004,"objects_trimmed":0,"snaptrim_duration":0.18459727300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"52'287","reported_seq":2379,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:05.398087+0000","last_change":"2023-11-28T21:57:05.398109+0000","last_active":"2023-11-28T21:57:05.398087+0000","last_peered":"2023-11-28T21:57:05.398087+0000","last_clean":"2023-11-28T21:57:05.398087+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:57:05.398087+0000","last_undegraded":"2023-11-28T21:57:05.398087+0000","last_fullsized":"2023-11-28T21:57:05.398087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'287","last_scrub_stamp":"2023-11-28T21:57:05.397991+0000","last_deep_scrub":"52'287","last_deep_scrub_stamp":"2023-11-28T21:57:05.397991+0000","last_clean_scrub_stamp":"2023-11-28T21:57:05.397991+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:09:47.792188+0000","scrub_duration":0.002983145,"objects_trimmed":0,"snaptrim_duration":0.29272911699999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"52'285","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:07.843268+0000","last_change":"2023-11-28T21:57:07.843319+0000","last_active":"2023-11-28T21:57:07.843268+0000","last_peered":"2023-11-28T21:57:07.843268+0000","last_clean":"2023-11-28T21:57:07.843268+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:57:07.843268+0000","last_undegraded":"2023-11-28T21:57:07.843268+0000","last_fullsized":"2023-11-28T21:57:07.843268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'285","last_scrub_stamp":"2023-11-28T21:57:07.843058+0000","last_deep_scrub":"52'285","last_deep_scrub_stamp":"2023-11-28T21:57:07.843058+0000","last_clean_scrub_stamp":"2023-11-28T21:57:07.843058+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:11:36.128065+0000","scrub_duration":0.0032873389999999998,"objects_trimmed":0,"snaptrim_duration":0.30579284099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"52'272","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:51.436185+0000","last_change":"2023-11-28T21:56:51.436185+0000","last_active":"2023-11-28T21:56:51.436185+0000","last_peered":"2023-11-28T21:56:51.436185+0000","last_clean":"2023-11-28T21:56:51.436185+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:56:51.436185+0000","last_undegraded":"2023-11-28T21:56:51.436185+0000","last_fullsized":"2023-11-28T21:56:51.436185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'272","last_scrub_stamp":"2023-11-28T21:56:51.436052+0000","last_deep_scrub":"52'272","last_deep_scrub_stamp":"2023-11-28T21:56:51.436052+0000","last_clean_scrub_stamp":"2023-11-28T21:56:51.436052+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:27.893115+0000","scrub_duration":0.0032975470000000001,"objects_trimmed":0,"snaptrim_duration":0.28793565599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"52'285","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.755908+0000","last_change":"2023-11-28T21:57:16.755960+0000","last_active":"2023-11-28T21:57:16.755908+0000","last_peered":"2023-11-28T21:57:16.755908+0000","last_clean":"2023-11-28T21:57:16.755908+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:57:16.755908+0000","last_undegraded":"2023-11-28T21:57:16.755908+0000","last_fullsized":"2023-11-28T21:57:16.755908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'285","last_scrub_stamp":"2023-11-28T21:57:16.755612+0000","last_deep_scrub":"52'285","last_deep_scrub_stamp":"2023-11-28T21:57:16.755612+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.755612+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:01.560436+0000","scrub_duration":0.0036730019999999999,"objects_trimmed":0,"snaptrim_duration":0.30636868099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.293494+0000","last_change":"2023-11-28T21:57:04.293494+0000","last_active":"2023-11-28T21:57:04.293494+0000","last_peered":"2023-11-28T21:57:04.293494+0000","last_clean":"2023-11-28T21:57:04.293494+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:57:04.293494+0000","last_undegraded":"2023-11-28T21:57:04.293494+0000","last_fullsized":"2023-11-28T21:57:04.293494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:04.293402+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:04.293402+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.293402+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:26:25.956930+0000","scrub_duration":0.0026303339999999998,"objects_trimmed":0,"snaptrim_duration":0.26835999100000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"52'297","reported_seq":2381,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:42.584648+0000","last_change":"2023-11-28T21:40:42.910839+0000","last_active":"2023-11-28T21:56:42.584648+0000","last_peered":"2023-11-28T21:56:42.584648+0000","last_clean":"2023-11-28T21:56:42.584648+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:56:42.584648+0000","last_undegraded":"2023-11-28T21:56:42.584648+0000","last_fullsized":"2023-11-28T21:56:42.584648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.194469013,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"52'293","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:18.231899+0000","last_change":"2023-11-28T21:57:18.231950+0000","last_active":"2023-11-28T21:57:18.231899+0000","last_peered":"2023-11-28T21:57:18.231899+0000","last_clean":"2023-11-28T21:57:18.231899+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:57:18.231899+0000","last_undegraded":"2023-11-28T21:57:18.231899+0000","last_fullsized":"2023-11-28T21:57:18.231899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'293","last_scrub_stamp":"2023-11-28T21:57:18.231694+0000","last_deep_scrub":"52'293","last_deep_scrub_stamp":"2023-11-28T21:57:18.231694+0000","last_clean_scrub_stamp":"2023-11-28T21:57:18.231694+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:39:32.183707+0000","scrub_duration":0.0026952199999999999,"objects_trimmed":0,"snaptrim_duration":0.21943309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"52'268","reported_seq":2360,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.255130+0000","last_change":"2023-11-28T21:57:04.255150+0000","last_active":"2023-11-28T21:57:04.255130+0000","last_peered":"2023-11-28T21:57:04.255130+0000","last_clean":"2023-11-28T21:57:04.255130+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:57:04.255130+0000","last_undegraded":"2023-11-28T21:57:04.255130+0000","last_fullsized":"2023-11-28T21:57:04.255130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'268","last_scrub_stamp":"2023-11-28T21:57:04.255048+0000","last_deep_scrub":"52'268","last_deep_scrub_stamp":"2023-11-28T21:57:04.255048+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.255048+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:40:34.130819+0000","scrub_duration":0.0027406069999999999,"objects_trimmed":0,"snaptrim_duration":0.14941532399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"52'313","reported_seq":2405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:42.478620+0000","last_change":"2023-11-28T21:56:42.478673+0000","last_active":"2023-11-28T21:56:42.478620+0000","last_peered":"2023-11-28T21:56:42.478620+0000","last_clean":"2023-11-28T21:56:42.478620+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:56:42.478620+0000","last_undegraded":"2023-11-28T21:56:42.478620+0000","last_fullsized":"2023-11-28T21:56:42.478620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'313","last_scrub_stamp":"2023-11-28T21:56:42.478435+0000","last_deep_scrub":"52'313","last_deep_scrub_stamp":"2023-11-28T21:56:42.478435+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.478435+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:59.037978+0000","scrub_duration":0.0037738749999999999,"objects_trimmed":0,"snaptrim_duration":0.30374592,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"52'301","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.566855+0000","last_change":"2023-11-28T21:56:45.566906+0000","last_active":"2023-11-28T21:56:45.566855+0000","last_peered":"2023-11-28T21:56:45.566855+0000","last_clean":"2023-11-28T21:56:45.566855+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:56:45.566855+0000","last_undegraded":"2023-11-28T21:56:45.566855+0000","last_fullsized":"2023-11-28T21:56:45.566855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'301","last_scrub_stamp":"2023-11-28T21:56:45.566667+0000","last_deep_scrub":"52'301","last_deep_scrub_stamp":"2023-11-28T21:56:45.566667+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.566667+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:37:12.783651+0000","scrub_duration":0.0036235360000000001,"objects_trimmed":0,"snaptrim_duration":0.30238551699999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"52'316","reported_seq":2408,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.741463+0000","last_change":"2023-11-28T21:57:04.741484+0000","last_active":"2023-11-28T21:57:04.741463+0000","last_peered":"2023-11-28T21:57:04.741463+0000","last_clean":"2023-11-28T21:57:04.741463+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:57:04.741463+0000","last_undegraded":"2023-11-28T21:57:04.741463+0000","last_fullsized":"2023-11-28T21:57:04.741463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'316","last_scrub_stamp":"2023-11-28T21:57:04.741375+0000","last_deep_scrub":"52'316","last_deep_scrub_stamp":"2023-11-28T21:57:04.741375+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.741375+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:53:03.728731+0000","scrub_duration":0.0023203569999999999,"objects_trimmed":0,"snaptrim_duration":0.246777987,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"52'301","reported_seq":2392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.425032+0000","last_change":"2023-11-28T21:56:55.425054+0000","last_active":"2023-11-28T21:56:55.425032+0000","last_peered":"2023-11-28T21:56:55.425032+0000","last_clean":"2023-11-28T21:56:55.425032+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:56:55.425032+0000","last_undegraded":"2023-11-28T21:56:55.425032+0000","last_fullsized":"2023-11-28T21:56:55.425032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'301","last_scrub_stamp":"2023-11-28T21:56:55.424936+0000","last_deep_scrub":"52'301","last_deep_scrub_stamp":"2023-11-28T21:56:55.424936+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.424936+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:58:23.742113+0000","scrub_duration":0.0019665469999999999,"objects_trimmed":0,"snaptrim_duration":0.21399896700000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"52'274","reported_seq":2365,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.388835+0000","last_change":"2023-11-28T21:56:48.388857+0000","last_active":"2023-11-28T21:56:48.388835+0000","last_peered":"2023-11-28T21:56:48.388835+0000","last_clean":"2023-11-28T21:56:48.388835+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:56:48.388835+0000","last_undegraded":"2023-11-28T21:56:48.388835+0000","last_fullsized":"2023-11-28T21:56:48.388835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'274","last_scrub_stamp":"2023-11-28T21:56:48.388743+0000","last_deep_scrub":"52'274","last_deep_scrub_stamp":"2023-11-28T21:56:48.388743+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.388743+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:55.401441+0000","scrub_duration":0.0030358709999999999,"objects_trimmed":0,"snaptrim_duration":0.287656776,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"52'261","reported_seq":2352,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.326425+0000","last_change":"2023-11-28T21:56:46.326533+0000","last_active":"2023-11-28T21:56:46.326425+0000","last_peered":"2023-11-28T21:56:46.326425+0000","last_clean":"2023-11-28T21:56:46.326425+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:56:46.326425+0000","last_undegraded":"2023-11-28T21:56:46.326425+0000","last_fullsized":"2023-11-28T21:56:46.326425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'261","last_scrub_stamp":"2023-11-28T21:56:46.326201+0000","last_deep_scrub":"52'261","last_deep_scrub_stamp":"2023-11-28T21:56:46.326201+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.326201+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:57.520054+0000","scrub_duration":0.0037943849999999999,"objects_trimmed":0,"snaptrim_duration":0.222804104,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"52'261","reported_seq":2344,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757320+0000","last_change":"2023-11-28T21:40:42.914448+0000","last_active":"2023-11-28T21:56:35.757320+0000","last_peered":"2023-11-28T21:56:35.757320+0000","last_clean":"2023-11-28T21:56:35.757320+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:56:35.757320+0000","last_undegraded":"2023-11-28T21:56:35.757320+0000","last_fullsized":"2023-11-28T21:56:35.757320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.19768582900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"52'283","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.372700+0000","last_change":"2023-11-28T21:57:16.372800+0000","last_active":"2023-11-28T21:57:16.372700+0000","last_peered":"2023-11-28T21:57:16.372700+0000","last_clean":"2023-11-28T21:57:16.372700+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:57:16.372700+0000","last_undegraded":"2023-11-28T21:57:16.372700+0000","last_fullsized":"2023-11-28T21:57:16.372700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:57:16.372401+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:57:16.372401+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.372401+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:12.888742+0000","scrub_duration":0.0041957210000000003,"objects_trimmed":0,"snaptrim_duration":0.18434383600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"52'314","reported_seq":2405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757872+0000","last_change":"2023-11-28T21:40:42.754000+0000","last_active":"2023-11-28T21:56:35.757872+0000","last_peered":"2023-11-28T21:56:35.757872+0000","last_clean":"2023-11-28T21:56:35.757872+0000","last_became_active":"2023-11-28T21:35:25.598839+0000","last_became_peered":"2023-11-28T21:35:25.598839+0000","last_unstale":"2023-11-28T21:56:35.757872+0000","last_undegraded":"2023-11-28T21:56:35.757872+0000","last_fullsized":"2023-11-28T21:56:35.757872+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22244017299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"52'291","reported_seq":2382,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:10.406800+0000","last_change":"2023-11-28T21:57:10.406825+0000","last_active":"2023-11-28T21:57:10.406800+0000","last_peered":"2023-11-28T21:57:10.406800+0000","last_clean":"2023-11-28T21:57:10.406800+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:57:10.406800+0000","last_undegraded":"2023-11-28T21:57:10.406800+0000","last_fullsized":"2023-11-28T21:57:10.406800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'291","last_scrub_stamp":"2023-11-28T21:57:10.406693+0000","last_deep_scrub":"52'291","last_deep_scrub_stamp":"2023-11-28T21:57:10.406693+0000","last_clean_scrub_stamp":"2023-11-28T21:57:10.406693+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:00:26.481166+0000","scrub_duration":0.0034554859999999998,"objects_trimmed":0,"snaptrim_duration":0.28637254899999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"52'292","reported_seq":2384,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:05.386536+0000","last_change":"2023-11-28T21:57:05.386536+0000","last_active":"2023-11-28T21:57:05.386536+0000","last_peered":"2023-11-28T21:57:05.386536+0000","last_clean":"2023-11-28T21:57:05.386536+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:57:05.386536+0000","last_undegraded":"2023-11-28T21:57:05.386536+0000","last_fullsized":"2023-11-28T21:57:05.386536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'292","last_scrub_stamp":"2023-11-28T21:57:05.386287+0000","last_deep_scrub":"52'292","last_deep_scrub_stamp":"2023-11-28T21:57:05.386287+0000","last_clean_scrub_stamp":"2023-11-28T21:57:05.386287+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:49.649068+0000","scrub_duration":0.0038884100000000001,"objects_trimmed":0,"snaptrim_duration":0.219142752,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"52'282","reported_seq":2366,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:42.048077+0000","last_change":"2023-11-28T21:40:42.831458+0000","last_active":"2023-11-28T21:56:42.048077+0000","last_peered":"2023-11-28T21:56:42.048077+0000","last_clean":"2023-11-28T21:56:42.048077+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:56:42.048077+0000","last_undegraded":"2023-11-28T21:56:42.048077+0000","last_fullsized":"2023-11-28T21:56:42.048077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.307113203,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.239031+0000","last_change":"2023-11-28T21:57:16.239138+0000","last_active":"2023-11-28T21:57:16.239031+0000","last_peered":"2023-11-28T21:57:16.239031+0000","last_clean":"2023-11-28T21:57:16.239031+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:57:16.239031+0000","last_undegraded":"2023-11-28T21:57:16.239031+0000","last_fullsized":"2023-11-28T21:57:16.239031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:16.238782+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:16.238782+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.238782+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:31.855622+0000","scrub_duration":0.0036200709999999999,"objects_trimmed":0,"snaptrim_duration":0.21561430200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"52'287","reported_seq":2373,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:40.216243+0000","last_change":"2023-11-28T21:40:42.829941+0000","last_active":"2023-11-28T21:56:40.216243+0000","last_peered":"2023-11-28T21:56:40.216243+0000","last_clean":"2023-11-28T21:56:40.216243+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:56:40.216243+0000","last_undegraded":"2023-11-28T21:56:40.216243+0000","last_fullsized":"2023-11-28T21:56:40.216243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30486287499999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"52'281","reported_seq":2373,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:18.200025+0000","last_change":"2023-11-28T21:57:18.200046+0000","last_active":"2023-11-28T21:57:18.200025+0000","last_peered":"2023-11-28T21:57:18.200025+0000","last_clean":"2023-11-28T21:57:18.200025+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:57:18.200025+0000","last_undegraded":"2023-11-28T21:57:18.200025+0000","last_fullsized":"2023-11-28T21:57:18.200025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'281","last_scrub_stamp":"2023-11-28T21:57:18.199943+0000","last_deep_scrub":"52'281","last_deep_scrub_stamp":"2023-11-28T21:57:18.199943+0000","last_clean_scrub_stamp":"2023-11-28T21:57:18.199943+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:04.947793+0000","scrub_duration":0.0015704720000000001,"objects_trimmed":0,"snaptrim_duration":0.26474323500000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"52'292","reported_seq":2385,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.804985+0000","last_change":"2023-11-28T21:57:09.805004+0000","last_active":"2023-11-28T21:57:09.804985+0000","last_peered":"2023-11-28T21:57:09.804985+0000","last_clean":"2023-11-28T21:57:09.804985+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:57:09.804985+0000","last_undegraded":"2023-11-28T21:57:09.804985+0000","last_fullsized":"2023-11-28T21:57:09.804985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'292","last_scrub_stamp":"2023-11-28T21:57:09.804904+0000","last_deep_scrub":"52'292","last_deep_scrub_stamp":"2023-11-28T21:57:09.804904+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.804904+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:06:20.229796+0000","scrub_duration":0.0016870489999999999,"objects_trimmed":0,"snaptrim_duration":0.24685295800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"52'289","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.757430+0000","last_change":"2023-11-28T21:40:42.753421+0000","last_active":"2023-11-28T21:56:35.757430+0000","last_peered":"2023-11-28T21:56:35.757430+0000","last_clean":"2023-11-28T21:56:35.757430+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:56:35.757430+0000","last_undegraded":"2023-11-28T21:56:35.757430+0000","last_fullsized":"2023-11-28T21:56:35.757430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:35.964297+0000","last_clean_scrub_stamp":"2023-11-28T21:34:35.964297+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.22257470400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":927864585,"num_objects":10340,"num_object_clones":0,"num_object_copies":41360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10340,"num_whiteouts":0,"num_read":7542,"num_read_kb":441308,"num_write":41596,"num_write_kb":1955411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1822244864,"data_stored":1974304768,"data_compressed":124280088,"data_compressed_allocated":140013568,"data_compressed_original":292073472,"omap_allocated":0,"internal_metadata":0},"log_size":42425,"ondisk_log_size":42425,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18675,"ondisk_log_size":18675,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":303274248,"num_objects":196,"num_object_clones":0,"num_object_copies":392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1486,"num_read_kb":1965797,"num_write":14881,"num_write_kb":402355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":687608,"num_omap_keys":1199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173113344,"data_stored":606548496,"data_compressed":82550738,"data_compressed_allocated":105840640,"data_compressed_original":539434902,"omap_allocated":0,"internal_metadata":0},"log_size":16085,"ondisk_log_size":16085,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575127,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329140,"kb_used_data":211884,"kb_used_omap":1,"kb_used_meta":117246,"kb_avail":94042700,"statfs":{"total":96636764160,"available":96299724800,"internally_reserved":0,"allocated":216969216,"data_stored":292707674,"data_compressed":24978755,"data_compressed_allocated":29880320,"data_compressed_original":105926656,"omap_allocated":1590,"internal_metadata":120060362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.874,"15min":0.98499999999999999},"min":{"1min":0.498,"5min":0.41399999999999998,"15min":0.35299999999999998},"max":{"1min":3.3719999999999999,"5min":5.056,"15min":47.475000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.85599999999999998,"15min":0.82999999999999996},"min":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.36199999999999999},"max":{"1min":1.8260000000000001,"5min":4.069,"15min":5.3310000000000004},"last":0.66100000000000003}]},{"osd":1,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0069999999999999,"15min":1.1060000000000001},"min":{"1min":0.65400000000000003,"5min":0.59699999999999998,"15min":0.39200000000000002},"max":{"1min":1.79,"5min":2.5630000000000002,"15min":47.529000000000003},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0860000000000001,"15min":1.0189999999999999},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.734,"5min":17.460000000000001,"15min":17.460000000000001},"last":0.71199999999999997}]},{"osd":2,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.83899999999999997,"15min":0.90500000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98899999999999999,"5min":2.9359999999999999,"15min":5.5140000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.008,"15min":1.0249999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.498},"max":{"1min":1.212,"5min":2.972,"15min":5.5979999999999999},"last":0.79600000000000004}]},{"osd":3,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.216,"15min":1.198},"min":{"1min":0.60699999999999998,"5min":0.57299999999999995,"15min":0.44600000000000001},"max":{"1min":7.3760000000000003,"5min":16.113,"15min":47.57},"last":1.159},{"interface":"front","average":{"1min":1.327,"5min":1.214,"15min":1.0920000000000001},"min":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.53800000000000003},"max":{"1min":7.4269999999999996,"5min":11.148999999999999,"15min":12.273},"last":0.94699999999999995}]},{"osd":4,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.125,"15min":1.0609999999999999},"min":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.35499999999999998},"max":{"1min":5.6859999999999999,"5min":17.542000000000002,"15min":17.542000000000002},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0429999999999999,"15min":1.1950000000000001},"min":{"1min":0.67200000000000004,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":49.106000000000002},"last":1.109}]},{"osd":5,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.2909999999999999},"min":{"1min":0.91700000000000004,"5min":0.65600000000000003,"15min":0.40899999999999997},"max":{"1min":2.0110000000000001,"5min":6.6420000000000003,"15min":64.855999999999995},"last":1.24},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.1779999999999999,"15min":1.206},"min":{"1min":0.79800000000000004,"5min":0.73499999999999999,"15min":0.56399999999999995},"max":{"1min":5.843,"5min":11.327,"15min":16.753},"last":0.997}]},{"osd":6,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.216,"15min":1.175},"min":{"1min":0.72999999999999998,"5min":0.67500000000000004,"15min":0.37},"max":{"1min":5.7679999999999998,"5min":11.641999999999999,"15min":11.641999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.1719999999999999,"15min":1.167},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.502},"max":{"1min":5.4210000000000003,"5min":5.4210000000000003,"15min":7.4909999999999997},"last":1.272}]}]},{"osd":6,"up_from":13,"seq":55834575125,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":380176,"kb_used_data":254600,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":93991664,"statfs":{"total":96636764160,"available":96247463936,"internally_reserved":0,"allocated":260710400,"data_stored":335877014,"data_compressed":24633140,"data_compressed_allocated":29302784,"data_compressed_original":104771531,"omap_allocated":1594,"internal_metadata":128580038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.276,"15min":1.0249999999999999},"min":{"1min":0.55800000000000005,"5min":0.441,"15min":0.27300000000000002},"max":{"1min":2.9119999999999999,"5min":40.999000000000002,"15min":40.999000000000002},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.042,"15min":1.04},"min":{"1min":0.62,"5min":0.42299999999999999,"15min":0.40000000000000002},"max":{"1min":2.3370000000000002,"5min":14.132999999999999,"15min":24.399999999999999},"last":0.70299999999999996}]},{"osd":1,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":0.99399999999999999,"15min":0.92000000000000004},"min":{"1min":0.55900000000000005,"5min":0.48599999999999999,"15min":0.35899999999999999},"max":{"1min":7.0709999999999997,"5min":7.0709999999999997,"15min":11.196},"last":1.089},{"interface":"front","average":{"1min":1.611,"5min":1.0389999999999999,"15min":1.012},"min":{"1min":0.57499999999999996,"5min":0.42799999999999999,"15min":0.40899999999999997},"max":{"1min":13.404,"5min":13.404,"15min":13.404},"last":0.64800000000000002}]},{"osd":2,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":0.94299999999999995,"15min":0.96399999999999997},"min":{"1min":0.74399999999999999,"5min":0.56999999999999995,"15min":0.39100000000000001},"max":{"1min":2.6829999999999998,"5min":3.855,"15min":21.986999999999998},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.004,"15min":1.024},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.40400000000000003},"max":{"1min":8.4730000000000008,"5min":8.4730000000000008,"15min":8.6170000000000009},"last":0.745}]},{"osd":3,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.131,"15min":1.0189999999999999},"min":{"1min":0.68400000000000005,"5min":0.54600000000000004,"15min":0.41999999999999998},"max":{"1min":15.964,"5min":15.964,"15min":15.964},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.897,"5min":1.1919999999999999,"15min":1.149},"min":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.47499999999999998},"max":{"1min":16.602,"5min":16.602,"15min":24.745000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.95999999999999996,"15min":1.018},"min":{"1min":0.69799999999999995,"5min":0.59799999999999998,"15min":0.30299999999999999},"max":{"1min":1.7050000000000001,"5min":2.4550000000000001,"15min":16.015000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.093,"15min":1.0900000000000001},"min":{"1min":0.91000000000000003,"5min":0.65100000000000002,"15min":0.35799999999999998},"max":{"1min":2.4929999999999999,"5min":2.5329999999999999,"15min":13.193},"last":1.0589999999999999}]},{"osd":5,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.0329999999999999,"15min":1.1200000000000001},"min":{"1min":0.86699999999999999,"5min":0.58699999999999997,"15min":0.307},"max":{"1min":1.661,"5min":2.069,"15min":25.006},"last":1.127},{"interface":"front","average":{"1min":1.276,"5min":1.1140000000000001,"15min":1.151},"min":{"1min":0.86799999999999999,"5min":0.46700000000000003,"15min":0.33000000000000002},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":7.843},"last":0.96399999999999997}]},{"osd":7,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.109,"15min":1.163},"min":{"1min":0.78600000000000003,"5min":0.68999999999999995,"15min":0.36099999999999999},"max":{"1min":5.6799999999999997,"5min":5.6799999999999997,"15min":7.883},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.246,"15min":1.304},"min":{"1min":0.95399999999999996,"5min":0.86499999999999999,"15min":0.432},"max":{"1min":5.7199999999999998,"5min":5.7199999999999998,"15min":27.942},"last":1.1499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575125,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363676,"kb_used_data":238100,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":94008164,"statfs":{"total":96636764160,"available":96264359936,"internally_reserved":0,"allocated":243814400,"data_stored":323612635,"data_compressed":28196796,"data_compressed_allocated":33439744,"data_compressed_original":113541120,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.99399999999999999,"15min":0.89800000000000002},"min":{"1min":0.47399999999999998,"5min":0.46500000000000002,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":15.005000000000001,"15min":15.005000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88800000000000001},"min":{"1min":0.501,"5min":0.47799999999999998,"15min":0.46700000000000003},"max":{"1min":1.2290000000000001,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":1.0429999999999999}]},{"osd":1,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.84799999999999998,"15min":0.84299999999999997},"min":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.48199999999999998},"max":{"1min":3.806,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":0.96399999999999997,"15min":0.93700000000000006},"min":{"1min":0.57999999999999996,"5min":0.57299999999999995,"15min":0.496},"max":{"1min":7.4930000000000003,"5min":7.4930000000000003,"15min":7.4930000000000003},"last":0.65600000000000003}]},{"osd":2,"last update":"Tue Nov 28 21:56:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.016,"15min":0.97399999999999998},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.105,"5min":14.775,"15min":14.775},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.1040000000000001,"15min":0.94599999999999995},"min":{"1min":0.72899999999999998,"5min":0.63800000000000001,"15min":0.57499999999999996},"max":{"1min":1.1919999999999999,"5min":14.901999999999999,"15min":14.901999999999999},"last":0.97099999999999997}]},{"osd":3,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.038,"15min":0.98999999999999999},"min":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.54400000000000004},"max":{"1min":1.347,"5min":15.154,"15min":15.154},"last":0.755},{"interface":"front","average":{"1min":1.02,"5min":0.996,"15min":1.0149999999999999},"min":{"1min":0.68600000000000005,"5min":0.60799999999999998,"15min":0.53500000000000003},"max":{"1min":1.3069999999999999,"5min":3.4289999999999998,"15min":5.5170000000000003},"last":0.88200000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.534,"15min":1.1870000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.52000000000000002},"max":{"1min":2.98,"5min":61.020000000000003,"15min":61.020000000000003},"last":1.107},{"interface":"front","average":{"1min":1.405,"5min":1.22,"15min":1.1160000000000001},"min":{"1min":0.67400000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":8.6029999999999998,"5min":15.085000000000001,"15min":15.085000000000001},"last":1.0800000000000001}]},{"osd":6,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.137,"15min":1.1259999999999999},"min":{"1min":0.72599999999999998,"5min":0.66000000000000003,"15min":0.56699999999999995},"max":{"1min":1.4099999999999999,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":1.204},{"interface":"front","average":{"1min":1.266,"5min":1.234,"15min":1.181},"min":{"1min":1.0589999999999999,"5min":0.73199999999999998,"15min":0.64800000000000002},"max":{"1min":1.4970000000000001,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.252}]},{"osd":7,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2070000000000001,"15min":1.1739999999999999},"min":{"1min":0.85599999999999998,"5min":0.75600000000000001,"15min":0.70399999999999996},"max":{"1min":1.5600000000000001,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.157},{"interface":"front","average":{"1min":1.042,"5min":1.0600000000000001,"15min":1.0289999999999999},"min":{"1min":0.69299999999999995,"5min":0.66800000000000004,"15min":0.60499999999999998},"max":{"1min":1.6779999999999999,"5min":6.2869999999999999,"15min":6.2869999999999999},"last":0.93799999999999994}]}]},{"osd":4,"up_from":13,"seq":55834575125,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357644,"kb_used_data":236228,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":94014196,"statfs":{"total":96636764160,"available":96270536704,"internally_reserved":0,"allocated":241897472,"data_stored":321511706,"data_compressed":25475129,"data_compressed_allocated":30593024,"data_compressed_original":110510027,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2210000000000001,"15min":1.004},"min":{"1min":0.38200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":9.2759999999999998,"5min":43.139000000000003,"15min":43.139000000000003},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.573,"5min":0.96599999999999997,"15min":0.94399999999999995},"min":{"1min":0.57299999999999995,"5min":0.499,"15min":0.39900000000000002},"max":{"1min":9.9009999999999998,"5min":9.9009999999999998,"15min":17.446000000000002},"last":1.0840000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.8200000000000001,"5min":1.153,"15min":1.1140000000000001},"min":{"1min":0.67100000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":15.432,"5min":15.432,"15min":21.763999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.72,"5min":1.145,"15min":1.5369999999999999},"min":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.46500000000000002},"max":{"1min":12.186999999999999,"5min":12.186999999999999,"15min":78.649000000000001},"last":0.94599999999999995}]},{"osd":2,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.9319999999999999,"5min":1.1439999999999999,"15min":1.1160000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":20.105,"5min":20.105,"15min":28.341000000000001},"last":1.302},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.2070000000000001,"15min":1.603},"min":{"1min":0.70399999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"max":{"1min":15.393000000000001,"5min":15.393000000000001,"15min":94.998999999999995},"last":0.995}]},{"osd":3,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":2.0720000000000001,"5min":1.4019999999999999,"15min":1.169},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":20.065000000000001,"5min":20.065000000000001,"15min":20.065000000000001},"last":1.222},{"interface":"front","average":{"1min":2.0419999999999998,"5min":1.246,"15min":1.1279999999999999},"min":{"1min":0.64200000000000002,"5min":0.54800000000000004,"15min":0.53200000000000003},"max":{"1min":18.588000000000001,"5min":18.588000000000001,"15min":18.588000000000001},"last":1.1779999999999999}]},{"osd":5,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.0600000000000001,"15min":1.159},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.376},"max":{"1min":12.159000000000001,"5min":12.159000000000001,"15min":17.033000000000001},"last":1.036},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.468,"15min":1.329},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.437},"max":{"1min":9.5500000000000007,"5min":38.939,"15min":38.939},"last":1.2629999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":2.3500000000000001,"5min":1.2929999999999999,"15min":1.2609999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.49299999999999999},"max":{"1min":28.289000000000001,"5min":28.289000000000001,"15min":28.289000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5640000000000001,"15min":1.367},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.55600000000000005},"max":{"1min":9.5120000000000005,"5min":43.420000000000002,"15min":43.420000000000002},"last":1.335}]},{"osd":7,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.1859999999999999,"15min":1.6499999999999999},"min":{"1min":0.55900000000000005,"5min":0.434,"15min":0.434},"max":{"1min":10.282,"5min":10.282,"15min":77.638999999999996},"last":1.375},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.3089999999999999,"15min":1.2070000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.51500000000000001},"max":{"1min":9.6080000000000005,"5min":9.6080000000000005,"15min":9.6080000000000005},"last":1.4079999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575126,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465164,"kb_used_data":307652,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93906676,"statfs":{"total":96636764160,"available":96160436224,"internally_reserved":0,"allocated":315035648,"data_stored":410723951,"data_compressed":36998413,"data_compressed_allocated":43757568,"data_compressed_original":139747328,"omap_allocated":1592,"internal_metadata":161282504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.89400000000000002,"15min":0.70899999999999996},"min":{"1min":0.312,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.77600000000000002,"5min":36.585000000000001,"15min":36.585000000000001},"last":0.316},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.88600000000000001,"15min":0.67600000000000005},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.33100000000000002},"max":{"1min":0.996,"5min":30.065000000000001,"15min":30.065000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.42699999999999999,"5min":0.33200000000000002,"15min":0.26300000000000001},"max":{"1min":0.80800000000000005,"5min":4.8099999999999996,"15min":10.449},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.58799999999999997,"15min":0.67100000000000004},"min":{"1min":0.33600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80400000000000005,"5min":4.8520000000000003,"15min":10.481999999999999},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57499999999999996,"15min":0.64800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.27900000000000003},"max":{"1min":0.90200000000000002,"5min":1.25,"15min":10.369},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.69099999999999995},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":10.406000000000001},"last":0.378}]},{"osd":4,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91700000000000004,"15min":1.3580000000000001},"min":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.376},"max":{"1min":1.036,"5min":4.2119999999999997,"15min":110.39700000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.97399999999999998,"15min":1.399},"min":{"1min":0.752,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.2609999999999999,"5min":7.2030000000000003,"15min":108.777},"last":0.85299999999999998}]},{"osd":5,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.871,"15min":0.92600000000000005},"min":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.39900000000000002},"max":{"1min":1.196,"5min":7.2679999999999998,"15min":17.513000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.85799999999999998,"15min":0.92900000000000005},"min":{"1min":0.64400000000000002,"5min":0.621,"15min":0.52300000000000002},"max":{"1min":1.323,"5min":1.7070000000000001,"15min":10.516999999999999},"last":0.83399999999999996}]},{"osd":6,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83199999999999996,"15min":1.399},"min":{"1min":0.62,"5min":0.52700000000000002,"15min":0.45200000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":138.91300000000001},"last":0.872},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.875,"15min":0.94799999999999995},"min":{"1min":0.65000000000000002,"5min":0.53300000000000003,"15min":0.48299999999999998},"max":{"1min":1.23,"5min":1.766,"15min":10.302},"last":0.66500000000000004}]},{"osd":7,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.92500000000000004,"15min":0.96099999999999997},"min":{"1min":0.66000000000000003,"5min":0.60999999999999999,"15min":0.45200000000000001},"max":{"1min":1.246,"5min":7.3040000000000003,"15min":17.263000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.72599999999999998,"5min":0.54900000000000004,"15min":0.49299999999999999},"max":{"1min":1.397,"5min":7.3339999999999996,"15min":28.613},"last":0.81799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575124,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288308,"kb_used_data":176620,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94083532,"statfs":{"total":96636764160,"available":96341536768,"internally_reserved":0,"allocated":180858880,"data_stored":223905413,"data_compressed":16019215,"data_compressed_allocated":18952192,"data_compressed_original":62304256,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.63500000000000001,"15min":0.58699999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.23599999999999999},"max":{"1min":1.6679999999999999,"5min":13.599,"15min":13.599},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.74299999999999999,"15min":0.63700000000000001},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.318},"max":{"1min":7.1159999999999997,"5min":8.0630000000000006,"15min":8.0630000000000006},"last":0.33800000000000002}]},{"osd":1,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.32000000000000001,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":7.8970000000000002,"5min":7.8970000000000002,"15min":9.3260000000000005},"last":0.313},{"interface":"front","average":{"1min":1.173,"5min":0.81200000000000006,"15min":0.71299999999999997},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.34200000000000003},"max":{"1min":10.657,"5min":10.657,"15min":10.657},"last":0.436}]},{"osd":3,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.046,"15min":0.81899999999999995},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.307},"max":{"1min":7.1639999999999997,"5min":29.123999999999999,"15min":29.123999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":1.498,"5min":0.80600000000000005,"15min":0.66500000000000004},"min":{"1min":0.36199999999999999,"5min":0.312,"15min":0.28000000000000003},"max":{"1min":10.18,"5min":10.18,"15min":10.18},"last":0.38}]},{"osd":4,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97299999999999998,"15min":0.93500000000000005},"min":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.49099999999999999},"max":{"1min":2.9769999999999999,"5min":5.6289999999999996,"15min":7.3630000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.002,"15min":0.92600000000000005},"min":{"1min":0.59299999999999997,"5min":0.51200000000000001,"15min":0.41399999999999998},"max":{"1min":12.723000000000001,"5min":12.723000000000001,"15min":12.723000000000001},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":0.94099999999999995,"15min":0.87},"min":{"1min":0.64900000000000002,"5min":0.501,"15min":0.501},"max":{"1min":6.875,"5min":6.875,"15min":6.875},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.2070000000000001,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.59399999999999997,"15min":0.48399999999999999},"max":{"1min":9.4809999999999999,"5min":9.4809999999999999,"15min":9.4809999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.093,"15min":0.92900000000000005},"min":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":3.5489999999999999,"5min":18.959,"15min":18.959},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.897,"5min":1.325,"15min":1.0369999999999999},"min":{"1min":0.77700000000000002,"5min":0.622,"15min":0.52200000000000002},"max":{"1min":9.4399999999999995,"5min":21.957999999999998,"15min":21.957999999999998},"last":0.81599999999999995}]},{"osd":7,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":1.0800000000000001,"15min":0.98099999999999998},"min":{"1min":0.64200000000000002,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":9.2249999999999996,"5min":9.2249999999999996,"15min":9.2799999999999994},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.7989999999999999,"5min":1.109,"15min":0.97199999999999998},"min":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":9.1679999999999993,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":0.59099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575124,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469316,"kb_used_data":311804,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93902524,"statfs":{"total":96636764160,"available":96156184576,"internally_reserved":0,"allocated":319287296,"data_stored":398560274,"data_compressed":30742535,"data_compressed_allocated":36515840,"data_compressed_original":116101120,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.75700000000000001,"15min":0.69299999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.246},"max":{"1min":0.65400000000000003,"5min":12.311999999999999,"15min":14.538},"last":0.252},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.621,"15min":0.64900000000000002},"min":{"1min":0.33400000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":8.2479999999999993,"15min":14.618},"last":0.41199999999999998}]},{"osd":2,"last update":"Tue Nov 28 21:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.66100000000000003,"15min":0.63100000000000001},"min":{"1min":0.40799999999999997,"5min":0.34899999999999998,"15min":0.31900000000000001},"max":{"1min":1.7150000000000001,"5min":5.1689999999999996,"15min":8.4749999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.44500000000000001,"5min":0.41199999999999998,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":4.7030000000000003,"15min":8.5150000000000006},"last":0.48499999999999999}]},{"osd":3,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.75700000000000001,"15min":0.84699999999999998},"min":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.27600000000000002},"max":{"1min":0.68500000000000005,"5min":13.385,"15min":43.536999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.82499999999999996,"15min":0.84499999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":8.2899999999999991,"15min":18.308},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.98499999999999999,"15min":0.95199999999999996},"min":{"1min":0.498,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":1.0900000000000001,"5min":9.9849999999999994,"15min":9.9849999999999994},"last":0.498},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.98399999999999999,"15min":1.0760000000000001},"min":{"1min":0.58199999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.149,"5min":10.287000000000001,"15min":43.497999999999998},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.93000000000000005,"15min":0.91700000000000004},"min":{"1min":0.72199999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.085,"5min":3.3740000000000001,"15min":5.5350000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.01,"15min":0.995},"min":{"1min":0.874,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1799999999999999,"5min":3.448,"15min":5.7409999999999997},"last":0.92600000000000005}]},{"osd":6,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":1.0349999999999999,"15min":1.0069999999999999},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0629999999999999,"5min":10.023,"15min":10.385},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.0509999999999999,"15min":1.2490000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.59599999999999997},"max":{"1min":1.1719999999999999,"5min":5.9349999999999996,"15min":43.448999999999998},"last":0.77900000000000003}]},{"osd":7,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":1.0580000000000001,"15min":1.02},"min":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.56899999999999995},"max":{"1min":1.135,"5min":5.9779999999999998,"15min":7.96},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.1830000000000001,"15min":1.161},"min":{"1min":0.65700000000000003,"5min":0.55800000000000005,"15min":0.51500000000000001},"max":{"1min":1.109,"5min":15.672000000000001,"15min":42.473999999999997},"last":0.81999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575125,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":346236,"kb_used_data":216500,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94025604,"statfs":{"total":96636764160,"available":96282218496,"internally_reserved":0,"allocated":221696000,"data_stored":277678898,"data_compressed":19875137,"data_compressed_allocated":23535616,"data_compressed_original":79818752,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.7909999999999999,"5min":0.75,"15min":0.56399999999999995},"min":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.215},"max":{"1min":21.814,"5min":21.814,"15min":21.814},"last":0.309},{"interface":"front","average":{"1min":1.8500000000000001,"5min":0.88300000000000001,"15min":0.624},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.248},"max":{"1min":21.925000000000001,"5min":21.925000000000001,"15min":21.925000000000001},"last":0.38800000000000001}]},{"osd":2,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.9399999999999999,"5min":0.93700000000000006,"15min":0.70199999999999996},"min":{"1min":0.47499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":21.852,"5min":21.852,"15min":21.852},"last":0.52200000000000002},{"interface":"front","average":{"1min":1.617,"5min":0.82099999999999995,"15min":0.67600000000000005},"min":{"1min":0.36399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":18.213000000000001,"5min":18.213000000000001,"15min":18.213000000000001},"last":0.34499999999999997}]},{"osd":3,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.083,"15min":0.72199999999999998},"min":{"1min":0.495,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":21.98,"5min":21.98,"15min":21.98},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.857,"5min":1.071,"15min":0.77600000000000002},"min":{"1min":0.437,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":20.931999999999999,"5min":20.931999999999999,"15min":20.931999999999999},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.98599999999999999,"15min":0.92500000000000004},"min":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.42899999999999999},"max":{"1min":4.0810000000000004,"5min":5.2069999999999999,"15min":5.734},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.9379999999999999,"5min":1.1000000000000001,"15min":0.97399999999999998},"min":{"1min":0.65100000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":17.98,"5min":17.98,"15min":17.98},"last":0.83699999999999997}]},{"osd":5,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":2.1960000000000002,"5min":1.1919999999999999,"15min":0.93400000000000005},"min":{"1min":0.65800000000000003,"5min":0.61799999999999999,"15min":0.40699999999999997},"max":{"1min":21.890999999999998,"5min":21.890999999999998,"15min":21.890999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.931,"5min":1.1519999999999999,"15min":0.95799999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":18.181000000000001,"5min":18.181000000000001,"15min":18.181000000000001},"last":0.77300000000000002}]},{"osd":6,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.056,"15min":0.91500000000000004},"min":{"1min":0.63,"5min":0.61799999999999999,"15min":0.48699999999999999},"max":{"1min":4.0410000000000004,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.9670000000000001,"5min":1.1399999999999999,"15min":0.95799999999999996},"min":{"1min":0.68600000000000005,"5min":0.60499999999999998,"15min":0.55100000000000005},"max":{"1min":18.030999999999999,"5min":18.030999999999999,"15min":18.030999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Tue Nov 28 21:56:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.95199999999999996,"15min":0.90800000000000003},"min":{"1min":0.627,"5min":0.627,"15min":0.48999999999999999},"max":{"1min":1.6950000000000001,"5min":4.4279999999999999,"15min":4.4279999999999999},"last":0.875},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.95299999999999996,"15min":0.89600000000000002},"min":{"1min":0.68799999999999994,"5min":0.63,"15min":0.55700000000000005},"max":{"1min":1.627,"5min":3.1819999999999999,"15min":3.806},"last":0.69099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14426112,"data_stored":54526612,"data_compressed":7767250,"data_compressed_allocated":9949184,"data_compressed_original":50065408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24444928,"data_stored":78864793,"data_compressed":10483869,"data_compressed_allocated":13647872,"data_compressed_original":68096000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15634432,"data_stored":48245772,"data_compressed":6171423,"data_compressed_allocated":8048640,"data_compressed_original":40681472,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25956352,"data_stored":94383094,"data_compressed":13023761,"data_compressed_allocated":16736256,"data_compressed_original":85180416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23392256,"data_stored":83887777,"data_compressed":11283626,"data_compressed_allocated":14446592,"data_compressed_original":74960843,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22241280,"data_stored":79693154,"data_compressed":11120127,"data_compressed_allocated":14139392,"data_compressed_original":71610368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":22917120,"data_stored":83888413,"data_compressed":11073880,"data_compressed_allocated":14143488,"data_compressed_original":75132875,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24100864,"data_stored":83058881,"data_compressed":11626802,"data_compressed_allocated":14729216,"data_compressed_original":73707520,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":206610432,"data_stored":222244864,"data_compressed":12071885,"data_compressed_allocated":13537280,"data_compressed_original":29171712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294244352,"data_stored":319377408,"data_compressed":20255951,"data_compressed_allocated":22863872,"data_compressed_original":47996928,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164626432,"data_stored":175341568,"data_compressed":9845077,"data_compressed_allocated":10899456,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":288481280,"data_stored":316022784,"data_compressed":23971937,"data_compressed_allocated":27017216,"data_compressed_original":54558720,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":217907200,"data_stored":237305856,"data_compressed":14188788,"data_compressed_allocated":16142336,"data_compressed_original":35540992,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":220975104,"data_stored":243601408,"data_compressed":17073954,"data_compressed_allocated":19296256,"data_compressed_original":41922560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":237195264,"data_stored":251670528,"data_compressed":13556545,"data_compressed_allocated":15155200,"data_compressed_original":29630464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":192204800,"data_stored":208740352,"data_compressed":13315951,"data_compressed_allocated":15101952,"data_compressed_original":31637504,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:57:24.351 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.352 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.352 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.352 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.352 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.353 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.353 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.353 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.353 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.353 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.354 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.354 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.354 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.354 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.355 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.355 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.355 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.355 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.356 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.356 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.356 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.356 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.357 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.357 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.357 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-11-28T21:34:35.964297+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=35, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:24.357 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T21:57:44.359 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:57:44.518 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bf04dd640 1 Processor -- start 2023-11-28T21:57:44.519 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bf04dd640 1 -- start start 2023-11-28T21:57:44.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bf04dd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5be8150c20 0x7f5be8156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:44.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bf04dd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5be806cf40 0x7f5be8158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:44.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bf04dd640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be8067410 con 0x7f5be806e300 2023-11-28T21:57:44.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bf04dd640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be806e090 con 0x7f5be8150c20 2023-11-28T21:57:44.520 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bf04dd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be8157f30 con 0x7f5be806cf40 2023-11-28T21:57:44.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bee279640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5be8150c20 0x7f5be8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:44.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bee279640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5be8150c20 0x7f5be8156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48046/0 (socket says 172.21.15.12:48046) 2023-11-28T21:57:44.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bee279640 1 -- 172.21.15.12:0/636292944 learned_addr learned my addr 172.21.15.12:0/636292944 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:57:44.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5beda78640 1 --1- 172.21.15.12:0/636292944 >> v1:172.21.15.12:6789/0 conn(0x7f5be806e300 0x7f5be80c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:52944/0 (socket says 172.21.15.12:52944) 2023-11-28T21:57:44.521 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5beea7a640 1 --2- 172.21.15.12:0/636292944 >> v2:172.21.15.144:3300/0 conn(0x7f5be806cf40 0x7f5be8158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:44.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bee279640 1 -- 172.21.15.12:0/636292944 >> v2:172.21.15.144:3300/0 conn(0x7f5be806cf40 msgr2=0x7f5be8158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:57:44.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bee279640 1 --2- 172.21.15.12:0/636292944 >> v2:172.21.15.144:3300/0 conn(0x7f5be806cf40 0x7f5be8158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:44.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bee279640 1 -- 172.21.15.12:0/636292944 >> v1:172.21.15.12:6789/0 conn(0x7f5be806e300 legacy=0x7f5be80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:44.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bee279640 1 -- 172.21.15.12:0/636292944 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be8151000 con 0x7f5be8150c20 2023-11-28T21:57:44.522 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.516+0000 7f5bed277640 1 -- 172.21.15.12:0/636292944 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397777845 0 0) 0x7f5be8067410 con 0x7f5be806e300 2023-11-28T21:57:44.523 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bee279640 1 --2- 172.21.15.12:0/636292944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be8150c20 0x7f5be8156910 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5be806b9d0 tx=0x7f5be401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f560965d5afb458 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:57:44.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bed277640 1 -- 172.21.15.12:0/636292944 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40ab610 con 0x7f5be8150c20 2023-11-28T21:57:44.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bed277640 1 -- 172.21.15.12:0/636292944 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be40ab770 con 0x7f5be8150c20 2023-11-28T21:57:44.524 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bed277640 1 -- 172.21.15.12:0/636292944 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40abb00 con 0x7f5be8150c20 2023-11-28T21:57:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bf04dd640 1 -- 172.21.15.12:0/636292944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be8150c20 msgr2=0x7f5be8156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bf04dd640 1 --2- 172.21.15.12:0/636292944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be8150c20 0x7f5be8156910 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5be806b9d0 tx=0x7f5be401c500 comp rx=0 tx=0).stop 2023-11-28T21:57:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bf04dd640 1 -- 172.21.15.12:0/636292944 shutdown_connections 2023-11-28T21:57:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bf04dd640 1 --2- 172.21.15.12:0/636292944 >> v2:172.21.15.144:3300/0 conn(0x7f5be806cf40 0x7f5be8158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:44.525 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bf04dd640 1 --2- 172.21.15.12:0/636292944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be8150c20 0x7f5be8156910 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bf04dd640 1 -- 172.21.15.12:0/636292944 >> 172.21.15.12:0/636292944 conn(0x7f5be8066360 msgr2=0x7f5be8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:57:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bf04dd640 1 -- 172.21.15.12:0/636292944 shutdown_connections 2023-11-28T21:57:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.520+0000 7f5bf04dd640 1 -- 172.21.15.12:0/636292944 wait complete. 2023-11-28T21:57:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 Processor -- start 2023-11-28T21:57:44.526 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 -- start start 2023-11-28T21:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be806cf40 0x7f5be81f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be806e300 0x7f5be80c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be8150c20 0x7f5be814e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be81ee920 con 0x7f5be806cf40 2023-11-28T21:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be814df80 con 0x7f5be806e300 2023-11-28T21:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be81f1900 con 0x7f5be8150c20 2023-11-28T21:57:44.528 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bee279640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be806cf40 0x7f5be81f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bee279640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be806cf40 0x7f5be81f13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48054/0 (socket says 172.21.15.12:48054) 2023-11-28T21:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bee279640 1 -- 172.21.15.12:0/3018273263 learned_addr learned my addr 172.21.15.12:0/3018273263 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5beda78640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be806e300 0x7f5be80c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5beea7a640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be8150c20 0x7f5be814e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bee279640 1 -- 172.21.15.12:0/3018273263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be8150c20 msgr2=0x7f5be814e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bee279640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be8150c20 0x7f5be814e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bee279640 1 -- 172.21.15.12:0/3018273263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be806e300 msgr2=0x7f5be80c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bee279640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be806e300 0x7f5be80c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:44.529 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bee279640 1 -- 172.21.15.12:0/3018273263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be40094d0 con 0x7f5be806cf40 2023-11-28T21:57:44.530 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bee279640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be806cf40 0x7f5be81f13c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5be401abb0 tx=0x7f5be401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:57:44.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bdeffd640 1 -- 172.21.15.12:0/3018273263 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40abcf0 con 0x7f5be806cf40 2023-11-28T21:57:44.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bdeffd640 1 -- 172.21.15.12:0/3018273263 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be40b8070 con 0x7f5be806cf40 2023-11-28T21:57:44.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bdeffd640 1 -- 172.21.15.12:0/3018273263 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4004220 con 0x7f5be806cf40 2023-11-28T21:57:44.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 -- 172.21.15.12:0/3018273263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be814dca0 con 0x7f5be806cf40 2023-11-28T21:57:44.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 -- 172.21.15.12:0/3018273263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be814f5a0 con 0x7f5be806cf40 2023-11-28T21:57:44.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.524+0000 7f5bf04dd640 1 -- 172.21.15.12:0/3018273263 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be8074f60 con 0x7f5be806cf40 2023-11-28T21:57:44.531 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.528+0000 7f5bdeffd640 1 -- 172.21.15.12:0/3018273263 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5be4004380 con 0x7f5be806cf40 2023-11-28T21:57:44.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.528+0000 7f5bdeffd640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5bcc066470 0x7f5bcc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:57:44.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.528+0000 7f5bdeffd640 1 -- 172.21.15.12:0/3018273263 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f5be411e710 con 0x7f5be806cf40 2023-11-28T21:57:44.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.528+0000 7f5beda78640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5bcc066470 0x7f5bcc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:57:44.532 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.528+0000 7f5beda78640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5bcc066470 0x7f5bcc068930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5be80c6060 tx=0x7f5bd8007b80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:57:44.535 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.532+0000 7f5bdeffd640 1 -- 172.21.15.12:0/3018273263 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5be40eb620 con 0x7f5be806cf40 2023-11-28T21:57:44.690 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.688+0000 7f5bf04dd640 1 -- 172.21.15.12:0/3018273263 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5be8158d60 con 0x7f5bcc066470 2023-11-28T21:57:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.696+0000 7f5bdeffd640 1 -- 172.21.15.12:0/3018273263 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+376769 (secure 0 0 0) 0x7f5bd80071f0 con 0x7f5bcc066470 2023-11-28T21:57:44.704 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:57:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:57:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 -- 172.21.15.12:0/3018273263 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5bcc066470 msgr2=0x7f5bcc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5bcc066470 0x7f5bcc068930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5be80c6060 tx=0x7f5bd8007b80 comp rx=0 tx=0).stop 2023-11-28T21:57:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 -- 172.21.15.12:0/3018273263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be806cf40 msgr2=0x7f5be81f13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be806cf40 0x7f5be81f13c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5be401abb0 tx=0x7f5be401ad40 comp rx=0 tx=0).stop 2023-11-28T21:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 -- 172.21.15.12:0/3018273263 shutdown_connections 2023-11-28T21:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5bcc066470 0x7f5bcc068930 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5be8150c20 0x7f5be814e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5be806e300 0x7f5be80c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 --2- 172.21.15.12:0/3018273263 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5be806cf40 0x7f5be81f13c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 -- 172.21.15.12:0/3018273263 >> 172.21.15.12:0/3018273263 conn(0x7f5be8066360 msgr2=0x7f5be8159070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:57:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 -- 172.21.15.12:0/3018273263 shutdown_connections 2023-11-28T21:57:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:57:44.700+0000 7f5bf04dd640 1 -- 172.21.15.12:0/3018273263 wait complete. 2023-11-28T21:57:44.723 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":742,"stamp":"2023-11-28T21:57:44.230060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1231787149,"num_objects":10538,"num_object_clones":0,"num_object_copies":41756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10538,"num_whiteouts":0,"num_read":9144,"num_read_kb":2407205,"num_write":56637,"num_write_kb":2360239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":687608,"num_omap_keys":1199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77305,"ondisk_log_size":77305,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2999684,"kb_used_data":1953412,"kb_used_omap":12,"kb_used_meta":1046195,"kb_avail":751975036,"statfs":{"total":773094113280,"available":770022436864,"internally_reserved":0,"allocated":2000293888,"data_stored":2584693461,"data_compressed":206930852,"data_compressed_allocated":245993472,"data_compressed_original":832827286,"omap_allocated":12751,"internal_metadata":1071304241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003289"},"pg_stats":[{"pgid":"2.1f","version":"52'347","reported_seq":443,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.479855+0000","last_change":"2023-11-28T21:56:50.479855+0000","last_active":"2023-11-28T21:56:50.479855+0000","last_peered":"2023-11-28T21:56:50.479855+0000","last_clean":"2023-11-28T21:56:50.479855+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:56:50.479855+0000","last_undegraded":"2023-11-28T21:56:50.479855+0000","last_fullsized":"2023-11-28T21:56:50.479855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'347","last_scrub_stamp":"2023-11-28T21:56:50.479765+0000","last_deep_scrub":"52'347","last_deep_scrub_stamp":"2023-11-28T21:56:50.479765+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.479765+0000","objects_scrubbed":6,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:00.234465+0000","scrub_duration":0.028234239000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":61536,"num_write":310,"num_write_kb":8499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":41082,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"52'277","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.442324+0000","last_change":"2023-11-28T21:57:04.442342+0000","last_active":"2023-11-28T21:57:04.442324+0000","last_peered":"2023-11-28T21:57:04.442324+0000","last_clean":"2023-11-28T21:57:04.442324+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:57:04.442324+0000","last_undegraded":"2023-11-28T21:57:04.442324+0000","last_fullsized":"2023-11-28T21:57:04.442324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'277","last_scrub_stamp":"2023-11-28T21:57:04.442253+0000","last_deep_scrub":"52'277","last_deep_scrub_stamp":"2023-11-28T21:57:04.442253+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.442253+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:47:27.933208+0000","scrub_duration":0.0025024589999999998,"objects_trimmed":0,"snaptrim_duration":0.28950874199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"52'1318","reported_seq":3650,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:22.822296+0000","last_change":"2023-11-28T21:57:22.822343+0000","last_active":"2023-11-28T21:57:22.822296+0000","last_peered":"2023-11-28T21:57:22.822296+0000","last_clean":"2023-11-28T21:57:22.822296+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:57:22.822296+0000","last_undegraded":"2023-11-28T21:57:22.822296+0000","last_fullsized":"2023-11-28T21:57:22.822296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1318","last_scrub_stamp":"2023-11-28T21:57:22.822202+0000","last_deep_scrub":"52'1318","last_deep_scrub_stamp":"2023-11-28T21:57:22.822202+0000","last_clean_scrub_stamp":"2023-11-28T21:57:22.822202+0000","objects_scrubbed":349,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:14:46.637231+0000","scrub_duration":0.186113631,"objects_trimmed":0,"snaptrim_duration":0.29329539100000002,"stat_sum":{"num_bytes":30193112,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":229,"num_read_kb":12319,"num_write":1291,"num_write_kb":57664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"52'534","reported_seq":618,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:54.494340+0000","last_change":"2023-11-28T21:56:54.494393+0000","last_active":"2023-11-28T21:56:54.494340+0000","last_peered":"2023-11-28T21:56:54.494340+0000","last_clean":"2023-11-28T21:56:54.494340+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:56:54.494340+0000","last_undegraded":"2023-11-28T21:56:54.494340+0000","last_fullsized":"2023-11-28T21:56:54.494340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'534","last_scrub_stamp":"2023-11-28T21:56:54.494241+0000","last_deep_scrub":"52'534","last_deep_scrub_stamp":"2023-11-28T21:56:54.494241+0000","last_clean_scrub_stamp":"2023-11-28T21:56:54.494241+0000","objects_scrubbed":7,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:51:32.158044+0000","scrub_duration":0.057898067999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":81920,"num_write":485,"num_write_kb":24906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8778,"num_omap_keys":17,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"52'269","reported_seq":2362,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:42.409939+0000","last_change":"2023-11-28T21:56:42.409939+0000","last_active":"2023-11-28T21:56:42.409939+0000","last_peered":"2023-11-28T21:56:42.409939+0000","last_clean":"2023-11-28T21:56:42.409939+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:56:42.409939+0000","last_undegraded":"2023-11-28T21:56:42.409939+0000","last_fullsized":"2023-11-28T21:56:42.409939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'269","last_scrub_stamp":"2023-11-28T21:56:42.409827+0000","last_deep_scrub":"52'269","last_deep_scrub_stamp":"2023-11-28T21:56:42.409827+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.409827+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:28:29.334975+0000","scrub_duration":0.0029008010000000002,"objects_trimmed":0,"snaptrim_duration":0.23094383900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"52'1401","reported_seq":3755,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.530886+0000","last_change":"2023-11-28T21:57:17.530942+0000","last_active":"2023-11-28T21:57:17.530886+0000","last_peered":"2023-11-28T21:57:17.530886+0000","last_clean":"2023-11-28T21:57:17.530886+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:57:17.530886+0000","last_undegraded":"2023-11-28T21:57:17.530886+0000","last_fullsized":"2023-11-28T21:57:17.530886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1401","last_scrub_stamp":"2023-11-28T21:57:17.530763+0000","last_deep_scrub":"52'1401","last_deep_scrub_stamp":"2023-11-28T21:57:17.530763+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.530763+0000","objects_scrubbed":330,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:05:20.452581+0000","scrub_duration":0.19727914399999999,"objects_trimmed":0,"snaptrim_duration":0.21613474099999999,"stat_sum":{"num_bytes":11729902,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":251,"num_read_kb":24615,"num_write":1375,"num_write_kb":48688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"52'764","reported_seq":867,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.611643+0000","last_change":"2023-11-28T21:56:50.611643+0000","last_active":"2023-11-28T21:56:50.611643+0000","last_peered":"2023-11-28T21:56:50.611643+0000","last_clean":"2023-11-28T21:56:50.611643+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:56:50.611643+0000","last_undegraded":"2023-11-28T21:56:50.611643+0000","last_fullsized":"2023-11-28T21:56:50.611643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'764","last_scrub_stamp":"2023-11-28T21:56:50.611223+0000","last_deep_scrub":"52'764","last_deep_scrub_stamp":"2023-11-28T21:56:50.611223+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.611223+0000","objects_scrubbed":5,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:28:25.261471+0000","scrub_duration":0.059433334999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":59,"num_read_kb":131099,"num_write":715,"num_write_kb":20835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13932,"num_omap_keys":27,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"52'294","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.715849+0000","last_change":"2023-11-28T21:57:17.715901+0000","last_active":"2023-11-28T21:57:17.715849+0000","last_peered":"2023-11-28T21:57:17.715849+0000","last_clean":"2023-11-28T21:57:17.715849+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:57:17.715849+0000","last_undegraded":"2023-11-28T21:57:17.715849+0000","last_fullsized":"2023-11-28T21:57:17.715849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'294","last_scrub_stamp":"2023-11-28T21:57:17.715645+0000","last_deep_scrub":"52'294","last_deep_scrub_stamp":"2023-11-28T21:57:17.715645+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.715645+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:16:17.284423+0000","scrub_duration":0.0037761639999999998,"objects_trimmed":0,"snaptrim_duration":0.30458173900000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"52'1324","reported_seq":3652,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.623143+0000","last_change":"2023-11-28T21:56:57.623143+0000","last_active":"2023-11-28T21:56:57.623143+0000","last_peered":"2023-11-28T21:56:57.623143+0000","last_clean":"2023-11-28T21:56:57.623143+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:56:57.623143+0000","last_undegraded":"2023-11-28T21:56:57.623143+0000","last_fullsized":"2023-11-28T21:56:57.623143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1324","last_scrub_stamp":"2023-11-28T21:56:57.622774+0000","last_deep_scrub":"52'1324","last_deep_scrub_stamp":"2023-11-28T21:56:57.622774+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.622774+0000","objects_scrubbed":314,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:08:12.479556+0000","scrub_duration":0.25549122899999999,"objects_trimmed":0,"snaptrim_duration":0.23712909900000001,"stat_sum":{"num_bytes":32661416,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":226,"num_read_kb":7690,"num_write":1300,"num_write_kb":56471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"52'96","reported_seq":176,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:47.599947+0000","last_change":"2023-11-28T21:56:47.599947+0000","last_active":"2023-11-28T21:56:47.599947+0000","last_peered":"2023-11-28T21:56:47.599947+0000","last_clean":"2023-11-28T21:56:47.599947+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:56:47.599947+0000","last_undegraded":"2023-11-28T21:56:47.599947+0000","last_fullsized":"2023-11-28T21:56:47.599947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'96","last_scrub_stamp":"2023-11-28T21:56:47.599552+0000","last_deep_scrub":"52'96","last_deep_scrub_stamp":"2023-11-28T21:56:47.599552+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.599552+0000","objects_scrubbed":2,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:41:39.184546+0000","scrub_duration":0.012005902000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36891,"num_write":51,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10256,"num_omap_keys":19,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"52'283","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.791963+0000","last_change":"2023-11-28T21:57:09.791963+0000","last_active":"2023-11-28T21:57:09.791963+0000","last_peered":"2023-11-28T21:57:09.791963+0000","last_clean":"2023-11-28T21:57:09.791963+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:57:09.791963+0000","last_undegraded":"2023-11-28T21:57:09.791963+0000","last_fullsized":"2023-11-28T21:57:09.791963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:57:09.791711+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:57:09.791711+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.791711+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:05.807760+0000","scrub_duration":0.0033624179999999998,"objects_trimmed":0,"snaptrim_duration":0.30416134,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"52'1294","reported_seq":3632,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:56.624661+0000","last_change":"2023-11-28T21:56:56.624661+0000","last_active":"2023-11-28T21:56:56.624661+0000","last_peered":"2023-11-28T21:56:56.624661+0000","last_clean":"2023-11-28T21:56:56.624661+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:56:56.624661+0000","last_undegraded":"2023-11-28T21:56:56.624661+0000","last_fullsized":"2023-11-28T21:56:56.624661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1294","last_scrub_stamp":"2023-11-28T21:56:56.624359+0000","last_deep_scrub":"52'1294","last_deep_scrub_stamp":"2023-11-28T21:56:56.624359+0000","last_clean_scrub_stamp":"2023-11-28T21:56:56.624359+0000","objects_scrubbed":312,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:59:20.046078+0000","scrub_duration":0.25756709,"objects_trimmed":0,"snaptrim_duration":0.26667665699999998,"stat_sum":{"num_bytes":37281632,"num_objects":312,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":0,"num_read":236,"num_read_kb":13169,"num_write":1275,"num_write_kb":61269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"52'303","reported_seq":2395,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:23.239181+0000","last_change":"2023-11-28T21:57:23.239234+0000","last_active":"2023-11-28T21:57:23.239181+0000","last_peered":"2023-11-28T21:57:23.239181+0000","last_clean":"2023-11-28T21:57:23.239181+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:57:23.239181+0000","last_undegraded":"2023-11-28T21:57:23.239181+0000","last_fullsized":"2023-11-28T21:57:23.239181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'303","last_scrub_stamp":"2023-11-28T21:57:23.238975+0000","last_deep_scrub":"52'303","last_deep_scrub_stamp":"2023-11-28T21:57:23.238975+0000","last_clean_scrub_stamp":"2023-11-28T21:57:23.238975+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:37:16.094494+0000","scrub_duration":0.0035136970000000001,"objects_trimmed":0,"snaptrim_duration":0.200606007,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"52'627","reported_seq":731,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.667245+0000","last_change":"2023-11-28T21:56:48.667245+0000","last_active":"2023-11-28T21:56:48.667245+0000","last_peered":"2023-11-28T21:56:48.667245+0000","last_clean":"2023-11-28T21:56:48.667245+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:56:48.667245+0000","last_undegraded":"2023-11-28T21:56:48.667245+0000","last_fullsized":"2023-11-28T21:56:48.667245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'627","last_scrub_stamp":"2023-11-28T21:56:48.666890+0000","last_deep_scrub":"52'627","last_deep_scrub_stamp":"2023-11-28T21:56:48.666890+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.666890+0000","objects_scrubbed":10,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:43:39.567306+0000","scrub_duration":0.095730051999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":114718,"num_write":571,"num_write_kb":20791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64058,"num_omap_keys":115,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"52'1331","reported_seq":3673,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.617376+0000","last_change":"2023-11-28T21:57:01.617376+0000","last_active":"2023-11-28T21:57:01.617376+0000","last_peered":"2023-11-28T21:57:01.617376+0000","last_clean":"2023-11-28T21:57:01.617376+0000","last_became_active":"2023-11-28T21:35:25.616978+0000","last_became_peered":"2023-11-28T21:35:25.616978+0000","last_unstale":"2023-11-28T21:57:01.617376+0000","last_undegraded":"2023-11-28T21:57:01.617376+0000","last_fullsized":"2023-11-28T21:57:01.617376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1331","last_scrub_stamp":"2023-11-28T21:57:01.617097+0000","last_deep_scrub":"52'1331","last_deep_scrub_stamp":"2023-11-28T21:57:01.617097+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.617097+0000","objects_scrubbed":332,"log_size":1331,"log_dups_size":0,"ondisk_log_size":1331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:25.432186+0000","scrub_duration":0.26810302699999999,"objects_trimmed":0,"snaptrim_duration":0.23129675,"stat_sum":{"num_bytes":32707084,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":229,"num_read_kb":4127,"num_write":1308,"num_write_kb":57872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"52'305","reported_seq":2397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:00.459939+0000","last_change":"2023-11-28T21:57:00.459939+0000","last_active":"2023-11-28T21:57:00.459939+0000","last_peered":"2023-11-28T21:57:00.459939+0000","last_clean":"2023-11-28T21:57:00.459939+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:57:00.459939+0000","last_undegraded":"2023-11-28T21:57:00.459939+0000","last_fullsized":"2023-11-28T21:57:00.459939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'305","last_scrub_stamp":"2023-11-28T21:57:00.459826+0000","last_deep_scrub":"52'305","last_deep_scrub_stamp":"2023-11-28T21:57:00.459826+0000","last_clean_scrub_stamp":"2023-11-28T21:57:00.459826+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:07.320593+0000","scrub_duration":0.0022599199999999999,"objects_trimmed":0,"snaptrim_duration":0.28777871399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"52'962","reported_seq":1049,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.736149+0000","last_change":"2023-11-28T21:57:01.736178+0000","last_active":"2023-11-28T21:57:01.736149+0000","last_peered":"2023-11-28T21:57:01.736149+0000","last_clean":"2023-11-28T21:57:01.736149+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:57:01.736149+0000","last_undegraded":"2023-11-28T21:57:01.736149+0000","last_fullsized":"2023-11-28T21:57:01.736149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'962","last_scrub_stamp":"2023-11-28T21:57:01.736083+0000","last_deep_scrub":"52'962","last_deep_scrub_stamp":"2023-11-28T21:57:01.736083+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.736083+0000","objects_scrubbed":7,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:06:38.504425+0000","scrub_duration":0.040000139999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":75782,"num_write":935,"num_write_kb":13441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31916,"num_omap_keys":58,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"52'1416","reported_seq":3764,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:59.657597+0000","last_change":"2023-11-28T21:56:59.657597+0000","last_active":"2023-11-28T21:56:59.657597+0000","last_peered":"2023-11-28T21:56:59.657597+0000","last_clean":"2023-11-28T21:56:59.657597+0000","last_became_active":"2023-11-28T21:35:25.790674+0000","last_became_peered":"2023-11-28T21:35:25.790674+0000","last_unstale":"2023-11-28T21:56:59.657597+0000","last_undegraded":"2023-11-28T21:56:59.657597+0000","last_fullsized":"2023-11-28T21:56:59.657597+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1416","last_scrub_stamp":"2023-11-28T21:56:59.657277+0000","last_deep_scrub":"52'1416","last_deep_scrub_stamp":"2023-11-28T21:56:59.657277+0000","last_clean_scrub_stamp":"2023-11-28T21:56:59.657277+0000","objects_scrubbed":354,"log_size":1416,"log_dups_size":0,"ondisk_log_size":1416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:47.719457+0000","scrub_duration":0.27211498200000001,"objects_trimmed":0,"snaptrim_duration":0.229623311,"stat_sum":{"num_bytes":17731185,"num_objects":354,"num_object_clones":0,"num_object_copies":1416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":234,"num_read_kb":20007,"num_write":1382,"num_write_kb":71863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"52'306","reported_seq":2398,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.421624+0000","last_change":"2023-11-28T21:56:43.421641+0000","last_active":"2023-11-28T21:56:43.421624+0000","last_peered":"2023-11-28T21:56:43.421624+0000","last_clean":"2023-11-28T21:56:43.421624+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:56:43.421624+0000","last_undegraded":"2023-11-28T21:56:43.421624+0000","last_fullsized":"2023-11-28T21:56:43.421624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'306","last_scrub_stamp":"2023-11-28T21:56:43.421546+0000","last_deep_scrub":"52'306","last_deep_scrub_stamp":"2023-11-28T21:56:43.421546+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.421546+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:18:49.309763+0000","scrub_duration":0.001616658,"objects_trimmed":0,"snaptrim_duration":0.29027087200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"52'741","reported_seq":818,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:58.924936+0000","last_change":"2023-11-28T21:56:42.774090+0000","last_active":"2023-11-28T21:56:58.924936+0000","last_peered":"2023-11-28T21:56:58.924936+0000","last_clean":"2023-11-28T21:56:58.924936+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:56:58.924936+0000","last_undegraded":"2023-11-28T21:56:58.924936+0000","last_fullsized":"2023-11-28T21:56:58.924936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'734","last_scrub_stamp":"2023-11-28T21:56:42.773996+0000","last_deep_scrub":"52'734","last_deep_scrub_stamp":"2023-11-28T21:56:42.773996+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.773996+0000","objects_scrubbed":6,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:44:34.363577+0000","scrub_duration":0.055636247999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11383300,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":32782,"num_write":691,"num_write_kb":11471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8864,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"52'1341","reported_seq":3707,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:23.899760+0000","last_change":"2023-11-28T21:57:23.899805+0000","last_active":"2023-11-28T21:57:23.899760+0000","last_peered":"2023-11-28T21:57:23.899760+0000","last_clean":"2023-11-28T21:57:23.899760+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:57:23.899760+0000","last_undegraded":"2023-11-28T21:57:23.899760+0000","last_fullsized":"2023-11-28T21:57:23.899760+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1341","last_scrub_stamp":"2023-11-28T21:57:23.899656+0000","last_deep_scrub":"52'1341","last_deep_scrub_stamp":"2023-11-28T21:57:23.899656+0000","last_clean_scrub_stamp":"2023-11-28T21:57:23.899656+0000","objects_scrubbed":336,"log_size":1341,"log_dups_size":0,"ondisk_log_size":1341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:35.370900+0000","scrub_duration":0.18968862,"objects_trimmed":0,"snaptrim_duration":0.24643899999999999,"stat_sum":{"num_bytes":14380393,"num_objects":336,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":262,"num_read_kb":16758,"num_write":1318,"num_write_kb":45879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"52'294","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.274947+0000","last_change":"2023-11-28T21:56:48.275049+0000","last_active":"2023-11-28T21:56:48.274947+0000","last_peered":"2023-11-28T21:56:48.274947+0000","last_clean":"2023-11-28T21:56:48.274947+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:56:48.274947+0000","last_undegraded":"2023-11-28T21:56:48.274947+0000","last_fullsized":"2023-11-28T21:56:48.274947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'294","last_scrub_stamp":"2023-11-28T21:56:48.274731+0000","last_deep_scrub":"52'294","last_deep_scrub_stamp":"2023-11-28T21:56:48.274731+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.274731+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:02:56.783147+0000","scrub_duration":0.0035010150000000001,"objects_trimmed":0,"snaptrim_duration":0.21610210399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"52'343","reported_seq":422,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.528829+0000","last_change":"2023-11-28T21:56:57.528897+0000","last_active":"2023-11-28T21:56:57.528829+0000","last_peered":"2023-11-28T21:56:57.528829+0000","last_clean":"2023-11-28T21:56:57.528829+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:56:57.528829+0000","last_undegraded":"2023-11-28T21:56:57.528829+0000","last_fullsized":"2023-11-28T21:56:57.528829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'343","last_scrub_stamp":"2023-11-28T21:56:57.528632+0000","last_deep_scrub":"52'343","last_deep_scrub_stamp":"2023-11-28T21:56:57.528632+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.528632+0000","objects_scrubbed":1,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:11:06.480768+0000","scrub_duration":0.0086859099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":49164,"num_write":266,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"52'1262","reported_seq":3571,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.616628+0000","last_change":"2023-11-28T21:57:16.616828+0000","last_active":"2023-11-28T21:57:16.616628+0000","last_peered":"2023-11-28T21:57:16.616628+0000","last_clean":"2023-11-28T21:57:16.616628+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:57:16.616628+0000","last_undegraded":"2023-11-28T21:57:16.616628+0000","last_fullsized":"2023-11-28T21:57:16.616628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1262","last_scrub_stamp":"2023-11-28T21:57:16.616299+0000","last_deep_scrub":"52'1262","last_deep_scrub_stamp":"2023-11-28T21:57:16.616299+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.616299+0000","objects_scrubbed":318,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:23:10.854185+0000","scrub_duration":0.284423806,"objects_trimmed":0,"snaptrim_duration":0.244586779,"stat_sum":{"num_bytes":50267474,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":206,"num_read_kb":15432,"num_write":1240,"num_write_kb":82194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"52'302","reported_seq":2394,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:05.285530+0000","last_change":"2023-11-28T21:57:05.285554+0000","last_active":"2023-11-28T21:57:05.285530+0000","last_peered":"2023-11-28T21:57:05.285530+0000","last_clean":"2023-11-28T21:57:05.285530+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:57:05.285530+0000","last_undegraded":"2023-11-28T21:57:05.285530+0000","last_fullsized":"2023-11-28T21:57:05.285530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'302","last_scrub_stamp":"2023-11-28T21:57:05.285424+0000","last_deep_scrub":"52'302","last_deep_scrub_stamp":"2023-11-28T21:57:05.285424+0000","last_clean_scrub_stamp":"2023-11-28T21:57:05.285424+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:27.928951+0000","scrub_duration":0.0020171199999999999,"objects_trimmed":0,"snaptrim_duration":0.26728515899999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"52'172","reported_seq":253,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.150811+0000","last_change":"2023-11-28T21:56:49.150879+0000","last_active":"2023-11-28T21:56:49.150811+0000","last_peered":"2023-11-28T21:56:49.150811+0000","last_clean":"2023-11-28T21:56:49.150811+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:56:49.150811+0000","last_undegraded":"2023-11-28T21:56:49.150811+0000","last_fullsized":"2023-11-28T21:56:49.150811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'172","last_scrub_stamp":"2023-11-28T21:56:49.150620+0000","last_deep_scrub":"52'172","last_deep_scrub_stamp":"2023-11-28T21:56:49.150620+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.150620+0000","objects_scrubbed":4,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:13:50.517958+0000","scrub_duration":0.035921929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":45,"num_read_kb":24615,"num_write":148,"num_write_kb":4216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6802,"num_omap_keys":13,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"52'1368","reported_seq":3722,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.407745+0000","last_change":"2023-11-28T21:57:11.407745+0000","last_active":"2023-11-28T21:57:11.407745+0000","last_peered":"2023-11-28T21:57:11.407745+0000","last_clean":"2023-11-28T21:57:11.407745+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:57:11.407745+0000","last_undegraded":"2023-11-28T21:57:11.407745+0000","last_fullsized":"2023-11-28T21:57:11.407745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1368","last_scrub_stamp":"2023-11-28T21:57:11.407483+0000","last_deep_scrub":"52'1368","last_deep_scrub_stamp":"2023-11-28T21:57:11.407483+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.407483+0000","objects_scrubbed":316,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:58:46.833003+0000","scrub_duration":0.207722514,"objects_trimmed":0,"snaptrim_duration":0.21672229300000001,"stat_sum":{"num_bytes":16299977,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":250,"num_read_kb":17253,"num_write":1343,"num_write_kb":69509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"52'268","reported_seq":2360,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.444623+0000","last_change":"2023-11-28T21:56:50.444642+0000","last_active":"2023-11-28T21:56:50.444623+0000","last_peered":"2023-11-28T21:56:50.444623+0000","last_clean":"2023-11-28T21:56:50.444623+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:56:50.444623+0000","last_undegraded":"2023-11-28T21:56:50.444623+0000","last_fullsized":"2023-11-28T21:56:50.444623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'268","last_scrub_stamp":"2023-11-28T21:56:50.444546+0000","last_deep_scrub":"52'268","last_deep_scrub_stamp":"2023-11-28T21:56:50.444546+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.444546+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:58:47.104569+0000","scrub_duration":0.0027754860000000002,"objects_trimmed":0,"snaptrim_duration":0.29009278799999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"52'248","reported_seq":326,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:47.891843+0000","last_change":"2023-11-28T21:56:47.891843+0000","last_active":"2023-11-28T21:56:47.891843+0000","last_peered":"2023-11-28T21:56:47.891843+0000","last_clean":"2023-11-28T21:56:47.891843+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:56:47.891843+0000","last_undegraded":"2023-11-28T21:56:47.891843+0000","last_fullsized":"2023-11-28T21:56:47.891843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'247","last_scrub_stamp":"2023-11-28T21:56:47.891725+0000","last_deep_scrub":"52'247","last_deep_scrub_stamp":"2023-11-28T21:56:47.891725+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.891725+0000","objects_scrubbed":3,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:49:45.140707+0000","scrub_duration":0.119752109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":607,"num_write":448,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8486,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"52'1386","reported_seq":3722,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.550973+0000","last_change":"2023-11-28T21:57:11.551001+0000","last_active":"2023-11-28T21:57:11.550973+0000","last_peered":"2023-11-28T21:57:11.550973+0000","last_clean":"2023-11-28T21:57:11.550973+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:57:11.550973+0000","last_undegraded":"2023-11-28T21:57:11.550973+0000","last_fullsized":"2023-11-28T21:57:11.550973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1386","last_scrub_stamp":"2023-11-28T21:57:11.550878+0000","last_deep_scrub":"52'1386","last_deep_scrub_stamp":"2023-11-28T21:57:11.550878+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.550878+0000","objects_scrubbed":313,"log_size":1386,"log_dups_size":0,"ondisk_log_size":1386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:03:55.485268+0000","scrub_duration":0.14493951099999999,"objects_trimmed":0,"snaptrim_duration":0.287791554,"stat_sum":{"num_bytes":19942162,"num_objects":313,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":232,"num_read_kb":16373,"num_write":1352,"num_write_kb":52080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"52'300","reported_seq":2394,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:12.806224+0000","last_change":"2023-11-28T21:57:12.806224+0000","last_active":"2023-11-28T21:57:12.806224+0000","last_peered":"2023-11-28T21:57:12.806224+0000","last_clean":"2023-11-28T21:57:12.806224+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:57:12.806224+0000","last_undegraded":"2023-11-28T21:57:12.806224+0000","last_fullsized":"2023-11-28T21:57:12.806224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'300","last_scrub_stamp":"2023-11-28T21:57:12.806104+0000","last_deep_scrub":"52'300","last_deep_scrub_stamp":"2023-11-28T21:57:12.806104+0000","last_clean_scrub_stamp":"2023-11-28T21:57:12.806104+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:07:35.352399+0000","scrub_duration":0.0019360530000000001,"objects_trimmed":0,"snaptrim_duration":0.24779062600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"52'593","reported_seq":681,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.579266+0000","last_change":"2023-11-28T21:56:55.579266+0000","last_active":"2023-11-28T21:56:55.579266+0000","last_peered":"2023-11-28T21:56:55.579266+0000","last_clean":"2023-11-28T21:56:55.579266+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:56:55.579266+0000","last_undegraded":"2023-11-28T21:56:55.579266+0000","last_fullsized":"2023-11-28T21:56:55.579266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'593","last_scrub_stamp":"2023-11-28T21:56:55.579027+0000","last_deep_scrub":"52'593","last_deep_scrub_stamp":"2023-11-28T21:56:55.579027+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.579027+0000","objects_scrubbed":6,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:56:30.723466+0000","scrub_duration":0.077952980000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":81538,"num_write":523,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17674,"num_omap_keys":32,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"52'1410","reported_seq":3764,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:34.517241+0000","last_change":"2023-11-28T21:57:34.517241+0000","last_active":"2023-11-28T21:57:34.517241+0000","last_peered":"2023-11-28T21:57:34.517241+0000","last_clean":"2023-11-28T21:57:34.517241+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:57:34.517241+0000","last_undegraded":"2023-11-28T21:57:34.517241+0000","last_fullsized":"2023-11-28T21:57:34.517241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1410","last_scrub_stamp":"2023-11-28T21:57:34.517108+0000","last_deep_scrub":"52'1410","last_deep_scrub_stamp":"2023-11-28T21:57:34.517108+0000","last_clean_scrub_stamp":"2023-11-28T21:57:34.517108+0000","objects_scrubbed":365,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:11:10.145700+0000","scrub_duration":0.20779793599999999,"objects_trimmed":0,"snaptrim_duration":0.27155133199999998,"stat_sum":{"num_bytes":49110889,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":248,"num_read_kb":24602,"num_write":1394,"num_write_kb":89036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"52'284","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:00.790304+0000","last_change":"2023-11-28T21:57:00.790304+0000","last_active":"2023-11-28T21:57:00.790304+0000","last_peered":"2023-11-28T21:57:00.790304+0000","last_clean":"2023-11-28T21:57:00.790304+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:57:00.790304+0000","last_undegraded":"2023-11-28T21:57:00.790304+0000","last_fullsized":"2023-11-28T21:57:00.790304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'284","last_scrub_stamp":"2023-11-28T21:57:00.790051+0000","last_deep_scrub":"52'284","last_deep_scrub_stamp":"2023-11-28T21:57:00.790051+0000","last_clean_scrub_stamp":"2023-11-28T21:57:00.790051+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:01:01.704329+0000","scrub_duration":0.0035905590000000001,"objects_trimmed":0,"snaptrim_duration":0.30430776599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"52'167","reported_seq":238,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.743275+0000","last_change":"2023-11-28T21:56:55.743275+0000","last_active":"2023-11-28T21:56:55.743275+0000","last_peered":"2023-11-28T21:56:55.743275+0000","last_clean":"2023-11-28T21:56:55.743275+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:56:55.743275+0000","last_undegraded":"2023-11-28T21:56:55.743275+0000","last_fullsized":"2023-11-28T21:56:55.743275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'167","last_scrub_stamp":"2023-11-28T21:56:55.743149+0000","last_deep_scrub":"52'167","last_deep_scrub_stamp":"2023-11-28T21:56:55.743149+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.743149+0000","objects_scrubbed":3,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:24.042531+0000","scrub_duration":0.015377216000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24576,"num_write":146,"num_write_kb":4262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19890,"num_omap_keys":36,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"52'1318","reported_seq":3644,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:14.990245+0000","last_change":"2023-11-28T21:57:14.990245+0000","last_active":"2023-11-28T21:57:14.990245+0000","last_peered":"2023-11-28T21:57:14.990245+0000","last_clean":"2023-11-28T21:57:14.990245+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:57:14.990245+0000","last_undegraded":"2023-11-28T21:57:14.990245+0000","last_fullsized":"2023-11-28T21:57:14.990245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1318","last_scrub_stamp":"2023-11-28T21:57:14.990114+0000","last_deep_scrub":"52'1318","last_deep_scrub_stamp":"2023-11-28T21:57:14.990114+0000","last_clean_scrub_stamp":"2023-11-28T21:57:14.990114+0000","objects_scrubbed":314,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:16:53.276374+0000","scrub_duration":0.197652458,"objects_trimmed":0,"snaptrim_duration":0.25659980999999998,"stat_sum":{"num_bytes":32626940,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":222,"num_read_kb":4128,"num_write":1285,"num_write_kb":60020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"52'300","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.336260+0000","last_change":"2023-11-28T21:56:44.336260+0000","last_active":"2023-11-28T21:56:44.336260+0000","last_peered":"2023-11-28T21:56:44.336260+0000","last_clean":"2023-11-28T21:56:44.336260+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:56:44.336260+0000","last_undegraded":"2023-11-28T21:56:44.336260+0000","last_fullsized":"2023-11-28T21:56:44.336260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'300","last_scrub_stamp":"2023-11-28T21:56:44.335964+0000","last_deep_scrub":"52'300","last_deep_scrub_stamp":"2023-11-28T21:56:44.335964+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.335964+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:50:07.131774+0000","scrub_duration":0.0032352040000000002,"objects_trimmed":0,"snaptrim_duration":0.23142385100000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"52'370","reported_seq":454,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:51.659286+0000","last_change":"2023-11-28T21:56:51.659286+0000","last_active":"2023-11-28T21:56:51.659286+0000","last_peered":"2023-11-28T21:56:51.659286+0000","last_clean":"2023-11-28T21:56:51.659286+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:56:51.659286+0000","last_undegraded":"2023-11-28T21:56:51.659286+0000","last_fullsized":"2023-11-28T21:56:51.659286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'370","last_scrub_stamp":"2023-11-28T21:56:51.659033+0000","last_deep_scrub":"52'370","last_deep_scrub_stamp":"2023-11-28T21:56:51.659033+0000","last_clean_scrub_stamp":"2023-11-28T21:56:51.659033+0000","objects_scrubbed":3,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:45:32.821715+0000","scrub_duration":0.043796694999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":81920,"num_write":339,"num_write_kb":16655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"52'1263","reported_seq":3576,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.935360+0000","last_change":"2023-11-28T21:57:13.935360+0000","last_active":"2023-11-28T21:57:13.935360+0000","last_peered":"2023-11-28T21:57:13.935360+0000","last_clean":"2023-11-28T21:57:13.935360+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:57:13.935360+0000","last_undegraded":"2023-11-28T21:57:13.935360+0000","last_fullsized":"2023-11-28T21:57:13.935360+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1263","last_scrub_stamp":"2023-11-28T21:57:13.935220+0000","last_deep_scrub":"52'1263","last_deep_scrub_stamp":"2023-11-28T21:57:13.935220+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.935220+0000","objects_scrubbed":288,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:20.055177+0000","scrub_duration":0.14496025300000001,"objects_trimmed":0,"snaptrim_duration":0.24685721299999999,"stat_sum":{"num_bytes":16718463,"num_objects":288,"num_object_clones":0,"num_object_copies":1152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":210,"num_read_kb":3302,"num_write":1235,"num_write_kb":35539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"52'299","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:07.792410+0000","last_change":"2023-11-28T21:57:07.792430+0000","last_active":"2023-11-28T21:57:07.792410+0000","last_peered":"2023-11-28T21:57:07.792410+0000","last_clean":"2023-11-28T21:57:07.792410+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:57:07.792410+0000","last_undegraded":"2023-11-28T21:57:07.792410+0000","last_fullsized":"2023-11-28T21:57:07.792410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'299","last_scrub_stamp":"2023-11-28T21:57:07.792311+0000","last_deep_scrub":"52'299","last_deep_scrub_stamp":"2023-11-28T21:57:07.792311+0000","last_clean_scrub_stamp":"2023-11-28T21:57:07.792311+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:44.781100+0000","scrub_duration":0.0021660130000000001,"objects_trimmed":0,"snaptrim_duration":0.24023003500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"52'943","reported_seq":1424,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.244727+0000","last_change":"2023-11-28T21:56:45.244727+0000","last_active":"2023-11-28T21:56:45.244727+0000","last_peered":"2023-11-28T21:56:45.244727+0000","last_clean":"2023-11-28T21:56:45.244727+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:56:45.244727+0000","last_undegraded":"2023-11-28T21:56:45.244727+0000","last_fullsized":"2023-11-28T21:56:45.244727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'943","last_scrub_stamp":"2023-11-28T21:56:45.244580+0000","last_deep_scrub":"52'943","last_deep_scrub_stamp":"2023-11-28T21:56:45.244580+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.244580+0000","objects_scrubbed":9,"log_size":943,"log_dups_size":0,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:47:30.047518+0000","scrub_duration":0.055949021000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18884916,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":417,"num_read_kb":34819,"num_write":869,"num_write_kb":18975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17074,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"52'1231","reported_seq":3577,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:19.911197+0000","last_change":"2023-11-28T21:57:19.911246+0000","last_active":"2023-11-28T21:57:19.911197+0000","last_peered":"2023-11-28T21:57:19.911197+0000","last_clean":"2023-11-28T21:57:19.911197+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:57:19.911197+0000","last_undegraded":"2023-11-28T21:57:19.911197+0000","last_fullsized":"2023-11-28T21:57:19.911197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1231","last_scrub_stamp":"2023-11-28T21:57:19.911063+0000","last_deep_scrub":"52'1231","last_deep_scrub_stamp":"2023-11-28T21:57:19.911063+0000","last_clean_scrub_stamp":"2023-11-28T21:57:19.911063+0000","objects_scrubbed":293,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:45:27.145851+0000","scrub_duration":0.164741481,"objects_trimmed":0,"snaptrim_duration":0.24393632200000001,"stat_sum":{"num_bytes":26023878,"num_objects":293,"num_object_clones":0,"num_object_copies":1172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":293,"num_whiteouts":0,"num_read":244,"num_read_kb":12314,"num_write":1202,"num_write_kb":62723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"52'277","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.370466+0000","last_change":"2023-11-28T21:57:01.370466+0000","last_active":"2023-11-28T21:57:01.370466+0000","last_peered":"2023-11-28T21:57:01.370466+0000","last_clean":"2023-11-28T21:57:01.370466+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:57:01.370466+0000","last_undegraded":"2023-11-28T21:57:01.370466+0000","last_fullsized":"2023-11-28T21:57:01.370466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'277","last_scrub_stamp":"2023-11-28T21:57:01.370337+0000","last_deep_scrub":"52'277","last_deep_scrub_stamp":"2023-11-28T21:57:01.370337+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.370337+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:30:28.221635+0000","scrub_duration":0.002672757,"objects_trimmed":0,"snaptrim_duration":0.25105937499999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"52'331","reported_seq":423,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:12.832467+0000","last_change":"2023-11-28T21:56:59.491989+0000","last_active":"2023-11-28T21:57:12.832467+0000","last_peered":"2023-11-28T21:57:12.832467+0000","last_clean":"2023-11-28T21:57:12.832467+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:57:12.832467+0000","last_undegraded":"2023-11-28T21:57:12.832467+0000","last_fullsized":"2023-11-28T21:57:12.832467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'325","last_scrub_stamp":"2023-11-28T21:56:59.491653+0000","last_deep_scrub":"52'325","last_deep_scrub_stamp":"2023-11-28T21:56:59.491653+0000","last_clean_scrub_stamp":"2023-11-28T21:56:59.491653+0000","objects_scrubbed":5,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:19:40.745953+0000","scrub_duration":0.050149041999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8818331,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":114688,"num_write":289,"num_write_kb":16977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5654,"num_omap_keys":11,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"52'1365","reported_seq":3732,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:21.294986+0000","last_change":"2023-11-28T21:57:21.295019+0000","last_active":"2023-11-28T21:57:21.294986+0000","last_peered":"2023-11-28T21:57:21.294986+0000","last_clean":"2023-11-28T21:57:21.294986+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:57:21.294986+0000","last_undegraded":"2023-11-28T21:57:21.294986+0000","last_fullsized":"2023-11-28T21:57:21.294986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1365","last_scrub_stamp":"2023-11-28T21:57:21.294885+0000","last_deep_scrub":"52'1365","last_deep_scrub_stamp":"2023-11-28T21:57:21.294885+0000","last_clean_scrub_stamp":"2023-11-28T21:57:21.294885+0000","objects_scrubbed":302,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:23:35.502385+0000","scrub_duration":0.15644470799999999,"objects_trimmed":0,"snaptrim_duration":0.270607394,"stat_sum":{"num_bytes":19916106,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":264,"num_read_kb":13370,"num_write":1336,"num_write_kb":59202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"52'283","reported_seq":2376,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.428177+0000","last_change":"2023-11-28T21:56:57.428177+0000","last_active":"2023-11-28T21:56:57.428177+0000","last_peered":"2023-11-28T21:56:57.428177+0000","last_clean":"2023-11-28T21:56:57.428177+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:56:57.428177+0000","last_undegraded":"2023-11-28T21:56:57.428177+0000","last_fullsized":"2023-11-28T21:56:57.428177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:56:57.428075+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:56:57.428075+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.428075+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:09:37.862848+0000","scrub_duration":0.0026023819999999999,"objects_trimmed":0,"snaptrim_duration":0.29152715000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"52'318","reported_seq":392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.764096+0000","last_change":"2023-11-28T21:56:57.764096+0000","last_active":"2023-11-28T21:56:57.764096+0000","last_peered":"2023-11-28T21:56:57.764096+0000","last_clean":"2023-11-28T21:56:57.764096+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:56:57.764096+0000","last_undegraded":"2023-11-28T21:56:57.764096+0000","last_fullsized":"2023-11-28T21:56:57.764096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'318","last_scrub_stamp":"2023-11-28T21:56:57.763819+0000","last_deep_scrub":"52'318","last_deep_scrub_stamp":"2023-11-28T21:56:57.763819+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.763819+0000","objects_scrubbed":7,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:20:01.143626+0000","scrub_duration":0.047864816999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":24594,"num_write":297,"num_write_kb":8480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":38186,"num_omap_keys":70,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"52'119","reported_seq":219,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.591456+0000","last_change":"2023-11-28T21:56:45.591517+0000","last_active":"2023-11-28T21:56:45.591456+0000","last_peered":"2023-11-28T21:56:45.591456+0000","last_clean":"2023-11-28T21:56:45.591456+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:56:45.591456+0000","last_undegraded":"2023-11-28T21:56:45.591456+0000","last_fullsized":"2023-11-28T21:56:45.591456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'119","last_scrub_stamp":"2023-11-28T21:56:45.591265+0000","last_deep_scrub":"52'119","last_deep_scrub_stamp":"2023-11-28T21:56:45.591265+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.591265+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:52:45.646774+0000","scrub_duration":0.0095868710000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"52'1342","reported_seq":3698,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:34.343916+0000","last_change":"2023-11-28T21:57:34.344047+0000","last_active":"2023-11-28T21:57:34.343916+0000","last_peered":"2023-11-28T21:57:34.343916+0000","last_clean":"2023-11-28T21:57:34.343916+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:57:34.343916+0000","last_undegraded":"2023-11-28T21:57:34.343916+0000","last_fullsized":"2023-11-28T21:57:34.343916+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1342","last_scrub_stamp":"2023-11-28T21:57:34.343779+0000","last_deep_scrub":"52'1342","last_deep_scrub_stamp":"2023-11-28T21:57:34.343779+0000","last_clean_scrub_stamp":"2023-11-28T21:57:34.343779+0000","objects_scrubbed":339,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:47.308742+0000","scrub_duration":0.26345470500000001,"objects_trimmed":0,"snaptrim_duration":0.19732549099999999,"stat_sum":{"num_bytes":29762487,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":252,"num_read_kb":20508,"num_write":1310,"num_write_kb":73242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"52'280","reported_seq":2373,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.333148+0000","last_change":"2023-11-28T21:57:09.333148+0000","last_active":"2023-11-28T21:57:09.333148+0000","last_peered":"2023-11-28T21:57:09.333148+0000","last_clean":"2023-11-28T21:57:09.333148+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:57:09.333148+0000","last_undegraded":"2023-11-28T21:57:09.333148+0000","last_fullsized":"2023-11-28T21:57:09.333148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'280","last_scrub_stamp":"2023-11-28T21:57:09.332900+0000","last_deep_scrub":"52'280","last_deep_scrub_stamp":"2023-11-28T21:57:09.332900+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.332900+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:46:05.592588+0000","scrub_duration":0.00363196,"objects_trimmed":0,"snaptrim_duration":0.219020891,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"52'482","reported_seq":551,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.334575+0000","last_change":"2023-11-28T21:57:01.334575+0000","last_active":"2023-11-28T21:57:01.334575+0000","last_peered":"2023-11-28T21:57:01.334575+0000","last_clean":"2023-11-28T21:57:01.334575+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:57:01.334575+0000","last_undegraded":"2023-11-28T21:57:01.334575+0000","last_fullsized":"2023-11-28T21:57:01.334575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'482","last_scrub_stamp":"2023-11-28T21:57:01.334419+0000","last_deep_scrub":"52'482","last_deep_scrub_stamp":"2023-11-28T21:57:01.334419+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.334419+0000","objects_scrubbed":6,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:10:08.252390+0000","scrub_duration":0.032982576999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":20485,"num_write":447,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":27630,"num_omap_keys":50,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"52'1372","reported_seq":3726,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:38.300326+0000","last_change":"2023-11-28T21:57:38.300460+0000","last_active":"2023-11-28T21:57:38.300326+0000","last_peered":"2023-11-28T21:57:38.300326+0000","last_clean":"2023-11-28T21:57:38.300326+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:57:38.300326+0000","last_undegraded":"2023-11-28T21:57:38.300326+0000","last_fullsized":"2023-11-28T21:57:38.300326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1372","last_scrub_stamp":"2023-11-28T21:57:38.300030+0000","last_deep_scrub":"52'1372","last_deep_scrub_stamp":"2023-11-28T21:57:38.300030+0000","last_clean_scrub_stamp":"2023-11-28T21:57:38.300030+0000","objects_scrubbed":335,"log_size":1372,"log_dups_size":0,"ondisk_log_size":1372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:53:38.148437+0000","scrub_duration":0.26399045500000001,"objects_trimmed":0,"snaptrim_duration":0.19929065100000001,"stat_sum":{"num_bytes":26470227,"num_objects":335,"num_object_clones":0,"num_object_copies":1340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":251,"num_read_kb":4134,"num_write":1344,"num_write_kb":47412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"52'330","reported_seq":2423,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:14.726189+0000","last_change":"2023-11-28T21:57:14.726238+0000","last_active":"2023-11-28T21:57:14.726189+0000","last_peered":"2023-11-28T21:57:14.726189+0000","last_clean":"2023-11-28T21:57:14.726189+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:57:14.726189+0000","last_undegraded":"2023-11-28T21:57:14.726189+0000","last_fullsized":"2023-11-28T21:57:14.726189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'330","last_scrub_stamp":"2023-11-28T21:57:14.726054+0000","last_deep_scrub":"52'330","last_deep_scrub_stamp":"2023-11-28T21:57:14.726054+0000","last_clean_scrub_stamp":"2023-11-28T21:57:14.726054+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:53:00.065479+0000","scrub_duration":0.0023321930000000002,"objects_trimmed":0,"snaptrim_duration":0.30458413400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"52'971","reported_seq":1061,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.753678+0000","last_change":"2023-11-28T21:56:49.753678+0000","last_active":"2023-11-28T21:56:49.753678+0000","last_peered":"2023-11-28T21:56:49.753678+0000","last_clean":"2023-11-28T21:56:49.753678+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:56:49.753678+0000","last_undegraded":"2023-11-28T21:56:49.753678+0000","last_fullsized":"2023-11-28T21:56:49.753678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'971","last_scrub_stamp":"2023-11-28T21:56:49.753562+0000","last_deep_scrub":"52'971","last_deep_scrub_stamp":"2023-11-28T21:56:49.753562+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.753562+0000","objects_scrubbed":7,"log_size":971,"log_dups_size":0,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:05.338455+0000","scrub_duration":0.053294293999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":27,"num_read_kb":68346,"num_write":931,"num_write_kb":16947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":38524,"num_omap_keys":71,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"52'1319","reported_seq":3687,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:22.364889+0000","last_change":"2023-11-28T21:57:22.364911+0000","last_active":"2023-11-28T21:57:22.364889+0000","last_peered":"2023-11-28T21:57:22.364889+0000","last_clean":"2023-11-28T21:57:22.364889+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:57:22.364889+0000","last_undegraded":"2023-11-28T21:57:22.364889+0000","last_fullsized":"2023-11-28T21:57:22.364889+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1319","last_scrub_stamp":"2023-11-28T21:57:22.364796+0000","last_deep_scrub":"52'1319","last_deep_scrub_stamp":"2023-11-28T21:57:22.364796+0000","last_clean_scrub_stamp":"2023-11-28T21:57:22.364796+0000","objects_scrubbed":302,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:58:07.675638+0000","scrub_duration":0.192509492,"objects_trimmed":0,"snaptrim_duration":0.27089875299999999,"stat_sum":{"num_bytes":28004524,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":265,"num_read_kb":16422,"num_write":1294,"num_write_kb":50614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"52'274","reported_seq":2359,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.276188+0000","last_change":"2023-11-28T21:57:16.276211+0000","last_active":"2023-11-28T21:57:16.276188+0000","last_peered":"2023-11-28T21:57:16.276188+0000","last_clean":"2023-11-28T21:57:16.276188+0000","last_became_active":"2023-11-28T21:35:25.594533+0000","last_became_peered":"2023-11-28T21:35:25.594533+0000","last_unstale":"2023-11-28T21:57:16.276188+0000","last_undegraded":"2023-11-28T21:57:16.276188+0000","last_fullsized":"2023-11-28T21:57:16.276188+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'274","last_scrub_stamp":"2023-11-28T21:57:16.276089+0000","last_deep_scrub":"52'274","last_deep_scrub_stamp":"2023-11-28T21:57:16.276089+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.276089+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:46:27.069531+0000","scrub_duration":0.0027233159999999999,"objects_trimmed":0,"snaptrim_duration":0.14880083199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"52'307","reported_seq":392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.178599+0000","last_change":"2023-11-28T21:56:48.178599+0000","last_active":"2023-11-28T21:56:48.178599+0000","last_peered":"2023-11-28T21:56:48.178599+0000","last_clean":"2023-11-28T21:56:48.178599+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:56:48.178599+0000","last_undegraded":"2023-11-28T21:56:48.178599+0000","last_fullsized":"2023-11-28T21:56:48.178599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'307","last_scrub_stamp":"2023-11-28T21:56:48.178460+0000","last_deep_scrub":"52'307","last_deep_scrub_stamp":"2023-11-28T21:56:48.178460+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.178460+0000","objects_scrubbed":5,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:26:15.702492+0000","scrub_duration":0.026539895000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":53275,"num_write":252,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21228,"num_omap_keys":39,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"52'1307","reported_seq":3611,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:32.770136+0000","last_change":"2023-11-28T21:57:32.770194+0000","last_active":"2023-11-28T21:57:32.770136+0000","last_peered":"2023-11-28T21:57:32.770136+0000","last_clean":"2023-11-28T21:57:32.770136+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:57:32.770136+0000","last_undegraded":"2023-11-28T21:57:32.770136+0000","last_fullsized":"2023-11-28T21:57:32.770136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1307","last_scrub_stamp":"2023-11-28T21:57:32.770017+0000","last_deep_scrub":"52'1307","last_deep_scrub_stamp":"2023-11-28T21:57:32.770017+0000","last_clean_scrub_stamp":"2023-11-28T21:57:32.770017+0000","objects_scrubbed":295,"log_size":1307,"log_dups_size":0,"ondisk_log_size":1307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:57:16.838574+0000","scrub_duration":0.22258078000000001,"objects_trimmed":0,"snaptrim_duration":0.30648515900000001,"stat_sum":{"num_bytes":28110180,"num_objects":295,"num_object_clones":0,"num_object_copies":1180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":0,"num_read":202,"num_read_kb":8227,"num_write":1283,"num_write_kb":51039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"52'277","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:24.329680+0000","last_change":"2023-11-28T21:57:24.329730+0000","last_active":"2023-11-28T21:57:24.329680+0000","last_peered":"2023-11-28T21:57:24.329680+0000","last_clean":"2023-11-28T21:57:24.329680+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:57:24.329680+0000","last_undegraded":"2023-11-28T21:57:24.329680+0000","last_fullsized":"2023-11-28T21:57:24.329680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'277","last_scrub_stamp":"2023-11-28T21:57:24.329487+0000","last_deep_scrub":"52'277","last_deep_scrub_stamp":"2023-11-28T21:57:24.329487+0000","last_clean_scrub_stamp":"2023-11-28T21:57:24.329487+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:27:26.819175+0000","scrub_duration":0.003383807,"objects_trimmed":0,"snaptrim_duration":0.21877398200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"52'197","reported_seq":280,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.311351+0000","last_change":"2023-11-28T21:56:43.311351+0000","last_active":"2023-11-28T21:56:43.311351+0000","last_peered":"2023-11-28T21:56:43.311351+0000","last_clean":"2023-11-28T21:56:43.311351+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:56:43.311351+0000","last_undegraded":"2023-11-28T21:56:43.311351+0000","last_fullsized":"2023-11-28T21:56:43.311351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'197","last_scrub_stamp":"2023-11-28T21:56:43.311087+0000","last_deep_scrub":"52'197","last_deep_scrub_stamp":"2023-11-28T21:56:43.311087+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.311087+0000","objects_scrubbed":4,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:17:01.116728+0000","scrub_duration":0.027225414999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":49161,"num_write":147,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11661,"num_omap_keys":22,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"52'1273","reported_seq":3595,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.508129+0000","last_change":"2023-11-28T21:57:13.508211+0000","last_active":"2023-11-28T21:57:13.508129+0000","last_peered":"2023-11-28T21:57:13.508129+0000","last_clean":"2023-11-28T21:57:13.508129+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:57:13.508129+0000","last_undegraded":"2023-11-28T21:57:13.508129+0000","last_fullsized":"2023-11-28T21:57:13.508129+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1273","last_scrub_stamp":"2023-11-28T21:57:13.507890+0000","last_deep_scrub":"52'1273","last_deep_scrub_stamp":"2023-11-28T21:57:13.507890+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.507890+0000","objects_scrubbed":308,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:10:17.277733+0000","scrub_duration":0.25249365600000001,"objects_trimmed":0,"snaptrim_duration":0.223266511,"stat_sum":{"num_bytes":11523503,"num_objects":308,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":219,"num_read_kb":12316,"num_write":1248,"num_write_kb":38807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"52'323","reported_seq":2415,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.543891+0000","last_change":"2023-11-28T21:56:43.543999+0000","last_active":"2023-11-28T21:56:43.543891+0000","last_peered":"2023-11-28T21:56:43.543891+0000","last_clean":"2023-11-28T21:56:43.543891+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:56:43.543891+0000","last_undegraded":"2023-11-28T21:56:43.543891+0000","last_fullsized":"2023-11-28T21:56:43.543891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'323","last_scrub_stamp":"2023-11-28T21:56:43.543658+0000","last_deep_scrub":"52'323","last_deep_scrub_stamp":"2023-11-28T21:56:43.543658+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.543658+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:59.532405+0000","scrub_duration":0.003404327,"objects_trimmed":0,"snaptrim_duration":0.21978382299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"52'868","reported_seq":987,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.270821+0000","last_change":"2023-11-28T21:56:50.270874+0000","last_active":"2023-11-28T21:56:50.270821+0000","last_peered":"2023-11-28T21:56:50.270821+0000","last_clean":"2023-11-28T21:56:50.270821+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:56:50.270821+0000","last_undegraded":"2023-11-28T21:56:50.270821+0000","last_fullsized":"2023-11-28T21:56:50.270821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'868","last_scrub_stamp":"2023-11-28T21:56:50.270733+0000","last_deep_scrub":"52'868","last_deep_scrub_stamp":"2023-11-28T21:56:50.270733+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.270733+0000","objects_scrubbed":14,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:25.187061+0000","scrub_duration":0.051670183000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":137,"num_read_kb":58709,"num_write":838,"num_write_kb":8642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34555,"num_omap_keys":61,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"52'1274","reported_seq":3607,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:35.000372+0000","last_change":"2023-11-28T21:40:43.252839+0000","last_active":"2023-11-28T21:56:35.000372+0000","last_peered":"2023-11-28T21:56:35.000372+0000","last_clean":"2023-11-28T21:56:35.000372+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:56:35.000372+0000","last_undegraded":"2023-11-28T21:56:35.000372+0000","last_fullsized":"2023-11-28T21:56:35.000372+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27152726799999999,"stat_sum":{"num_bytes":36307339,"num_objects":283,"num_object_clones":0,"num_object_copies":1132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":283,"num_whiteouts":0,"num_read":251,"num_read_kb":8046,"num_write":1253,"num_write_kb":60298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"52'284","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.131960+0000","last_change":"2023-11-28T21:57:01.132013+0000","last_active":"2023-11-28T21:57:01.131960+0000","last_peered":"2023-11-28T21:57:01.131960+0000","last_clean":"2023-11-28T21:57:01.131960+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:57:01.131960+0000","last_undegraded":"2023-11-28T21:57:01.131960+0000","last_fullsized":"2023-11-28T21:57:01.131960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'284","last_scrub_stamp":"2023-11-28T21:57:01.131764+0000","last_deep_scrub":"52'284","last_deep_scrub_stamp":"2023-11-28T21:57:01.131764+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.131764+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:44:58.129846+0000","scrub_duration":0.0035951529999999998,"objects_trimmed":0,"snaptrim_duration":0.18244292200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"52'243","reported_seq":306,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.132811+0000","last_change":"2023-11-28T21:56:48.132928+0000","last_active":"2023-11-28T21:56:48.132811+0000","last_peered":"2023-11-28T21:56:48.132811+0000","last_clean":"2023-11-28T21:56:48.132811+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:56:48.132811+0000","last_undegraded":"2023-11-28T21:56:48.132811+0000","last_fullsized":"2023-11-28T21:56:48.132811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'243","last_scrub_stamp":"2023-11-28T21:56:48.132597+0000","last_deep_scrub":"52'243","last_deep_scrub_stamp":"2023-11-28T21:56:48.132597+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.132597+0000","objects_scrubbed":4,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:38.258199+0000","scrub_duration":0.035274727999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195875,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":194,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12188,"num_omap_keys":22,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"52'1346","reported_seq":3683,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:20.814918+0000","last_change":"2023-11-28T21:57:20.814952+0000","last_active":"2023-11-28T21:57:20.814918+0000","last_peered":"2023-11-28T21:57:20.814918+0000","last_clean":"2023-11-28T21:57:20.814918+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:57:20.814918+0000","last_undegraded":"2023-11-28T21:57:20.814918+0000","last_fullsized":"2023-11-28T21:57:20.814918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1346","last_scrub_stamp":"2023-11-28T21:57:20.814818+0000","last_deep_scrub":"52'1346","last_deep_scrub_stamp":"2023-11-28T21:57:20.814818+0000","last_clean_scrub_stamp":"2023-11-28T21:57:20.814818+0000","objects_scrubbed":324,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:05.852021+0000","scrub_duration":0.22313440300000001,"objects_trimmed":0,"snaptrim_duration":0.29439885100000002,"stat_sum":{"num_bytes":34458204,"num_objects":324,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":324,"num_whiteouts":0,"num_read":235,"num_read_kb":27797,"num_write":1320,"num_write_kb":70332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"52'313","reported_seq":2405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:22.236772+0000","last_change":"2023-11-28T21:57:22.236867+0000","last_active":"2023-11-28T21:57:22.236772+0000","last_peered":"2023-11-28T21:57:22.236772+0000","last_clean":"2023-11-28T21:57:22.236772+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:57:22.236772+0000","last_undegraded":"2023-11-28T21:57:22.236772+0000","last_fullsized":"2023-11-28T21:57:22.236772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'313","last_scrub_stamp":"2023-11-28T21:57:22.236522+0000","last_deep_scrub":"52'313","last_deep_scrub_stamp":"2023-11-28T21:57:22.236522+0000","last_clean_scrub_stamp":"2023-11-28T21:57:22.236522+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:39:11.637946+0000","scrub_duration":0.0037346620000000001,"objects_trimmed":0,"snaptrim_duration":0.218364582,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"52'160","reported_seq":227,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.715676+0000","last_change":"2023-11-28T21:56:55.715743+0000","last_active":"2023-11-28T21:56:55.715676+0000","last_peered":"2023-11-28T21:56:55.715676+0000","last_clean":"2023-11-28T21:56:55.715676+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:56:55.715676+0000","last_undegraded":"2023-11-28T21:56:55.715676+0000","last_fullsized":"2023-11-28T21:56:55.715676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'160","last_scrub_stamp":"2023-11-28T21:56:55.715502+0000","last_deep_scrub":"52'160","last_deep_scrub_stamp":"2023-11-28T21:56:55.715502+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.715502+0000","objects_scrubbed":5,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:08:17.898435+0000","scrub_duration":0.024710507999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":107,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23214,"num_omap_keys":42,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"52'1271","reported_seq":3593,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.743280+0000","last_change":"2023-11-28T21:57:16.743301+0000","last_active":"2023-11-28T21:57:16.743280+0000","last_peered":"2023-11-28T21:57:16.743280+0000","last_clean":"2023-11-28T21:57:16.743280+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:57:16.743280+0000","last_undegraded":"2023-11-28T21:57:16.743280+0000","last_fullsized":"2023-11-28T21:57:16.743280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1271","last_scrub_stamp":"2023-11-28T21:57:16.743187+0000","last_deep_scrub":"52'1271","last_deep_scrub_stamp":"2023-11-28T21:57:16.743187+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.743187+0000","objects_scrubbed":350,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:32.276987+0000","scrub_duration":0.193506183,"objects_trimmed":0,"snaptrim_duration":0.29571303700000001,"stat_sum":{"num_bytes":33678171,"num_objects":350,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":350,"num_whiteouts":0,"num_read":218,"num_read_kb":4129,"num_write":1246,"num_write_kb":52865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"52'279","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.192213+0000","last_change":"2023-11-28T21:57:13.192234+0000","last_active":"2023-11-28T21:57:13.192213+0000","last_peered":"2023-11-28T21:57:13.192213+0000","last_clean":"2023-11-28T21:57:13.192213+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:57:13.192213+0000","last_undegraded":"2023-11-28T21:57:13.192213+0000","last_fullsized":"2023-11-28T21:57:13.192213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'279","last_scrub_stamp":"2023-11-28T21:57:13.192121+0000","last_deep_scrub":"52'279","last_deep_scrub_stamp":"2023-11-28T21:57:13.192121+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.192121+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:37:10.470818+0000","scrub_duration":0.003054668,"objects_trimmed":0,"snaptrim_duration":0.21362616300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"52'313","reported_seq":415,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.422213+0000","last_change":"2023-11-28T21:56:44.422244+0000","last_active":"2023-11-28T21:56:44.422213+0000","last_peered":"2023-11-28T21:56:44.422213+0000","last_clean":"2023-11-28T21:56:44.422213+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:56:44.422213+0000","last_undegraded":"2023-11-28T21:56:44.422213+0000","last_fullsized":"2023-11-28T21:56:44.422213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'313","last_scrub_stamp":"2023-11-28T21:56:44.422134+0000","last_deep_scrub":"52'313","last_deep_scrub_stamp":"2023-11-28T21:56:44.422134+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.422134+0000","objects_scrubbed":9,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:50.028108+0000","scrub_duration":0.030421210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":67,"num_read_kb":77908,"num_write":272,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57622,"num_omap_keys":105,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"52'1265","reported_seq":3558,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:27.943077+0000","last_change":"2023-11-28T21:57:27.943198+0000","last_active":"2023-11-28T21:57:27.943077+0000","last_peered":"2023-11-28T21:57:27.943077+0000","last_clean":"2023-11-28T21:57:27.943077+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:57:27.943077+0000","last_undegraded":"2023-11-28T21:57:27.943077+0000","last_fullsized":"2023-11-28T21:57:27.943077+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1265","last_scrub_stamp":"2023-11-28T21:57:27.942825+0000","last_deep_scrub":"52'1265","last_deep_scrub_stamp":"2023-11-28T21:57:27.942825+0000","last_clean_scrub_stamp":"2023-11-28T21:57:27.942825+0000","objects_scrubbed":340,"log_size":1265,"log_dups_size":0,"ondisk_log_size":1265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:57:42.550943+0000","scrub_duration":0.26899315400000001,"objects_trimmed":0,"snaptrim_duration":0.30778537500000003,"stat_sum":{"num_bytes":30037804,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":190,"num_read_kb":33,"num_write":1241,"num_write_kb":36631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.264901+0000","last_change":"2023-11-28T21:57:17.264953+0000","last_active":"2023-11-28T21:57:17.264901+0000","last_peered":"2023-11-28T21:57:17.264901+0000","last_clean":"2023-11-28T21:57:17.264901+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:57:17.264901+0000","last_undegraded":"2023-11-28T21:57:17.264901+0000","last_fullsized":"2023-11-28T21:57:17.264901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:17.264706+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:17.264706+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.264706+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:11:48.499371+0000","scrub_duration":0.0034211150000000002,"objects_trimmed":0,"snaptrim_duration":0.21990301600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"52'759","reported_seq":882,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.931156+0000","last_change":"2023-11-28T21:56:47.144324+0000","last_active":"2023-11-28T21:56:53.931156+0000","last_peered":"2023-11-28T21:56:53.931156+0000","last_clean":"2023-11-28T21:56:53.931156+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:56:53.931156+0000","last_undegraded":"2023-11-28T21:56:53.931156+0000","last_fullsized":"2023-11-28T21:56:53.931156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'757","last_scrub_stamp":"2023-11-28T21:56:47.144054+0000","last_deep_scrub":"52'757","last_deep_scrub_stamp":"2023-11-28T21:56:47.144054+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.144054+0000","objects_scrubbed":12,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:36:20.172063+0000","scrub_duration":0.085472306999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":90,"num_read_kb":122940,"num_write":684,"num_write_kb":20915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":36218,"num_omap_keys":66,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"52'1318","reported_seq":3626,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:33.779319+0000","last_change":"2023-11-28T21:57:33.779362+0000","last_active":"2023-11-28T21:57:33.779319+0000","last_peered":"2023-11-28T21:57:33.779319+0000","last_clean":"2023-11-28T21:57:33.779319+0000","last_became_active":"2023-11-28T21:35:25.790293+0000","last_became_peered":"2023-11-28T21:35:25.790293+0000","last_unstale":"2023-11-28T21:57:33.779319+0000","last_undegraded":"2023-11-28T21:57:33.779319+0000","last_fullsized":"2023-11-28T21:57:33.779319+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1318","last_scrub_stamp":"2023-11-28T21:57:33.779155+0000","last_deep_scrub":"52'1318","last_deep_scrub_stamp":"2023-11-28T21:57:33.779155+0000","last_clean_scrub_stamp":"2023-11-28T21:57:33.779155+0000","objects_scrubbed":330,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:20:10.097131+0000","scrub_duration":0.27549114499999999,"objects_trimmed":0,"snaptrim_duration":0.30978138799999999,"stat_sum":{"num_bytes":36258609,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":208,"num_read_kb":8227,"num_write":1285,"num_write_kb":56637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"52'283","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.299355+0000","last_change":"2023-11-28T21:56:50.299408+0000","last_active":"2023-11-28T21:56:50.299355+0000","last_peered":"2023-11-28T21:56:50.299355+0000","last_clean":"2023-11-28T21:56:50.299355+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:56:50.299355+0000","last_undegraded":"2023-11-28T21:56:50.299355+0000","last_fullsized":"2023-11-28T21:56:50.299355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:56:50.299160+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:56:50.299160+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.299160+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:20.213618+0000","scrub_duration":0.003897317,"objects_trimmed":0,"snaptrim_duration":0.22435063599999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"52'525","reported_seq":625,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.790155+0000","last_change":"2023-11-28T21:56:53.790210+0000","last_active":"2023-11-28T21:56:53.790155+0000","last_peered":"2023-11-28T21:56:53.790155+0000","last_clean":"2023-11-28T21:56:53.790155+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:56:53.790155+0000","last_undegraded":"2023-11-28T21:56:53.790155+0000","last_fullsized":"2023-11-28T21:56:53.790155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'525","last_scrub_stamp":"2023-11-28T21:56:53.790048+0000","last_deep_scrub":"52'525","last_deep_scrub_stamp":"2023-11-28T21:56:53.790048+0000","last_clean_scrub_stamp":"2023-11-28T21:56:53.790048+0000","objects_scrubbed":9,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:47:27.057628+0000","scrub_duration":0.097465174000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":69670,"num_write":485,"num_write_kb":16676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24376,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"52'1329","reported_seq":3690,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:52.591960+0000","last_change":"2023-11-28T21:56:52.592017+0000","last_active":"2023-11-28T21:56:52.591960+0000","last_peered":"2023-11-28T21:56:52.591960+0000","last_clean":"2023-11-28T21:56:52.591960+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:56:52.591960+0000","last_undegraded":"2023-11-28T21:56:52.591960+0000","last_fullsized":"2023-11-28T21:56:52.591960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1329","last_scrub_stamp":"2023-11-28T21:56:52.591801+0000","last_deep_scrub":"52'1329","last_deep_scrub_stamp":"2023-11-28T21:56:52.591801+0000","last_clean_scrub_stamp":"2023-11-28T21:56:52.591801+0000","objects_scrubbed":342,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:47:53.831632+0000","scrub_duration":0.25412254099999998,"objects_trimmed":0,"snaptrim_duration":0.2340457,"stat_sum":{"num_bytes":46614746,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":259,"num_read_kb":20536,"num_write":1294,"num_write_kb":83351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"52'265","reported_seq":2356,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.218640+0000","last_change":"2023-11-28T21:57:11.218663+0000","last_active":"2023-11-28T21:57:11.218640+0000","last_peered":"2023-11-28T21:57:11.218640+0000","last_clean":"2023-11-28T21:57:11.218640+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:57:11.218640+0000","last_undegraded":"2023-11-28T21:57:11.218640+0000","last_fullsized":"2023-11-28T21:57:11.218640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'265","last_scrub_stamp":"2023-11-28T21:57:11.218547+0000","last_deep_scrub":"52'265","last_deep_scrub_stamp":"2023-11-28T21:57:11.218547+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.218547+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:14.792134+0000","scrub_duration":0.0033770470000000002,"objects_trimmed":0,"snaptrim_duration":0.228570471,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"52'997","reported_seq":1085,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.686854+0000","last_change":"2023-11-28T21:56:49.686931+0000","last_active":"2023-11-28T21:56:49.686854+0000","last_peered":"2023-11-28T21:56:49.686854+0000","last_clean":"2023-11-28T21:56:49.686854+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:56:49.686854+0000","last_undegraded":"2023-11-28T21:56:49.686854+0000","last_fullsized":"2023-11-28T21:56:49.686854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'997","last_scrub_stamp":"2023-11-28T21:56:49.686647+0000","last_deep_scrub":"52'997","last_deep_scrub_stamp":"2023-11-28T21:56:49.686647+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.686647+0000","objects_scrubbed":7,"log_size":997,"log_dups_size":0,"ondisk_log_size":997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:07:13.091549+0000","scrub_duration":0.104808501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":90124,"num_write":955,"num_write_kb":20985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9418,"num_omap_keys":17,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"52'1334","reported_seq":3694,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.715355+0000","last_change":"2023-11-28T21:57:17.715381+0000","last_active":"2023-11-28T21:57:17.715355+0000","last_peered":"2023-11-28T21:57:17.715355+0000","last_clean":"2023-11-28T21:57:17.715355+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:57:17.715355+0000","last_undegraded":"2023-11-28T21:57:17.715355+0000","last_fullsized":"2023-11-28T21:57:17.715355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1334","last_scrub_stamp":"2023-11-28T21:57:17.715260+0000","last_deep_scrub":"52'1334","last_deep_scrub_stamp":"2023-11-28T21:57:17.715260+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.715260+0000","objects_scrubbed":314,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:34:07.662604+0000","scrub_duration":0.20735551499999999,"objects_trimmed":0,"snaptrim_duration":0.29518589499999998,"stat_sum":{"num_bytes":25457761,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":258,"num_read_kb":22214,"num_write":1317,"num_write_kb":78783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"52'308","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:15.248158+0000","last_change":"2023-11-28T21:57:15.248196+0000","last_active":"2023-11-28T21:57:15.248158+0000","last_peered":"2023-11-28T21:57:15.248158+0000","last_clean":"2023-11-28T21:57:15.248158+0000","last_became_active":"2023-11-28T21:35:25.595386+0000","last_became_peered":"2023-11-28T21:35:25.595386+0000","last_unstale":"2023-11-28T21:57:15.248158+0000","last_undegraded":"2023-11-28T21:57:15.248158+0000","last_fullsized":"2023-11-28T21:57:15.248158+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'308","last_scrub_stamp":"2023-11-28T21:57:15.248071+0000","last_deep_scrub":"52'308","last_deep_scrub_stamp":"2023-11-28T21:57:15.248071+0000","last_clean_scrub_stamp":"2023-11-28T21:57:15.248071+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:34:19.589282+0000","scrub_duration":0.0030527179999999998,"objects_trimmed":0,"snaptrim_duration":0.148630489,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"52'112","reported_seq":179,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.072726+0000","last_change":"2023-11-28T21:56:55.072786+0000","last_active":"2023-11-28T21:56:55.072726+0000","last_peered":"2023-11-28T21:56:55.072726+0000","last_clean":"2023-11-28T21:56:55.072726+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:56:55.072726+0000","last_undegraded":"2023-11-28T21:56:55.072726+0000","last_fullsized":"2023-11-28T21:56:55.072726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'112","last_scrub_stamp":"2023-11-28T21:56:55.072507+0000","last_deep_scrub":"52'112","last_deep_scrub_stamp":"2023-11-28T21:56:55.072507+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.072507+0000","objects_scrubbed":3,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:04:39.066804+0000","scrub_duration":0.0092242439999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":68,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"52'1427","reported_seq":3792,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:14.697621+0000","last_change":"2023-11-28T21:57:14.697652+0000","last_active":"2023-11-28T21:57:14.697621+0000","last_peered":"2023-11-28T21:57:14.697621+0000","last_clean":"2023-11-28T21:57:14.697621+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:57:14.697621+0000","last_undegraded":"2023-11-28T21:57:14.697621+0000","last_fullsized":"2023-11-28T21:57:14.697621+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1427","last_scrub_stamp":"2023-11-28T21:57:14.697528+0000","last_deep_scrub":"52'1427","last_deep_scrub_stamp":"2023-11-28T21:57:14.697528+0000","last_clean_scrub_stamp":"2023-11-28T21:57:14.697528+0000","objects_scrubbed":317,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:15:02.264993+0000","scrub_duration":0.20879229699999999,"objects_trimmed":0,"snaptrim_duration":0.28498848100000002,"stat_sum":{"num_bytes":33245588,"num_objects":317,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":263,"num_read_kb":28708,"num_write":1396,"num_write_kb":77612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"52'1309","reported_seq":3641,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:39.342045+0000","last_change":"2023-11-28T21:40:43.251187+0000","last_active":"2023-11-28T21:56:39.342045+0000","last_peered":"2023-11-28T21:56:39.342045+0000","last_clean":"2023-11-28T21:56:39.342045+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:56:39.342045+0000","last_undegraded":"2023-11-28T21:56:39.342045+0000","last_fullsized":"2023-11-28T21:56:39.342045+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":0,"log_size":1309,"log_dups_size":0,"ondisk_log_size":1309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.27353859400000002,"stat_sum":{"num_bytes":25285382,"num_objects":331,"num_object_clones":0,"num_object_copies":1324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":248,"num_read_kb":4117,"num_write":1287,"num_write_kb":58337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"52'285","reported_seq":2384,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:28.663920+0000","last_change":"2023-11-28T21:57:28.663963+0000","last_active":"2023-11-28T21:57:28.663920+0000","last_peered":"2023-11-28T21:57:28.663920+0000","last_clean":"2023-11-28T21:57:28.663920+0000","last_became_active":"2023-11-28T21:35:25.589378+0000","last_became_peered":"2023-11-28T21:35:25.589378+0000","last_unstale":"2023-11-28T21:57:28.663920+0000","last_undegraded":"2023-11-28T21:57:28.663920+0000","last_fullsized":"2023-11-28T21:57:28.663920+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'285","last_scrub_stamp":"2023-11-28T21:57:28.663780+0000","last_deep_scrub":"52'285","last_deep_scrub_stamp":"2023-11-28T21:57:28.663780+0000","last_clean_scrub_stamp":"2023-11-28T21:57:28.663780+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:58:59.855623+0000","scrub_duration":0.0020765459999999999,"objects_trimmed":0,"snaptrim_duration":0.246426856,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"52'951","reported_seq":1047,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.694770+0000","last_change":"2023-11-28T21:56:50.694839+0000","last_active":"2023-11-28T21:56:50.694770+0000","last_peered":"2023-11-28T21:56:50.694770+0000","last_clean":"2023-11-28T21:56:50.694770+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:56:50.694770+0000","last_undegraded":"2023-11-28T21:56:50.694770+0000","last_fullsized":"2023-11-28T21:56:50.694770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'951","last_scrub_stamp":"2023-11-28T21:56:50.694579+0000","last_deep_scrub":"52'951","last_deep_scrub_stamp":"2023-11-28T21:56:50.694579+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.694579+0000","objects_scrubbed":5,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:57:02.007325+0000","scrub_duration":0.095599400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":899,"num_write_kb":25075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12742,"num_omap_keys":23,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"52'1266","reported_seq":3563,"reported_epoch":52,"state":"active+clean+scrubbing+deep","last_fresh":"2023-11-28T21:57:40.136047+0000","last_change":"2023-11-28T21:57:40.038794+0000","last_active":"2023-11-28T21:57:40.136047+0000","last_peered":"2023-11-28T21:57:40.136047+0000","last_clean":"2023-11-28T21:57:40.136047+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:57:40.136047+0000","last_undegraded":"2023-11-28T21:57:40.136047+0000","last_fullsized":"2023-11-28T21:57:40.136047+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T21:34:40.321594+0000","last_clean_scrub_stamp":"2023-11-28T21:34:40.321594+0000","objects_scrubbed":124,"log_size":1266,"log_dups_size":0,"ondisk_log_size":1266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.16839808100000001,"stat_sum":{"num_bytes":18660789,"num_objects":319,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":205,"num_read_kb":12320,"num_write":1249,"num_write_kb":51464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"52'321","reported_seq":2412,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:58.470634+0000","last_change":"2023-11-28T21:56:58.470657+0000","last_active":"2023-11-28T21:56:58.470634+0000","last_peered":"2023-11-28T21:56:58.470634+0000","last_clean":"2023-11-28T21:56:58.470634+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:56:58.470634+0000","last_undegraded":"2023-11-28T21:56:58.470634+0000","last_fullsized":"2023-11-28T21:56:58.470634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'321","last_scrub_stamp":"2023-11-28T21:56:58.470530+0000","last_deep_scrub":"52'321","last_deep_scrub_stamp":"2023-11-28T21:56:58.470530+0000","last_clean_scrub_stamp":"2023-11-28T21:56:58.470530+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:47:39.793970+0000","scrub_duration":0.0019542520000000001,"objects_trimmed":0,"snaptrim_duration":0.28961016299999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"52'91","reported_seq":162,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:52.421023+0000","last_change":"2023-11-28T21:56:52.421071+0000","last_active":"2023-11-28T21:56:52.421023+0000","last_peered":"2023-11-28T21:56:52.421023+0000","last_clean":"2023-11-28T21:56:52.421023+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:56:52.421023+0000","last_undegraded":"2023-11-28T21:56:52.421023+0000","last_fullsized":"2023-11-28T21:56:52.421023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'91","last_scrub_stamp":"2023-11-28T21:56:52.420930+0000","last_deep_scrub":"52'91","last_deep_scrub_stamp":"2023-11-28T21:56:52.420930+0000","last_clean_scrub_stamp":"2023-11-28T21:56:52.420930+0000","objects_scrubbed":6,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:15:04.674788+0000","scrub_duration":0.030405108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":63,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":37270,"num_omap_keys":68,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"52'1081","reported_seq":1182,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.227886+0000","last_change":"2023-11-28T21:56:46.227886+0000","last_active":"2023-11-28T21:56:46.227886+0000","last_peered":"2023-11-28T21:56:46.227886+0000","last_clean":"2023-11-28T21:56:46.227886+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:56:46.227886+0000","last_undegraded":"2023-11-28T21:56:46.227886+0000","last_fullsized":"2023-11-28T21:56:46.227886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1081","last_scrub_stamp":"2023-11-28T21:56:46.227627+0000","last_deep_scrub":"52'1081","last_deep_scrub_stamp":"2023-11-28T21:56:46.227627+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.227627+0000","objects_scrubbed":10,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:49:00.930323+0000","scrub_duration":0.124157213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":146592,"num_write":1020,"num_write_kb":29113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":33716,"num_omap_keys":61,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"52'284","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:19.161483+0000","last_change":"2023-11-28T21:57:19.161506+0000","last_active":"2023-11-28T21:57:19.161483+0000","last_peered":"2023-11-28T21:57:19.161483+0000","last_clean":"2023-11-28T21:57:19.161483+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:57:19.161483+0000","last_undegraded":"2023-11-28T21:57:19.161483+0000","last_fullsized":"2023-11-28T21:57:19.161483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'284","last_scrub_stamp":"2023-11-28T21:57:19.161376+0000","last_deep_scrub":"52'284","last_deep_scrub_stamp":"2023-11-28T21:57:19.161376+0000","last_clean_scrub_stamp":"2023-11-28T21:57:19.161376+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:45.359179+0000","scrub_duration":0.0032904639999999999,"objects_trimmed":0,"snaptrim_duration":0.26886027800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"52'1325","reported_seq":3646,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.515073+0000","last_change":"2023-11-28T21:57:09.515073+0000","last_active":"2023-11-28T21:57:09.515073+0000","last_peered":"2023-11-28T21:57:09.515073+0000","last_clean":"2023-11-28T21:57:09.515073+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:57:09.515073+0000","last_undegraded":"2023-11-28T21:57:09.515073+0000","last_fullsized":"2023-11-28T21:57:09.515073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1325","last_scrub_stamp":"2023-11-28T21:57:09.514950+0000","last_deep_scrub":"52'1325","last_deep_scrub_stamp":"2023-11-28T21:57:09.514950+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.514950+0000","objects_scrubbed":342,"log_size":1325,"log_dups_size":0,"ondisk_log_size":1325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:43:36.262680+0000","scrub_duration":0.21339559899999999,"objects_trimmed":0,"snaptrim_duration":0.13232148399999999,"stat_sum":{"num_bytes":46546650,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":218,"num_read_kb":9401,"num_write":1296,"num_write_kb":74280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"52'473","reported_seq":550,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.461705+0000","last_change":"2023-11-28T21:56:53.461705+0000","last_active":"2023-11-28T21:56:53.461705+0000","last_peered":"2023-11-28T21:56:53.461705+0000","last_clean":"2023-11-28T21:56:53.461705+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:56:53.461705+0000","last_undegraded":"2023-11-28T21:56:53.461705+0000","last_fullsized":"2023-11-28T21:56:53.461705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'473","last_scrub_stamp":"2023-11-28T21:56:53.461589+0000","last_deep_scrub":"52'473","last_deep_scrub_stamp":"2023-11-28T21:56:53.461589+0000","last_clean_scrub_stamp":"2023-11-28T21:56:53.461589+0000","objects_scrubbed":6,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:21:39.580720+0000","scrub_duration":0.057920754999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":48896,"num_write":437,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":14594,"num_omap_keys":28,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"52'279","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.249131+0000","last_change":"2023-11-28T21:56:49.249131+0000","last_active":"2023-11-28T21:56:49.249131+0000","last_peered":"2023-11-28T21:56:49.249131+0000","last_clean":"2023-11-28T21:56:49.249131+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:56:49.249131+0000","last_undegraded":"2023-11-28T21:56:49.249131+0000","last_fullsized":"2023-11-28T21:56:49.249131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'279","last_scrub_stamp":"2023-11-28T21:56:49.248890+0000","last_deep_scrub":"52'279","last_deep_scrub_stamp":"2023-11-28T21:56:49.248890+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.248890+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:20:21.740665+0000","scrub_duration":0.0035163680000000002,"objects_trimmed":0,"snaptrim_duration":0.24793844200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"52'1320","reported_seq":3680,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:26.896353+0000","last_change":"2023-11-28T21:57:26.896377+0000","last_active":"2023-11-28T21:57:26.896353+0000","last_peered":"2023-11-28T21:57:26.896353+0000","last_clean":"2023-11-28T21:57:26.896353+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:57:26.896353+0000","last_undegraded":"2023-11-28T21:57:26.896353+0000","last_fullsized":"2023-11-28T21:57:26.896353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1320","last_scrub_stamp":"2023-11-28T21:57:26.896247+0000","last_deep_scrub":"52'1320","last_deep_scrub_stamp":"2023-11-28T21:57:26.896247+0000","last_clean_scrub_stamp":"2023-11-28T21:57:26.896247+0000","objects_scrubbed":333,"log_size":1320,"log_dups_size":0,"ondisk_log_size":1320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:24.699472+0000","scrub_duration":0.18315010500000001,"objects_trimmed":0,"snaptrim_duration":0.29562192300000001,"stat_sum":{"num_bytes":29852008,"num_objects":333,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":257,"num_read_kb":28714,"num_write":1297,"num_write_kb":83698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"52'321","reported_seq":2414,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:10.252806+0000","last_change":"2023-11-28T21:57:10.252806+0000","last_active":"2023-11-28T21:57:10.252806+0000","last_peered":"2023-11-28T21:57:10.252806+0000","last_clean":"2023-11-28T21:57:10.252806+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:57:10.252806+0000","last_undegraded":"2023-11-28T21:57:10.252806+0000","last_fullsized":"2023-11-28T21:57:10.252806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'321","last_scrub_stamp":"2023-11-28T21:57:10.252559+0000","last_deep_scrub":"52'321","last_deep_scrub_stamp":"2023-11-28T21:57:10.252559+0000","last_clean_scrub_stamp":"2023-11-28T21:57:10.252559+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:04.185942+0000","scrub_duration":0.0035793169999999998,"objects_trimmed":0,"snaptrim_duration":0.18187520500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"52'328","reported_seq":2422,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:03.303115+0000","last_change":"2023-11-28T21:57:03.303115+0000","last_active":"2023-11-28T21:57:03.303115+0000","last_peered":"2023-11-28T21:57:03.303115+0000","last_clean":"2023-11-28T21:57:03.303115+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:57:03.303115+0000","last_undegraded":"2023-11-28T21:57:03.303115+0000","last_fullsized":"2023-11-28T21:57:03.303115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'328","last_scrub_stamp":"2023-11-28T21:57:03.302989+0000","last_deep_scrub":"52'328","last_deep_scrub_stamp":"2023-11-28T21:57:03.302989+0000","last_clean_scrub_stamp":"2023-11-28T21:57:03.302989+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:52:13.280846+0000","scrub_duration":0.0030730839999999998,"objects_trimmed":0,"snaptrim_duration":0.14391041299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"52'303","reported_seq":2397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.533536+0000","last_change":"2023-11-28T21:56:44.533536+0000","last_active":"2023-11-28T21:56:44.533536+0000","last_peered":"2023-11-28T21:56:44.533536+0000","last_clean":"2023-11-28T21:56:44.533536+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:56:44.533536+0000","last_undegraded":"2023-11-28T21:56:44.533536+0000","last_fullsized":"2023-11-28T21:56:44.533536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'303","last_scrub_stamp":"2023-11-28T21:56:44.533290+0000","last_deep_scrub":"52'303","last_deep_scrub_stamp":"2023-11-28T21:56:44.533290+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.533290+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:11:39.585022+0000","scrub_duration":0.0035272839999999999,"objects_trimmed":0,"snaptrim_duration":0.30621145,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"52'294","reported_seq":2387,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.242644+0000","last_change":"2023-11-28T21:57:09.242644+0000","last_active":"2023-11-28T21:57:09.242644+0000","last_peered":"2023-11-28T21:57:09.242644+0000","last_clean":"2023-11-28T21:57:09.242644+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:57:09.242644+0000","last_undegraded":"2023-11-28T21:57:09.242644+0000","last_fullsized":"2023-11-28T21:57:09.242644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'294","last_scrub_stamp":"2023-11-28T21:57:09.242522+0000","last_deep_scrub":"52'294","last_deep_scrub_stamp":"2023-11-28T21:57:09.242522+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.242522+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:12:54.800329+0000","scrub_duration":0.001978142,"objects_trimmed":0,"snaptrim_duration":0.26967866099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"52'275","reported_seq":2367,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.387085+0000","last_change":"2023-11-28T21:56:46.387109+0000","last_active":"2023-11-28T21:56:46.387085+0000","last_peered":"2023-11-28T21:56:46.387085+0000","last_clean":"2023-11-28T21:56:46.387085+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:56:46.387085+0000","last_undegraded":"2023-11-28T21:56:46.387085+0000","last_fullsized":"2023-11-28T21:56:46.387085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'275","last_scrub_stamp":"2023-11-28T21:56:46.386968+0000","last_deep_scrub":"52'275","last_deep_scrub_stamp":"2023-11-28T21:56:46.386968+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.386968+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:09:10.256531+0000","scrub_duration":0.0028373539999999998,"objects_trimmed":0,"snaptrim_duration":0.29058787400000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"52'293","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.787300+0000","last_change":"2023-11-28T21:57:11.787318+0000","last_active":"2023-11-28T21:57:11.787300+0000","last_peered":"2023-11-28T21:57:11.787300+0000","last_clean":"2023-11-28T21:57:11.787300+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:57:11.787300+0000","last_undegraded":"2023-11-28T21:57:11.787300+0000","last_fullsized":"2023-11-28T21:57:11.787300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'293","last_scrub_stamp":"2023-11-28T21:57:11.787231+0000","last_deep_scrub":"52'293","last_deep_scrub_stamp":"2023-11-28T21:57:11.787231+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.787231+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:35:53.843661+0000","scrub_duration":0.0015063699999999999,"objects_trimmed":0,"snaptrim_duration":0.246741036,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"52'316","reported_seq":2408,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.330062+0000","last_change":"2023-11-28T21:57:13.330127+0000","last_active":"2023-11-28T21:57:13.330062+0000","last_peered":"2023-11-28T21:57:13.330062+0000","last_clean":"2023-11-28T21:57:13.330062+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:57:13.330062+0000","last_undegraded":"2023-11-28T21:57:13.330062+0000","last_fullsized":"2023-11-28T21:57:13.330062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'316","last_scrub_stamp":"2023-11-28T21:57:13.329847+0000","last_deep_scrub":"52'316","last_deep_scrub_stamp":"2023-11-28T21:57:13.329847+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.329847+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:32:35.225710+0000","scrub_duration":0.0039367100000000004,"objects_trimmed":0,"snaptrim_duration":0.18459727300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"52'287","reported_seq":2379,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:05.398087+0000","last_change":"2023-11-28T21:57:05.398109+0000","last_active":"2023-11-28T21:57:05.398087+0000","last_peered":"2023-11-28T21:57:05.398087+0000","last_clean":"2023-11-28T21:57:05.398087+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:57:05.398087+0000","last_undegraded":"2023-11-28T21:57:05.398087+0000","last_fullsized":"2023-11-28T21:57:05.398087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'287","last_scrub_stamp":"2023-11-28T21:57:05.397991+0000","last_deep_scrub":"52'287","last_deep_scrub_stamp":"2023-11-28T21:57:05.397991+0000","last_clean_scrub_stamp":"2023-11-28T21:57:05.397991+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:09:47.792188+0000","scrub_duration":0.002983145,"objects_trimmed":0,"snaptrim_duration":0.29272911699999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"52'285","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:07.843268+0000","last_change":"2023-11-28T21:57:07.843319+0000","last_active":"2023-11-28T21:57:07.843268+0000","last_peered":"2023-11-28T21:57:07.843268+0000","last_clean":"2023-11-28T21:57:07.843268+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:57:07.843268+0000","last_undegraded":"2023-11-28T21:57:07.843268+0000","last_fullsized":"2023-11-28T21:57:07.843268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'285","last_scrub_stamp":"2023-11-28T21:57:07.843058+0000","last_deep_scrub":"52'285","last_deep_scrub_stamp":"2023-11-28T21:57:07.843058+0000","last_clean_scrub_stamp":"2023-11-28T21:57:07.843058+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:11:36.128065+0000","scrub_duration":0.0032873389999999998,"objects_trimmed":0,"snaptrim_duration":0.30579284099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"52'272","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:51.436185+0000","last_change":"2023-11-28T21:56:51.436185+0000","last_active":"2023-11-28T21:56:51.436185+0000","last_peered":"2023-11-28T21:56:51.436185+0000","last_clean":"2023-11-28T21:56:51.436185+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:56:51.436185+0000","last_undegraded":"2023-11-28T21:56:51.436185+0000","last_fullsized":"2023-11-28T21:56:51.436185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'272","last_scrub_stamp":"2023-11-28T21:56:51.436052+0000","last_deep_scrub":"52'272","last_deep_scrub_stamp":"2023-11-28T21:56:51.436052+0000","last_clean_scrub_stamp":"2023-11-28T21:56:51.436052+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:27.893115+0000","scrub_duration":0.0032975470000000001,"objects_trimmed":0,"snaptrim_duration":0.28793565599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"52'285","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.755908+0000","last_change":"2023-11-28T21:57:16.755960+0000","last_active":"2023-11-28T21:57:16.755908+0000","last_peered":"2023-11-28T21:57:16.755908+0000","last_clean":"2023-11-28T21:57:16.755908+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:57:16.755908+0000","last_undegraded":"2023-11-28T21:57:16.755908+0000","last_fullsized":"2023-11-28T21:57:16.755908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'285","last_scrub_stamp":"2023-11-28T21:57:16.755612+0000","last_deep_scrub":"52'285","last_deep_scrub_stamp":"2023-11-28T21:57:16.755612+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.755612+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:01.560436+0000","scrub_duration":0.0036730019999999999,"objects_trimmed":0,"snaptrim_duration":0.30636868099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.293494+0000","last_change":"2023-11-28T21:57:04.293494+0000","last_active":"2023-11-28T21:57:04.293494+0000","last_peered":"2023-11-28T21:57:04.293494+0000","last_clean":"2023-11-28T21:57:04.293494+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:57:04.293494+0000","last_undegraded":"2023-11-28T21:57:04.293494+0000","last_fullsized":"2023-11-28T21:57:04.293494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:04.293402+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:04.293402+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.293402+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:26:25.956930+0000","scrub_duration":0.0026303339999999998,"objects_trimmed":0,"snaptrim_duration":0.26835999100000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:22.280875+0000","last_change":"2023-11-28T21:57:22.280955+0000","last_active":"2023-11-28T21:57:22.280875+0000","last_peered":"2023-11-28T21:57:22.280875+0000","last_clean":"2023-11-28T21:57:22.280875+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:57:22.280875+0000","last_undegraded":"2023-11-28T21:57:22.280875+0000","last_fullsized":"2023-11-28T21:57:22.280875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:22.280604+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:22.280604+0000","last_clean_scrub_stamp":"2023-11-28T21:57:22.280604+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:15:11.070749+0000","scrub_duration":0.0040196090000000004,"objects_trimmed":0,"snaptrim_duration":0.194469013,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"52'293","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:18.231899+0000","last_change":"2023-11-28T21:57:18.231950+0000","last_active":"2023-11-28T21:57:18.231899+0000","last_peered":"2023-11-28T21:57:18.231899+0000","last_clean":"2023-11-28T21:57:18.231899+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:57:18.231899+0000","last_undegraded":"2023-11-28T21:57:18.231899+0000","last_fullsized":"2023-11-28T21:57:18.231899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'293","last_scrub_stamp":"2023-11-28T21:57:18.231694+0000","last_deep_scrub":"52'293","last_deep_scrub_stamp":"2023-11-28T21:57:18.231694+0000","last_clean_scrub_stamp":"2023-11-28T21:57:18.231694+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:39:32.183707+0000","scrub_duration":0.0026952199999999999,"objects_trimmed":0,"snaptrim_duration":0.21943309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"52'268","reported_seq":2360,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.255130+0000","last_change":"2023-11-28T21:57:04.255150+0000","last_active":"2023-11-28T21:57:04.255130+0000","last_peered":"2023-11-28T21:57:04.255130+0000","last_clean":"2023-11-28T21:57:04.255130+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:57:04.255130+0000","last_undegraded":"2023-11-28T21:57:04.255130+0000","last_fullsized":"2023-11-28T21:57:04.255130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'268","last_scrub_stamp":"2023-11-28T21:57:04.255048+0000","last_deep_scrub":"52'268","last_deep_scrub_stamp":"2023-11-28T21:57:04.255048+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.255048+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:40:34.130819+0000","scrub_duration":0.0027406069999999999,"objects_trimmed":0,"snaptrim_duration":0.14941532399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"52'313","reported_seq":2405,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:42.478620+0000","last_change":"2023-11-28T21:56:42.478673+0000","last_active":"2023-11-28T21:56:42.478620+0000","last_peered":"2023-11-28T21:56:42.478620+0000","last_clean":"2023-11-28T21:56:42.478620+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:56:42.478620+0000","last_undegraded":"2023-11-28T21:56:42.478620+0000","last_fullsized":"2023-11-28T21:56:42.478620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'313","last_scrub_stamp":"2023-11-28T21:56:42.478435+0000","last_deep_scrub":"52'313","last_deep_scrub_stamp":"2023-11-28T21:56:42.478435+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.478435+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:59.037978+0000","scrub_duration":0.0037738749999999999,"objects_trimmed":0,"snaptrim_duration":0.30374592,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"52'301","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.566855+0000","last_change":"2023-11-28T21:56:45.566906+0000","last_active":"2023-11-28T21:56:45.566855+0000","last_peered":"2023-11-28T21:56:45.566855+0000","last_clean":"2023-11-28T21:56:45.566855+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:56:45.566855+0000","last_undegraded":"2023-11-28T21:56:45.566855+0000","last_fullsized":"2023-11-28T21:56:45.566855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'301","last_scrub_stamp":"2023-11-28T21:56:45.566667+0000","last_deep_scrub":"52'301","last_deep_scrub_stamp":"2023-11-28T21:56:45.566667+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.566667+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:37:12.783651+0000","scrub_duration":0.0036235360000000001,"objects_trimmed":0,"snaptrim_duration":0.30238551699999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"52'316","reported_seq":2408,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.741463+0000","last_change":"2023-11-28T21:57:04.741484+0000","last_active":"2023-11-28T21:57:04.741463+0000","last_peered":"2023-11-28T21:57:04.741463+0000","last_clean":"2023-11-28T21:57:04.741463+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:57:04.741463+0000","last_undegraded":"2023-11-28T21:57:04.741463+0000","last_fullsized":"2023-11-28T21:57:04.741463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'316","last_scrub_stamp":"2023-11-28T21:57:04.741375+0000","last_deep_scrub":"52'316","last_deep_scrub_stamp":"2023-11-28T21:57:04.741375+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.741375+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:53:03.728731+0000","scrub_duration":0.0023203569999999999,"objects_trimmed":0,"snaptrim_duration":0.246777987,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"52'301","reported_seq":2392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.425032+0000","last_change":"2023-11-28T21:56:55.425054+0000","last_active":"2023-11-28T21:56:55.425032+0000","last_peered":"2023-11-28T21:56:55.425032+0000","last_clean":"2023-11-28T21:56:55.425032+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:56:55.425032+0000","last_undegraded":"2023-11-28T21:56:55.425032+0000","last_fullsized":"2023-11-28T21:56:55.425032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'301","last_scrub_stamp":"2023-11-28T21:56:55.424936+0000","last_deep_scrub":"52'301","last_deep_scrub_stamp":"2023-11-28T21:56:55.424936+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.424936+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:58:23.742113+0000","scrub_duration":0.0019665469999999999,"objects_trimmed":0,"snaptrim_duration":0.21399896700000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"52'274","reported_seq":2365,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.388835+0000","last_change":"2023-11-28T21:56:48.388857+0000","last_active":"2023-11-28T21:56:48.388835+0000","last_peered":"2023-11-28T21:56:48.388835+0000","last_clean":"2023-11-28T21:56:48.388835+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:56:48.388835+0000","last_undegraded":"2023-11-28T21:56:48.388835+0000","last_fullsized":"2023-11-28T21:56:48.388835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'274","last_scrub_stamp":"2023-11-28T21:56:48.388743+0000","last_deep_scrub":"52'274","last_deep_scrub_stamp":"2023-11-28T21:56:48.388743+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.388743+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:55.401441+0000","scrub_duration":0.0030358709999999999,"objects_trimmed":0,"snaptrim_duration":0.287656776,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"52'261","reported_seq":2352,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.326425+0000","last_change":"2023-11-28T21:56:46.326533+0000","last_active":"2023-11-28T21:56:46.326425+0000","last_peered":"2023-11-28T21:56:46.326425+0000","last_clean":"2023-11-28T21:56:46.326425+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:56:46.326425+0000","last_undegraded":"2023-11-28T21:56:46.326425+0000","last_fullsized":"2023-11-28T21:56:46.326425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'261","last_scrub_stamp":"2023-11-28T21:56:46.326201+0000","last_deep_scrub":"52'261","last_deep_scrub_stamp":"2023-11-28T21:56:46.326201+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.326201+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:57.520054+0000","scrub_duration":0.0037943849999999999,"objects_trimmed":0,"snaptrim_duration":0.222804104,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"52'261","reported_seq":2352,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:33.057770+0000","last_change":"2023-11-28T21:57:33.057822+0000","last_active":"2023-11-28T21:57:33.057770+0000","last_peered":"2023-11-28T21:57:33.057770+0000","last_clean":"2023-11-28T21:57:33.057770+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:57:33.057770+0000","last_undegraded":"2023-11-28T21:57:33.057770+0000","last_fullsized":"2023-11-28T21:57:33.057770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'261","last_scrub_stamp":"2023-11-28T21:57:33.057492+0000","last_deep_scrub":"52'261","last_deep_scrub_stamp":"2023-11-28T21:57:33.057492+0000","last_clean_scrub_stamp":"2023-11-28T21:57:33.057492+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:41.660862+0000","scrub_duration":0.0037777190000000001,"objects_trimmed":0,"snaptrim_duration":0.19768582900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"52'283","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.372700+0000","last_change":"2023-11-28T21:57:16.372800+0000","last_active":"2023-11-28T21:57:16.372700+0000","last_peered":"2023-11-28T21:57:16.372700+0000","last_clean":"2023-11-28T21:57:16.372700+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:57:16.372700+0000","last_undegraded":"2023-11-28T21:57:16.372700+0000","last_fullsized":"2023-11-28T21:57:16.372700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:57:16.372401+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:57:16.372401+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.372401+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:12.888742+0000","scrub_duration":0.0041957210000000003,"objects_trimmed":0,"snaptrim_duration":0.18434383600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"52'314","reported_seq":2413,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:26.291975+0000","last_change":"2023-11-28T21:57:26.292027+0000","last_active":"2023-11-28T21:57:26.291975+0000","last_peered":"2023-11-28T21:57:26.291975+0000","last_clean":"2023-11-28T21:57:26.291975+0000","last_became_active":"2023-11-28T21:35:25.598839+0000","last_became_peered":"2023-11-28T21:35:25.598839+0000","last_unstale":"2023-11-28T21:57:26.291975+0000","last_undegraded":"2023-11-28T21:57:26.291975+0000","last_fullsized":"2023-11-28T21:57:26.291975+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'314","last_scrub_stamp":"2023-11-28T21:57:26.291780+0000","last_deep_scrub":"52'314","last_deep_scrub_stamp":"2023-11-28T21:57:26.291780+0000","last_clean_scrub_stamp":"2023-11-28T21:57:26.291780+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:05:40.948354+0000","scrub_duration":0.0037891439999999999,"objects_trimmed":0,"snaptrim_duration":0.22244017299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"52'291","reported_seq":2382,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:10.406800+0000","last_change":"2023-11-28T21:57:10.406825+0000","last_active":"2023-11-28T21:57:10.406800+0000","last_peered":"2023-11-28T21:57:10.406800+0000","last_clean":"2023-11-28T21:57:10.406800+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:57:10.406800+0000","last_undegraded":"2023-11-28T21:57:10.406800+0000","last_fullsized":"2023-11-28T21:57:10.406800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'291","last_scrub_stamp":"2023-11-28T21:57:10.406693+0000","last_deep_scrub":"52'291","last_deep_scrub_stamp":"2023-11-28T21:57:10.406693+0000","last_clean_scrub_stamp":"2023-11-28T21:57:10.406693+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:00:26.481166+0000","scrub_duration":0.0034554859999999998,"objects_trimmed":0,"snaptrim_duration":0.28637254899999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"52'292","reported_seq":2384,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:05.386536+0000","last_change":"2023-11-28T21:57:05.386536+0000","last_active":"2023-11-28T21:57:05.386536+0000","last_peered":"2023-11-28T21:57:05.386536+0000","last_clean":"2023-11-28T21:57:05.386536+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:57:05.386536+0000","last_undegraded":"2023-11-28T21:57:05.386536+0000","last_fullsized":"2023-11-28T21:57:05.386536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'292","last_scrub_stamp":"2023-11-28T21:57:05.386287+0000","last_deep_scrub":"52'292","last_deep_scrub_stamp":"2023-11-28T21:57:05.386287+0000","last_clean_scrub_stamp":"2023-11-28T21:57:05.386287+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:49.649068+0000","scrub_duration":0.0038884100000000001,"objects_trimmed":0,"snaptrim_duration":0.219142752,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"52'282","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:24.707483+0000","last_change":"2023-11-28T21:57:24.707536+0000","last_active":"2023-11-28T21:57:24.707483+0000","last_peered":"2023-11-28T21:57:24.707483+0000","last_clean":"2023-11-28T21:57:24.707483+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:57:24.707483+0000","last_undegraded":"2023-11-28T21:57:24.707483+0000","last_fullsized":"2023-11-28T21:57:24.707483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'282","last_scrub_stamp":"2023-11-28T21:57:24.707271+0000","last_deep_scrub":"52'282","last_deep_scrub_stamp":"2023-11-28T21:57:24.707271+0000","last_clean_scrub_stamp":"2023-11-28T21:57:24.707271+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:31:32.444772+0000","scrub_duration":0.0040746819999999996,"objects_trimmed":0,"snaptrim_duration":0.307113203,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.239031+0000","last_change":"2023-11-28T21:57:16.239138+0000","last_active":"2023-11-28T21:57:16.239031+0000","last_peered":"2023-11-28T21:57:16.239031+0000","last_clean":"2023-11-28T21:57:16.239031+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:57:16.239031+0000","last_undegraded":"2023-11-28T21:57:16.239031+0000","last_fullsized":"2023-11-28T21:57:16.239031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:16.238782+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:16.238782+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.238782+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:31.855622+0000","scrub_duration":0.0036200709999999999,"objects_trimmed":0,"snaptrim_duration":0.21561430200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"52'287","reported_seq":2381,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:21.659207+0000","last_change":"2023-11-28T21:57:21.659264+0000","last_active":"2023-11-28T21:57:21.659207+0000","last_peered":"2023-11-28T21:57:21.659207+0000","last_clean":"2023-11-28T21:57:21.659207+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:57:21.659207+0000","last_undegraded":"2023-11-28T21:57:21.659207+0000","last_fullsized":"2023-11-28T21:57:21.659207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'287","last_scrub_stamp":"2023-11-28T21:57:21.658984+0000","last_deep_scrub":"52'287","last_deep_scrub_stamp":"2023-11-28T21:57:21.658984+0000","last_clean_scrub_stamp":"2023-11-28T21:57:21.658984+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:42:40.185184+0000","scrub_duration":0.0032803480000000002,"objects_trimmed":0,"snaptrim_duration":0.30486287499999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"52'281","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:18.200046+0000","last_change":"2023-11-28T21:57:18.200046+0000","last_active":"2023-11-28T21:57:18.200046+0000","last_peered":"2023-11-28T21:57:18.200046+0000","last_clean":"2023-11-28T21:57:18.200046+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:57:18.200046+0000","last_undegraded":"2023-11-28T21:57:18.200046+0000","last_fullsized":"2023-11-28T21:57:18.200046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'281","last_scrub_stamp":"2023-11-28T21:57:18.199943+0000","last_deep_scrub":"52'281","last_deep_scrub_stamp":"2023-11-28T21:57:18.199943+0000","last_clean_scrub_stamp":"2023-11-28T21:57:18.199943+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:04.947793+0000","scrub_duration":0.0015704720000000001,"objects_trimmed":0,"snaptrim_duration":0.26474323500000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"52'292","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.805004+0000","last_change":"2023-11-28T21:57:09.805004+0000","last_active":"2023-11-28T21:57:09.805004+0000","last_peered":"2023-11-28T21:57:09.805004+0000","last_clean":"2023-11-28T21:57:09.805004+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:57:09.805004+0000","last_undegraded":"2023-11-28T21:57:09.805004+0000","last_fullsized":"2023-11-28T21:57:09.805004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'292","last_scrub_stamp":"2023-11-28T21:57:09.804904+0000","last_deep_scrub":"52'292","last_deep_scrub_stamp":"2023-11-28T21:57:09.804904+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.804904+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:06:20.229796+0000","scrub_duration":0.0016870489999999999,"objects_trimmed":0,"snaptrim_duration":0.24685295800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"52'289","reported_seq":2383,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:27.327882+0000","last_change":"2023-11-28T21:57:27.327882+0000","last_active":"2023-11-28T21:57:27.327882+0000","last_peered":"2023-11-28T21:57:27.327882+0000","last_clean":"2023-11-28T21:57:27.327882+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:57:27.327882+0000","last_undegraded":"2023-11-28T21:57:27.327882+0000","last_fullsized":"2023-11-28T21:57:27.327882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'289","last_scrub_stamp":"2023-11-28T21:57:27.327611+0000","last_deep_scrub":"52'289","last_deep_scrub_stamp":"2023-11-28T21:57:27.327611+0000","last_clean_scrub_stamp":"2023-11-28T21:57:27.327611+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:57:34.455926+0000","scrub_duration":0.003750448,"objects_trimmed":0,"snaptrim_duration":0.22257470400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":927864585,"num_objects":10340,"num_object_clones":0,"num_object_copies":41360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10340,"num_whiteouts":0,"num_read":7542,"num_read_kb":441308,"num_write":41596,"num_write_kb":1955411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1822244864,"data_stored":1974304768,"data_compressed":124280088,"data_compressed_allocated":140013568,"data_compressed_original":292073472,"omap_allocated":0,"internal_metadata":0},"log_size":42425,"ondisk_log_size":42425,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18675,"ondisk_log_size":18675,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":303332196,"num_objects":196,"num_object_clones":0,"num_object_copies":392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1486,"num_read_kb":1965797,"num_write":14882,"num_write_kb":402412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":687608,"num_omap_keys":1199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173137920,"data_stored":606664392,"data_compressed":82562470,"data_compressed_allocated":105857024,"data_compressed_original":539541398,"omap_allocated":0,"internal_metadata":0},"log_size":16086,"ondisk_log_size":16086,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575131,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329152,"kb_used_data":211896,"kb_used_omap":1,"kb_used_meta":117246,"kb_avail":94042688,"statfs":{"total":96636764160,"available":96299712512,"internally_reserved":0,"allocated":216981504,"data_stored":292765622,"data_compressed":24984621,"data_compressed_allocated":29888512,"data_compressed_original":105979904,"omap_allocated":1590,"internal_metadata":120060362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.874,"15min":0.98499999999999999},"min":{"1min":0.498,"5min":0.41399999999999998,"15min":0.35299999999999998},"max":{"1min":3.3719999999999999,"5min":5.056,"15min":47.475000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.85599999999999998,"15min":0.82999999999999996},"min":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.36199999999999999},"max":{"1min":1.8260000000000001,"5min":4.069,"15min":5.3310000000000004},"last":1.1950000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0069999999999999,"15min":1.1060000000000001},"min":{"1min":0.65400000000000003,"5min":0.59699999999999998,"15min":0.39200000000000002},"max":{"1min":1.79,"5min":2.5630000000000002,"15min":47.529000000000003},"last":1.139},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0860000000000001,"15min":1.0189999999999999},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.734,"5min":17.460000000000001,"15min":17.460000000000001},"last":0.85199999999999998}]},{"osd":2,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.83899999999999997,"15min":0.90500000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98899999999999999,"5min":2.9359999999999999,"15min":5.5140000000000002},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.008,"15min":1.0249999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.498},"max":{"1min":1.212,"5min":2.972,"15min":5.5979999999999999},"last":1.0780000000000001}]},{"osd":3,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.216,"15min":1.198},"min":{"1min":0.60699999999999998,"5min":0.57299999999999995,"15min":0.44600000000000001},"max":{"1min":7.3760000000000003,"5min":16.113,"15min":47.57},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.327,"5min":1.214,"15min":1.0920000000000001},"min":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.53800000000000003},"max":{"1min":7.4269999999999996,"5min":11.148999999999999,"15min":12.273},"last":1.171}]},{"osd":4,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.125,"15min":1.0609999999999999},"min":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.35499999999999998},"max":{"1min":5.6859999999999999,"5min":17.542000000000002,"15min":17.542000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0429999999999999,"15min":1.1950000000000001},"min":{"1min":0.67200000000000004,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":49.106000000000002},"last":0.89800000000000002}]},{"osd":5,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.2909999999999999},"min":{"1min":0.91700000000000004,"5min":0.65600000000000003,"15min":0.40899999999999997},"max":{"1min":2.0110000000000001,"5min":6.6420000000000003,"15min":64.855999999999995},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.1779999999999999,"15min":1.206},"min":{"1min":0.79800000000000004,"5min":0.73499999999999999,"15min":0.56399999999999995},"max":{"1min":5.843,"5min":11.327,"15min":16.753},"last":0.98699999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.216,"15min":1.175},"min":{"1min":0.72999999999999998,"5min":0.67500000000000004,"15min":0.37},"max":{"1min":5.7679999999999998,"5min":11.641999999999999,"15min":11.641999999999999},"last":1.105},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.1719999999999999,"15min":1.167},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.502},"max":{"1min":5.4210000000000003,"5min":5.4210000000000003,"15min":7.4909999999999997},"last":1.2350000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575129,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":380176,"kb_used_data":254600,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":93991664,"statfs":{"total":96636764160,"available":96247463936,"internally_reserved":0,"allocated":260710400,"data_stored":335877014,"data_compressed":24633140,"data_compressed_allocated":29302784,"data_compressed_original":104771531,"omap_allocated":1594,"internal_metadata":128580038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.276,"15min":1.0249999999999999},"min":{"1min":0.55800000000000005,"5min":0.441,"15min":0.27300000000000002},"max":{"1min":2.9119999999999999,"5min":40.999000000000002,"15min":40.999000000000002},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.042,"15min":1.04},"min":{"1min":0.62,"5min":0.42299999999999999,"15min":0.40000000000000002},"max":{"1min":2.3370000000000002,"5min":14.132999999999999,"15min":24.399999999999999},"last":1.1639999999999999}]},{"osd":1,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":0.99399999999999999,"15min":0.92000000000000004},"min":{"1min":0.55900000000000005,"5min":0.48599999999999999,"15min":0.35899999999999999},"max":{"1min":7.0709999999999997,"5min":7.0709999999999997,"15min":11.196},"last":1.101},{"interface":"front","average":{"1min":1.611,"5min":1.0389999999999999,"15min":1.012},"min":{"1min":0.57499999999999996,"5min":0.42799999999999999,"15min":0.40899999999999997},"max":{"1min":13.404,"5min":13.404,"15min":13.404},"last":1.0289999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":0.94299999999999995,"15min":0.96399999999999997},"min":{"1min":0.74399999999999999,"5min":0.56999999999999995,"15min":0.39100000000000001},"max":{"1min":2.6829999999999998,"5min":3.855,"15min":21.986999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.004,"15min":1.024},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.40400000000000003},"max":{"1min":8.4730000000000008,"5min":8.4730000000000008,"15min":8.6170000000000009},"last":0.91600000000000004}]},{"osd":3,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.131,"15min":1.0189999999999999},"min":{"1min":0.68400000000000005,"5min":0.54600000000000004,"15min":0.41999999999999998},"max":{"1min":15.964,"5min":15.964,"15min":15.964},"last":1.135},{"interface":"front","average":{"1min":1.897,"5min":1.1919999999999999,"15min":1.149},"min":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.47499999999999998},"max":{"1min":16.602,"5min":16.602,"15min":24.745000000000001},"last":1.056}]},{"osd":4,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.95999999999999996,"15min":1.018},"min":{"1min":0.69799999999999995,"5min":0.59799999999999998,"15min":0.30299999999999999},"max":{"1min":1.7050000000000001,"5min":2.4550000000000001,"15min":16.015000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.093,"15min":1.0900000000000001},"min":{"1min":0.91000000000000003,"5min":0.65100000000000002,"15min":0.35799999999999998},"max":{"1min":2.4929999999999999,"5min":2.5329999999999999,"15min":13.193},"last":0.95099999999999996}]},{"osd":5,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.0329999999999999,"15min":1.1200000000000001},"min":{"1min":0.86699999999999999,"5min":0.58699999999999997,"15min":0.307},"max":{"1min":1.661,"5min":2.069,"15min":25.006},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.276,"5min":1.1140000000000001,"15min":1.151},"min":{"1min":0.86799999999999999,"5min":0.46700000000000003,"15min":0.33000000000000002},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":7.843},"last":0.72699999999999998}]},{"osd":7,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.109,"15min":1.163},"min":{"1min":0.78600000000000003,"5min":0.68999999999999995,"15min":0.36099999999999999},"max":{"1min":5.6799999999999997,"5min":5.6799999999999997,"15min":7.883},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.246,"15min":1.304},"min":{"1min":0.95399999999999996,"5min":0.86499999999999999,"15min":0.432},"max":{"1min":5.7199999999999998,"5min":5.7199999999999998,"15min":27.942},"last":1.27}]}]},{"osd":5,"up_from":13,"seq":55834575129,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363676,"kb_used_data":238100,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":94008164,"statfs":{"total":96636764160,"available":96264359936,"internally_reserved":0,"allocated":243814400,"data_stored":323612635,"data_compressed":28196796,"data_compressed_allocated":33439744,"data_compressed_original":113541120,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:25 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0349999999999999,"15min":0.90800000000000003},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.40600000000000003},"max":{"1min":3.3980000000000001,"5min":15.005000000000001,"15min":15.005000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0009999999999999,"15min":0.90800000000000003},"min":{"1min":0.54200000000000004,"5min":0.495,"15min":0.46700000000000003},"max":{"1min":3.419,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":1.0740000000000001}]},{"osd":1,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.84799999999999998,"15min":0.84299999999999997},"min":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.48199999999999998},"max":{"1min":3.806,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":0.96399999999999997,"15min":0.93700000000000006},"min":{"1min":0.57999999999999996,"5min":0.57299999999999995,"15min":0.496},"max":{"1min":7.4930000000000003,"5min":7.4930000000000003,"15min":7.4930000000000003},"last":0.91500000000000004}]},{"osd":2,"last update":"Tue Nov 28 21:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0109999999999999,"15min":0.97699999999999998},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2649999999999999,"5min":14.775,"15min":14.775},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1379999999999999,"15min":0.96299999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.57499999999999996},"max":{"1min":1.462,"5min":14.901999999999999,"15min":14.901999999999999},"last":1.115}]},{"osd":3,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.038,"15min":0.98999999999999999},"min":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.54400000000000004},"max":{"1min":1.347,"5min":15.154,"15min":15.154},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.02,"5min":0.996,"15min":1.0149999999999999},"min":{"1min":0.68600000000000005,"5min":0.60799999999999998,"15min":0.53500000000000003},"max":{"1min":1.3069999999999999,"5min":3.4289999999999998,"15min":5.5170000000000003},"last":1.163}]},{"osd":4,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.534,"15min":1.1870000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.52000000000000002},"max":{"1min":2.98,"5min":61.020000000000003,"15min":61.020000000000003},"last":0.747},{"interface":"front","average":{"1min":1.405,"5min":1.22,"15min":1.1160000000000001},"min":{"1min":0.67400000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":8.6029999999999998,"5min":15.085000000000001,"15min":15.085000000000001},"last":1.2010000000000001}]},{"osd":6,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.137,"15min":1.1259999999999999},"min":{"1min":0.72599999999999998,"5min":0.66000000000000003,"15min":0.56699999999999995},"max":{"1min":1.4099999999999999,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.234,"15min":1.181},"min":{"1min":1.0589999999999999,"5min":0.73199999999999998,"15min":0.64800000000000002},"max":{"1min":1.4970000000000001,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.413}]},{"osd":7,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2070000000000001,"15min":1.1739999999999999},"min":{"1min":0.85599999999999998,"5min":0.75600000000000001,"15min":0.70399999999999996},"max":{"1min":1.5600000000000001,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.333},{"interface":"front","average":{"1min":1.042,"5min":1.0600000000000001,"15min":1.0289999999999999},"min":{"1min":0.69299999999999995,"5min":0.66800000000000004,"15min":0.60499999999999998},"max":{"1min":1.6779999999999999,"5min":6.2869999999999999,"15min":6.2869999999999999},"last":0.80300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575129,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357644,"kb_used_data":236228,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":94014196,"statfs":{"total":96636764160,"available":96270536704,"internally_reserved":0,"allocated":241897472,"data_stored":321511706,"data_compressed":25475129,"data_compressed_allocated":30593024,"data_compressed_original":110510027,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2210000000000001,"15min":1.004},"min":{"1min":0.38200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":9.2759999999999998,"5min":43.139000000000003,"15min":43.139000000000003},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.573,"5min":0.96599999999999997,"15min":0.94399999999999995},"min":{"1min":0.57299999999999995,"5min":0.499,"15min":0.39900000000000002},"max":{"1min":9.9009999999999998,"5min":9.9009999999999998,"15min":17.446000000000002},"last":0.628}]},{"osd":1,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.8200000000000001,"5min":1.153,"15min":1.1140000000000001},"min":{"1min":0.67100000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":15.432,"5min":15.432,"15min":21.763999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.72,"5min":1.145,"15min":1.5369999999999999},"min":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.46500000000000002},"max":{"1min":12.186999999999999,"5min":12.186999999999999,"15min":78.649000000000001},"last":1.1319999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.9319999999999999,"5min":1.1439999999999999,"15min":1.1160000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":20.105,"5min":20.105,"15min":28.341000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.2070000000000001,"15min":1.603},"min":{"1min":0.70399999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"max":{"1min":15.393000000000001,"5min":15.393000000000001,"15min":94.998999999999995},"last":0.90400000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":2.0720000000000001,"5min":1.4019999999999999,"15min":1.169},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":20.065000000000001,"5min":20.065000000000001,"15min":20.065000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":2.0419999999999998,"5min":1.246,"15min":1.1279999999999999},"min":{"1min":0.64200000000000002,"5min":0.54800000000000004,"15min":0.53200000000000003},"max":{"1min":18.588000000000001,"5min":18.588000000000001,"15min":18.588000000000001},"last":0.748}]},{"osd":5,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.0600000000000001,"15min":1.159},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.376},"max":{"1min":12.159000000000001,"5min":12.159000000000001,"15min":17.033000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.468,"15min":1.329},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.437},"max":{"1min":9.5500000000000007,"5min":38.939,"15min":38.939},"last":1.1859999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":2.3500000000000001,"5min":1.2929999999999999,"15min":1.2609999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.49299999999999999},"max":{"1min":28.289000000000001,"5min":28.289000000000001,"15min":28.289000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5640000000000001,"15min":1.367},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.55600000000000005},"max":{"1min":9.5120000000000005,"5min":43.420000000000002,"15min":43.420000000000002},"last":1.2350000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.1859999999999999,"15min":1.6499999999999999},"min":{"1min":0.55900000000000005,"5min":0.434,"15min":0.434},"max":{"1min":10.282,"5min":10.282,"15min":77.638999999999996},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.3089999999999999,"15min":1.2070000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.51500000000000001},"max":{"1min":9.6080000000000005,"5min":9.6080000000000005,"15min":9.6080000000000005},"last":1.2789999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575130,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465164,"kb_used_data":307652,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93906676,"statfs":{"total":96636764160,"available":96160436224,"internally_reserved":0,"allocated":315035648,"data_stored":410723951,"data_compressed":36998413,"data_compressed_allocated":43757568,"data_compressed_original":139747328,"omap_allocated":1592,"internal_metadata":161282504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.89400000000000002,"15min":0.70899999999999996},"min":{"1min":0.312,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.77600000000000002,"5min":36.585000000000001,"15min":36.585000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.88600000000000001,"15min":0.67600000000000005},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.33100000000000002},"max":{"1min":0.996,"5min":30.065000000000001,"15min":30.065000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.42699999999999999,"5min":0.33200000000000002,"15min":0.26300000000000001},"max":{"1min":0.80800000000000005,"5min":4.8099999999999996,"15min":10.449},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.58799999999999997,"15min":0.67100000000000004},"min":{"1min":0.33600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80400000000000005,"5min":4.8520000000000003,"15min":10.481999999999999},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57499999999999996,"15min":0.64800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.27900000000000003},"max":{"1min":0.90200000000000002,"5min":1.25,"15min":10.369},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.69099999999999995},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":10.406000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91700000000000004,"15min":1.3580000000000001},"min":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.376},"max":{"1min":1.036,"5min":4.2119999999999997,"15min":110.39700000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.97399999999999998,"15min":1.399},"min":{"1min":0.752,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.2609999999999999,"5min":7.2030000000000003,"15min":108.777},"last":0.96699999999999997}]},{"osd":5,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.871,"15min":0.92600000000000005},"min":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.39900000000000002},"max":{"1min":1.196,"5min":7.2679999999999998,"15min":17.513000000000002},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.85799999999999998,"15min":0.92900000000000005},"min":{"1min":0.64400000000000002,"5min":0.621,"15min":0.52300000000000002},"max":{"1min":1.323,"5min":1.7070000000000001,"15min":10.516999999999999},"last":0.99199999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83199999999999996,"15min":1.399},"min":{"1min":0.62,"5min":0.52700000000000002,"15min":0.45200000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":138.91300000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.875,"15min":0.94799999999999995},"min":{"1min":0.65000000000000002,"5min":0.53300000000000003,"15min":0.48299999999999998},"max":{"1min":1.23,"5min":1.766,"15min":10.302},"last":0.88200000000000001}]},{"osd":7,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.92500000000000004,"15min":0.96099999999999997},"min":{"1min":0.66000000000000003,"5min":0.60999999999999999,"15min":0.45200000000000001},"max":{"1min":1.246,"5min":7.3040000000000003,"15min":17.263000000000002},"last":1.028},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.72599999999999998,"5min":0.54900000000000004,"15min":0.49299999999999999},"max":{"1min":1.397,"5min":7.3339999999999996,"15min":28.613},"last":0.90500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575128,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288308,"kb_used_data":176620,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94083532,"statfs":{"total":96636764160,"available":96341536768,"internally_reserved":0,"allocated":180858880,"data_stored":223905413,"data_compressed":16019215,"data_compressed_allocated":18952192,"data_compressed_original":62304256,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.63500000000000001,"15min":0.58699999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.23599999999999999},"max":{"1min":1.6679999999999999,"5min":13.599,"15min":13.599},"last":0.37},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.74299999999999999,"15min":0.63700000000000001},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.318},"max":{"1min":7.1159999999999997,"5min":8.0630000000000006,"15min":8.0630000000000006},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.32000000000000001,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":7.8970000000000002,"5min":7.8970000000000002,"15min":9.3260000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.173,"5min":0.81200000000000006,"15min":0.71299999999999997},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.34200000000000003},"max":{"1min":10.657,"5min":10.657,"15min":10.657},"last":0.53300000000000003}]},{"osd":3,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.046,"15min":0.81899999999999995},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.307},"max":{"1min":7.1639999999999997,"5min":29.123999999999999,"15min":29.123999999999999},"last":0.505},{"interface":"front","average":{"1min":1.498,"5min":0.80600000000000005,"15min":0.66500000000000004},"min":{"1min":0.36199999999999999,"5min":0.312,"15min":0.28000000000000003},"max":{"1min":10.18,"5min":10.18,"15min":10.18},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97299999999999998,"15min":0.93500000000000005},"min":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.49099999999999999},"max":{"1min":2.9769999999999999,"5min":5.6289999999999996,"15min":7.3630000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.002,"15min":0.92600000000000005},"min":{"1min":0.59299999999999997,"5min":0.51200000000000001,"15min":0.41399999999999998},"max":{"1min":12.723000000000001,"5min":12.723000000000001,"15min":12.723000000000001},"last":0.90000000000000002}]},{"osd":5,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":0.94099999999999995,"15min":0.87},"min":{"1min":0.64900000000000002,"5min":0.501,"15min":0.501},"max":{"1min":6.875,"5min":6.875,"15min":6.875},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.2070000000000001,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.59399999999999997,"15min":0.48399999999999999},"max":{"1min":9.4809999999999999,"5min":9.4809999999999999,"15min":9.4809999999999999},"last":0.69599999999999995}]},{"osd":6,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.093,"15min":0.92900000000000005},"min":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":3.5489999999999999,"5min":18.959,"15min":18.959},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.897,"5min":1.325,"15min":1.0369999999999999},"min":{"1min":0.77700000000000002,"5min":0.622,"15min":0.52200000000000002},"max":{"1min":9.4399999999999995,"5min":21.957999999999998,"15min":21.957999999999998},"last":0.82999999999999996}]},{"osd":7,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":1.0800000000000001,"15min":0.98099999999999998},"min":{"1min":0.64200000000000002,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":9.2249999999999996,"5min":9.2249999999999996,"15min":9.2799999999999994},"last":0.88},{"interface":"front","average":{"1min":1.7989999999999999,"5min":1.109,"15min":0.97199999999999998},"min":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":9.1679999999999993,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":0.73299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575128,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469328,"kb_used_data":311816,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93902512,"statfs":{"total":96636764160,"available":96156172288,"internally_reserved":0,"allocated":319299584,"data_stored":398618222,"data_compressed":30748401,"data_compressed_allocated":36524032,"data_compressed_original":116154368,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.75700000000000001,"15min":0.69299999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.246},"max":{"1min":0.65400000000000003,"5min":12.311999999999999,"15min":14.538},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.621,"15min":0.64900000000000002},"min":{"1min":0.33400000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":8.2479999999999993,"15min":14.618},"last":0.36699999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.65200000000000002,"15min":0.627},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.77700000000000002,"5min":5.1689999999999996,"15min":8.4749999999999996},"last":0.438},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36899999999999999},"max":{"1min":0.72599999999999998,"5min":4.7030000000000003,"15min":8.5150000000000006},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.75700000000000001,"15min":0.84699999999999998},"min":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.27600000000000002},"max":{"1min":0.68500000000000005,"5min":13.385,"15min":43.536999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.82499999999999996,"15min":0.84499999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":8.2899999999999991,"15min":18.308},"last":0.502}]},{"osd":4,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.98499999999999999,"15min":0.95199999999999996},"min":{"1min":0.498,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":1.0900000000000001,"5min":9.9849999999999994,"15min":9.9849999999999994},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.98399999999999999,"15min":1.0760000000000001},"min":{"1min":0.58199999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.149,"5min":10.287000000000001,"15min":43.497999999999998},"last":0.82199999999999995}]},{"osd":5,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.93000000000000005,"15min":0.91700000000000004},"min":{"1min":0.72199999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.085,"5min":3.3740000000000001,"15min":5.5350000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.01,"15min":0.995},"min":{"1min":0.874,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1799999999999999,"5min":3.448,"15min":5.7409999999999997},"last":0.94299999999999995}]},{"osd":6,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":1.0349999999999999,"15min":1.0069999999999999},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0629999999999999,"5min":10.023,"15min":10.385},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.0509999999999999,"15min":1.2490000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.59599999999999997},"max":{"1min":1.1719999999999999,"5min":5.9349999999999996,"15min":43.448999999999998},"last":0.67100000000000004}]},{"osd":7,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":1.0580000000000001,"15min":1.02},"min":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.56899999999999995},"max":{"1min":1.135,"5min":5.9779999999999998,"15min":7.96},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.1830000000000001,"15min":1.161},"min":{"1min":0.65700000000000003,"5min":0.55800000000000005,"15min":0.51500000000000001},"max":{"1min":1.109,"5min":15.672000000000001,"15min":42.473999999999997},"last":0.72599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575129,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":346236,"kb_used_data":216500,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94025604,"statfs":{"total":96636764160,"available":96282218496,"internally_reserved":0,"allocated":221696000,"data_stored":277678898,"data_compressed":19875137,"data_compressed_allocated":23535616,"data_compressed_original":79818752,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.7909999999999999,"5min":0.75,"15min":0.56399999999999995},"min":{"1min":0.27900000000000003,"5min":0.27800000000000002,"15min":0.215},"max":{"1min":21.814,"5min":21.814,"15min":21.814},"last":0.44900000000000001},{"interface":"front","average":{"1min":1.8500000000000001,"5min":0.88300000000000001,"15min":0.624},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.248},"max":{"1min":21.925000000000001,"5min":21.925000000000001,"15min":21.925000000000001},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.9399999999999999,"5min":0.93700000000000006,"15min":0.70199999999999996},"min":{"1min":0.47499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":21.852,"5min":21.852,"15min":21.852},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.617,"5min":0.82099999999999995,"15min":0.67600000000000005},"min":{"1min":0.36399999999999999,"5min":0.318,"15min":0.318},"max":{"1min":18.213000000000001,"5min":18.213000000000001,"15min":18.213000000000001},"last":0.65100000000000002}]},{"osd":3,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.083,"15min":0.72199999999999998},"min":{"1min":0.495,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":21.98,"5min":21.98,"15min":21.98},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.857,"5min":1.071,"15min":0.77600000000000002},"min":{"1min":0.437,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":20.931999999999999,"5min":20.931999999999999,"15min":20.931999999999999},"last":0.53700000000000003}]},{"osd":4,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.98599999999999999,"15min":0.92500000000000004},"min":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.42899999999999999},"max":{"1min":4.0810000000000004,"5min":5.2069999999999999,"15min":5.734},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.9379999999999999,"5min":1.1000000000000001,"15min":0.97399999999999998},"min":{"1min":0.65100000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":17.98,"5min":17.98,"15min":17.98},"last":0.98799999999999999}]},{"osd":5,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":2.1960000000000002,"5min":1.1919999999999999,"15min":0.93400000000000005},"min":{"1min":0.65800000000000003,"5min":0.61799999999999999,"15min":0.40699999999999997},"max":{"1min":21.890999999999998,"5min":21.890999999999998,"15min":21.890999999999998},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.931,"5min":1.1519999999999999,"15min":0.95799999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":18.181000000000001,"5min":18.181000000000001,"15min":18.181000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Tue Nov 28 21:56:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.056,"15min":0.91500000000000004},"min":{"1min":0.63,"5min":0.61799999999999999,"15min":0.48699999999999999},"max":{"1min":4.0410000000000004,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.9670000000000001,"5min":1.1399999999999999,"15min":0.95799999999999996},"min":{"1min":0.68600000000000005,"5min":0.60499999999999998,"15min":0.55100000000000005},"max":{"1min":18.030999999999999,"5min":18.030999999999999,"15min":18.030999999999999},"last":1.03}]},{"osd":7,"last update":"Tue Nov 28 21:57:20 2023","interfaces":[{"interface":"back","average":{"1min":1.7769999999999999,"5min":1.131,"15min":0.96899999999999997},"min":{"1min":0.65400000000000003,"5min":0.627,"15min":0.48999999999999999},"max":{"1min":18.140999999999998,"5min":18.140999999999998,"15min":18.140999999999998},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.7909999999999999,"5min":1.143,"15min":0.96099999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.55700000000000005},"max":{"1min":18.071000000000002,"5min":18.071000000000002,"15min":18.071000000000002},"last":1.0669999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14426112,"data_stored":54526612,"data_compressed":7767250,"data_compressed_allocated":9949184,"data_compressed_original":50065408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24457216,"data_stored":78922741,"data_compressed":10489735,"data_compressed_allocated":13656064,"data_compressed_original":68149248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15634432,"data_stored":48245772,"data_compressed":6171423,"data_compressed_allocated":8048640,"data_compressed_original":40681472,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25956352,"data_stored":94383094,"data_compressed":13023761,"data_compressed_allocated":16736256,"data_compressed_original":85180416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23392256,"data_stored":83887777,"data_compressed":11283626,"data_compressed_allocated":14446592,"data_compressed_original":74960843,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22241280,"data_stored":79693154,"data_compressed":11120127,"data_compressed_allocated":14139392,"data_compressed_original":71610368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":22917120,"data_stored":83888413,"data_compressed":11073880,"data_compressed_allocated":14143488,"data_compressed_original":75132875,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24113152,"data_stored":83116829,"data_compressed":11632668,"data_compressed_allocated":14737408,"data_compressed_original":73760768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":206610432,"data_stored":222244864,"data_compressed":12071885,"data_compressed_allocated":13537280,"data_compressed_original":29171712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294244352,"data_stored":319377408,"data_compressed":20255951,"data_compressed_allocated":22863872,"data_compressed_original":47996928,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164626432,"data_stored":175341568,"data_compressed":9845077,"data_compressed_allocated":10899456,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":288481280,"data_stored":316022784,"data_compressed":23971937,"data_compressed_allocated":27017216,"data_compressed_original":54558720,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":217907200,"data_stored":237305856,"data_compressed":14188788,"data_compressed_allocated":16142336,"data_compressed_original":35540992,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":220975104,"data_stored":243601408,"data_compressed":17073954,"data_compressed_allocated":19296256,"data_compressed_original":41922560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":237195264,"data_stored":251670528,"data_compressed":13556545,"data_compressed_allocated":15155200,"data_compressed_original":29630464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":192204800,"data_stored":208740352,"data_compressed":13315951,"data_compressed_allocated":15101952,"data_compressed_original":31637504,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:57:44.731 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:44.731 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:44.731 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-11-28T21:34:40.321594+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=34, tm_sec=40, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=21, tm_min=56, tm_sec=37, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T21:57:44.732 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T21:57:47.782 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:57:47.787 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:58:04.733 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T21:58:04.889 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.884+0000 7f98701b8640 1 Processor -- start 2023-11-28T21:58:04.890 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.884+0000 7f98701b8640 1 -- start start 2023-11-28T21:58:04.891 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f98701b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98681500d0 0x7f98680c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f98701b8640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f986806d7d0 0x7f986806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f98701b8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98680c6ce0 con 0x7f9868156af0 2023-11-28T21:58:04.893 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f98701b8640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98680c7170 con 0x7f986806d7d0 2023-11-28T21:58:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f98701b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98680c5df0 con 0x7f98681500d0 2023-11-28T21:58:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986d753640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9868156af0 0x7f98680c2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60770/0 (socket says 172.21.15.12:60770) 2023-11-28T21:58:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986d753640 1 -- 172.21.15.12:0/3692540105 learned_addr learned my addr 172.21.15.12:0/3692540105 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:58:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986df54640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98681500d0 0x7f98680c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:04.894 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986e755640 1 --2- 172.21.15.12:0/3692540105 >> v2:172.21.15.144:3300/0 conn(0x7f986806d7d0 0x7f986806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111401932 0 0) 0x7f98680c6ce0 con 0x7f9868156af0 2023-11-28T21:58:04.896 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9854005740 con 0x7f9868156af0 2023-11-28T21:58:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2830666271 0 0) 0x7f9854005740 con 0x7f9868156af0 2023-11-28T21:58:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98681500d0 msgr2=0x7f98680c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98681500d0 0x7f98680c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 >> v2:172.21.15.144:3300/0 conn(0x7f986806d7d0 msgr2=0x7f986806dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 --2- 172.21.15.12:0/3692540105 >> v2:172.21.15.144:3300/0 conn(0x7f986806d7d0 0x7f986806dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680c60d0 con 0x7f9868156af0 2023-11-28T21:58:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f9858001680 con 0x7f9868156af0 2023-11-28T21:58:04.897 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 >> v1:172.21.15.12:6789/0 conn(0x7f9868156af0 legacy=0x7f98680c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9854005c00 0x7f9854006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98540069e0 0x7f9854006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98681500d0 0x7f98540073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98680c7170 con 0x7f98540069e0 2023-11-28T21:58:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98680c5df0 con 0x7f9854005c00 2023-11-28T21:58:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985400dba0 con 0x7f98681500d0 2023-11-28T21:58:04.898 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986d753640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98540069e0 0x7f9854006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:04.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986df54640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9854005c00 0x7f9854006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:04.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986e755640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98681500d0 0x7f98540073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:04.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986d753640 1 -- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98681500d0 msgr2=0x7f98540073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:04.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986d753640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98681500d0 0x7f98540073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:04.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986d753640 1 -- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9854005c00 msgr2=0x7f9854006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:04.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986d753640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9854005c00 0x7f9854006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:04.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.888+0000 7f986d753640 1 -- 172.21.15.12:0/3692540105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680c60d0 con 0x7f98540069e0 2023-11-28T21:58:04.899 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f986d753640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98540069e0 0x7f9854006e60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f985800c290 tx=0x7f985801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fe91072bcd293e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:04.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580a8ce0 con 0x7f98540069e0 2023-11-28T21:58:04.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98580a8e60 con 0x7f98540069e0 2023-11-28T21:58:04.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f986cf52640 1 -- 172.21.15.12:0/3692540105 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858005b20 con 0x7f98540069e0 2023-11-28T21:58:04.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f98701b8640 1 -- 172.21.15.12:0/3692540105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98540069e0 msgr2=0x7f9854006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:04.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f98701b8640 1 --2- 172.21.15.12:0/3692540105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f98540069e0 0x7f9854006e60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f985800c290 tx=0x7f985801f3b0 comp rx=0 tx=0).stop 2023-11-28T21:58:04.900 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f986e755640 1 -- 172.21.15.12:0/3692540105 reap_dead start 2023-11-28T21:58:04.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f98701b8640 1 -- 172.21.15.12:0/3692540105 shutdown_connections 2023-11-28T21:58:04.901 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f98701b8640 1 -- 172.21.15.12:0/3692540105 >> 172.21.15.12:0/3692540105 conn(0x7f9868066360 msgr2=0x7f9868066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:04.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f98701b8640 1 -- 172.21.15.12:0/3692540105 shutdown_connections 2023-11-28T21:58:04.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.892+0000 7f98701b8640 1 -- 172.21.15.12:0/3692540105 wait complete. 2023-11-28T21:58:04.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f98701b8640 1 Processor -- start 2023-11-28T21:58:04.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f98701b8640 1 -- start start 2023-11-28T21:58:04.903 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f98701b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98540069e0 0x7f98681efe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:04.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f98701b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9868152dc0 0x7f98681f0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:04.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f98701b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9868071470 0x7f986814aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:04.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f98701b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98680c9d40 con 0x7f9868152dc0 2023-11-28T21:58:04.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f98701b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98681f0380 con 0x7f9868071470 2023-11-28T21:58:04.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f98701b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98681519b0 con 0x7f98540069e0 2023-11-28T21:58:04.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986d753640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9868152dc0 0x7f98681f0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:04.904 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986d753640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9868152dc0 0x7f98681f0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37796/0 (socket says 172.21.15.12:37796) 2023-11-28T21:58:04.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986d753640 1 -- 172.21.15.12:0/3529222552 learned_addr learned my addr 172.21.15.12:0/3529222552 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:04.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986e755640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9868071470 0x7f986814aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:04.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986df54640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98540069e0 0x7f98681efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:04.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986d753640 1 -- 172.21.15.12:0/3529222552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98540069e0 msgr2=0x7f98681efe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:04.905 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986d753640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98540069e0 0x7f98681efe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:04.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986d753640 1 -- 172.21.15.12:0/3529222552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9868071470 msgr2=0x7f986814aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:04.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986d753640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9868071470 0x7f986814aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:04.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986d753640 1 -- 172.21.15.12:0/3529222552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9868153590 con 0x7f9868152dc0 2023-11-28T21:58:04.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f986d753640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9868152dc0 0x7f98681f0870 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9858007040 tx=0x7f9854007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:04.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f985e7fc640 1 -- 172.21.15.12:0/3529222552 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580049f0 con 0x7f9868152dc0 2023-11-28T21:58:04.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f985e7fc640 1 -- 172.21.15.12:0/3529222552 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98580b6070 con 0x7f9868152dc0 2023-11-28T21:58:04.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f985e7fc640 1 -- 172.21.15.12:0/3529222552 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858004d90 con 0x7f9868152dc0 2023-11-28T21:58:04.906 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f98701b8640 1 -- 172.21.15.12:0/3529222552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9868153870 con 0x7f9868152dc0 2023-11-28T21:58:04.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.896+0000 7f98701b8640 1 -- 172.21.15.12:0/3529222552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9868151f30 con 0x7f9868152dc0 2023-11-28T21:58:04.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.900+0000 7f98701b8640 1 -- 172.21.15.12:0/3529222552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9868074f60 con 0x7f9868152dc0 2023-11-28T21:58:04.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.900+0000 7f985e7fc640 1 -- 172.21.15.12:0/3529222552 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f98580070d0 con 0x7f9868152dc0 2023-11-28T21:58:04.907 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.900+0000 7f985e7fc640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f984c0664a0 0x7f984c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:04.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.900+0000 7f985e7fc640 1 -- 172.21.15.12:0/3529222552 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f98580b8070 con 0x7f9868152dc0 2023-11-28T21:58:04.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.900+0000 7f986df54640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f984c0664a0 0x7f984c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:04.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.900+0000 7f986df54640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f984c0664a0 0x7f984c068960 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9864009a80 tx=0x7f9864009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:04.908 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:04.904+0000 7f985e7fc640 1 -- 172.21.15.12:0/3529222552 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98580ee280 con 0x7f9868152dc0 2023-11-28T21:58:05.065 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.060+0000 7f98701b8640 1 -- 172.21.15.12:0/3529222552 --> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98680c6970 con 0x7f984c0664a0 2023-11-28T21:58:05.075 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.072+0000 7f985e7fc640 1 -- 172.21.15.12:0/3529222552 <== mgr.4104 v2:172.21.15.12:6832/3598850103 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+376867 (secure 0 0 0) 0x7f9864019070 con 0x7f984c0664a0 2023-11-28T21:58:05.080 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:58:05.082 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2023-11-28T21:58:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 -- 172.21.15.12:0/3529222552 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f984c0664a0 msgr2=0x7f984c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f984c0664a0 0x7f984c068960 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9864009a80 tx=0x7f9864009190 comp rx=0 tx=0).stop 2023-11-28T21:58:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 -- 172.21.15.12:0/3529222552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9868152dc0 msgr2=0x7f98681f0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9868152dc0 0x7f98681f0870 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9858007040 tx=0x7f9854007a10 comp rx=0 tx=0).stop 2023-11-28T21:58:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 -- 172.21.15.12:0/3529222552 shutdown_connections 2023-11-28T21:58:05.084 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f984c0664a0 0x7f984c068960 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9868071470 0x7f986814aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9868152dc0 0x7f98681f0870 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 --2- 172.21.15.12:0/3529222552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98540069e0 0x7f98681efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 -- 172.21.15.12:0/3529222552 >> 172.21.15.12:0/3529222552 conn(0x7f9868066360 msgr2=0x7f986806ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 -- 172.21.15.12:0/3529222552 shutdown_connections 2023-11-28T21:58:05.085 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.076+0000 7f98701b8640 1 -- 172.21.15.12:0/3529222552 wait complete. 2023-11-28T21:58:05.102 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":752,"stamp":"2023-11-28T21:58:04.236171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1231787149,"num_objects":10538,"num_object_clones":0,"num_object_copies":41756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10538,"num_whiteouts":0,"num_read":9144,"num_read_kb":2407205,"num_write":56637,"num_write_kb":2360239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":687608,"num_omap_keys":1199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77305,"ondisk_log_size":77305,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2999684,"kb_used_data":1953412,"kb_used_omap":12,"kb_used_meta":1046195,"kb_avail":751975036,"statfs":{"total":773094113280,"available":770022436864,"internally_reserved":0,"allocated":2000293888,"data_stored":2584693461,"data_compressed":206930852,"data_compressed_allocated":245993472,"data_compressed_original":832827286,"omap_allocated":12751,"internal_metadata":1071304241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003558"},"pg_stats":[{"pgid":"2.1f","version":"52'347","reported_seq":443,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.479855+0000","last_change":"2023-11-28T21:56:50.479855+0000","last_active":"2023-11-28T21:56:50.479855+0000","last_peered":"2023-11-28T21:56:50.479855+0000","last_clean":"2023-11-28T21:56:50.479855+0000","last_became_active":"2023-11-28T21:34:36.011714+0000","last_became_peered":"2023-11-28T21:34:36.011714+0000","last_unstale":"2023-11-28T21:56:50.479855+0000","last_undegraded":"2023-11-28T21:56:50.479855+0000","last_fullsized":"2023-11-28T21:56:50.479855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'347","last_scrub_stamp":"2023-11-28T21:56:50.479765+0000","last_deep_scrub":"52'347","last_deep_scrub_stamp":"2023-11-28T21:56:50.479765+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.479765+0000","objects_scrubbed":6,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:38:00.234465+0000","scrub_duration":0.028234239000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":61536,"num_write":310,"num_write_kb":8499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":41082,"num_omap_keys":18,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"52'277","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.442324+0000","last_change":"2023-11-28T21:57:04.442342+0000","last_active":"2023-11-28T21:57:04.442324+0000","last_peered":"2023-11-28T21:57:04.442324+0000","last_clean":"2023-11-28T21:57:04.442324+0000","last_became_active":"2023-11-28T21:34:37.335392+0000","last_became_peered":"2023-11-28T21:34:37.335392+0000","last_unstale":"2023-11-28T21:57:04.442324+0000","last_undegraded":"2023-11-28T21:57:04.442324+0000","last_fullsized":"2023-11-28T21:57:04.442324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'277","last_scrub_stamp":"2023-11-28T21:57:04.442253+0000","last_deep_scrub":"52'277","last_deep_scrub_stamp":"2023-11-28T21:57:04.442253+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.442253+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:47:27.933208+0000","scrub_duration":0.0025024589999999998,"objects_trimmed":0,"snaptrim_duration":0.28950874199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"52'1318","reported_seq":3650,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:22.822296+0000","last_change":"2023-11-28T21:57:22.822343+0000","last_active":"2023-11-28T21:57:22.822296+0000","last_peered":"2023-11-28T21:57:22.822296+0000","last_clean":"2023-11-28T21:57:22.822296+0000","last_became_active":"2023-11-28T21:34:41.483388+0000","last_became_peered":"2023-11-28T21:34:41.483388+0000","last_unstale":"2023-11-28T21:57:22.822296+0000","last_undegraded":"2023-11-28T21:57:22.822296+0000","last_fullsized":"2023-11-28T21:57:22.822296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1318","last_scrub_stamp":"2023-11-28T21:57:22.822202+0000","last_deep_scrub":"52'1318","last_deep_scrub_stamp":"2023-11-28T21:57:22.822202+0000","last_clean_scrub_stamp":"2023-11-28T21:57:22.822202+0000","objects_scrubbed":349,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:14:46.637231+0000","scrub_duration":0.186113631,"objects_trimmed":0,"snaptrim_duration":0.29329539100000002,"stat_sum":{"num_bytes":30193112,"num_objects":349,"num_object_clones":0,"num_object_copies":1396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":229,"num_read_kb":12319,"num_write":1291,"num_write_kb":57664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"52'534","reported_seq":618,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:54.494340+0000","last_change":"2023-11-28T21:56:54.494393+0000","last_active":"2023-11-28T21:56:54.494340+0000","last_peered":"2023-11-28T21:56:54.494340+0000","last_clean":"2023-11-28T21:56:54.494340+0000","last_became_active":"2023-11-28T21:34:35.995194+0000","last_became_peered":"2023-11-28T21:34:35.995194+0000","last_unstale":"2023-11-28T21:56:54.494340+0000","last_undegraded":"2023-11-28T21:56:54.494340+0000","last_fullsized":"2023-11-28T21:56:54.494340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'534","last_scrub_stamp":"2023-11-28T21:56:54.494241+0000","last_deep_scrub":"52'534","last_deep_scrub_stamp":"2023-11-28T21:56:54.494241+0000","last_clean_scrub_stamp":"2023-11-28T21:56:54.494241+0000","objects_scrubbed":7,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:51:32.158044+0000","scrub_duration":0.057898067999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":81920,"num_write":485,"num_write_kb":24906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8778,"num_omap_keys":17,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"52'269","reported_seq":2362,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:42.409939+0000","last_change":"2023-11-28T21:56:42.409939+0000","last_active":"2023-11-28T21:56:42.409939+0000","last_peered":"2023-11-28T21:56:42.409939+0000","last_clean":"2023-11-28T21:56:42.409939+0000","last_became_active":"2023-11-28T21:34:37.333391+0000","last_became_peered":"2023-11-28T21:34:37.333391+0000","last_unstale":"2023-11-28T21:56:42.409939+0000","last_undegraded":"2023-11-28T21:56:42.409939+0000","last_fullsized":"2023-11-28T21:56:42.409939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'269","last_scrub_stamp":"2023-11-28T21:56:42.409827+0000","last_deep_scrub":"52'269","last_deep_scrub_stamp":"2023-11-28T21:56:42.409827+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.409827+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:28:29.334975+0000","scrub_duration":0.0029008010000000002,"objects_trimmed":0,"snaptrim_duration":0.23094383900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"52'1401","reported_seq":3755,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.530886+0000","last_change":"2023-11-28T21:57:17.530942+0000","last_active":"2023-11-28T21:57:17.530886+0000","last_peered":"2023-11-28T21:57:17.530886+0000","last_clean":"2023-11-28T21:57:17.530886+0000","last_became_active":"2023-11-28T21:34:41.387138+0000","last_became_peered":"2023-11-28T21:34:41.387138+0000","last_unstale":"2023-11-28T21:57:17.530886+0000","last_undegraded":"2023-11-28T21:57:17.530886+0000","last_fullsized":"2023-11-28T21:57:17.530886+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1401","last_scrub_stamp":"2023-11-28T21:57:17.530763+0000","last_deep_scrub":"52'1401","last_deep_scrub_stamp":"2023-11-28T21:57:17.530763+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.530763+0000","objects_scrubbed":330,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:05:20.452581+0000","scrub_duration":0.19727914399999999,"objects_trimmed":0,"snaptrim_duration":0.21613474099999999,"stat_sum":{"num_bytes":11729902,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":251,"num_read_kb":24615,"num_write":1375,"num_write_kb":48688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"52'764","reported_seq":867,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.611643+0000","last_change":"2023-11-28T21:56:50.611643+0000","last_active":"2023-11-28T21:56:50.611643+0000","last_peered":"2023-11-28T21:56:50.611643+0000","last_clean":"2023-11-28T21:56:50.611643+0000","last_became_active":"2023-11-28T21:34:36.034198+0000","last_became_peered":"2023-11-28T21:34:36.034198+0000","last_unstale":"2023-11-28T21:56:50.611643+0000","last_undegraded":"2023-11-28T21:56:50.611643+0000","last_fullsized":"2023-11-28T21:56:50.611643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'764","last_scrub_stamp":"2023-11-28T21:56:50.611223+0000","last_deep_scrub":"52'764","last_deep_scrub_stamp":"2023-11-28T21:56:50.611223+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.611223+0000","objects_scrubbed":5,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:28:25.261471+0000","scrub_duration":0.059433334999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":59,"num_read_kb":131099,"num_write":715,"num_write_kb":20835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13932,"num_omap_keys":27,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"52'294","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.715849+0000","last_change":"2023-11-28T21:57:17.715901+0000","last_active":"2023-11-28T21:57:17.715849+0000","last_peered":"2023-11-28T21:57:17.715849+0000","last_clean":"2023-11-28T21:57:17.715849+0000","last_became_active":"2023-11-28T21:34:37.346414+0000","last_became_peered":"2023-11-28T21:34:37.346414+0000","last_unstale":"2023-11-28T21:57:17.715849+0000","last_undegraded":"2023-11-28T21:57:17.715849+0000","last_fullsized":"2023-11-28T21:57:17.715849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'294","last_scrub_stamp":"2023-11-28T21:57:17.715645+0000","last_deep_scrub":"52'294","last_deep_scrub_stamp":"2023-11-28T21:57:17.715645+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.715645+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:16:17.284423+0000","scrub_duration":0.0037761639999999998,"objects_trimmed":0,"snaptrim_duration":0.30458173900000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"52'1324","reported_seq":3652,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.623143+0000","last_change":"2023-11-28T21:56:57.623143+0000","last_active":"2023-11-28T21:56:57.623143+0000","last_peered":"2023-11-28T21:56:57.623143+0000","last_clean":"2023-11-28T21:56:57.623143+0000","last_became_active":"2023-11-28T21:34:41.383175+0000","last_became_peered":"2023-11-28T21:34:41.383175+0000","last_unstale":"2023-11-28T21:56:57.623143+0000","last_undegraded":"2023-11-28T21:56:57.623143+0000","last_fullsized":"2023-11-28T21:56:57.623143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1324","last_scrub_stamp":"2023-11-28T21:56:57.622774+0000","last_deep_scrub":"52'1324","last_deep_scrub_stamp":"2023-11-28T21:56:57.622774+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.622774+0000","objects_scrubbed":314,"log_size":1324,"log_dups_size":0,"ondisk_log_size":1324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:08:12.479556+0000","scrub_duration":0.25549122899999999,"objects_trimmed":0,"snaptrim_duration":0.23712909900000001,"stat_sum":{"num_bytes":32661416,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":226,"num_read_kb":7690,"num_write":1300,"num_write_kb":56471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"52'96","reported_seq":176,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:47.599947+0000","last_change":"2023-11-28T21:56:47.599947+0000","last_active":"2023-11-28T21:56:47.599947+0000","last_peered":"2023-11-28T21:56:47.599947+0000","last_clean":"2023-11-28T21:56:47.599947+0000","last_became_active":"2023-11-28T21:34:36.019971+0000","last_became_peered":"2023-11-28T21:34:36.019971+0000","last_unstale":"2023-11-28T21:56:47.599947+0000","last_undegraded":"2023-11-28T21:56:47.599947+0000","last_fullsized":"2023-11-28T21:56:47.599947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'96","last_scrub_stamp":"2023-11-28T21:56:47.599552+0000","last_deep_scrub":"52'96","last_deep_scrub_stamp":"2023-11-28T21:56:47.599552+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.599552+0000","objects_scrubbed":2,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:41:39.184546+0000","scrub_duration":0.012005902000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36891,"num_write":51,"num_write_kb":4126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10256,"num_omap_keys":19,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"52'283","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.791963+0000","last_change":"2023-11-28T21:57:09.791963+0000","last_active":"2023-11-28T21:57:09.791963+0000","last_peered":"2023-11-28T21:57:09.791963+0000","last_clean":"2023-11-28T21:57:09.791963+0000","last_became_active":"2023-11-28T21:34:37.336971+0000","last_became_peered":"2023-11-28T21:34:37.336971+0000","last_unstale":"2023-11-28T21:57:09.791963+0000","last_undegraded":"2023-11-28T21:57:09.791963+0000","last_fullsized":"2023-11-28T21:57:09.791963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:57:09.791711+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:57:09.791711+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.791711+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:05.807760+0000","scrub_duration":0.0033624179999999998,"objects_trimmed":0,"snaptrim_duration":0.30416134,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"52'1294","reported_seq":3632,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:56.624661+0000","last_change":"2023-11-28T21:56:56.624661+0000","last_active":"2023-11-28T21:56:56.624661+0000","last_peered":"2023-11-28T21:56:56.624661+0000","last_clean":"2023-11-28T21:56:56.624661+0000","last_became_active":"2023-11-28T21:34:41.370967+0000","last_became_peered":"2023-11-28T21:34:41.370967+0000","last_unstale":"2023-11-28T21:56:56.624661+0000","last_undegraded":"2023-11-28T21:56:56.624661+0000","last_fullsized":"2023-11-28T21:56:56.624661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1294","last_scrub_stamp":"2023-11-28T21:56:56.624359+0000","last_deep_scrub":"52'1294","last_deep_scrub_stamp":"2023-11-28T21:56:56.624359+0000","last_clean_scrub_stamp":"2023-11-28T21:56:56.624359+0000","objects_scrubbed":312,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:59:20.046078+0000","scrub_duration":0.25756709,"objects_trimmed":0,"snaptrim_duration":0.26667665699999998,"stat_sum":{"num_bytes":37281632,"num_objects":312,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":0,"num_read":236,"num_read_kb":13169,"num_write":1275,"num_write_kb":61269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"52'303","reported_seq":2395,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:23.239181+0000","last_change":"2023-11-28T21:57:23.239234+0000","last_active":"2023-11-28T21:57:23.239181+0000","last_peered":"2023-11-28T21:57:23.239181+0000","last_clean":"2023-11-28T21:57:23.239181+0000","last_became_active":"2023-11-28T21:34:37.352779+0000","last_became_peered":"2023-11-28T21:34:37.352779+0000","last_unstale":"2023-11-28T21:57:23.239181+0000","last_undegraded":"2023-11-28T21:57:23.239181+0000","last_fullsized":"2023-11-28T21:57:23.239181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'303","last_scrub_stamp":"2023-11-28T21:57:23.238975+0000","last_deep_scrub":"52'303","last_deep_scrub_stamp":"2023-11-28T21:57:23.238975+0000","last_clean_scrub_stamp":"2023-11-28T21:57:23.238975+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:37:16.094494+0000","scrub_duration":0.0035136970000000001,"objects_trimmed":0,"snaptrim_duration":0.200606007,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"52'627","reported_seq":731,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.667245+0000","last_change":"2023-11-28T21:56:48.667245+0000","last_active":"2023-11-28T21:56:48.667245+0000","last_peered":"2023-11-28T21:56:48.667245+0000","last_clean":"2023-11-28T21:56:48.667245+0000","last_became_active":"2023-11-28T21:34:36.026522+0000","last_became_peered":"2023-11-28T21:34:36.026522+0000","last_unstale":"2023-11-28T21:56:48.667245+0000","last_undegraded":"2023-11-28T21:56:48.667245+0000","last_fullsized":"2023-11-28T21:56:48.667245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'627","last_scrub_stamp":"2023-11-28T21:56:48.666890+0000","last_deep_scrub":"52'627","last_deep_scrub_stamp":"2023-11-28T21:56:48.666890+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.666890+0000","objects_scrubbed":10,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:43:39.567306+0000","scrub_duration":0.095730051999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":114718,"num_write":571,"num_write_kb":20791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":64058,"num_omap_keys":115,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"52'1331","reported_seq":3673,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.617376+0000","last_change":"2023-11-28T21:57:01.617376+0000","last_active":"2023-11-28T21:57:01.617376+0000","last_peered":"2023-11-28T21:57:01.617376+0000","last_clean":"2023-11-28T21:57:01.617376+0000","last_became_active":"2023-11-28T21:35:25.616978+0000","last_became_peered":"2023-11-28T21:35:25.616978+0000","last_unstale":"2023-11-28T21:57:01.617376+0000","last_undegraded":"2023-11-28T21:57:01.617376+0000","last_fullsized":"2023-11-28T21:57:01.617376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1331","last_scrub_stamp":"2023-11-28T21:57:01.617097+0000","last_deep_scrub":"52'1331","last_deep_scrub_stamp":"2023-11-28T21:57:01.617097+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.617097+0000","objects_scrubbed":332,"log_size":1331,"log_dups_size":0,"ondisk_log_size":1331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:29:25.432186+0000","scrub_duration":0.26810302699999999,"objects_trimmed":0,"snaptrim_duration":0.23129675,"stat_sum":{"num_bytes":32707084,"num_objects":332,"num_object_clones":0,"num_object_copies":1328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":332,"num_whiteouts":0,"num_read":229,"num_read_kb":4127,"num_write":1308,"num_write_kb":57872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,5],"acting":[4,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"52'305","reported_seq":2397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:00.459939+0000","last_change":"2023-11-28T21:57:00.459939+0000","last_active":"2023-11-28T21:57:00.459939+0000","last_peered":"2023-11-28T21:57:00.459939+0000","last_clean":"2023-11-28T21:57:00.459939+0000","last_became_active":"2023-11-28T21:34:37.312099+0000","last_became_peered":"2023-11-28T21:34:37.312099+0000","last_unstale":"2023-11-28T21:57:00.459939+0000","last_undegraded":"2023-11-28T21:57:00.459939+0000","last_fullsized":"2023-11-28T21:57:00.459939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'305","last_scrub_stamp":"2023-11-28T21:57:00.459826+0000","last_deep_scrub":"52'305","last_deep_scrub_stamp":"2023-11-28T21:57:00.459826+0000","last_clean_scrub_stamp":"2023-11-28T21:57:00.459826+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:29:07.320593+0000","scrub_duration":0.0022599199999999999,"objects_trimmed":0,"snaptrim_duration":0.28777871399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"52'962","reported_seq":1049,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.736149+0000","last_change":"2023-11-28T21:57:01.736178+0000","last_active":"2023-11-28T21:57:01.736149+0000","last_peered":"2023-11-28T21:57:01.736149+0000","last_clean":"2023-11-28T21:57:01.736149+0000","last_became_active":"2023-11-28T21:34:36.000416+0000","last_became_peered":"2023-11-28T21:34:36.000416+0000","last_unstale":"2023-11-28T21:57:01.736149+0000","last_undegraded":"2023-11-28T21:57:01.736149+0000","last_fullsized":"2023-11-28T21:57:01.736149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'962","last_scrub_stamp":"2023-11-28T21:57:01.736083+0000","last_deep_scrub":"52'962","last_deep_scrub_stamp":"2023-11-28T21:57:01.736083+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.736083+0000","objects_scrubbed":7,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:06:38.504425+0000","scrub_duration":0.040000139999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":75782,"num_write":935,"num_write_kb":13441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31916,"num_omap_keys":58,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"52'1416","reported_seq":3764,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:59.657597+0000","last_change":"2023-11-28T21:56:59.657597+0000","last_active":"2023-11-28T21:56:59.657597+0000","last_peered":"2023-11-28T21:56:59.657597+0000","last_clean":"2023-11-28T21:56:59.657597+0000","last_became_active":"2023-11-28T21:35:25.790674+0000","last_became_peered":"2023-11-28T21:35:25.790674+0000","last_unstale":"2023-11-28T21:56:59.657597+0000","last_undegraded":"2023-11-28T21:56:59.657597+0000","last_fullsized":"2023-11-28T21:56:59.657597+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1416","last_scrub_stamp":"2023-11-28T21:56:59.657277+0000","last_deep_scrub":"52'1416","last_deep_scrub_stamp":"2023-11-28T21:56:59.657277+0000","last_clean_scrub_stamp":"2023-11-28T21:56:59.657277+0000","objects_scrubbed":354,"log_size":1416,"log_dups_size":0,"ondisk_log_size":1416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:11:47.719457+0000","scrub_duration":0.27211498200000001,"objects_trimmed":0,"snaptrim_duration":0.229623311,"stat_sum":{"num_bytes":17731185,"num_objects":354,"num_object_clones":0,"num_object_copies":1416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":234,"num_read_kb":20007,"num_write":1382,"num_write_kb":71863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"52'306","reported_seq":2398,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.421624+0000","last_change":"2023-11-28T21:56:43.421641+0000","last_active":"2023-11-28T21:56:43.421624+0000","last_peered":"2023-11-28T21:56:43.421624+0000","last_clean":"2023-11-28T21:56:43.421624+0000","last_became_active":"2023-11-28T21:34:37.326239+0000","last_became_peered":"2023-11-28T21:34:37.326239+0000","last_unstale":"2023-11-28T21:56:43.421624+0000","last_undegraded":"2023-11-28T21:56:43.421624+0000","last_fullsized":"2023-11-28T21:56:43.421624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'306","last_scrub_stamp":"2023-11-28T21:56:43.421546+0000","last_deep_scrub":"52'306","last_deep_scrub_stamp":"2023-11-28T21:56:43.421546+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.421546+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:18:49.309763+0000","scrub_duration":0.001616658,"objects_trimmed":0,"snaptrim_duration":0.29027087200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"52'741","reported_seq":818,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:58.924936+0000","last_change":"2023-11-28T21:56:42.774090+0000","last_active":"2023-11-28T21:56:58.924936+0000","last_peered":"2023-11-28T21:56:58.924936+0000","last_clean":"2023-11-28T21:56:58.924936+0000","last_became_active":"2023-11-28T21:34:36.013043+0000","last_became_peered":"2023-11-28T21:34:36.013043+0000","last_unstale":"2023-11-28T21:56:58.924936+0000","last_undegraded":"2023-11-28T21:56:58.924936+0000","last_fullsized":"2023-11-28T21:56:58.924936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'734","last_scrub_stamp":"2023-11-28T21:56:42.773996+0000","last_deep_scrub":"52'734","last_deep_scrub_stamp":"2023-11-28T21:56:42.773996+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.773996+0000","objects_scrubbed":6,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:44:34.363577+0000","scrub_duration":0.055636247999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11383300,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":32782,"num_write":691,"num_write_kb":11471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8864,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"52'1341","reported_seq":3707,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:23.899760+0000","last_change":"2023-11-28T21:57:23.899805+0000","last_active":"2023-11-28T21:57:23.899760+0000","last_peered":"2023-11-28T21:57:23.899760+0000","last_clean":"2023-11-28T21:57:23.899760+0000","last_became_active":"2023-11-28T21:34:41.483157+0000","last_became_peered":"2023-11-28T21:34:41.483157+0000","last_unstale":"2023-11-28T21:57:23.899760+0000","last_undegraded":"2023-11-28T21:57:23.899760+0000","last_fullsized":"2023-11-28T21:57:23.899760+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1341","last_scrub_stamp":"2023-11-28T21:57:23.899656+0000","last_deep_scrub":"52'1341","last_deep_scrub_stamp":"2023-11-28T21:57:23.899656+0000","last_clean_scrub_stamp":"2023-11-28T21:57:23.899656+0000","objects_scrubbed":336,"log_size":1341,"log_dups_size":0,"ondisk_log_size":1341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:35.370900+0000","scrub_duration":0.18968862,"objects_trimmed":0,"snaptrim_duration":0.24643899999999999,"stat_sum":{"num_bytes":14380393,"num_objects":336,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":262,"num_read_kb":16758,"num_write":1318,"num_write_kb":45879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"52'294","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.274947+0000","last_change":"2023-11-28T21:56:48.275049+0000","last_active":"2023-11-28T21:56:48.274947+0000","last_peered":"2023-11-28T21:56:48.274947+0000","last_clean":"2023-11-28T21:56:48.274947+0000","last_became_active":"2023-11-28T21:34:37.336157+0000","last_became_peered":"2023-11-28T21:34:37.336157+0000","last_unstale":"2023-11-28T21:56:48.274947+0000","last_undegraded":"2023-11-28T21:56:48.274947+0000","last_fullsized":"2023-11-28T21:56:48.274947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'294","last_scrub_stamp":"2023-11-28T21:56:48.274731+0000","last_deep_scrub":"52'294","last_deep_scrub_stamp":"2023-11-28T21:56:48.274731+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.274731+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:02:56.783147+0000","scrub_duration":0.0035010150000000001,"objects_trimmed":0,"snaptrim_duration":0.21610210399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"52'343","reported_seq":422,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.528829+0000","last_change":"2023-11-28T21:56:57.528897+0000","last_active":"2023-11-28T21:56:57.528829+0000","last_peered":"2023-11-28T21:56:57.528829+0000","last_clean":"2023-11-28T21:56:57.528829+0000","last_became_active":"2023-11-28T21:34:36.015499+0000","last_became_peered":"2023-11-28T21:34:36.015499+0000","last_unstale":"2023-11-28T21:56:57.528829+0000","last_undegraded":"2023-11-28T21:56:57.528829+0000","last_fullsized":"2023-11-28T21:56:57.528829+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'343","last_scrub_stamp":"2023-11-28T21:56:57.528632+0000","last_deep_scrub":"52'343","last_deep_scrub_stamp":"2023-11-28T21:56:57.528632+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.528632+0000","objects_scrubbed":1,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:11:06.480768+0000","scrub_duration":0.0086859099999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":49164,"num_write":266,"num_write_kb":4296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"52'1262","reported_seq":3571,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.616628+0000","last_change":"2023-11-28T21:57:16.616828+0000","last_active":"2023-11-28T21:57:16.616628+0000","last_peered":"2023-11-28T21:57:16.616628+0000","last_clean":"2023-11-28T21:57:16.616628+0000","last_became_active":"2023-11-28T21:34:41.381340+0000","last_became_peered":"2023-11-28T21:34:41.381340+0000","last_unstale":"2023-11-28T21:57:16.616628+0000","last_undegraded":"2023-11-28T21:57:16.616628+0000","last_fullsized":"2023-11-28T21:57:16.616628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1262","last_scrub_stamp":"2023-11-28T21:57:16.616299+0000","last_deep_scrub":"52'1262","last_deep_scrub_stamp":"2023-11-28T21:57:16.616299+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.616299+0000","objects_scrubbed":318,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:23:10.854185+0000","scrub_duration":0.284423806,"objects_trimmed":0,"snaptrim_duration":0.244586779,"stat_sum":{"num_bytes":50267474,"num_objects":318,"num_object_clones":0,"num_object_copies":1272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":318,"num_whiteouts":0,"num_read":206,"num_read_kb":15432,"num_write":1240,"num_write_kb":82194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"52'302","reported_seq":2394,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:05.285530+0000","last_change":"2023-11-28T21:57:05.285554+0000","last_active":"2023-11-28T21:57:05.285530+0000","last_peered":"2023-11-28T21:57:05.285530+0000","last_clean":"2023-11-28T21:57:05.285530+0000","last_became_active":"2023-11-28T21:34:37.320144+0000","last_became_peered":"2023-11-28T21:34:37.320144+0000","last_unstale":"2023-11-28T21:57:05.285530+0000","last_undegraded":"2023-11-28T21:57:05.285530+0000","last_fullsized":"2023-11-28T21:57:05.285530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'302","last_scrub_stamp":"2023-11-28T21:57:05.285424+0000","last_deep_scrub":"52'302","last_deep_scrub_stamp":"2023-11-28T21:57:05.285424+0000","last_clean_scrub_stamp":"2023-11-28T21:57:05.285424+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:27.928951+0000","scrub_duration":0.0020171199999999999,"objects_trimmed":0,"snaptrim_duration":0.26728515899999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"52'172","reported_seq":253,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.150811+0000","last_change":"2023-11-28T21:56:49.150879+0000","last_active":"2023-11-28T21:56:49.150811+0000","last_peered":"2023-11-28T21:56:49.150811+0000","last_clean":"2023-11-28T21:56:49.150811+0000","last_became_active":"2023-11-28T21:34:36.029817+0000","last_became_peered":"2023-11-28T21:34:36.029817+0000","last_unstale":"2023-11-28T21:56:49.150811+0000","last_undegraded":"2023-11-28T21:56:49.150811+0000","last_fullsized":"2023-11-28T21:56:49.150811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'172","last_scrub_stamp":"2023-11-28T21:56:49.150620+0000","last_deep_scrub":"52'172","last_deep_scrub_stamp":"2023-11-28T21:56:49.150620+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.150620+0000","objects_scrubbed":4,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:13:50.517958+0000","scrub_duration":0.035921929999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":45,"num_read_kb":24615,"num_write":148,"num_write_kb":4216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6802,"num_omap_keys":13,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"52'1368","reported_seq":3722,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.407745+0000","last_change":"2023-11-28T21:57:11.407745+0000","last_active":"2023-11-28T21:57:11.407745+0000","last_peered":"2023-11-28T21:57:11.407745+0000","last_clean":"2023-11-28T21:57:11.407745+0000","last_became_active":"2023-11-28T21:34:41.484083+0000","last_became_peered":"2023-11-28T21:34:41.484083+0000","last_unstale":"2023-11-28T21:57:11.407745+0000","last_undegraded":"2023-11-28T21:57:11.407745+0000","last_fullsized":"2023-11-28T21:57:11.407745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1368","last_scrub_stamp":"2023-11-28T21:57:11.407483+0000","last_deep_scrub":"52'1368","last_deep_scrub_stamp":"2023-11-28T21:57:11.407483+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.407483+0000","objects_scrubbed":316,"log_size":1368,"log_dups_size":0,"ondisk_log_size":1368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:58:46.833003+0000","scrub_duration":0.207722514,"objects_trimmed":0,"snaptrim_duration":0.21672229300000001,"stat_sum":{"num_bytes":16299977,"num_objects":316,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":250,"num_read_kb":17253,"num_write":1343,"num_write_kb":69509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"52'268","reported_seq":2360,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.444623+0000","last_change":"2023-11-28T21:56:50.444642+0000","last_active":"2023-11-28T21:56:50.444623+0000","last_peered":"2023-11-28T21:56:50.444623+0000","last_clean":"2023-11-28T21:56:50.444623+0000","last_became_active":"2023-11-28T21:34:37.344790+0000","last_became_peered":"2023-11-28T21:34:37.344790+0000","last_unstale":"2023-11-28T21:56:50.444623+0000","last_undegraded":"2023-11-28T21:56:50.444623+0000","last_fullsized":"2023-11-28T21:56:50.444623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'268","last_scrub_stamp":"2023-11-28T21:56:50.444546+0000","last_deep_scrub":"52'268","last_deep_scrub_stamp":"2023-11-28T21:56:50.444546+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.444546+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:58:47.104569+0000","scrub_duration":0.0027754860000000002,"objects_trimmed":0,"snaptrim_duration":0.29009278799999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"52'248","reported_seq":326,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:47.891843+0000","last_change":"2023-11-28T21:56:47.891843+0000","last_active":"2023-11-28T21:56:47.891843+0000","last_peered":"2023-11-28T21:56:47.891843+0000","last_clean":"2023-11-28T21:56:47.891843+0000","last_became_active":"2023-11-28T21:34:36.028297+0000","last_became_peered":"2023-11-28T21:34:36.028297+0000","last_unstale":"2023-11-28T21:56:47.891843+0000","last_undegraded":"2023-11-28T21:56:47.891843+0000","last_fullsized":"2023-11-28T21:56:47.891843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'247","last_scrub_stamp":"2023-11-28T21:56:47.891725+0000","last_deep_scrub":"52'247","last_deep_scrub_stamp":"2023-11-28T21:56:47.891725+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.891725+0000","objects_scrubbed":3,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:49:45.140707+0000","scrub_duration":0.119752109,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":607,"num_write":448,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8486,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"52'1386","reported_seq":3722,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.550973+0000","last_change":"2023-11-28T21:57:11.551001+0000","last_active":"2023-11-28T21:57:11.550973+0000","last_peered":"2023-11-28T21:57:11.550973+0000","last_clean":"2023-11-28T21:57:11.550973+0000","last_became_active":"2023-11-28T21:34:41.474971+0000","last_became_peered":"2023-11-28T21:34:41.474971+0000","last_unstale":"2023-11-28T21:57:11.550973+0000","last_undegraded":"2023-11-28T21:57:11.550973+0000","last_fullsized":"2023-11-28T21:57:11.550973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1386","last_scrub_stamp":"2023-11-28T21:57:11.550878+0000","last_deep_scrub":"52'1386","last_deep_scrub_stamp":"2023-11-28T21:57:11.550878+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.550878+0000","objects_scrubbed":313,"log_size":1386,"log_dups_size":0,"ondisk_log_size":1386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:03:55.485268+0000","scrub_duration":0.14493951099999999,"objects_trimmed":0,"snaptrim_duration":0.287791554,"stat_sum":{"num_bytes":19942162,"num_objects":313,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":232,"num_read_kb":16373,"num_write":1352,"num_write_kb":52080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"52'300","reported_seq":2394,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:12.806224+0000","last_change":"2023-11-28T21:57:12.806224+0000","last_active":"2023-11-28T21:57:12.806224+0000","last_peered":"2023-11-28T21:57:12.806224+0000","last_clean":"2023-11-28T21:57:12.806224+0000","last_became_active":"2023-11-28T21:34:37.329396+0000","last_became_peered":"2023-11-28T21:34:37.329396+0000","last_unstale":"2023-11-28T21:57:12.806224+0000","last_undegraded":"2023-11-28T21:57:12.806224+0000","last_fullsized":"2023-11-28T21:57:12.806224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'300","last_scrub_stamp":"2023-11-28T21:57:12.806104+0000","last_deep_scrub":"52'300","last_deep_scrub_stamp":"2023-11-28T21:57:12.806104+0000","last_clean_scrub_stamp":"2023-11-28T21:57:12.806104+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:07:35.352399+0000","scrub_duration":0.0019360530000000001,"objects_trimmed":0,"snaptrim_duration":0.24779062600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"52'593","reported_seq":681,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.579266+0000","last_change":"2023-11-28T21:56:55.579266+0000","last_active":"2023-11-28T21:56:55.579266+0000","last_peered":"2023-11-28T21:56:55.579266+0000","last_clean":"2023-11-28T21:56:55.579266+0000","last_became_active":"2023-11-28T21:34:36.012710+0000","last_became_peered":"2023-11-28T21:34:36.012710+0000","last_unstale":"2023-11-28T21:56:55.579266+0000","last_undegraded":"2023-11-28T21:56:55.579266+0000","last_fullsized":"2023-11-28T21:56:55.579266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'593","last_scrub_stamp":"2023-11-28T21:56:55.579027+0000","last_deep_scrub":"52'593","last_deep_scrub_stamp":"2023-11-28T21:56:55.579027+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.579027+0000","objects_scrubbed":6,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:56:30.723466+0000","scrub_duration":0.077952980000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":81538,"num_write":523,"num_write_kb":16632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17674,"num_omap_keys":32,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"52'1410","reported_seq":3764,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:34.517241+0000","last_change":"2023-11-28T21:57:34.517241+0000","last_active":"2023-11-28T21:57:34.517241+0000","last_peered":"2023-11-28T21:57:34.517241+0000","last_clean":"2023-11-28T21:57:34.517241+0000","last_became_active":"2023-11-28T21:34:41.474642+0000","last_became_peered":"2023-11-28T21:34:41.474642+0000","last_unstale":"2023-11-28T21:57:34.517241+0000","last_undegraded":"2023-11-28T21:57:34.517241+0000","last_fullsized":"2023-11-28T21:57:34.517241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1410","last_scrub_stamp":"2023-11-28T21:57:34.517108+0000","last_deep_scrub":"52'1410","last_deep_scrub_stamp":"2023-11-28T21:57:34.517108+0000","last_clean_scrub_stamp":"2023-11-28T21:57:34.517108+0000","objects_scrubbed":365,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:11:10.145700+0000","scrub_duration":0.20779793599999999,"objects_trimmed":0,"snaptrim_duration":0.27155133199999998,"stat_sum":{"num_bytes":49110889,"num_objects":365,"num_object_clones":0,"num_object_copies":1460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":248,"num_read_kb":24602,"num_write":1394,"num_write_kb":89036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"52'284","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:00.790304+0000","last_change":"2023-11-28T21:57:00.790304+0000","last_active":"2023-11-28T21:57:00.790304+0000","last_peered":"2023-11-28T21:57:00.790304+0000","last_clean":"2023-11-28T21:57:00.790304+0000","last_became_active":"2023-11-28T21:34:37.337215+0000","last_became_peered":"2023-11-28T21:34:37.337215+0000","last_unstale":"2023-11-28T21:57:00.790304+0000","last_undegraded":"2023-11-28T21:57:00.790304+0000","last_fullsized":"2023-11-28T21:57:00.790304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'284","last_scrub_stamp":"2023-11-28T21:57:00.790051+0000","last_deep_scrub":"52'284","last_deep_scrub_stamp":"2023-11-28T21:57:00.790051+0000","last_clean_scrub_stamp":"2023-11-28T21:57:00.790051+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:01:01.704329+0000","scrub_duration":0.0035905590000000001,"objects_trimmed":0,"snaptrim_duration":0.30430776599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"52'167","reported_seq":238,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.743275+0000","last_change":"2023-11-28T21:56:55.743275+0000","last_active":"2023-11-28T21:56:55.743275+0000","last_peered":"2023-11-28T21:56:55.743275+0000","last_clean":"2023-11-28T21:56:55.743275+0000","last_became_active":"2023-11-28T21:34:35.997628+0000","last_became_peered":"2023-11-28T21:34:35.997628+0000","last_unstale":"2023-11-28T21:56:55.743275+0000","last_undegraded":"2023-11-28T21:56:55.743275+0000","last_fullsized":"2023-11-28T21:56:55.743275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'167","last_scrub_stamp":"2023-11-28T21:56:55.743149+0000","last_deep_scrub":"52'167","last_deep_scrub_stamp":"2023-11-28T21:56:55.743149+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.743149+0000","objects_scrubbed":3,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:41:24.042531+0000","scrub_duration":0.015377216000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24576,"num_write":146,"num_write_kb":4262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":19890,"num_omap_keys":36,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"52'1318","reported_seq":3644,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:14.990245+0000","last_change":"2023-11-28T21:57:14.990245+0000","last_active":"2023-11-28T21:57:14.990245+0000","last_peered":"2023-11-28T21:57:14.990245+0000","last_clean":"2023-11-28T21:57:14.990245+0000","last_became_active":"2023-11-28T21:34:41.369935+0000","last_became_peered":"2023-11-28T21:34:41.369935+0000","last_unstale":"2023-11-28T21:57:14.990245+0000","last_undegraded":"2023-11-28T21:57:14.990245+0000","last_fullsized":"2023-11-28T21:57:14.990245+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1318","last_scrub_stamp":"2023-11-28T21:57:14.990114+0000","last_deep_scrub":"52'1318","last_deep_scrub_stamp":"2023-11-28T21:57:14.990114+0000","last_clean_scrub_stamp":"2023-11-28T21:57:14.990114+0000","objects_scrubbed":314,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:16:53.276374+0000","scrub_duration":0.197652458,"objects_trimmed":0,"snaptrim_duration":0.25659980999999998,"stat_sum":{"num_bytes":32626940,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":222,"num_read_kb":4128,"num_write":1285,"num_write_kb":60020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"52'300","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.336260+0000","last_change":"2023-11-28T21:56:44.336260+0000","last_active":"2023-11-28T21:56:44.336260+0000","last_peered":"2023-11-28T21:56:44.336260+0000","last_clean":"2023-11-28T21:56:44.336260+0000","last_became_active":"2023-11-28T21:34:37.323641+0000","last_became_peered":"2023-11-28T21:34:37.323641+0000","last_unstale":"2023-11-28T21:56:44.336260+0000","last_undegraded":"2023-11-28T21:56:44.336260+0000","last_fullsized":"2023-11-28T21:56:44.336260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'300","last_scrub_stamp":"2023-11-28T21:56:44.335964+0000","last_deep_scrub":"52'300","last_deep_scrub_stamp":"2023-11-28T21:56:44.335964+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.335964+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:50:07.131774+0000","scrub_duration":0.0032352040000000002,"objects_trimmed":0,"snaptrim_duration":0.23142385100000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"52'370","reported_seq":454,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:51.659286+0000","last_change":"2023-11-28T21:56:51.659286+0000","last_active":"2023-11-28T21:56:51.659286+0000","last_peered":"2023-11-28T21:56:51.659286+0000","last_clean":"2023-11-28T21:56:51.659286+0000","last_became_active":"2023-11-28T21:34:35.998482+0000","last_became_peered":"2023-11-28T21:34:35.998482+0000","last_unstale":"2023-11-28T21:56:51.659286+0000","last_undegraded":"2023-11-28T21:56:51.659286+0000","last_fullsized":"2023-11-28T21:56:51.659286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'370","last_scrub_stamp":"2023-11-28T21:56:51.659033+0000","last_deep_scrub":"52'370","last_deep_scrub_stamp":"2023-11-28T21:56:51.659033+0000","last_clean_scrub_stamp":"2023-11-28T21:56:51.659033+0000","objects_scrubbed":3,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:45:32.821715+0000","scrub_duration":0.043796694999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":81920,"num_write":339,"num_write_kb":16655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"52'1263","reported_seq":3576,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.935360+0000","last_change":"2023-11-28T21:57:13.935360+0000","last_active":"2023-11-28T21:57:13.935360+0000","last_peered":"2023-11-28T21:57:13.935360+0000","last_clean":"2023-11-28T21:57:13.935360+0000","last_became_active":"2023-11-28T21:34:41.379525+0000","last_became_peered":"2023-11-28T21:34:41.379525+0000","last_unstale":"2023-11-28T21:57:13.935360+0000","last_undegraded":"2023-11-28T21:57:13.935360+0000","last_fullsized":"2023-11-28T21:57:13.935360+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1263","last_scrub_stamp":"2023-11-28T21:57:13.935220+0000","last_deep_scrub":"52'1263","last_deep_scrub_stamp":"2023-11-28T21:57:13.935220+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.935220+0000","objects_scrubbed":288,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:20.055177+0000","scrub_duration":0.14496025300000001,"objects_trimmed":0,"snaptrim_duration":0.24685721299999999,"stat_sum":{"num_bytes":16718463,"num_objects":288,"num_object_clones":0,"num_object_copies":1152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":210,"num_read_kb":3302,"num_write":1235,"num_write_kb":35539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"52'299","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:07.792410+0000","last_change":"2023-11-28T21:57:07.792430+0000","last_active":"2023-11-28T21:57:07.792410+0000","last_peered":"2023-11-28T21:57:07.792410+0000","last_clean":"2023-11-28T21:57:07.792410+0000","last_became_active":"2023-11-28T21:34:37.330546+0000","last_became_peered":"2023-11-28T21:34:37.330546+0000","last_unstale":"2023-11-28T21:57:07.792410+0000","last_undegraded":"2023-11-28T21:57:07.792410+0000","last_fullsized":"2023-11-28T21:57:07.792410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'299","last_scrub_stamp":"2023-11-28T21:57:07.792311+0000","last_deep_scrub":"52'299","last_deep_scrub_stamp":"2023-11-28T21:57:07.792311+0000","last_clean_scrub_stamp":"2023-11-28T21:57:07.792311+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:44.781100+0000","scrub_duration":0.0021660130000000001,"objects_trimmed":0,"snaptrim_duration":0.24023003500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"52'943","reported_seq":1424,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.244727+0000","last_change":"2023-11-28T21:56:45.244727+0000","last_active":"2023-11-28T21:56:45.244727+0000","last_peered":"2023-11-28T21:56:45.244727+0000","last_clean":"2023-11-28T21:56:45.244727+0000","last_became_active":"2023-11-28T21:34:35.989589+0000","last_became_peered":"2023-11-28T21:34:35.989589+0000","last_unstale":"2023-11-28T21:56:45.244727+0000","last_undegraded":"2023-11-28T21:56:45.244727+0000","last_fullsized":"2023-11-28T21:56:45.244727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'943","last_scrub_stamp":"2023-11-28T21:56:45.244580+0000","last_deep_scrub":"52'943","last_deep_scrub_stamp":"2023-11-28T21:56:45.244580+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.244580+0000","objects_scrubbed":9,"log_size":943,"log_dups_size":0,"ondisk_log_size":943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:47:30.047518+0000","scrub_duration":0.055949021000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18884916,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":417,"num_read_kb":34819,"num_write":869,"num_write_kb":18975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":17074,"num_omap_keys":32,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"52'1231","reported_seq":3577,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:19.911197+0000","last_change":"2023-11-28T21:57:19.911246+0000","last_active":"2023-11-28T21:57:19.911197+0000","last_peered":"2023-11-28T21:57:19.911197+0000","last_clean":"2023-11-28T21:57:19.911197+0000","last_became_active":"2023-11-28T21:34:41.371503+0000","last_became_peered":"2023-11-28T21:34:41.371503+0000","last_unstale":"2023-11-28T21:57:19.911197+0000","last_undegraded":"2023-11-28T21:57:19.911197+0000","last_fullsized":"2023-11-28T21:57:19.911197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1231","last_scrub_stamp":"2023-11-28T21:57:19.911063+0000","last_deep_scrub":"52'1231","last_deep_scrub_stamp":"2023-11-28T21:57:19.911063+0000","last_clean_scrub_stamp":"2023-11-28T21:57:19.911063+0000","objects_scrubbed":293,"log_size":1231,"log_dups_size":0,"ondisk_log_size":1231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:45:27.145851+0000","scrub_duration":0.164741481,"objects_trimmed":0,"snaptrim_duration":0.24393632200000001,"stat_sum":{"num_bytes":26023878,"num_objects":293,"num_object_clones":0,"num_object_copies":1172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":293,"num_whiteouts":0,"num_read":244,"num_read_kb":12314,"num_write":1202,"num_write_kb":62723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"52'277","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.370466+0000","last_change":"2023-11-28T21:57:01.370466+0000","last_active":"2023-11-28T21:57:01.370466+0000","last_peered":"2023-11-28T21:57:01.370466+0000","last_clean":"2023-11-28T21:57:01.370466+0000","last_became_active":"2023-11-28T21:34:37.319444+0000","last_became_peered":"2023-11-28T21:34:37.319444+0000","last_unstale":"2023-11-28T21:57:01.370466+0000","last_undegraded":"2023-11-28T21:57:01.370466+0000","last_fullsized":"2023-11-28T21:57:01.370466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'277","last_scrub_stamp":"2023-11-28T21:57:01.370337+0000","last_deep_scrub":"52'277","last_deep_scrub_stamp":"2023-11-28T21:57:01.370337+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.370337+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:30:28.221635+0000","scrub_duration":0.002672757,"objects_trimmed":0,"snaptrim_duration":0.25105937499999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"52'331","reported_seq":423,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:12.832467+0000","last_change":"2023-11-28T21:56:59.491989+0000","last_active":"2023-11-28T21:57:12.832467+0000","last_peered":"2023-11-28T21:57:12.832467+0000","last_clean":"2023-11-28T21:57:12.832467+0000","last_became_active":"2023-11-28T21:34:36.016813+0000","last_became_peered":"2023-11-28T21:34:36.016813+0000","last_unstale":"2023-11-28T21:57:12.832467+0000","last_undegraded":"2023-11-28T21:57:12.832467+0000","last_fullsized":"2023-11-28T21:57:12.832467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'325","last_scrub_stamp":"2023-11-28T21:56:59.491653+0000","last_deep_scrub":"52'325","last_deep_scrub_stamp":"2023-11-28T21:56:59.491653+0000","last_clean_scrub_stamp":"2023-11-28T21:56:59.491653+0000","objects_scrubbed":5,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:19:40.745953+0000","scrub_duration":0.050149041999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8818331,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":114688,"num_write":289,"num_write_kb":16977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5654,"num_omap_keys":11,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"52'1365","reported_seq":3732,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:21.294986+0000","last_change":"2023-11-28T21:57:21.295019+0000","last_active":"2023-11-28T21:57:21.294986+0000","last_peered":"2023-11-28T21:57:21.294986+0000","last_clean":"2023-11-28T21:57:21.294986+0000","last_became_active":"2023-11-28T21:34:41.474605+0000","last_became_peered":"2023-11-28T21:34:41.474605+0000","last_unstale":"2023-11-28T21:57:21.294986+0000","last_undegraded":"2023-11-28T21:57:21.294986+0000","last_fullsized":"2023-11-28T21:57:21.294986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1365","last_scrub_stamp":"2023-11-28T21:57:21.294885+0000","last_deep_scrub":"52'1365","last_deep_scrub_stamp":"2023-11-28T21:57:21.294885+0000","last_clean_scrub_stamp":"2023-11-28T21:57:21.294885+0000","objects_scrubbed":302,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:23:35.502385+0000","scrub_duration":0.15644470799999999,"objects_trimmed":0,"snaptrim_duration":0.270607394,"stat_sum":{"num_bytes":19916106,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":264,"num_read_kb":13370,"num_write":1336,"num_write_kb":59202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"52'283","reported_seq":2376,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.428177+0000","last_change":"2023-11-28T21:56:57.428177+0000","last_active":"2023-11-28T21:56:57.428177+0000","last_peered":"2023-11-28T21:56:57.428177+0000","last_clean":"2023-11-28T21:56:57.428177+0000","last_became_active":"2023-11-28T21:34:37.358638+0000","last_became_peered":"2023-11-28T21:34:37.358638+0000","last_unstale":"2023-11-28T21:56:57.428177+0000","last_undegraded":"2023-11-28T21:56:57.428177+0000","last_fullsized":"2023-11-28T21:56:57.428177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:56:57.428075+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:56:57.428075+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.428075+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:09:37.862848+0000","scrub_duration":0.0026023819999999999,"objects_trimmed":0,"snaptrim_duration":0.29152715000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"52'318","reported_seq":392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:57.764096+0000","last_change":"2023-11-28T21:56:57.764096+0000","last_active":"2023-11-28T21:56:57.764096+0000","last_peered":"2023-11-28T21:56:57.764096+0000","last_clean":"2023-11-28T21:56:57.764096+0000","last_became_active":"2023-11-28T21:34:35.995360+0000","last_became_peered":"2023-11-28T21:34:35.995360+0000","last_unstale":"2023-11-28T21:56:57.764096+0000","last_undegraded":"2023-11-28T21:56:57.764096+0000","last_fullsized":"2023-11-28T21:56:57.764096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'318","last_scrub_stamp":"2023-11-28T21:56:57.763819+0000","last_deep_scrub":"52'318","last_deep_scrub_stamp":"2023-11-28T21:56:57.763819+0000","last_clean_scrub_stamp":"2023-11-28T21:56:57.763819+0000","objects_scrubbed":7,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:20:01.143626+0000","scrub_duration":0.047864816999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":16,"num_read_kb":24594,"num_write":297,"num_write_kb":8480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":38186,"num_omap_keys":70,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"52'119","reported_seq":219,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.591456+0000","last_change":"2023-11-28T21:56:45.591517+0000","last_active":"2023-11-28T21:56:45.591456+0000","last_peered":"2023-11-28T21:56:45.591456+0000","last_clean":"2023-11-28T21:56:45.591456+0000","last_became_active":"2023-11-28T21:34:31.334116+0000","last_became_peered":"2023-11-28T21:34:31.334116+0000","last_unstale":"2023-11-28T21:56:45.591456+0000","last_undegraded":"2023-11-28T21:56:45.591456+0000","last_fullsized":"2023-11-28T21:56:45.591456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'119","last_scrub_stamp":"2023-11-28T21:56:45.591265+0000","last_deep_scrub":"52'119","last_deep_scrub_stamp":"2023-11-28T21:56:45.591265+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.591265+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:52:45.646774+0000","scrub_duration":0.0095868710000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"52'1342","reported_seq":3699,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:34.344047+0000","last_change":"2023-11-28T21:57:34.344047+0000","last_active":"2023-11-28T21:57:34.344047+0000","last_peered":"2023-11-28T21:57:34.344047+0000","last_clean":"2023-11-28T21:57:34.344047+0000","last_became_active":"2023-11-28T21:34:41.476238+0000","last_became_peered":"2023-11-28T21:34:41.476238+0000","last_unstale":"2023-11-28T21:57:34.344047+0000","last_undegraded":"2023-11-28T21:57:34.344047+0000","last_fullsized":"2023-11-28T21:57:34.344047+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1342","last_scrub_stamp":"2023-11-28T21:57:34.343779+0000","last_deep_scrub":"52'1342","last_deep_scrub_stamp":"2023-11-28T21:57:34.343779+0000","last_clean_scrub_stamp":"2023-11-28T21:57:34.343779+0000","objects_scrubbed":339,"log_size":1342,"log_dups_size":0,"ondisk_log_size":1342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:43:47.308742+0000","scrub_duration":0.26345470500000001,"objects_trimmed":0,"snaptrim_duration":0.19732549099999999,"stat_sum":{"num_bytes":29762487,"num_objects":339,"num_object_clones":0,"num_object_copies":1356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":252,"num_read_kb":20508,"num_write":1310,"num_write_kb":73242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"52'280","reported_seq":2373,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.333148+0000","last_change":"2023-11-28T21:57:09.333148+0000","last_active":"2023-11-28T21:57:09.333148+0000","last_peered":"2023-11-28T21:57:09.333148+0000","last_clean":"2023-11-28T21:57:09.333148+0000","last_became_active":"2023-11-28T21:34:37.350139+0000","last_became_peered":"2023-11-28T21:34:37.350139+0000","last_unstale":"2023-11-28T21:57:09.333148+0000","last_undegraded":"2023-11-28T21:57:09.333148+0000","last_fullsized":"2023-11-28T21:57:09.333148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'280","last_scrub_stamp":"2023-11-28T21:57:09.332900+0000","last_deep_scrub":"52'280","last_deep_scrub_stamp":"2023-11-28T21:57:09.332900+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.332900+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:46:05.592588+0000","scrub_duration":0.00363196,"objects_trimmed":0,"snaptrim_duration":0.219020891,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"52'482","reported_seq":551,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.334575+0000","last_change":"2023-11-28T21:57:01.334575+0000","last_active":"2023-11-28T21:57:01.334575+0000","last_peered":"2023-11-28T21:57:01.334575+0000","last_clean":"2023-11-28T21:57:01.334575+0000","last_became_active":"2023-11-28T21:34:35.997092+0000","last_became_peered":"2023-11-28T21:34:35.997092+0000","last_unstale":"2023-11-28T21:57:01.334575+0000","last_undegraded":"2023-11-28T21:57:01.334575+0000","last_fullsized":"2023-11-28T21:57:01.334575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'482","last_scrub_stamp":"2023-11-28T21:57:01.334419+0000","last_deep_scrub":"52'482","last_deep_scrub_stamp":"2023-11-28T21:57:01.334419+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.334419+0000","objects_scrubbed":6,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:10:08.252390+0000","scrub_duration":0.032982576999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":8,"num_read_kb":20485,"num_write":447,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":27630,"num_omap_keys":50,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"52'1372","reported_seq":3726,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:38.300326+0000","last_change":"2023-11-28T21:57:38.300460+0000","last_active":"2023-11-28T21:57:38.300326+0000","last_peered":"2023-11-28T21:57:38.300326+0000","last_clean":"2023-11-28T21:57:38.300326+0000","last_became_active":"2023-11-28T21:34:41.484034+0000","last_became_peered":"2023-11-28T21:34:41.484034+0000","last_unstale":"2023-11-28T21:57:38.300326+0000","last_undegraded":"2023-11-28T21:57:38.300326+0000","last_fullsized":"2023-11-28T21:57:38.300326+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1372","last_scrub_stamp":"2023-11-28T21:57:38.300030+0000","last_deep_scrub":"52'1372","last_deep_scrub_stamp":"2023-11-28T21:57:38.300030+0000","last_clean_scrub_stamp":"2023-11-28T21:57:38.300030+0000","objects_scrubbed":335,"log_size":1372,"log_dups_size":0,"ondisk_log_size":1372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:53:38.148437+0000","scrub_duration":0.26399045500000001,"objects_trimmed":0,"snaptrim_duration":0.19929065100000001,"stat_sum":{"num_bytes":26470227,"num_objects":335,"num_object_clones":0,"num_object_copies":1340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":251,"num_read_kb":4134,"num_write":1344,"num_write_kb":47412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"52'330","reported_seq":2423,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:14.726189+0000","last_change":"2023-11-28T21:57:14.726238+0000","last_active":"2023-11-28T21:57:14.726189+0000","last_peered":"2023-11-28T21:57:14.726189+0000","last_clean":"2023-11-28T21:57:14.726189+0000","last_became_active":"2023-11-28T21:34:37.339491+0000","last_became_peered":"2023-11-28T21:34:37.339491+0000","last_unstale":"2023-11-28T21:57:14.726189+0000","last_undegraded":"2023-11-28T21:57:14.726189+0000","last_fullsized":"2023-11-28T21:57:14.726189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'330","last_scrub_stamp":"2023-11-28T21:57:14.726054+0000","last_deep_scrub":"52'330","last_deep_scrub_stamp":"2023-11-28T21:57:14.726054+0000","last_clean_scrub_stamp":"2023-11-28T21:57:14.726054+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:53:00.065479+0000","scrub_duration":0.0023321930000000002,"objects_trimmed":0,"snaptrim_duration":0.30458413400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"52'971","reported_seq":1061,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.753678+0000","last_change":"2023-11-28T21:56:49.753678+0000","last_active":"2023-11-28T21:56:49.753678+0000","last_peered":"2023-11-28T21:56:49.753678+0000","last_clean":"2023-11-28T21:56:49.753678+0000","last_became_active":"2023-11-28T21:34:36.018852+0000","last_became_peered":"2023-11-28T21:34:36.018852+0000","last_unstale":"2023-11-28T21:56:49.753678+0000","last_undegraded":"2023-11-28T21:56:49.753678+0000","last_fullsized":"2023-11-28T21:56:49.753678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'971","last_scrub_stamp":"2023-11-28T21:56:49.753562+0000","last_deep_scrub":"52'971","last_deep_scrub_stamp":"2023-11-28T21:56:49.753562+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.753562+0000","objects_scrubbed":7,"log_size":971,"log_dups_size":0,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:05.338455+0000","scrub_duration":0.053294293999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582982,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":27,"num_read_kb":68346,"num_write":931,"num_write_kb":16947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":38524,"num_omap_keys":71,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"52'1319","reported_seq":3687,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:22.364889+0000","last_change":"2023-11-28T21:57:22.364911+0000","last_active":"2023-11-28T21:57:22.364889+0000","last_peered":"2023-11-28T21:57:22.364889+0000","last_clean":"2023-11-28T21:57:22.364889+0000","last_became_active":"2023-11-28T21:34:41.370330+0000","last_became_peered":"2023-11-28T21:34:41.370330+0000","last_unstale":"2023-11-28T21:57:22.364889+0000","last_undegraded":"2023-11-28T21:57:22.364889+0000","last_fullsized":"2023-11-28T21:57:22.364889+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1319","last_scrub_stamp":"2023-11-28T21:57:22.364796+0000","last_deep_scrub":"52'1319","last_deep_scrub_stamp":"2023-11-28T21:57:22.364796+0000","last_clean_scrub_stamp":"2023-11-28T21:57:22.364796+0000","objects_scrubbed":302,"log_size":1319,"log_dups_size":0,"ondisk_log_size":1319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:58:07.675638+0000","scrub_duration":0.192509492,"objects_trimmed":0,"snaptrim_duration":0.27089875299999999,"stat_sum":{"num_bytes":28004524,"num_objects":302,"num_object_clones":0,"num_object_copies":1208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":265,"num_read_kb":16422,"num_write":1294,"num_write_kb":50614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"52'274","reported_seq":2360,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.276211+0000","last_change":"2023-11-28T21:57:16.276211+0000","last_active":"2023-11-28T21:57:16.276211+0000","last_peered":"2023-11-28T21:57:16.276211+0000","last_clean":"2023-11-28T21:57:16.276211+0000","last_became_active":"2023-11-28T21:35:25.594533+0000","last_became_peered":"2023-11-28T21:35:25.594533+0000","last_unstale":"2023-11-28T21:57:16.276211+0000","last_undegraded":"2023-11-28T21:57:16.276211+0000","last_fullsized":"2023-11-28T21:57:16.276211+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'274","last_scrub_stamp":"2023-11-28T21:57:16.276089+0000","last_deep_scrub":"52'274","last_deep_scrub_stamp":"2023-11-28T21:57:16.276089+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.276089+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:46:27.069531+0000","scrub_duration":0.0027233159999999999,"objects_trimmed":0,"snaptrim_duration":0.14880083199999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"52'307","reported_seq":392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.178599+0000","last_change":"2023-11-28T21:56:48.178599+0000","last_active":"2023-11-28T21:56:48.178599+0000","last_peered":"2023-11-28T21:56:48.178599+0000","last_clean":"2023-11-28T21:56:48.178599+0000","last_became_active":"2023-11-28T21:34:35.989426+0000","last_became_peered":"2023-11-28T21:34:35.989426+0000","last_unstale":"2023-11-28T21:56:48.178599+0000","last_undegraded":"2023-11-28T21:56:48.178599+0000","last_fullsized":"2023-11-28T21:56:48.178599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'307","last_scrub_stamp":"2023-11-28T21:56:48.178460+0000","last_deep_scrub":"52'307","last_deep_scrub_stamp":"2023-11-28T21:56:48.178460+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.178460+0000","objects_scrubbed":5,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:26:15.702492+0000","scrub_duration":0.026539895000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":53275,"num_write":252,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21228,"num_omap_keys":39,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"52'1307","reported_seq":3611,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:32.770136+0000","last_change":"2023-11-28T21:57:32.770194+0000","last_active":"2023-11-28T21:57:32.770136+0000","last_peered":"2023-11-28T21:57:32.770136+0000","last_clean":"2023-11-28T21:57:32.770136+0000","last_became_active":"2023-11-28T21:34:41.475934+0000","last_became_peered":"2023-11-28T21:34:41.475934+0000","last_unstale":"2023-11-28T21:57:32.770136+0000","last_undegraded":"2023-11-28T21:57:32.770136+0000","last_fullsized":"2023-11-28T21:57:32.770136+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1307","last_scrub_stamp":"2023-11-28T21:57:32.770017+0000","last_deep_scrub":"52'1307","last_deep_scrub_stamp":"2023-11-28T21:57:32.770017+0000","last_clean_scrub_stamp":"2023-11-28T21:57:32.770017+0000","objects_scrubbed":295,"log_size":1307,"log_dups_size":0,"ondisk_log_size":1307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:57:16.838574+0000","scrub_duration":0.22258078000000001,"objects_trimmed":0,"snaptrim_duration":0.30648515900000001,"stat_sum":{"num_bytes":28110180,"num_objects":295,"num_object_clones":0,"num_object_copies":1180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":0,"num_read":202,"num_read_kb":8227,"num_write":1283,"num_write_kb":51039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"52'277","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:24.329680+0000","last_change":"2023-11-28T21:57:24.329730+0000","last_active":"2023-11-28T21:57:24.329680+0000","last_peered":"2023-11-28T21:57:24.329680+0000","last_clean":"2023-11-28T21:57:24.329680+0000","last_became_active":"2023-11-28T21:34:37.355400+0000","last_became_peered":"2023-11-28T21:34:37.355400+0000","last_unstale":"2023-11-28T21:57:24.329680+0000","last_undegraded":"2023-11-28T21:57:24.329680+0000","last_fullsized":"2023-11-28T21:57:24.329680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'277","last_scrub_stamp":"2023-11-28T21:57:24.329487+0000","last_deep_scrub":"52'277","last_deep_scrub_stamp":"2023-11-28T21:57:24.329487+0000","last_clean_scrub_stamp":"2023-11-28T21:57:24.329487+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:27:26.819175+0000","scrub_duration":0.003383807,"objects_trimmed":0,"snaptrim_duration":0.21877398200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"52'197","reported_seq":280,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.311351+0000","last_change":"2023-11-28T21:56:43.311351+0000","last_active":"2023-11-28T21:56:43.311351+0000","last_peered":"2023-11-28T21:56:43.311351+0000","last_clean":"2023-11-28T21:56:43.311351+0000","last_became_active":"2023-11-28T21:34:36.005909+0000","last_became_peered":"2023-11-28T21:34:36.005909+0000","last_unstale":"2023-11-28T21:56:43.311351+0000","last_undegraded":"2023-11-28T21:56:43.311351+0000","last_fullsized":"2023-11-28T21:56:43.311351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'197","last_scrub_stamp":"2023-11-28T21:56:43.311087+0000","last_deep_scrub":"52'197","last_deep_scrub_stamp":"2023-11-28T21:56:43.311087+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.311087+0000","objects_scrubbed":4,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:17:01.116728+0000","scrub_duration":0.027225414999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":49161,"num_write":147,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11661,"num_omap_keys":22,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"52'1273","reported_seq":3595,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.508129+0000","last_change":"2023-11-28T21:57:13.508211+0000","last_active":"2023-11-28T21:57:13.508129+0000","last_peered":"2023-11-28T21:57:13.508129+0000","last_clean":"2023-11-28T21:57:13.508129+0000","last_became_active":"2023-11-28T21:34:41.381291+0000","last_became_peered":"2023-11-28T21:34:41.381291+0000","last_unstale":"2023-11-28T21:57:13.508129+0000","last_undegraded":"2023-11-28T21:57:13.508129+0000","last_fullsized":"2023-11-28T21:57:13.508129+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1273","last_scrub_stamp":"2023-11-28T21:57:13.507890+0000","last_deep_scrub":"52'1273","last_deep_scrub_stamp":"2023-11-28T21:57:13.507890+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.507890+0000","objects_scrubbed":308,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:10:17.277733+0000","scrub_duration":0.25249365600000001,"objects_trimmed":0,"snaptrim_duration":0.223266511,"stat_sum":{"num_bytes":11523503,"num_objects":308,"num_object_clones":0,"num_object_copies":1232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":219,"num_read_kb":12316,"num_write":1248,"num_write_kb":38807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"52'323","reported_seq":2416,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:43.543999+0000","last_change":"2023-11-28T21:56:43.543999+0000","last_active":"2023-11-28T21:56:43.543999+0000","last_peered":"2023-11-28T21:56:43.543999+0000","last_clean":"2023-11-28T21:56:43.543999+0000","last_became_active":"2023-11-28T21:34:37.343220+0000","last_became_peered":"2023-11-28T21:34:37.343220+0000","last_unstale":"2023-11-28T21:56:43.543999+0000","last_undegraded":"2023-11-28T21:56:43.543999+0000","last_fullsized":"2023-11-28T21:56:43.543999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'323","last_scrub_stamp":"2023-11-28T21:56:43.543658+0000","last_deep_scrub":"52'323","last_deep_scrub_stamp":"2023-11-28T21:56:43.543658+0000","last_clean_scrub_stamp":"2023-11-28T21:56:43.543658+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:59.532405+0000","scrub_duration":0.003404327,"objects_trimmed":0,"snaptrim_duration":0.21978382299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"52'868","reported_seq":988,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.270874+0000","last_change":"2023-11-28T21:56:50.270874+0000","last_active":"2023-11-28T21:56:50.270874+0000","last_peered":"2023-11-28T21:56:50.270874+0000","last_clean":"2023-11-28T21:56:50.270874+0000","last_became_active":"2023-11-28T21:34:35.998459+0000","last_became_peered":"2023-11-28T21:34:35.998459+0000","last_unstale":"2023-11-28T21:56:50.270874+0000","last_undegraded":"2023-11-28T21:56:50.270874+0000","last_fullsized":"2023-11-28T21:56:50.270874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'868","last_scrub_stamp":"2023-11-28T21:56:50.270733+0000","last_deep_scrub":"52'868","last_deep_scrub_stamp":"2023-11-28T21:56:50.270733+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.270733+0000","objects_scrubbed":14,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:08:25.187061+0000","scrub_duration":0.051670183000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":137,"num_read_kb":58709,"num_write":838,"num_write_kb":8642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34555,"num_omap_keys":61,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"52'1274","reported_seq":3626,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:40.510826+0000","last_change":"2023-11-28T21:57:40.510850+0000","last_active":"2023-11-28T21:57:40.510826+0000","last_peered":"2023-11-28T21:57:40.510826+0000","last_clean":"2023-11-28T21:57:40.510826+0000","last_became_active":"2023-11-28T21:34:41.384944+0000","last_became_peered":"2023-11-28T21:34:41.384944+0000","last_unstale":"2023-11-28T21:57:40.510826+0000","last_undegraded":"2023-11-28T21:57:40.510826+0000","last_fullsized":"2023-11-28T21:57:40.510826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1274","last_scrub_stamp":"2023-11-28T21:57:40.510731+0000","last_deep_scrub":"52'1274","last_deep_scrub_stamp":"2023-11-28T21:57:40.510731+0000","last_clean_scrub_stamp":"2023-11-28T21:57:40.510731+0000","objects_scrubbed":283,"log_size":1274,"log_dups_size":0,"ondisk_log_size":1274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:34:17.834945+0000","scrub_duration":0.16243864999999999,"objects_trimmed":0,"snaptrim_duration":0.27152726799999999,"stat_sum":{"num_bytes":36307339,"num_objects":283,"num_object_clones":0,"num_object_copies":1132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":283,"num_whiteouts":0,"num_read":251,"num_read_kb":8046,"num_write":1253,"num_write_kb":60298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"52'284","reported_seq":2376,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:01.132013+0000","last_change":"2023-11-28T21:57:01.132013+0000","last_active":"2023-11-28T21:57:01.132013+0000","last_peered":"2023-11-28T21:57:01.132013+0000","last_clean":"2023-11-28T21:57:01.132013+0000","last_became_active":"2023-11-28T21:34:37.332895+0000","last_became_peered":"2023-11-28T21:34:37.332895+0000","last_unstale":"2023-11-28T21:57:01.132013+0000","last_undegraded":"2023-11-28T21:57:01.132013+0000","last_fullsized":"2023-11-28T21:57:01.132013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'284","last_scrub_stamp":"2023-11-28T21:57:01.131764+0000","last_deep_scrub":"52'284","last_deep_scrub_stamp":"2023-11-28T21:57:01.131764+0000","last_clean_scrub_stamp":"2023-11-28T21:57:01.131764+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:44:58.129846+0000","scrub_duration":0.0035951529999999998,"objects_trimmed":0,"snaptrim_duration":0.18244292200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"52'243","reported_seq":307,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.132928+0000","last_change":"2023-11-28T21:56:48.132928+0000","last_active":"2023-11-28T21:56:48.132928+0000","last_peered":"2023-11-28T21:56:48.132928+0000","last_clean":"2023-11-28T21:56:48.132928+0000","last_became_active":"2023-11-28T21:34:36.025388+0000","last_became_peered":"2023-11-28T21:34:36.025388+0000","last_unstale":"2023-11-28T21:56:48.132928+0000","last_undegraded":"2023-11-28T21:56:48.132928+0000","last_fullsized":"2023-11-28T21:56:48.132928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'243","last_scrub_stamp":"2023-11-28T21:56:48.132597+0000","last_deep_scrub":"52'243","last_deep_scrub_stamp":"2023-11-28T21:56:48.132597+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.132597+0000","objects_scrubbed":4,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:59:38.258199+0000","scrub_duration":0.035274727999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195875,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":194,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12188,"num_omap_keys":22,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"52'1346","reported_seq":3683,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:20.814918+0000","last_change":"2023-11-28T21:57:20.814952+0000","last_active":"2023-11-28T21:57:20.814918+0000","last_peered":"2023-11-28T21:57:20.814918+0000","last_clean":"2023-11-28T21:57:20.814918+0000","last_became_active":"2023-11-28T21:34:41.479105+0000","last_became_peered":"2023-11-28T21:34:41.479105+0000","last_unstale":"2023-11-28T21:57:20.814918+0000","last_undegraded":"2023-11-28T21:57:20.814918+0000","last_fullsized":"2023-11-28T21:57:20.814918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1346","last_scrub_stamp":"2023-11-28T21:57:20.814818+0000","last_deep_scrub":"52'1346","last_deep_scrub_stamp":"2023-11-28T21:57:20.814818+0000","last_clean_scrub_stamp":"2023-11-28T21:57:20.814818+0000","objects_scrubbed":324,"log_size":1346,"log_dups_size":0,"ondisk_log_size":1346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:05.852021+0000","scrub_duration":0.22313440300000001,"objects_trimmed":0,"snaptrim_duration":0.29439885100000002,"stat_sum":{"num_bytes":34458204,"num_objects":324,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":324,"num_whiteouts":0,"num_read":235,"num_read_kb":27797,"num_write":1320,"num_write_kb":70332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"52'313","reported_seq":2406,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:22.236867+0000","last_change":"2023-11-28T21:57:22.236867+0000","last_active":"2023-11-28T21:57:22.236867+0000","last_peered":"2023-11-28T21:57:22.236867+0000","last_clean":"2023-11-28T21:57:22.236867+0000","last_became_active":"2023-11-28T21:34:37.355421+0000","last_became_peered":"2023-11-28T21:34:37.355421+0000","last_unstale":"2023-11-28T21:57:22.236867+0000","last_undegraded":"2023-11-28T21:57:22.236867+0000","last_fullsized":"2023-11-28T21:57:22.236867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'313","last_scrub_stamp":"2023-11-28T21:57:22.236522+0000","last_deep_scrub":"52'313","last_deep_scrub_stamp":"2023-11-28T21:57:22.236522+0000","last_clean_scrub_stamp":"2023-11-28T21:57:22.236522+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:39:11.637946+0000","scrub_duration":0.0037346620000000001,"objects_trimmed":0,"snaptrim_duration":0.218364582,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"52'160","reported_seq":228,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.715743+0000","last_change":"2023-11-28T21:56:55.715743+0000","last_active":"2023-11-28T21:56:55.715743+0000","last_peered":"2023-11-28T21:56:55.715743+0000","last_clean":"2023-11-28T21:56:55.715743+0000","last_became_active":"2023-11-28T21:34:35.993347+0000","last_became_peered":"2023-11-28T21:34:35.993347+0000","last_unstale":"2023-11-28T21:56:55.715743+0000","last_undegraded":"2023-11-28T21:56:55.715743+0000","last_fullsized":"2023-11-28T21:56:55.715743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'160","last_scrub_stamp":"2023-11-28T21:56:55.715502+0000","last_deep_scrub":"52'160","last_deep_scrub_stamp":"2023-11-28T21:56:55.715502+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.715502+0000","objects_scrubbed":5,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:08:17.898435+0000","scrub_duration":0.024710507999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":4108,"num_write":107,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23214,"num_omap_keys":42,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"52'1271","reported_seq":3593,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.743280+0000","last_change":"2023-11-28T21:57:16.743301+0000","last_active":"2023-11-28T21:57:16.743280+0000","last_peered":"2023-11-28T21:57:16.743280+0000","last_clean":"2023-11-28T21:57:16.743280+0000","last_became_active":"2023-11-28T21:34:41.477398+0000","last_became_peered":"2023-11-28T21:34:41.477398+0000","last_unstale":"2023-11-28T21:57:16.743280+0000","last_undegraded":"2023-11-28T21:57:16.743280+0000","last_fullsized":"2023-11-28T21:57:16.743280+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1271","last_scrub_stamp":"2023-11-28T21:57:16.743187+0000","last_deep_scrub":"52'1271","last_deep_scrub_stamp":"2023-11-28T21:57:16.743187+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.743187+0000","objects_scrubbed":350,"log_size":1271,"log_dups_size":0,"ondisk_log_size":1271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:16:32.276987+0000","scrub_duration":0.193506183,"objects_trimmed":0,"snaptrim_duration":0.29571303700000001,"stat_sum":{"num_bytes":33678171,"num_objects":350,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":350,"num_whiteouts":0,"num_read":218,"num_read_kb":4129,"num_write":1246,"num_write_kb":52865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"52'279","reported_seq":2371,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.192234+0000","last_change":"2023-11-28T21:57:13.192234+0000","last_active":"2023-11-28T21:57:13.192234+0000","last_peered":"2023-11-28T21:57:13.192234+0000","last_clean":"2023-11-28T21:57:13.192234+0000","last_became_active":"2023-11-28T21:34:37.345199+0000","last_became_peered":"2023-11-28T21:34:37.345199+0000","last_unstale":"2023-11-28T21:57:13.192234+0000","last_undegraded":"2023-11-28T21:57:13.192234+0000","last_fullsized":"2023-11-28T21:57:13.192234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'279","last_scrub_stamp":"2023-11-28T21:57:13.192121+0000","last_deep_scrub":"52'279","last_deep_scrub_stamp":"2023-11-28T21:57:13.192121+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.192121+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:37:10.470818+0000","scrub_duration":0.003054668,"objects_trimmed":0,"snaptrim_duration":0.21362616300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"52'313","reported_seq":416,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.422244+0000","last_change":"2023-11-28T21:56:44.422244+0000","last_active":"2023-11-28T21:56:44.422244+0000","last_peered":"2023-11-28T21:56:44.422244+0000","last_clean":"2023-11-28T21:56:44.422244+0000","last_became_active":"2023-11-28T21:34:36.018866+0000","last_became_peered":"2023-11-28T21:34:36.018866+0000","last_unstale":"2023-11-28T21:56:44.422244+0000","last_undegraded":"2023-11-28T21:56:44.422244+0000","last_fullsized":"2023-11-28T21:56:44.422244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'313","last_scrub_stamp":"2023-11-28T21:56:44.422134+0000","last_deep_scrub":"52'313","last_deep_scrub_stamp":"2023-11-28T21:56:44.422134+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.422134+0000","objects_scrubbed":9,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:29:50.028108+0000","scrub_duration":0.030421210000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":67,"num_read_kb":77908,"num_write":272,"num_write_kb":12523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57622,"num_omap_keys":105,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"52'1265","reported_seq":3558,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:27.943077+0000","last_change":"2023-11-28T21:57:27.943198+0000","last_active":"2023-11-28T21:57:27.943077+0000","last_peered":"2023-11-28T21:57:27.943077+0000","last_clean":"2023-11-28T21:57:27.943077+0000","last_became_active":"2023-11-28T21:34:41.475981+0000","last_became_peered":"2023-11-28T21:34:41.475981+0000","last_unstale":"2023-11-28T21:57:27.943077+0000","last_undegraded":"2023-11-28T21:57:27.943077+0000","last_fullsized":"2023-11-28T21:57:27.943077+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1265","last_scrub_stamp":"2023-11-28T21:57:27.942825+0000","last_deep_scrub":"52'1265","last_deep_scrub_stamp":"2023-11-28T21:57:27.942825+0000","last_clean_scrub_stamp":"2023-11-28T21:57:27.942825+0000","objects_scrubbed":340,"log_size":1265,"log_dups_size":0,"ondisk_log_size":1265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:57:42.550943+0000","scrub_duration":0.26899315400000001,"objects_trimmed":0,"snaptrim_duration":0.30778537500000003,"stat_sum":{"num_bytes":30037804,"num_objects":340,"num_object_clones":0,"num_object_copies":1360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":0,"num_read":190,"num_read_kb":33,"num_write":1241,"num_write_kb":36631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.264901+0000","last_change":"2023-11-28T21:57:17.264953+0000","last_active":"2023-11-28T21:57:17.264901+0000","last_peered":"2023-11-28T21:57:17.264901+0000","last_clean":"2023-11-28T21:57:17.264901+0000","last_became_active":"2023-11-28T21:34:37.341854+0000","last_became_peered":"2023-11-28T21:34:37.341854+0000","last_unstale":"2023-11-28T21:57:17.264901+0000","last_undegraded":"2023-11-28T21:57:17.264901+0000","last_fullsized":"2023-11-28T21:57:17.264901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:17.264706+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:17.264706+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.264706+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:11:48.499371+0000","scrub_duration":0.0034211150000000002,"objects_trimmed":0,"snaptrim_duration":0.21990301600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"52'759","reported_seq":882,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.931156+0000","last_change":"2023-11-28T21:56:47.144324+0000","last_active":"2023-11-28T21:56:53.931156+0000","last_peered":"2023-11-28T21:56:53.931156+0000","last_clean":"2023-11-28T21:56:53.931156+0000","last_became_active":"2023-11-28T21:34:36.028087+0000","last_became_peered":"2023-11-28T21:34:36.028087+0000","last_unstale":"2023-11-28T21:56:53.931156+0000","last_undegraded":"2023-11-28T21:56:53.931156+0000","last_fullsized":"2023-11-28T21:56:53.931156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'757","last_scrub_stamp":"2023-11-28T21:56:47.144054+0000","last_deep_scrub":"52'757","last_deep_scrub_stamp":"2023-11-28T21:56:47.144054+0000","last_clean_scrub_stamp":"2023-11-28T21:56:47.144054+0000","objects_scrubbed":12,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:36:20.172063+0000","scrub_duration":0.085472306999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":90,"num_read_kb":122940,"num_write":684,"num_write_kb":20915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":36218,"num_omap_keys":66,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"52'1318","reported_seq":3626,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:33.779319+0000","last_change":"2023-11-28T21:57:33.779362+0000","last_active":"2023-11-28T21:57:33.779319+0000","last_peered":"2023-11-28T21:57:33.779319+0000","last_clean":"2023-11-28T21:57:33.779319+0000","last_became_active":"2023-11-28T21:35:25.790293+0000","last_became_peered":"2023-11-28T21:35:25.790293+0000","last_unstale":"2023-11-28T21:57:33.779319+0000","last_undegraded":"2023-11-28T21:57:33.779319+0000","last_fullsized":"2023-11-28T21:57:33.779319+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1318","last_scrub_stamp":"2023-11-28T21:57:33.779155+0000","last_deep_scrub":"52'1318","last_deep_scrub_stamp":"2023-11-28T21:57:33.779155+0000","last_clean_scrub_stamp":"2023-11-28T21:57:33.779155+0000","objects_scrubbed":330,"log_size":1318,"log_dups_size":0,"ondisk_log_size":1318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:20:10.097131+0000","scrub_duration":0.27549114499999999,"objects_trimmed":0,"snaptrim_duration":0.30978138799999999,"stat_sum":{"num_bytes":36258609,"num_objects":330,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":208,"num_read_kb":8227,"num_write":1285,"num_write_kb":56637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"52'283","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.299355+0000","last_change":"2023-11-28T21:56:50.299408+0000","last_active":"2023-11-28T21:56:50.299355+0000","last_peered":"2023-11-28T21:56:50.299355+0000","last_clean":"2023-11-28T21:56:50.299355+0000","last_became_active":"2023-11-28T21:34:37.344094+0000","last_became_peered":"2023-11-28T21:34:37.344094+0000","last_unstale":"2023-11-28T21:56:50.299355+0000","last_undegraded":"2023-11-28T21:56:50.299355+0000","last_fullsized":"2023-11-28T21:56:50.299355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:56:50.299160+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:56:50.299160+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.299160+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:51:20.213618+0000","scrub_duration":0.003897317,"objects_trimmed":0,"snaptrim_duration":0.22435063599999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"52'525","reported_seq":625,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.790155+0000","last_change":"2023-11-28T21:56:53.790210+0000","last_active":"2023-11-28T21:56:53.790155+0000","last_peered":"2023-11-28T21:56:53.790155+0000","last_clean":"2023-11-28T21:56:53.790155+0000","last_became_active":"2023-11-28T21:34:35.997877+0000","last_became_peered":"2023-11-28T21:34:35.997877+0000","last_unstale":"2023-11-28T21:56:53.790155+0000","last_undegraded":"2023-11-28T21:56:53.790155+0000","last_fullsized":"2023-11-28T21:56:53.790155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'525","last_scrub_stamp":"2023-11-28T21:56:53.790048+0000","last_deep_scrub":"52'525","last_deep_scrub_stamp":"2023-11-28T21:56:53.790048+0000","last_clean_scrub_stamp":"2023-11-28T21:56:53.790048+0000","objects_scrubbed":9,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:47:27.057628+0000","scrub_duration":0.097465174000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777888,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":69670,"num_write":485,"num_write_kb":16676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24376,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"52'1329","reported_seq":3690,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:52.591960+0000","last_change":"2023-11-28T21:56:52.592017+0000","last_active":"2023-11-28T21:56:52.591960+0000","last_peered":"2023-11-28T21:56:52.591960+0000","last_clean":"2023-11-28T21:56:52.591960+0000","last_became_active":"2023-11-28T21:34:41.364631+0000","last_became_peered":"2023-11-28T21:34:41.364631+0000","last_unstale":"2023-11-28T21:56:52.591960+0000","last_undegraded":"2023-11-28T21:56:52.591960+0000","last_fullsized":"2023-11-28T21:56:52.591960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1329","last_scrub_stamp":"2023-11-28T21:56:52.591801+0000","last_deep_scrub":"52'1329","last_deep_scrub_stamp":"2023-11-28T21:56:52.591801+0000","last_clean_scrub_stamp":"2023-11-28T21:56:52.591801+0000","objects_scrubbed":342,"log_size":1329,"log_dups_size":0,"ondisk_log_size":1329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:47:53.831632+0000","scrub_duration":0.25412254099999998,"objects_trimmed":0,"snaptrim_duration":0.2340457,"stat_sum":{"num_bytes":46614746,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":259,"num_read_kb":20536,"num_write":1294,"num_write_kb":83351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"52'265","reported_seq":2356,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.218640+0000","last_change":"2023-11-28T21:57:11.218663+0000","last_active":"2023-11-28T21:57:11.218640+0000","last_peered":"2023-11-28T21:57:11.218640+0000","last_clean":"2023-11-28T21:57:11.218640+0000","last_became_active":"2023-11-28T21:34:37.333293+0000","last_became_peered":"2023-11-28T21:34:37.333293+0000","last_unstale":"2023-11-28T21:57:11.218640+0000","last_undegraded":"2023-11-28T21:57:11.218640+0000","last_fullsized":"2023-11-28T21:57:11.218640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'265","last_scrub_stamp":"2023-11-28T21:57:11.218547+0000","last_deep_scrub":"52'265","last_deep_scrub_stamp":"2023-11-28T21:57:11.218547+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.218547+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:45:14.792134+0000","scrub_duration":0.0033770470000000002,"objects_trimmed":0,"snaptrim_duration":0.228570471,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"52'997","reported_seq":1085,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.686854+0000","last_change":"2023-11-28T21:56:49.686931+0000","last_active":"2023-11-28T21:56:49.686854+0000","last_peered":"2023-11-28T21:56:49.686854+0000","last_clean":"2023-11-28T21:56:49.686854+0000","last_became_active":"2023-11-28T21:34:36.030052+0000","last_became_peered":"2023-11-28T21:34:36.030052+0000","last_unstale":"2023-11-28T21:56:49.686854+0000","last_undegraded":"2023-11-28T21:56:49.686854+0000","last_fullsized":"2023-11-28T21:56:49.686854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'997","last_scrub_stamp":"2023-11-28T21:56:49.686647+0000","last_deep_scrub":"52'997","last_deep_scrub_stamp":"2023-11-28T21:56:49.686647+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.686647+0000","objects_scrubbed":7,"log_size":997,"log_dups_size":0,"ondisk_log_size":997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:07:13.091549+0000","scrub_duration":0.104808501,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":34,"num_read_kb":90124,"num_write":955,"num_write_kb":20985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9418,"num_omap_keys":17,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"52'1334","reported_seq":3694,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:17.715355+0000","last_change":"2023-11-28T21:57:17.715381+0000","last_active":"2023-11-28T21:57:17.715355+0000","last_peered":"2023-11-28T21:57:17.715355+0000","last_clean":"2023-11-28T21:57:17.715355+0000","last_became_active":"2023-11-28T21:34:41.379961+0000","last_became_peered":"2023-11-28T21:34:41.379961+0000","last_unstale":"2023-11-28T21:57:17.715355+0000","last_undegraded":"2023-11-28T21:57:17.715355+0000","last_fullsized":"2023-11-28T21:57:17.715355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1334","last_scrub_stamp":"2023-11-28T21:57:17.715260+0000","last_deep_scrub":"52'1334","last_deep_scrub_stamp":"2023-11-28T21:57:17.715260+0000","last_clean_scrub_stamp":"2023-11-28T21:57:17.715260+0000","objects_scrubbed":314,"log_size":1334,"log_dups_size":0,"ondisk_log_size":1334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:34:07.662604+0000","scrub_duration":0.20735551499999999,"objects_trimmed":0,"snaptrim_duration":0.29518589499999998,"stat_sum":{"num_bytes":25457761,"num_objects":314,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":258,"num_read_kb":22214,"num_write":1317,"num_write_kb":78783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"52'308","reported_seq":2393,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:15.248158+0000","last_change":"2023-11-28T21:57:15.248196+0000","last_active":"2023-11-28T21:57:15.248158+0000","last_peered":"2023-11-28T21:57:15.248158+0000","last_clean":"2023-11-28T21:57:15.248158+0000","last_became_active":"2023-11-28T21:35:25.595386+0000","last_became_peered":"2023-11-28T21:35:25.595386+0000","last_unstale":"2023-11-28T21:57:15.248158+0000","last_undegraded":"2023-11-28T21:57:15.248158+0000","last_fullsized":"2023-11-28T21:57:15.248158+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'308","last_scrub_stamp":"2023-11-28T21:57:15.248071+0000","last_deep_scrub":"52'308","last_deep_scrub_stamp":"2023-11-28T21:57:15.248071+0000","last_clean_scrub_stamp":"2023-11-28T21:57:15.248071+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:34:19.589282+0000","scrub_duration":0.0030527179999999998,"objects_trimmed":0,"snaptrim_duration":0.148630489,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"52'112","reported_seq":179,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.072726+0000","last_change":"2023-11-28T21:56:55.072786+0000","last_active":"2023-11-28T21:56:55.072726+0000","last_peered":"2023-11-28T21:56:55.072726+0000","last_clean":"2023-11-28T21:56:55.072726+0000","last_became_active":"2023-11-28T21:34:36.030453+0000","last_became_peered":"2023-11-28T21:34:36.030453+0000","last_unstale":"2023-11-28T21:56:55.072726+0000","last_undegraded":"2023-11-28T21:56:55.072726+0000","last_fullsized":"2023-11-28T21:56:55.072726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'112","last_scrub_stamp":"2023-11-28T21:56:55.072507+0000","last_deep_scrub":"52'112","last_deep_scrub_stamp":"2023-11-28T21:56:55.072507+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.072507+0000","objects_scrubbed":3,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T09:04:39.066804+0000","scrub_duration":0.0092242439999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":68,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"52'1427","reported_seq":3792,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:14.697621+0000","last_change":"2023-11-28T21:57:14.697652+0000","last_active":"2023-11-28T21:57:14.697621+0000","last_peered":"2023-11-28T21:57:14.697621+0000","last_clean":"2023-11-28T21:57:14.697621+0000","last_became_active":"2023-11-28T21:34:41.474799+0000","last_became_peered":"2023-11-28T21:34:41.474799+0000","last_unstale":"2023-11-28T21:57:14.697621+0000","last_undegraded":"2023-11-28T21:57:14.697621+0000","last_fullsized":"2023-11-28T21:57:14.697621+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1427","last_scrub_stamp":"2023-11-28T21:57:14.697528+0000","last_deep_scrub":"52'1427","last_deep_scrub_stamp":"2023-11-28T21:57:14.697528+0000","last_clean_scrub_stamp":"2023-11-28T21:57:14.697528+0000","objects_scrubbed":317,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:15:02.264993+0000","scrub_duration":0.20879229699999999,"objects_trimmed":0,"snaptrim_duration":0.28498848100000002,"stat_sum":{"num_bytes":33245588,"num_objects":317,"num_object_clones":0,"num_object_copies":1268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":263,"num_read_kb":28708,"num_write":1396,"num_write_kb":77612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"52'1309","reported_seq":3662,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:46.426749+0000","last_change":"2023-11-28T21:57:46.426774+0000","last_active":"2023-11-28T21:57:46.426749+0000","last_peered":"2023-11-28T21:57:46.426749+0000","last_clean":"2023-11-28T21:57:46.426749+0000","last_became_active":"2023-11-28T21:34:41.479577+0000","last_became_peered":"2023-11-28T21:34:41.479577+0000","last_unstale":"2023-11-28T21:57:46.426749+0000","last_undegraded":"2023-11-28T21:57:46.426749+0000","last_fullsized":"2023-11-28T21:57:46.426749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1309","last_scrub_stamp":"2023-11-28T21:57:46.426651+0000","last_deep_scrub":"52'1309","last_deep_scrub_stamp":"2023-11-28T21:57:46.426651+0000","last_clean_scrub_stamp":"2023-11-28T21:57:46.426651+0000","objects_scrubbed":331,"log_size":1309,"log_dups_size":0,"ondisk_log_size":1309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:33.889619+0000","scrub_duration":0.17816349400000001,"objects_trimmed":0,"snaptrim_duration":0.27353859400000002,"stat_sum":{"num_bytes":25285382,"num_objects":331,"num_object_clones":0,"num_object_copies":1324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":331,"num_whiteouts":0,"num_read":248,"num_read_kb":4117,"num_write":1287,"num_write_kb":58337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"52'285","reported_seq":2384,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:28.663920+0000","last_change":"2023-11-28T21:57:28.663963+0000","last_active":"2023-11-28T21:57:28.663920+0000","last_peered":"2023-11-28T21:57:28.663920+0000","last_clean":"2023-11-28T21:57:28.663920+0000","last_became_active":"2023-11-28T21:35:25.589378+0000","last_became_peered":"2023-11-28T21:35:25.589378+0000","last_unstale":"2023-11-28T21:57:28.663920+0000","last_undegraded":"2023-11-28T21:57:28.663920+0000","last_fullsized":"2023-11-28T21:57:28.663920+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'285","last_scrub_stamp":"2023-11-28T21:57:28.663780+0000","last_deep_scrub":"52'285","last_deep_scrub_stamp":"2023-11-28T21:57:28.663780+0000","last_clean_scrub_stamp":"2023-11-28T21:57:28.663780+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:58:59.855623+0000","scrub_duration":0.0020765459999999999,"objects_trimmed":0,"snaptrim_duration":0.246426856,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"52'951","reported_seq":1047,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:50.694770+0000","last_change":"2023-11-28T21:56:50.694839+0000","last_active":"2023-11-28T21:56:50.694770+0000","last_peered":"2023-11-28T21:56:50.694770+0000","last_clean":"2023-11-28T21:56:50.694770+0000","last_became_active":"2023-11-28T21:34:36.005447+0000","last_became_peered":"2023-11-28T21:34:36.005447+0000","last_unstale":"2023-11-28T21:56:50.694770+0000","last_undegraded":"2023-11-28T21:56:50.694770+0000","last_fullsized":"2023-11-28T21:56:50.694770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'951","last_scrub_stamp":"2023-11-28T21:56:50.694579+0000","last_deep_scrub":"52'951","last_deep_scrub_stamp":"2023-11-28T21:56:50.694579+0000","last_clean_scrub_stamp":"2023-11-28T21:56:50.694579+0000","objects_scrubbed":5,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:57:02.007325+0000","scrub_duration":0.095599400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":139264,"num_write":899,"num_write_kb":25075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":12742,"num_omap_keys":23,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"52'1266","reported_seq":3575,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:40.285833+0000","last_change":"2023-11-28T21:57:40.285920+0000","last_active":"2023-11-28T21:57:40.285833+0000","last_peered":"2023-11-28T21:57:40.285833+0000","last_clean":"2023-11-28T21:57:40.285833+0000","last_became_active":"2023-11-28T21:34:41.384796+0000","last_became_peered":"2023-11-28T21:34:41.384796+0000","last_unstale":"2023-11-28T21:57:40.285833+0000","last_undegraded":"2023-11-28T21:57:40.285833+0000","last_fullsized":"2023-11-28T21:57:40.285833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1266","last_scrub_stamp":"2023-11-28T21:57:40.285575+0000","last_deep_scrub":"52'1266","last_deep_scrub_stamp":"2023-11-28T21:57:40.285575+0000","last_clean_scrub_stamp":"2023-11-28T21:57:40.285575+0000","objects_scrubbed":319,"log_size":1266,"log_dups_size":0,"ondisk_log_size":1266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:19:13.230680+0000","scrub_duration":0.24648260999999999,"objects_trimmed":0,"snaptrim_duration":0.16839808100000001,"stat_sum":{"num_bytes":18660789,"num_objects":319,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":319,"num_whiteouts":0,"num_read":205,"num_read_kb":12320,"num_write":1249,"num_write_kb":51464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"52'321","reported_seq":2412,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:58.470634+0000","last_change":"2023-11-28T21:56:58.470657+0000","last_active":"2023-11-28T21:56:58.470634+0000","last_peered":"2023-11-28T21:56:58.470634+0000","last_clean":"2023-11-28T21:56:58.470634+0000","last_became_active":"2023-11-28T21:34:37.313782+0000","last_became_peered":"2023-11-28T21:34:37.313782+0000","last_unstale":"2023-11-28T21:56:58.470634+0000","last_undegraded":"2023-11-28T21:56:58.470634+0000","last_fullsized":"2023-11-28T21:56:58.470634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'321","last_scrub_stamp":"2023-11-28T21:56:58.470530+0000","last_deep_scrub":"52'321","last_deep_scrub_stamp":"2023-11-28T21:56:58.470530+0000","last_clean_scrub_stamp":"2023-11-28T21:56:58.470530+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:47:39.793970+0000","scrub_duration":0.0019542520000000001,"objects_trimmed":0,"snaptrim_duration":0.28961016299999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"52'91","reported_seq":162,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:52.421023+0000","last_change":"2023-11-28T21:56:52.421071+0000","last_active":"2023-11-28T21:56:52.421023+0000","last_peered":"2023-11-28T21:56:52.421023+0000","last_clean":"2023-11-28T21:56:52.421023+0000","last_became_active":"2023-11-28T21:34:36.007894+0000","last_became_peered":"2023-11-28T21:34:36.007894+0000","last_unstale":"2023-11-28T21:56:52.421023+0000","last_undegraded":"2023-11-28T21:56:52.421023+0000","last_fullsized":"2023-11-28T21:56:52.421023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'91","last_scrub_stamp":"2023-11-28T21:56:52.420930+0000","last_deep_scrub":"52'91","last_deep_scrub_stamp":"2023-11-28T21:56:52.420930+0000","last_clean_scrub_stamp":"2023-11-28T21:56:52.420930+0000","objects_scrubbed":6,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:15:04.674788+0000","scrub_duration":0.030405108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":63,"num_write_kb":4162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":37270,"num_omap_keys":68,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"52'1081","reported_seq":1182,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.227886+0000","last_change":"2023-11-28T21:56:46.227886+0000","last_active":"2023-11-28T21:56:46.227886+0000","last_peered":"2023-11-28T21:56:46.227886+0000","last_clean":"2023-11-28T21:56:46.227886+0000","last_became_active":"2023-11-28T21:34:36.025347+0000","last_became_peered":"2023-11-28T21:34:36.025347+0000","last_unstale":"2023-11-28T21:56:46.227886+0000","last_undegraded":"2023-11-28T21:56:46.227886+0000","last_fullsized":"2023-11-28T21:56:46.227886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1081","last_scrub_stamp":"2023-11-28T21:56:46.227627+0000","last_deep_scrub":"52'1081","last_deep_scrub_stamp":"2023-11-28T21:56:46.227627+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.227627+0000","objects_scrubbed":10,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:49:00.930323+0000","scrub_duration":0.124157213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":146592,"num_write":1020,"num_write_kb":29113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":33716,"num_omap_keys":61,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"52'284","reported_seq":2375,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:19.161483+0000","last_change":"2023-11-28T21:57:19.161506+0000","last_active":"2023-11-28T21:57:19.161483+0000","last_peered":"2023-11-28T21:57:19.161483+0000","last_clean":"2023-11-28T21:57:19.161483+0000","last_became_active":"2023-11-28T21:34:37.319449+0000","last_became_peered":"2023-11-28T21:34:37.319449+0000","last_unstale":"2023-11-28T21:57:19.161483+0000","last_undegraded":"2023-11-28T21:57:19.161483+0000","last_fullsized":"2023-11-28T21:57:19.161483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'284","last_scrub_stamp":"2023-11-28T21:57:19.161376+0000","last_deep_scrub":"52'284","last_deep_scrub_stamp":"2023-11-28T21:57:19.161376+0000","last_clean_scrub_stamp":"2023-11-28T21:57:19.161376+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:45.359179+0000","scrub_duration":0.0032904639999999999,"objects_trimmed":0,"snaptrim_duration":0.26886027800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"52'1325","reported_seq":3646,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.515073+0000","last_change":"2023-11-28T21:57:09.515073+0000","last_active":"2023-11-28T21:57:09.515073+0000","last_peered":"2023-11-28T21:57:09.515073+0000","last_clean":"2023-11-28T21:57:09.515073+0000","last_became_active":"2023-11-28T21:34:41.370094+0000","last_became_peered":"2023-11-28T21:34:41.370094+0000","last_unstale":"2023-11-28T21:57:09.515073+0000","last_undegraded":"2023-11-28T21:57:09.515073+0000","last_fullsized":"2023-11-28T21:57:09.515073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1325","last_scrub_stamp":"2023-11-28T21:57:09.514950+0000","last_deep_scrub":"52'1325","last_deep_scrub_stamp":"2023-11-28T21:57:09.514950+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.514950+0000","objects_scrubbed":342,"log_size":1325,"log_dups_size":0,"ondisk_log_size":1325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:43:36.262680+0000","scrub_duration":0.21339559899999999,"objects_trimmed":0,"snaptrim_duration":0.13232148399999999,"stat_sum":{"num_bytes":46546650,"num_objects":342,"num_object_clones":0,"num_object_copies":1368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":218,"num_read_kb":9401,"num_write":1296,"num_write_kb":74280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"52'473","reported_seq":550,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:53.461705+0000","last_change":"2023-11-28T21:56:53.461705+0000","last_active":"2023-11-28T21:56:53.461705+0000","last_peered":"2023-11-28T21:56:53.461705+0000","last_clean":"2023-11-28T21:56:53.461705+0000","last_became_active":"2023-11-28T21:34:36.010523+0000","last_became_peered":"2023-11-28T21:34:36.010523+0000","last_unstale":"2023-11-28T21:56:53.461705+0000","last_undegraded":"2023-11-28T21:56:53.461705+0000","last_fullsized":"2023-11-28T21:56:53.461705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'473","last_scrub_stamp":"2023-11-28T21:56:53.461589+0000","last_deep_scrub":"52'473","last_deep_scrub_stamp":"2023-11-28T21:56:53.461589+0000","last_clean_scrub_stamp":"2023-11-28T21:56:53.461589+0000","objects_scrubbed":6,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:21:39.580720+0000","scrub_duration":0.057920754999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582946,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":48896,"num_write":437,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":14594,"num_omap_keys":28,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"52'279","reported_seq":2370,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:49.249131+0000","last_change":"2023-11-28T21:56:49.249131+0000","last_active":"2023-11-28T21:56:49.249131+0000","last_peered":"2023-11-28T21:56:49.249131+0000","last_clean":"2023-11-28T21:56:49.249131+0000","last_became_active":"2023-11-28T21:34:37.327993+0000","last_became_peered":"2023-11-28T21:34:37.327993+0000","last_unstale":"2023-11-28T21:56:49.249131+0000","last_undegraded":"2023-11-28T21:56:49.249131+0000","last_fullsized":"2023-11-28T21:56:49.249131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'279","last_scrub_stamp":"2023-11-28T21:56:49.248890+0000","last_deep_scrub":"52'279","last_deep_scrub_stamp":"2023-11-28T21:56:49.248890+0000","last_clean_scrub_stamp":"2023-11-28T21:56:49.248890+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:20:21.740665+0000","scrub_duration":0.0035163680000000002,"objects_trimmed":0,"snaptrim_duration":0.24793844200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"52'1320","reported_seq":3680,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:26.896353+0000","last_change":"2023-11-28T21:57:26.896377+0000","last_active":"2023-11-28T21:57:26.896353+0000","last_peered":"2023-11-28T21:57:26.896353+0000","last_clean":"2023-11-28T21:57:26.896353+0000","last_became_active":"2023-11-28T21:34:41.362466+0000","last_became_peered":"2023-11-28T21:34:41.362466+0000","last_unstale":"2023-11-28T21:57:26.896353+0000","last_undegraded":"2023-11-28T21:57:26.896353+0000","last_fullsized":"2023-11-28T21:57:26.896353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'1320","last_scrub_stamp":"2023-11-28T21:57:26.896247+0000","last_deep_scrub":"52'1320","last_deep_scrub_stamp":"2023-11-28T21:57:26.896247+0000","last_clean_scrub_stamp":"2023-11-28T21:57:26.896247+0000","objects_scrubbed":333,"log_size":1320,"log_dups_size":0,"ondisk_log_size":1320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:41:24.699472+0000","scrub_duration":0.18315010500000001,"objects_trimmed":0,"snaptrim_duration":0.29562192300000001,"stat_sum":{"num_bytes":29852008,"num_objects":333,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":257,"num_read_kb":28714,"num_write":1297,"num_write_kb":83698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"52'321","reported_seq":2414,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:10.252806+0000","last_change":"2023-11-28T21:57:10.252806+0000","last_active":"2023-11-28T21:57:10.252806+0000","last_peered":"2023-11-28T21:57:10.252806+0000","last_clean":"2023-11-28T21:57:10.252806+0000","last_became_active":"2023-11-28T21:34:37.333195+0000","last_became_peered":"2023-11-28T21:34:37.333195+0000","last_unstale":"2023-11-28T21:57:10.252806+0000","last_undegraded":"2023-11-28T21:57:10.252806+0000","last_fullsized":"2023-11-28T21:57:10.252806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'321","last_scrub_stamp":"2023-11-28T21:57:10.252559+0000","last_deep_scrub":"52'321","last_deep_scrub_stamp":"2023-11-28T21:57:10.252559+0000","last_clean_scrub_stamp":"2023-11-28T21:57:10.252559+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:21:04.185942+0000","scrub_duration":0.0035793169999999998,"objects_trimmed":0,"snaptrim_duration":0.18187520500000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"52'328","reported_seq":2422,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:03.303115+0000","last_change":"2023-11-28T21:57:03.303115+0000","last_active":"2023-11-28T21:57:03.303115+0000","last_peered":"2023-11-28T21:57:03.303115+0000","last_clean":"2023-11-28T21:57:03.303115+0000","last_became_active":"2023-11-28T21:34:37.346871+0000","last_became_peered":"2023-11-28T21:34:37.346871+0000","last_unstale":"2023-11-28T21:57:03.303115+0000","last_undegraded":"2023-11-28T21:57:03.303115+0000","last_fullsized":"2023-11-28T21:57:03.303115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'328","last_scrub_stamp":"2023-11-28T21:57:03.302989+0000","last_deep_scrub":"52'328","last_deep_scrub_stamp":"2023-11-28T21:57:03.302989+0000","last_clean_scrub_stamp":"2023-11-28T21:57:03.302989+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:52:13.280846+0000","scrub_duration":0.0030730839999999998,"objects_trimmed":0,"snaptrim_duration":0.14391041299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"52'303","reported_seq":2397,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:44.533536+0000","last_change":"2023-11-28T21:56:44.533536+0000","last_active":"2023-11-28T21:56:44.533536+0000","last_peered":"2023-11-28T21:56:44.533536+0000","last_clean":"2023-11-28T21:56:44.533536+0000","last_became_active":"2023-11-28T21:34:37.338268+0000","last_became_peered":"2023-11-28T21:34:37.338268+0000","last_unstale":"2023-11-28T21:56:44.533536+0000","last_undegraded":"2023-11-28T21:56:44.533536+0000","last_fullsized":"2023-11-28T21:56:44.533536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'303","last_scrub_stamp":"2023-11-28T21:56:44.533290+0000","last_deep_scrub":"52'303","last_deep_scrub_stamp":"2023-11-28T21:56:44.533290+0000","last_clean_scrub_stamp":"2023-11-28T21:56:44.533290+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:11:39.585022+0000","scrub_duration":0.0035272839999999999,"objects_trimmed":0,"snaptrim_duration":0.30621145,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"52'294","reported_seq":2387,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.242644+0000","last_change":"2023-11-28T21:57:09.242644+0000","last_active":"2023-11-28T21:57:09.242644+0000","last_peered":"2023-11-28T21:57:09.242644+0000","last_clean":"2023-11-28T21:57:09.242644+0000","last_became_active":"2023-11-28T21:34:37.315893+0000","last_became_peered":"2023-11-28T21:34:37.315893+0000","last_unstale":"2023-11-28T21:57:09.242644+0000","last_undegraded":"2023-11-28T21:57:09.242644+0000","last_fullsized":"2023-11-28T21:57:09.242644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'294","last_scrub_stamp":"2023-11-28T21:57:09.242522+0000","last_deep_scrub":"52'294","last_deep_scrub_stamp":"2023-11-28T21:57:09.242522+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.242522+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:12:54.800329+0000","scrub_duration":0.001978142,"objects_trimmed":0,"snaptrim_duration":0.26967866099999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"52'275","reported_seq":2367,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.387085+0000","last_change":"2023-11-28T21:56:46.387109+0000","last_active":"2023-11-28T21:56:46.387085+0000","last_peered":"2023-11-28T21:56:46.387085+0000","last_clean":"2023-11-28T21:56:46.387085+0000","last_became_active":"2023-11-28T21:34:37.334645+0000","last_became_peered":"2023-11-28T21:34:37.334645+0000","last_unstale":"2023-11-28T21:56:46.387085+0000","last_undegraded":"2023-11-28T21:56:46.387085+0000","last_fullsized":"2023-11-28T21:56:46.387085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'275","last_scrub_stamp":"2023-11-28T21:56:46.386968+0000","last_deep_scrub":"52'275","last_deep_scrub_stamp":"2023-11-28T21:56:46.386968+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.386968+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T04:09:10.256531+0000","scrub_duration":0.0028373539999999998,"objects_trimmed":0,"snaptrim_duration":0.29058787400000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"52'293","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:11.787300+0000","last_change":"2023-11-28T21:57:11.787318+0000","last_active":"2023-11-28T21:57:11.787300+0000","last_peered":"2023-11-28T21:57:11.787300+0000","last_clean":"2023-11-28T21:57:11.787300+0000","last_became_active":"2023-11-28T21:34:37.317783+0000","last_became_peered":"2023-11-28T21:34:37.317783+0000","last_unstale":"2023-11-28T21:57:11.787300+0000","last_undegraded":"2023-11-28T21:57:11.787300+0000","last_fullsized":"2023-11-28T21:57:11.787300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'293","last_scrub_stamp":"2023-11-28T21:57:11.787231+0000","last_deep_scrub":"52'293","last_deep_scrub_stamp":"2023-11-28T21:57:11.787231+0000","last_clean_scrub_stamp":"2023-11-28T21:57:11.787231+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T08:35:53.843661+0000","scrub_duration":0.0015063699999999999,"objects_trimmed":0,"snaptrim_duration":0.246741036,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"52'316","reported_seq":2408,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:13.330062+0000","last_change":"2023-11-28T21:57:13.330127+0000","last_active":"2023-11-28T21:57:13.330062+0000","last_peered":"2023-11-28T21:57:13.330062+0000","last_clean":"2023-11-28T21:57:13.330062+0000","last_became_active":"2023-11-28T21:34:37.345466+0000","last_became_peered":"2023-11-28T21:34:37.345466+0000","last_unstale":"2023-11-28T21:57:13.330062+0000","last_undegraded":"2023-11-28T21:57:13.330062+0000","last_fullsized":"2023-11-28T21:57:13.330062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'316","last_scrub_stamp":"2023-11-28T21:57:13.329847+0000","last_deep_scrub":"52'316","last_deep_scrub_stamp":"2023-11-28T21:57:13.329847+0000","last_clean_scrub_stamp":"2023-11-28T21:57:13.329847+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:32:35.225710+0000","scrub_duration":0.0039367100000000004,"objects_trimmed":0,"snaptrim_duration":0.18459727300000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"52'287","reported_seq":2379,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:05.398087+0000","last_change":"2023-11-28T21:57:05.398109+0000","last_active":"2023-11-28T21:57:05.398087+0000","last_peered":"2023-11-28T21:57:05.398087+0000","last_clean":"2023-11-28T21:57:05.398087+0000","last_became_active":"2023-11-28T21:34:37.334663+0000","last_became_peered":"2023-11-28T21:34:37.334663+0000","last_unstale":"2023-11-28T21:57:05.398087+0000","last_undegraded":"2023-11-28T21:57:05.398087+0000","last_fullsized":"2023-11-28T21:57:05.398087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'287","last_scrub_stamp":"2023-11-28T21:57:05.397991+0000","last_deep_scrub":"52'287","last_deep_scrub_stamp":"2023-11-28T21:57:05.397991+0000","last_clean_scrub_stamp":"2023-11-28T21:57:05.397991+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:09:47.792188+0000","scrub_duration":0.002983145,"objects_trimmed":0,"snaptrim_duration":0.29272911699999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"52'285","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:07.843268+0000","last_change":"2023-11-28T21:57:07.843319+0000","last_active":"2023-11-28T21:57:07.843268+0000","last_peered":"2023-11-28T21:57:07.843268+0000","last_clean":"2023-11-28T21:57:07.843268+0000","last_became_active":"2023-11-28T21:34:37.331756+0000","last_became_peered":"2023-11-28T21:34:37.331756+0000","last_unstale":"2023-11-28T21:57:07.843268+0000","last_undegraded":"2023-11-28T21:57:07.843268+0000","last_fullsized":"2023-11-28T21:57:07.843268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'285","last_scrub_stamp":"2023-11-28T21:57:07.843058+0000","last_deep_scrub":"52'285","last_deep_scrub_stamp":"2023-11-28T21:57:07.843058+0000","last_clean_scrub_stamp":"2023-11-28T21:57:07.843058+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:11:36.128065+0000","scrub_duration":0.0032873389999999998,"objects_trimmed":0,"snaptrim_duration":0.30579284099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"52'272","reported_seq":2364,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:51.436185+0000","last_change":"2023-11-28T21:56:51.436185+0000","last_active":"2023-11-28T21:56:51.436185+0000","last_peered":"2023-11-28T21:56:51.436185+0000","last_clean":"2023-11-28T21:56:51.436185+0000","last_became_active":"2023-11-28T21:34:37.336587+0000","last_became_peered":"2023-11-28T21:34:37.336587+0000","last_unstale":"2023-11-28T21:56:51.436185+0000","last_undegraded":"2023-11-28T21:56:51.436185+0000","last_fullsized":"2023-11-28T21:56:51.436185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'272","last_scrub_stamp":"2023-11-28T21:56:51.436052+0000","last_deep_scrub":"52'272","last_deep_scrub_stamp":"2023-11-28T21:56:51.436052+0000","last_clean_scrub_stamp":"2023-11-28T21:56:51.436052+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:27.893115+0000","scrub_duration":0.0032975470000000001,"objects_trimmed":0,"snaptrim_duration":0.28793565599999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"52'285","reported_seq":2378,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.755908+0000","last_change":"2023-11-28T21:57:16.755960+0000","last_active":"2023-11-28T21:57:16.755908+0000","last_peered":"2023-11-28T21:57:16.755908+0000","last_clean":"2023-11-28T21:57:16.755908+0000","last_became_active":"2023-11-28T21:34:37.349325+0000","last_became_peered":"2023-11-28T21:34:37.349325+0000","last_unstale":"2023-11-28T21:57:16.755908+0000","last_undegraded":"2023-11-28T21:57:16.755908+0000","last_fullsized":"2023-11-28T21:57:16.755908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'285","last_scrub_stamp":"2023-11-28T21:57:16.755612+0000","last_deep_scrub":"52'285","last_deep_scrub_stamp":"2023-11-28T21:57:16.755612+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.755612+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:01.560436+0000","scrub_duration":0.0036730019999999999,"objects_trimmed":0,"snaptrim_duration":0.30636868099999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.293494+0000","last_change":"2023-11-28T21:57:04.293494+0000","last_active":"2023-11-28T21:57:04.293494+0000","last_peered":"2023-11-28T21:57:04.293494+0000","last_clean":"2023-11-28T21:57:04.293494+0000","last_became_active":"2023-11-28T21:34:37.343883+0000","last_became_peered":"2023-11-28T21:34:37.343883+0000","last_unstale":"2023-11-28T21:57:04.293494+0000","last_undegraded":"2023-11-28T21:57:04.293494+0000","last_fullsized":"2023-11-28T21:57:04.293494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:04.293402+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:04.293402+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.293402+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:26:25.956930+0000","scrub_duration":0.0026303339999999998,"objects_trimmed":0,"snaptrim_duration":0.26835999100000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:22.280875+0000","last_change":"2023-11-28T21:57:22.280955+0000","last_active":"2023-11-28T21:57:22.280875+0000","last_peered":"2023-11-28T21:57:22.280875+0000","last_clean":"2023-11-28T21:57:22.280875+0000","last_became_active":"2023-11-28T21:34:37.352726+0000","last_became_peered":"2023-11-28T21:34:37.352726+0000","last_unstale":"2023-11-28T21:57:22.280875+0000","last_undegraded":"2023-11-28T21:57:22.280875+0000","last_fullsized":"2023-11-28T21:57:22.280875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:22.280604+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:22.280604+0000","last_clean_scrub_stamp":"2023-11-28T21:57:22.280604+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:15:11.070749+0000","scrub_duration":0.0040196090000000004,"objects_trimmed":0,"snaptrim_duration":0.194469013,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"52'293","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:18.231899+0000","last_change":"2023-11-28T21:57:18.231950+0000","last_active":"2023-11-28T21:57:18.231899+0000","last_peered":"2023-11-28T21:57:18.231899+0000","last_clean":"2023-11-28T21:57:18.231899+0000","last_became_active":"2023-11-28T21:34:37.351995+0000","last_became_peered":"2023-11-28T21:34:37.351995+0000","last_unstale":"2023-11-28T21:57:18.231899+0000","last_undegraded":"2023-11-28T21:57:18.231899+0000","last_fullsized":"2023-11-28T21:57:18.231899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'293","last_scrub_stamp":"2023-11-28T21:57:18.231694+0000","last_deep_scrub":"52'293","last_deep_scrub_stamp":"2023-11-28T21:57:18.231694+0000","last_clean_scrub_stamp":"2023-11-28T21:57:18.231694+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:39:32.183707+0000","scrub_duration":0.0026952199999999999,"objects_trimmed":0,"snaptrim_duration":0.21943309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"52'268","reported_seq":2361,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.255150+0000","last_change":"2023-11-28T21:57:04.255150+0000","last_active":"2023-11-28T21:57:04.255150+0000","last_peered":"2023-11-28T21:57:04.255150+0000","last_clean":"2023-11-28T21:57:04.255150+0000","last_became_active":"2023-11-28T21:34:37.336495+0000","last_became_peered":"2023-11-28T21:34:37.336495+0000","last_unstale":"2023-11-28T21:57:04.255150+0000","last_undegraded":"2023-11-28T21:57:04.255150+0000","last_fullsized":"2023-11-28T21:57:04.255150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'268","last_scrub_stamp":"2023-11-28T21:57:04.255048+0000","last_deep_scrub":"52'268","last_deep_scrub_stamp":"2023-11-28T21:57:04.255048+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.255048+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:40:34.130819+0000","scrub_duration":0.0027406069999999999,"objects_trimmed":0,"snaptrim_duration":0.14941532399999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"52'313","reported_seq":2406,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:42.478673+0000","last_change":"2023-11-28T21:56:42.478673+0000","last_active":"2023-11-28T21:56:42.478673+0000","last_peered":"2023-11-28T21:56:42.478673+0000","last_clean":"2023-11-28T21:56:42.478673+0000","last_became_active":"2023-11-28T21:34:37.348890+0000","last_became_peered":"2023-11-28T21:34:37.348890+0000","last_unstale":"2023-11-28T21:56:42.478673+0000","last_undegraded":"2023-11-28T21:56:42.478673+0000","last_fullsized":"2023-11-28T21:56:42.478673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'313","last_scrub_stamp":"2023-11-28T21:56:42.478435+0000","last_deep_scrub":"52'313","last_deep_scrub_stamp":"2023-11-28T21:56:42.478435+0000","last_clean_scrub_stamp":"2023-11-28T21:56:42.478435+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:09:59.037978+0000","scrub_duration":0.0037738749999999999,"objects_trimmed":0,"snaptrim_duration":0.30374592,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"52'301","reported_seq":2394,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:45.566906+0000","last_change":"2023-11-28T21:56:45.566906+0000","last_active":"2023-11-28T21:56:45.566906+0000","last_peered":"2023-11-28T21:56:45.566906+0000","last_clean":"2023-11-28T21:56:45.566906+0000","last_became_active":"2023-11-28T21:34:37.335734+0000","last_became_peered":"2023-11-28T21:34:37.335734+0000","last_unstale":"2023-11-28T21:56:45.566906+0000","last_undegraded":"2023-11-28T21:56:45.566906+0000","last_fullsized":"2023-11-28T21:56:45.566906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'301","last_scrub_stamp":"2023-11-28T21:56:45.566667+0000","last_deep_scrub":"52'301","last_deep_scrub_stamp":"2023-11-28T21:56:45.566667+0000","last_clean_scrub_stamp":"2023-11-28T21:56:45.566667+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:37:12.783651+0000","scrub_duration":0.0036235360000000001,"objects_trimmed":0,"snaptrim_duration":0.30238551699999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"52'316","reported_seq":2409,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:04.741484+0000","last_change":"2023-11-28T21:57:04.741484+0000","last_active":"2023-11-28T21:57:04.741484+0000","last_peered":"2023-11-28T21:57:04.741484+0000","last_clean":"2023-11-28T21:57:04.741484+0000","last_became_active":"2023-11-28T21:34:37.332509+0000","last_became_peered":"2023-11-28T21:34:37.332509+0000","last_unstale":"2023-11-28T21:57:04.741484+0000","last_undegraded":"2023-11-28T21:57:04.741484+0000","last_fullsized":"2023-11-28T21:57:04.741484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'316","last_scrub_stamp":"2023-11-28T21:57:04.741375+0000","last_deep_scrub":"52'316","last_deep_scrub_stamp":"2023-11-28T21:57:04.741375+0000","last_clean_scrub_stamp":"2023-11-28T21:57:04.741375+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:53:03.728731+0000","scrub_duration":0.0023203569999999999,"objects_trimmed":0,"snaptrim_duration":0.246777987,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"52'301","reported_seq":2392,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:55.425032+0000","last_change":"2023-11-28T21:56:55.425054+0000","last_active":"2023-11-28T21:56:55.425032+0000","last_peered":"2023-11-28T21:56:55.425032+0000","last_clean":"2023-11-28T21:56:55.425032+0000","last_became_active":"2023-11-28T21:34:37.320550+0000","last_became_peered":"2023-11-28T21:34:37.320550+0000","last_unstale":"2023-11-28T21:56:55.425032+0000","last_undegraded":"2023-11-28T21:56:55.425032+0000","last_fullsized":"2023-11-28T21:56:55.425032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'301","last_scrub_stamp":"2023-11-28T21:56:55.424936+0000","last_deep_scrub":"52'301","last_deep_scrub_stamp":"2023-11-28T21:56:55.424936+0000","last_clean_scrub_stamp":"2023-11-28T21:56:55.424936+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:58:23.742113+0000","scrub_duration":0.0019665469999999999,"objects_trimmed":0,"snaptrim_duration":0.21399896700000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"52'274","reported_seq":2365,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:48.388835+0000","last_change":"2023-11-28T21:56:48.388857+0000","last_active":"2023-11-28T21:56:48.388835+0000","last_peered":"2023-11-28T21:56:48.388835+0000","last_clean":"2023-11-28T21:56:48.388835+0000","last_became_active":"2023-11-28T21:34:37.337449+0000","last_became_peered":"2023-11-28T21:34:37.337449+0000","last_unstale":"2023-11-28T21:56:48.388835+0000","last_undegraded":"2023-11-28T21:56:48.388835+0000","last_fullsized":"2023-11-28T21:56:48.388835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'274","last_scrub_stamp":"2023-11-28T21:56:48.388743+0000","last_deep_scrub":"52'274","last_deep_scrub_stamp":"2023-11-28T21:56:48.388743+0000","last_clean_scrub_stamp":"2023-11-28T21:56:48.388743+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:55.401441+0000","scrub_duration":0.0030358709999999999,"objects_trimmed":0,"snaptrim_duration":0.287656776,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"52'261","reported_seq":2352,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:56:46.326425+0000","last_change":"2023-11-28T21:56:46.326533+0000","last_active":"2023-11-28T21:56:46.326425+0000","last_peered":"2023-11-28T21:56:46.326425+0000","last_clean":"2023-11-28T21:56:46.326425+0000","last_became_active":"2023-11-28T21:34:37.334339+0000","last_became_peered":"2023-11-28T21:34:37.334339+0000","last_unstale":"2023-11-28T21:56:46.326425+0000","last_undegraded":"2023-11-28T21:56:46.326425+0000","last_fullsized":"2023-11-28T21:56:46.326425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'261","last_scrub_stamp":"2023-11-28T21:56:46.326201+0000","last_deep_scrub":"52'261","last_deep_scrub_stamp":"2023-11-28T21:56:46.326201+0000","last_clean_scrub_stamp":"2023-11-28T21:56:46.326201+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:10:57.520054+0000","scrub_duration":0.0037943849999999999,"objects_trimmed":0,"snaptrim_duration":0.222804104,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"52'261","reported_seq":2352,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:33.057770+0000","last_change":"2023-11-28T21:57:33.057822+0000","last_active":"2023-11-28T21:57:33.057770+0000","last_peered":"2023-11-28T21:57:33.057770+0000","last_clean":"2023-11-28T21:57:33.057770+0000","last_became_active":"2023-11-28T21:34:37.354430+0000","last_became_peered":"2023-11-28T21:34:37.354430+0000","last_unstale":"2023-11-28T21:57:33.057770+0000","last_undegraded":"2023-11-28T21:57:33.057770+0000","last_fullsized":"2023-11-28T21:57:33.057770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'261","last_scrub_stamp":"2023-11-28T21:57:33.057492+0000","last_deep_scrub":"52'261","last_deep_scrub_stamp":"2023-11-28T21:57:33.057492+0000","last_clean_scrub_stamp":"2023-11-28T21:57:33.057492+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:57:41.660862+0000","scrub_duration":0.0037777190000000001,"objects_trimmed":0,"snaptrim_duration":0.19768582900000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"52'283","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.372700+0000","last_change":"2023-11-28T21:57:16.372800+0000","last_active":"2023-11-28T21:57:16.372700+0000","last_peered":"2023-11-28T21:57:16.372700+0000","last_clean":"2023-11-28T21:57:16.372700+0000","last_became_active":"2023-11-28T21:34:37.347375+0000","last_became_peered":"2023-11-28T21:34:37.347375+0000","last_unstale":"2023-11-28T21:57:16.372700+0000","last_undegraded":"2023-11-28T21:57:16.372700+0000","last_fullsized":"2023-11-28T21:57:16.372700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'283","last_scrub_stamp":"2023-11-28T21:57:16.372401+0000","last_deep_scrub":"52'283","last_deep_scrub_stamp":"2023-11-28T21:57:16.372401+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.372401+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:12.888742+0000","scrub_duration":0.0041957210000000003,"objects_trimmed":0,"snaptrim_duration":0.18434383600000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"52'314","reported_seq":2413,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:26.291975+0000","last_change":"2023-11-28T21:57:26.292027+0000","last_active":"2023-11-28T21:57:26.291975+0000","last_peered":"2023-11-28T21:57:26.291975+0000","last_clean":"2023-11-28T21:57:26.291975+0000","last_became_active":"2023-11-28T21:35:25.598839+0000","last_became_peered":"2023-11-28T21:35:25.598839+0000","last_unstale":"2023-11-28T21:57:26.291975+0000","last_undegraded":"2023-11-28T21:57:26.291975+0000","last_fullsized":"2023-11-28T21:57:26.291975+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'314","last_scrub_stamp":"2023-11-28T21:57:26.291780+0000","last_deep_scrub":"52'314","last_deep_scrub_stamp":"2023-11-28T21:57:26.291780+0000","last_clean_scrub_stamp":"2023-11-28T21:57:26.291780+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T02:05:40.948354+0000","scrub_duration":0.0037891439999999999,"objects_trimmed":0,"snaptrim_duration":0.22244017299999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"52'291","reported_seq":2382,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:10.406800+0000","last_change":"2023-11-28T21:57:10.406825+0000","last_active":"2023-11-28T21:57:10.406800+0000","last_peered":"2023-11-28T21:57:10.406800+0000","last_clean":"2023-11-28T21:57:10.406800+0000","last_became_active":"2023-11-28T21:34:37.358550+0000","last_became_peered":"2023-11-28T21:34:37.358550+0000","last_unstale":"2023-11-28T21:57:10.406800+0000","last_undegraded":"2023-11-28T21:57:10.406800+0000","last_fullsized":"2023-11-28T21:57:10.406800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'291","last_scrub_stamp":"2023-11-28T21:57:10.406693+0000","last_deep_scrub":"52'291","last_deep_scrub_stamp":"2023-11-28T21:57:10.406693+0000","last_clean_scrub_stamp":"2023-11-28T21:57:10.406693+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:00:26.481166+0000","scrub_duration":0.0034554859999999998,"objects_trimmed":0,"snaptrim_duration":0.28637254899999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"52'292","reported_seq":2384,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:05.386536+0000","last_change":"2023-11-28T21:57:05.386536+0000","last_active":"2023-11-28T21:57:05.386536+0000","last_peered":"2023-11-28T21:57:05.386536+0000","last_clean":"2023-11-28T21:57:05.386536+0000","last_became_active":"2023-11-28T21:34:37.343152+0000","last_became_peered":"2023-11-28T21:34:37.343152+0000","last_unstale":"2023-11-28T21:57:05.386536+0000","last_undegraded":"2023-11-28T21:57:05.386536+0000","last_fullsized":"2023-11-28T21:57:05.386536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'292","last_scrub_stamp":"2023-11-28T21:57:05.386287+0000","last_deep_scrub":"52'292","last_deep_scrub_stamp":"2023-11-28T21:57:05.386287+0000","last_clean_scrub_stamp":"2023-11-28T21:57:05.386287+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:00:49.649068+0000","scrub_duration":0.0038884100000000001,"objects_trimmed":0,"snaptrim_duration":0.219142752,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"52'282","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:24.707483+0000","last_change":"2023-11-28T21:57:24.707536+0000","last_active":"2023-11-28T21:57:24.707483+0000","last_peered":"2023-11-28T21:57:24.707483+0000","last_clean":"2023-11-28T21:57:24.707483+0000","last_became_active":"2023-11-28T21:34:37.345500+0000","last_became_peered":"2023-11-28T21:34:37.345500+0000","last_unstale":"2023-11-28T21:57:24.707483+0000","last_undegraded":"2023-11-28T21:57:24.707483+0000","last_fullsized":"2023-11-28T21:57:24.707483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'282","last_scrub_stamp":"2023-11-28T21:57:24.707271+0000","last_deep_scrub":"52'282","last_deep_scrub_stamp":"2023-11-28T21:57:24.707271+0000","last_clean_scrub_stamp":"2023-11-28T21:57:24.707271+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:31:32.444772+0000","scrub_duration":0.0040746819999999996,"objects_trimmed":0,"snaptrim_duration":0.307113203,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"52'297","reported_seq":2389,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:16.239031+0000","last_change":"2023-11-28T21:57:16.239138+0000","last_active":"2023-11-28T21:57:16.239031+0000","last_peered":"2023-11-28T21:57:16.239031+0000","last_clean":"2023-11-28T21:57:16.239031+0000","last_became_active":"2023-11-28T21:34:37.343125+0000","last_became_peered":"2023-11-28T21:34:37.343125+0000","last_unstale":"2023-11-28T21:57:16.239031+0000","last_undegraded":"2023-11-28T21:57:16.239031+0000","last_fullsized":"2023-11-28T21:57:16.239031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'297","last_scrub_stamp":"2023-11-28T21:57:16.238782+0000","last_deep_scrub":"52'297","last_deep_scrub_stamp":"2023-11-28T21:57:16.238782+0000","last_clean_scrub_stamp":"2023-11-28T21:57:16.238782+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T05:06:31.855622+0000","scrub_duration":0.0036200709999999999,"objects_trimmed":0,"snaptrim_duration":0.21561430200000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"52'287","reported_seq":2381,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:21.659207+0000","last_change":"2023-11-28T21:57:21.659264+0000","last_active":"2023-11-28T21:57:21.659207+0000","last_peered":"2023-11-28T21:57:21.659207+0000","last_clean":"2023-11-28T21:57:21.659207+0000","last_became_active":"2023-11-28T21:34:37.347251+0000","last_became_peered":"2023-11-28T21:34:37.347251+0000","last_unstale":"2023-11-28T21:57:21.659207+0000","last_undegraded":"2023-11-28T21:57:21.659207+0000","last_fullsized":"2023-11-28T21:57:21.659207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'287","last_scrub_stamp":"2023-11-28T21:57:21.658984+0000","last_deep_scrub":"52'287","last_deep_scrub_stamp":"2023-11-28T21:57:21.658984+0000","last_clean_scrub_stamp":"2023-11-28T21:57:21.658984+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T06:42:40.185184+0000","scrub_duration":0.0032803480000000002,"objects_trimmed":0,"snaptrim_duration":0.30486287499999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"52'281","reported_seq":2374,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:18.200046+0000","last_change":"2023-11-28T21:57:18.200046+0000","last_active":"2023-11-28T21:57:18.200046+0000","last_peered":"2023-11-28T21:57:18.200046+0000","last_clean":"2023-11-28T21:57:18.200046+0000","last_became_active":"2023-11-28T21:34:37.330438+0000","last_became_peered":"2023-11-28T21:34:37.330438+0000","last_unstale":"2023-11-28T21:57:18.200046+0000","last_undegraded":"2023-11-28T21:57:18.200046+0000","last_fullsized":"2023-11-28T21:57:18.200046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'281","last_scrub_stamp":"2023-11-28T21:57:18.199943+0000","last_deep_scrub":"52'281","last_deep_scrub_stamp":"2023-11-28T21:57:18.199943+0000","last_clean_scrub_stamp":"2023-11-28T21:57:18.199943+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:24:04.947793+0000","scrub_duration":0.0015704720000000001,"objects_trimmed":0,"snaptrim_duration":0.26474323500000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"52'292","reported_seq":2386,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:09.805004+0000","last_change":"2023-11-28T21:57:09.805004+0000","last_active":"2023-11-28T21:57:09.805004+0000","last_peered":"2023-11-28T21:57:09.805004+0000","last_clean":"2023-11-28T21:57:09.805004+0000","last_became_active":"2023-11-28T21:34:37.318453+0000","last_became_peered":"2023-11-28T21:34:37.318453+0000","last_unstale":"2023-11-28T21:57:09.805004+0000","last_undegraded":"2023-11-28T21:57:09.805004+0000","last_fullsized":"2023-11-28T21:57:09.805004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'292","last_scrub_stamp":"2023-11-28T21:57:09.804904+0000","last_deep_scrub":"52'292","last_deep_scrub_stamp":"2023-11-28T21:57:09.804904+0000","last_clean_scrub_stamp":"2023-11-28T21:57:09.804904+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T07:06:20.229796+0000","scrub_duration":0.0016870489999999999,"objects_trimmed":0,"snaptrim_duration":0.24685295800000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"52'289","reported_seq":2383,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-28T21:57:27.327882+0000","last_change":"2023-11-28T21:57:27.327882+0000","last_active":"2023-11-28T21:57:27.327882+0000","last_peered":"2023-11-28T21:57:27.327882+0000","last_clean":"2023-11-28T21:57:27.327882+0000","last_became_active":"2023-11-28T21:34:37.349438+0000","last_became_peered":"2023-11-28T21:34:37.349438+0000","last_unstale":"2023-11-28T21:57:27.327882+0000","last_undegraded":"2023-11-28T21:57:27.327882+0000","last_fullsized":"2023-11-28T21:57:27.327882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"52'289","last_scrub_stamp":"2023-11-28T21:57:27.327611+0000","last_deep_scrub":"52'289","last_deep_scrub_stamp":"2023-11-28T21:57:27.327611+0000","last_clean_scrub_stamp":"2023-11-28T21:57:27.327611+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T03:57:34.455926+0000","scrub_duration":0.003750448,"objects_trimmed":0,"snaptrim_duration":0.22257470400000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":927864585,"num_objects":10340,"num_object_clones":0,"num_object_copies":41360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10340,"num_whiteouts":0,"num_read":7542,"num_read_kb":441308,"num_write":41596,"num_write_kb":1955411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1822244864,"data_stored":1974304768,"data_compressed":124280088,"data_compressed_allocated":140013568,"data_compressed_original":292073472,"omap_allocated":0,"internal_metadata":0},"log_size":42425,"ondisk_log_size":42425,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18675,"ondisk_log_size":18675,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":303332196,"num_objects":196,"num_object_clones":0,"num_object_copies":392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":1486,"num_read_kb":1965797,"num_write":14882,"num_write_kb":402412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":97,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":687608,"num_omap_keys":1199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173137920,"data_stored":606664392,"data_compressed":82562470,"data_compressed_allocated":105857024,"data_compressed_original":539541398,"omap_allocated":0,"internal_metadata":0},"log_size":16086,"ondisk_log_size":16086,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66574,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575135,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":329152,"kb_used_data":211896,"kb_used_omap":1,"kb_used_meta":117246,"kb_avail":94042688,"statfs":{"total":96636764160,"available":96299712512,"internally_reserved":0,"allocated":216981504,"data_stored":292765622,"data_compressed":24984621,"data_compressed_allocated":29888512,"data_compressed_original":105979904,"omap_allocated":1590,"internal_metadata":120060362},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.874,"15min":0.98499999999999999},"min":{"1min":0.498,"5min":0.41399999999999998,"15min":0.35299999999999998},"max":{"1min":3.3719999999999999,"5min":5.056,"15min":47.475000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.85599999999999998,"15min":0.82999999999999996},"min":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.36199999999999999},"max":{"1min":1.8260000000000001,"5min":4.069,"15min":5.3310000000000004},"last":1.014}]},{"osd":1,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0069999999999999,"15min":1.1060000000000001},"min":{"1min":0.65400000000000003,"5min":0.59699999999999998,"15min":0.39200000000000002},"max":{"1min":1.79,"5min":2.5630000000000002,"15min":47.529000000000003},"last":1.228},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0860000000000001,"15min":1.0189999999999999},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.734,"5min":17.460000000000001,"15min":17.460000000000001},"last":1.198}]},{"osd":2,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.83899999999999997,"15min":0.90500000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98899999999999999,"5min":2.9359999999999999,"15min":5.5140000000000002},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.008,"15min":1.0249999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.498},"max":{"1min":1.212,"5min":2.972,"15min":5.5979999999999999},"last":0.97599999999999998}]},{"osd":3,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.216,"15min":1.198},"min":{"1min":0.60699999999999998,"5min":0.57299999999999995,"15min":0.44600000000000001},"max":{"1min":7.3760000000000003,"5min":16.113,"15min":47.57},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.327,"5min":1.214,"15min":1.0920000000000001},"min":{"1min":0.72599999999999998,"5min":0.65400000000000003,"15min":0.53800000000000003},"max":{"1min":7.4269999999999996,"5min":11.148999999999999,"15min":12.273},"last":1.0660000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.125,"15min":1.0609999999999999},"min":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.35499999999999998},"max":{"1min":5.6859999999999999,"5min":17.542000000000002,"15min":17.542000000000002},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0429999999999999,"15min":1.1950000000000001},"min":{"1min":0.67200000000000004,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":49.106000000000002},"last":0.78900000000000003}]},{"osd":5,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.2909999999999999},"min":{"1min":0.91700000000000004,"5min":0.65600000000000003,"15min":0.40899999999999997},"max":{"1min":2.0110000000000001,"5min":6.6420000000000003,"15min":64.855999999999995},"last":1.115},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.1779999999999999,"15min":1.206},"min":{"1min":0.79800000000000004,"5min":0.73499999999999999,"15min":0.56399999999999995},"max":{"1min":5.843,"5min":11.327,"15min":16.753},"last":1.3169999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.216,"15min":1.175},"min":{"1min":0.72999999999999998,"5min":0.67500000000000004,"15min":0.37},"max":{"1min":5.7679999999999998,"5min":11.641999999999999,"15min":11.641999999999999},"last":1.268},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.1719999999999999,"15min":1.167},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.502},"max":{"1min":5.4210000000000003,"5min":5.4210000000000003,"15min":7.4909999999999997},"last":1.173}]}]},{"osd":6,"up_from":13,"seq":55834575133,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":380176,"kb_used_data":254600,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":93991664,"statfs":{"total":96636764160,"available":96247463936,"internally_reserved":0,"allocated":260710400,"data_stored":335877014,"data_compressed":24633140,"data_compressed_allocated":29302784,"data_compressed_original":104771531,"omap_allocated":1594,"internal_metadata":128580038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.276,"15min":1.0249999999999999},"min":{"1min":0.55800000000000005,"5min":0.441,"15min":0.27300000000000002},"max":{"1min":2.9119999999999999,"5min":40.999000000000002,"15min":40.999000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.042,"15min":1.04},"min":{"1min":0.62,"5min":0.42299999999999999,"15min":0.40000000000000002},"max":{"1min":2.3370000000000002,"5min":14.132999999999999,"15min":24.399999999999999},"last":0.93000000000000005}]},{"osd":1,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":0.99399999999999999,"15min":0.92000000000000004},"min":{"1min":0.55900000000000005,"5min":0.48599999999999999,"15min":0.35899999999999999},"max":{"1min":7.0709999999999997,"5min":7.0709999999999997,"15min":11.196},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.611,"5min":1.0389999999999999,"15min":1.012},"min":{"1min":0.57499999999999996,"5min":0.42799999999999999,"15min":0.40899999999999997},"max":{"1min":13.404,"5min":13.404,"15min":13.404},"last":0.82499999999999996}]},{"osd":2,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":0.94299999999999995,"15min":0.96399999999999997},"min":{"1min":0.74399999999999999,"5min":0.56999999999999995,"15min":0.39100000000000001},"max":{"1min":2.6829999999999998,"5min":3.855,"15min":21.986999999999998},"last":1.173},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.004,"15min":1.024},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.40400000000000003},"max":{"1min":8.4730000000000008,"5min":8.4730000000000008,"15min":8.6170000000000009},"last":1.208}]},{"osd":3,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.131,"15min":1.0189999999999999},"min":{"1min":0.68400000000000005,"5min":0.54600000000000004,"15min":0.41999999999999998},"max":{"1min":15.964,"5min":15.964,"15min":15.964},"last":0.873},{"interface":"front","average":{"1min":1.897,"5min":1.1919999999999999,"15min":1.149},"min":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.47499999999999998},"max":{"1min":16.602,"5min":16.602,"15min":24.745000000000001},"last":1.302}]},{"osd":4,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.95999999999999996,"15min":1.018},"min":{"1min":0.69799999999999995,"5min":0.59799999999999998,"15min":0.30299999999999999},"max":{"1min":1.7050000000000001,"5min":2.4550000000000001,"15min":16.015000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.093,"15min":1.0900000000000001},"min":{"1min":0.91000000000000003,"5min":0.65100000000000002,"15min":0.35799999999999998},"max":{"1min":2.4929999999999999,"5min":2.5329999999999999,"15min":13.193},"last":1.135}]},{"osd":5,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.0329999999999999,"15min":1.1200000000000001},"min":{"1min":0.86699999999999999,"5min":0.58699999999999997,"15min":0.307},"max":{"1min":1.661,"5min":2.069,"15min":25.006},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.1140000000000001,"15min":1.151},"min":{"1min":0.86799999999999999,"5min":0.46700000000000003,"15min":0.33000000000000002},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":7.843},"last":1.2649999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.109,"15min":1.163},"min":{"1min":0.78600000000000003,"5min":0.68999999999999995,"15min":0.36099999999999999},"max":{"1min":5.6799999999999997,"5min":5.6799999999999997,"15min":7.883},"last":1.339},{"interface":"front","average":{"1min":1.554,"5min":1.246,"15min":1.304},"min":{"1min":0.95399999999999996,"5min":0.86499999999999999,"15min":0.432},"max":{"1min":5.7199999999999998,"5min":5.7199999999999998,"15min":27.942},"last":1.3959999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575133,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363676,"kb_used_data":238100,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":94008164,"statfs":{"total":96636764160,"available":96264359936,"internally_reserved":0,"allocated":243814400,"data_stored":323612635,"data_compressed":28196796,"data_compressed_allocated":33439744,"data_compressed_original":113541120,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:25 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0349999999999999,"15min":0.90800000000000003},"min":{"1min":0.51000000000000001,"5min":0.46500000000000002,"15min":0.40600000000000003},"max":{"1min":3.3980000000000001,"5min":15.005000000000001,"15min":15.005000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.0009999999999999,"15min":0.90800000000000003},"min":{"1min":0.54200000000000004,"5min":0.495,"15min":0.46700000000000003},"max":{"1min":3.419,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.98199999999999998}]},{"osd":1,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.84799999999999998,"15min":0.84299999999999997},"min":{"1min":0.54800000000000004,"5min":0.51800000000000002,"15min":0.48199999999999998},"max":{"1min":3.806,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.3029999999999999,"5min":0.96399999999999997,"15min":0.93700000000000006},"min":{"1min":0.57999999999999996,"5min":0.57299999999999995,"15min":0.496},"max":{"1min":7.4930000000000003,"5min":7.4930000000000003,"15min":7.4930000000000003},"last":1.0469999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:57:25 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0109999999999999,"15min":0.97699999999999998},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.2649999999999999,"5min":14.775,"15min":14.775},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1379999999999999,"15min":0.96299999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.57499999999999996},"max":{"1min":1.462,"5min":14.901999999999999,"15min":14.901999999999999},"last":1.0109999999999999}]},{"osd":3,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.038,"15min":0.98999999999999999},"min":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.54400000000000004},"max":{"1min":1.347,"5min":15.154,"15min":15.154},"last":1.254},{"interface":"front","average":{"1min":1.02,"5min":0.996,"15min":1.0149999999999999},"min":{"1min":0.68600000000000005,"5min":0.60799999999999998,"15min":0.53500000000000003},"max":{"1min":1.3069999999999999,"5min":3.4289999999999998,"15min":5.5170000000000003},"last":0.84499999999999997}]},{"osd":4,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.534,"15min":1.1870000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.52000000000000002},"max":{"1min":2.98,"5min":61.020000000000003,"15min":61.020000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.22,"15min":1.1160000000000001},"min":{"1min":0.67400000000000004,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":8.6029999999999998,"5min":15.085000000000001,"15min":15.085000000000001},"last":1.143}]},{"osd":6,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.137,"15min":1.1259999999999999},"min":{"1min":0.72599999999999998,"5min":0.66000000000000003,"15min":0.56699999999999995},"max":{"1min":1.4099999999999999,"5min":3.8290000000000002,"15min":3.8290000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.266,"5min":1.234,"15min":1.181},"min":{"1min":1.0589999999999999,"5min":0.73199999999999998,"15min":0.64800000000000002},"max":{"1min":1.4970000000000001,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":1.218}]},{"osd":7,"last update":"Tue Nov 28 21:57:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2070000000000001,"15min":1.1739999999999999},"min":{"1min":0.85599999999999998,"5min":0.75600000000000001,"15min":0.70399999999999996},"max":{"1min":1.5600000000000001,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":1.115},{"interface":"front","average":{"1min":1.042,"5min":1.0600000000000001,"15min":1.0289999999999999},"min":{"1min":0.69299999999999995,"5min":0.66800000000000004,"15min":0.60499999999999998},"max":{"1min":1.6779999999999999,"5min":6.2869999999999999,"15min":6.2869999999999999},"last":1.1830000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575133,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":357644,"kb_used_data":236228,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":94014196,"statfs":{"total":96636764160,"available":96270536704,"internally_reserved":0,"allocated":241897472,"data_stored":321511706,"data_compressed":25475129,"data_compressed_allocated":30593024,"data_compressed_original":110510027,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2210000000000001,"15min":1.004},"min":{"1min":0.38200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":9.2759999999999998,"5min":43.139000000000003,"15min":43.139000000000003},"last":1.097},{"interface":"front","average":{"1min":1.573,"5min":0.96599999999999997,"15min":0.94399999999999995},"min":{"1min":0.57299999999999995,"5min":0.499,"15min":0.39900000000000002},"max":{"1min":9.9009999999999998,"5min":9.9009999999999998,"15min":17.446000000000002},"last":1.2649999999999999}]},{"osd":1,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.8200000000000001,"5min":1.153,"15min":1.1140000000000001},"min":{"1min":0.67100000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":15.432,"5min":15.432,"15min":21.763999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.72,"5min":1.145,"15min":1.5369999999999999},"min":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.46500000000000002},"max":{"1min":12.186999999999999,"5min":12.186999999999999,"15min":78.649000000000001},"last":1.2130000000000001}]},{"osd":2,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.9319999999999999,"5min":1.1439999999999999,"15min":1.1160000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":20.105,"5min":20.105,"15min":28.341000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.2070000000000001,"15min":1.603},"min":{"1min":0.70399999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"max":{"1min":15.393000000000001,"5min":15.393000000000001,"15min":94.998999999999995},"last":1.038}]},{"osd":3,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":2.0720000000000001,"5min":1.4019999999999999,"15min":1.169},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":20.065000000000001,"5min":20.065000000000001,"15min":20.065000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":2.0419999999999998,"5min":1.246,"15min":1.1279999999999999},"min":{"1min":0.64200000000000002,"5min":0.54800000000000004,"15min":0.53200000000000003},"max":{"1min":18.588000000000001,"5min":18.588000000000001,"15min":18.588000000000001},"last":0.875}]},{"osd":5,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.0600000000000001,"15min":1.159},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.376},"max":{"1min":12.159000000000001,"5min":12.159000000000001,"15min":17.033000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.468,"15min":1.329},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.437},"max":{"1min":9.5500000000000007,"5min":38.939,"15min":38.939},"last":1.3129999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":2.3500000000000001,"5min":1.2929999999999999,"15min":1.2609999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.49299999999999999},"max":{"1min":28.289000000000001,"5min":28.289000000000001,"15min":28.289000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.5640000000000001,"15min":1.367},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.55600000000000005},"max":{"1min":9.5120000000000005,"5min":43.420000000000002,"15min":43.420000000000002},"last":1.369}]},{"osd":7,"last update":"Tue Nov 28 21:57:10 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.1859999999999999,"15min":1.6499999999999999},"min":{"1min":0.55900000000000005,"5min":0.434,"15min":0.434},"max":{"1min":10.282,"5min":10.282,"15min":77.638999999999996},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.3089999999999999,"15min":1.2070000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.51500000000000001},"max":{"1min":9.6080000000000005,"5min":9.6080000000000005,"15min":9.6080000000000005},"last":1.4139999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575134,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465164,"kb_used_data":307652,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93906676,"statfs":{"total":96636764160,"available":96160436224,"internally_reserved":0,"allocated":315035648,"data_stored":410723951,"data_compressed":36998413,"data_compressed_allocated":43757568,"data_compressed_original":139747328,"omap_allocated":1592,"internal_metadata":161282504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.89400000000000002,"15min":0.70899999999999996},"min":{"1min":0.312,"5min":0.246,"15min":0.24399999999999999},"max":{"1min":0.77600000000000002,"5min":36.585000000000001,"15min":36.585000000000001},"last":0.434},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.88600000000000001,"15min":0.67600000000000005},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.33100000000000002},"max":{"1min":0.996,"5min":30.065000000000001,"15min":30.065000000000001},"last":0.498}]},{"osd":1,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.42699999999999999,"5min":0.33200000000000002,"15min":0.26300000000000001},"max":{"1min":0.80800000000000005,"5min":4.8099999999999996,"15min":10.449},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.58799999999999997,"15min":0.67100000000000004},"min":{"1min":0.33600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80400000000000005,"5min":4.8520000000000003,"15min":10.481999999999999},"last":0.36299999999999999}]},{"osd":2,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57499999999999996,"15min":0.64800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.27900000000000003},"max":{"1min":0.90200000000000002,"5min":1.25,"15min":10.369},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.69099999999999995},"min":{"1min":0.36599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":10.406000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91700000000000004,"15min":1.3580000000000001},"min":{"1min":0.59499999999999997,"5min":0.55500000000000005,"15min":0.376},"max":{"1min":1.036,"5min":4.2119999999999997,"15min":110.39700000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.97399999999999998,"15min":1.399},"min":{"1min":0.752,"5min":0.56100000000000005,"15min":0.40500000000000003},"max":{"1min":1.2609999999999999,"5min":7.2030000000000003,"15min":108.777},"last":0.75900000000000001}]},{"osd":5,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.871,"15min":0.92600000000000005},"min":{"1min":0.61599999999999999,"5min":0.56299999999999994,"15min":0.39900000000000002},"max":{"1min":1.196,"5min":7.2679999999999998,"15min":17.513000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.85799999999999998,"15min":0.92900000000000005},"min":{"1min":0.64400000000000002,"5min":0.621,"15min":0.52300000000000002},"max":{"1min":1.323,"5min":1.7070000000000001,"15min":10.516999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83199999999999996,"15min":1.399},"min":{"1min":0.62,"5min":0.52700000000000002,"15min":0.45200000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":138.91300000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.875,"15min":0.94799999999999995},"min":{"1min":0.65000000000000002,"5min":0.53300000000000003,"15min":0.48299999999999998},"max":{"1min":1.23,"5min":1.766,"15min":10.302},"last":0.77800000000000002}]},{"osd":7,"last update":"Tue Nov 28 21:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.92500000000000004,"15min":0.96099999999999997},"min":{"1min":0.66000000000000003,"5min":0.60999999999999999,"15min":0.45200000000000001},"max":{"1min":1.246,"5min":7.3040000000000003,"15min":17.263000000000002},"last":0.877},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98399999999999999,"15min":1.038},"min":{"1min":0.72599999999999998,"5min":0.54900000000000004,"15min":0.49299999999999999},"max":{"1min":1.397,"5min":7.3339999999999996,"15min":28.613},"last":0.81299999999999994}]}]},{"osd":2,"up_from":13,"seq":55834575132,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288308,"kb_used_data":176620,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94083532,"statfs":{"total":96636764160,"available":96341536768,"internally_reserved":0,"allocated":180858880,"data_stored":223905413,"data_compressed":16019215,"data_compressed_allocated":18952192,"data_compressed_original":62304256,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.63500000000000001,"15min":0.58699999999999997},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.23599999999999999},"max":{"1min":1.6679999999999999,"5min":13.599,"15min":13.599},"last":0.442},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.74299999999999999,"15min":0.63700000000000001},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.318},"max":{"1min":7.1159999999999997,"5min":8.0630000000000006,"15min":8.0630000000000006},"last":0.65700000000000003}]},{"osd":1,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.32000000000000001,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":7.8970000000000002,"5min":7.8970000000000002,"15min":9.3260000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":1.173,"5min":0.81200000000000006,"15min":0.71299999999999997},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.34200000000000003},"max":{"1min":10.657,"5min":10.657,"15min":10.657},"last":0.73699999999999999}]},{"osd":3,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.046,"15min":0.81899999999999995},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.307},"max":{"1min":7.1639999999999997,"5min":29.123999999999999,"15min":29.123999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.498,"5min":0.80600000000000005,"15min":0.66500000000000004},"min":{"1min":0.36199999999999999,"5min":0.312,"15min":0.28000000000000003},"max":{"1min":10.18,"5min":10.18,"15min":10.18},"last":0.70399999999999996}]},{"osd":4,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.97299999999999998,"15min":0.93500000000000005},"min":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.49099999999999999},"max":{"1min":2.9769999999999999,"5min":5.6289999999999996,"15min":7.3630000000000004},"last":0.995},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.002,"15min":0.92600000000000005},"min":{"1min":0.59299999999999997,"5min":0.51200000000000001,"15min":0.41399999999999998},"max":{"1min":12.723000000000001,"5min":12.723000000000001,"15min":12.723000000000001},"last":0.752}]},{"osd":5,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":0.94099999999999995,"15min":0.87},"min":{"1min":0.64900000000000002,"5min":0.501,"15min":0.501},"max":{"1min":6.875,"5min":6.875,"15min":6.875},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.8520000000000001,"5min":1.2070000000000001,"15min":1.006},"min":{"1min":0.65000000000000002,"5min":0.59399999999999997,"15min":0.48399999999999999},"max":{"1min":9.4809999999999999,"5min":9.4809999999999999,"15min":9.4809999999999999},"last":0.81599999999999995}]},{"osd":6,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.093,"15min":0.92900000000000005},"min":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.55400000000000005},"max":{"1min":3.5489999999999999,"5min":18.959,"15min":18.959},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.897,"5min":1.325,"15min":1.0369999999999999},"min":{"1min":0.77700000000000002,"5min":0.622,"15min":0.52200000000000002},"max":{"1min":9.4399999999999995,"5min":21.957999999999998,"15min":21.957999999999998},"last":1.02}]},{"osd":7,"last update":"Tue Nov 28 21:56:59 2023","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":1.0800000000000001,"15min":0.98099999999999998},"min":{"1min":0.64200000000000002,"5min":0.58799999999999997,"15min":0.57699999999999996},"max":{"1min":9.2249999999999996,"5min":9.2249999999999996,"15min":9.2799999999999994},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.7989999999999999,"5min":1.109,"15min":0.97199999999999998},"min":{"1min":0.61799999999999999,"5min":0.59499999999999997,"15min":0.58199999999999996},"max":{"1min":9.1679999999999993,"5min":9.1679999999999993,"15min":9.1679999999999993},"last":0.88}]}]},{"osd":1,"up_from":13,"seq":55834575132,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469328,"kb_used_data":311816,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93902512,"statfs":{"total":96636764160,"available":96156172288,"internally_reserved":0,"allocated":319299584,"data_stored":398618222,"data_compressed":30748401,"data_compressed_allocated":36524032,"data_compressed_original":116154368,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.75700000000000001,"15min":0.69299999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.246},"max":{"1min":0.65400000000000003,"5min":12.311999999999999,"15min":14.538},"last":0.753},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.621,"15min":0.64900000000000002},"min":{"1min":0.33400000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65000000000000002,"5min":8.2479999999999993,"15min":14.618},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Nov 28 21:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.65200000000000002,"15min":0.627},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.31900000000000001},"max":{"1min":0.77700000000000002,"5min":5.1689999999999996,"15min":8.4749999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36899999999999999},"max":{"1min":0.72599999999999998,"5min":4.7030000000000003,"15min":8.5150000000000006},"last":0.68400000000000005}]},{"osd":3,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.75700000000000001,"15min":0.84699999999999998},"min":{"1min":0.33400000000000002,"5min":0.32200000000000001,"15min":0.27600000000000002},"max":{"1min":0.68500000000000005,"5min":13.385,"15min":43.536999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.82499999999999996,"15min":0.84499999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":8.2899999999999991,"15min":18.308},"last":0.78300000000000003}]},{"osd":4,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.98499999999999999,"15min":0.95199999999999996},"min":{"1min":0.498,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":1.0900000000000001,"5min":9.9849999999999994,"15min":9.9849999999999994},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.98399999999999999,"15min":1.0760000000000001},"min":{"1min":0.58199999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.149,"5min":10.287000000000001,"15min":43.497999999999998},"last":0.92800000000000005}]},{"osd":5,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.93000000000000005,"15min":0.91700000000000004},"min":{"1min":0.72199999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.085,"5min":3.3740000000000001,"15min":5.5350000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.01,"15min":0.995},"min":{"1min":0.874,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1799999999999999,"5min":3.448,"15min":5.7409999999999997},"last":0.86399999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":1.0349999999999999,"15min":1.0069999999999999},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0629999999999999,"5min":10.023,"15min":10.385},"last":1.004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":1.0509999999999999,"15min":1.2490000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.59599999999999997},"max":{"1min":1.1719999999999999,"5min":5.9349999999999996,"15min":43.448999999999998},"last":0.84799999999999998}]},{"osd":7,"last update":"Tue Nov 28 21:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":1.0580000000000001,"15min":1.02},"min":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.56899999999999995},"max":{"1min":1.135,"5min":5.9779999999999998,"15min":7.96},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":1.1830000000000001,"15min":1.161},"min":{"1min":0.65700000000000003,"5min":0.55800000000000005,"15min":0.51500000000000001},"max":{"1min":1.109,"5min":15.672000000000001,"15min":42.473999999999997},"last":0.89700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575133,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":346236,"kb_used_data":216500,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":94025604,"statfs":{"total":96636764160,"available":96282218496,"internally_reserved":0,"allocated":221696000,"data_stored":277678898,"data_compressed":19875137,"data_compressed_allocated":23535616,"data_compressed_original":79818752,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 21:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.72099999999999997,"15min":0.55700000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.215},"max":{"1min":0.64300000000000002,"5min":21.814,"15min":21.814},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.88300000000000001,"15min":0.621},"min":{"1min":0.315,"5min":0.311,"15min":0.248},"max":{"1min":0.71299999999999997,"5min":21.925000000000001,"15min":21.925000000000001},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Nov 28 21:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.92800000000000005,"15min":0.70099999999999996},"min":{"1min":0.443,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77200000000000002,"5min":21.852,"15min":21.852},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.78900000000000003,"15min":0.66600000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.68300000000000005,"5min":18.213000000000001,"15min":18.213000000000001},"last":0.49399999999999999}]},{"osd":3,"last update":"Tue Nov 28 21:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.94899999999999995,"15min":0.72799999999999998},"min":{"1min":0.46700000000000003,"5min":0.29299999999999998,"15min":0.27600000000000002},"max":{"1min":0.85099999999999998,"5min":21.98,"15min":21.98},"last":0.753},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.91600000000000004,"15min":0.77200000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":20.931999999999999,"15min":20.931999999999999},"last":0.81799999999999995}]},{"osd":4,"last update":"Tue Nov 28 21:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.95599999999999996,"15min":0.91600000000000004},"min":{"1min":0.60199999999999998,"5min":0.56699999999999995,"15min":0.42899999999999999},"max":{"1min":1.0269999999999999,"5min":5.2069999999999999,"15min":5.734},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":1.0740000000000001,"15min":0.96499999999999997},"min":{"1min":0.55400000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98799999999999999,"5min":17.98,"15min":17.98},"last":0.877}]},{"osd":5,"last update":"Tue Nov 28 21:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":1.1839999999999999,"15min":0.93300000000000005},"min":{"1min":0.67200000000000004,"5min":0.61799999999999999,"15min":0.40699999999999997},"max":{"1min":1.014,"5min":21.890999999999998,"15min":21.890999999999998},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.1339999999999999,"15min":0.95899999999999996},"min":{"1min":0.61399999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0900000000000001,"5min":18.181000000000001,"15min":18.181000000000001},"last":1.0369999999999999}]},{"osd":6,"last update":"Tue Nov 28 21:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":1.02,"15min":0.91600000000000004},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.48699999999999999},"max":{"1min":1.1459999999999999,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.1279999999999999,"15min":0.95699999999999996},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.55100000000000005},"max":{"1min":1.0529999999999999,"5min":18.030999999999999,"15min":18.030999999999999},"last":0.99199999999999999}]},{"osd":7,"last update":"Tue Nov 28 21:57:20 2023","interfaces":[{"interface":"back","average":{"1min":1.7769999999999999,"5min":1.131,"15min":0.96899999999999997},"min":{"1min":0.65400000000000003,"5min":0.627,"15min":0.48999999999999999},"max":{"1min":18.140999999999998,"5min":18.140999999999998,"15min":18.140999999999998},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.7909999999999999,"5min":1.143,"15min":0.96099999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.55700000000000005},"max":{"1min":18.071000000000002,"5min":18.071000000000002,"15min":18.071000000000002},"last":1.167}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33287,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14426112,"data_stored":54526612,"data_compressed":7767250,"data_compressed_allocated":9949184,"data_compressed_original":50065408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24457216,"data_stored":78922741,"data_compressed":10489735,"data_compressed_allocated":13656064,"data_compressed_original":68149248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15634432,"data_stored":48245772,"data_compressed":6171423,"data_compressed_allocated":8048640,"data_compressed_original":40681472,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":25956352,"data_stored":94383094,"data_compressed":13023761,"data_compressed_allocated":16736256,"data_compressed_original":85180416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23392256,"data_stored":83887777,"data_compressed":11283626,"data_compressed_allocated":14446592,"data_compressed_original":74960843,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22241280,"data_stored":79693154,"data_compressed":11120127,"data_compressed_allocated":14139392,"data_compressed_original":71610368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":22917120,"data_stored":83888413,"data_compressed":11073880,"data_compressed_allocated":14143488,"data_compressed_original":75132875,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24113152,"data_stored":83116829,"data_compressed":11632668,"data_compressed_allocated":14737408,"data_compressed_original":73760768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":206610432,"data_stored":222244864,"data_compressed":12071885,"data_compressed_allocated":13537280,"data_compressed_original":29171712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294244352,"data_stored":319377408,"data_compressed":20255951,"data_compressed_allocated":22863872,"data_compressed_original":47996928,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":164626432,"data_stored":175341568,"data_compressed":9845077,"data_compressed_allocated":10899456,"data_compressed_original":21614592,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":288481280,"data_stored":316022784,"data_compressed":23971937,"data_compressed_allocated":27017216,"data_compressed_original":54558720,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":217907200,"data_stored":237305856,"data_compressed":14188788,"data_compressed_allocated":16142336,"data_compressed_original":35540992,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":220975104,"data_stored":243601408,"data_compressed":17073954,"data_compressed_allocated":19296256,"data_compressed_original":41922560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":237195264,"data_stored":251670528,"data_compressed":13556545,"data_compressed_allocated":15155200,"data_compressed_original":29630464,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":192204800,"data_stored":208740352,"data_compressed":13315951,"data_compressed_allocated":15101952,"data_compressed_original":31637504,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T21:58:05.112 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T21:58:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d701b8640 1 Processor -- start 2023-11-28T21:58:05.291 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d701b8640 1 -- start start 2023-11-28T21:58:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d701b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d680c4000 0x7f8d6806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d701b8640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8d68161600 0x7f8d68161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d701b8640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d680c3460 con 0x7f8d68158420 2023-11-28T21:58:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d701b8640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d68158800 con 0x7f8d68161600 2023-11-28T21:58:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d701b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680c52c0 con 0x7f8d680c4000 2023-11-28T21:58:05.293 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d6e755640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8d68161600 0x7f8d68161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d6df54640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8d68158420 0x7f8d680c38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60786/0 (socket says 172.21.15.12:60786) 2023-11-28T21:58:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d6df54640 1 -- 172.21.15.12:0/2246395958 learned_addr learned my addr 172.21.15.12:0/2246395958 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:58:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d6e755640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8d68161600 0x7f8d68161a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37806/0 (socket says 172.21.15.12:37806) 2023-11-28T21:58:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d6d753640 1 --2- 172.21.15.12:0/2246395958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d680c4000 0x7f8d6806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d6e755640 1 -- 172.21.15.12:0/2246395958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d680c4000 msgr2=0x7f8d6806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d6e755640 1 --2- 172.21.15.12:0/2246395958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d680c4000 0x7f8d6806d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.294 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d6e755640 1 -- 172.21.15.12:0/2246395958 >> v1:172.21.15.12:6789/0 conn(0x7f8d68158420 legacy=0x7f8d680c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.288+0000 7f8d6e755640 1 -- 172.21.15.12:0/2246395958 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680c2b70 con 0x7f8d68161600 2023-11-28T21:58:05.295 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d6e755640 1 --2- 172.21.15.12:0/2246395958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d68161600 0x7f8d68161a10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009d90 tx=0x7f8d6001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4025011ffa46b9ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:05.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d6cf52640 1 -- 172.21.15.12:0/2246395958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600a5810 con 0x7f8d68161600 2023-11-28T21:58:05.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d6cf52640 1 -- 172.21.15.12:0/2246395958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d600a5970 con 0x7f8d68161600 2023-11-28T21:58:05.296 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d6cf52640 1 -- 172.21.15.12:0/2246395958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600a5d00 con 0x7f8d68161600 2023-11-28T21:58:05.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d701b8640 1 -- 172.21.15.12:0/2246395958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d68161600 msgr2=0x7f8d68161a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.297 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d701b8640 1 --2- 172.21.15.12:0/2246395958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d68161600 0x7f8d68161a10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009d90 tx=0x7f8d6001c950 comp rx=0 tx=0).stop 2023-11-28T21:58:05.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d701b8640 1 -- 172.21.15.12:0/2246395958 shutdown_connections 2023-11-28T21:58:05.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d701b8640 1 --2- 172.21.15.12:0/2246395958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d68161600 0x7f8d68161a10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d701b8640 1 --2- 172.21.15.12:0/2246395958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d680c4000 0x7f8d6806d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d701b8640 1 -- 172.21.15.12:0/2246395958 >> 172.21.15.12:0/2246395958 conn(0x7f8d68066360 msgr2=0x7f8d68066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:05.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d701b8640 1 -- 172.21.15.12:0/2246395958 shutdown_connections 2023-11-28T21:58:05.298 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.292+0000 7f8d701b8640 1 -- 172.21.15.12:0/2246395958 wait complete. 2023-11-28T21:58:05.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d701b8640 1 Processor -- start 2023-11-28T21:58:05.299 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d701b8640 1 -- start start 2023-11-28T21:58:05.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d701b8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d680c4000 0x7f8d68154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d701b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d68158420 0x7f8d681efec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d701b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d68161600 0x7f8d681f0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d701b8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6815f1d0 con 0x7f8d680c4000 2023-11-28T21:58:05.301 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d701b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d681fa870 con 0x7f8d68161600 2023-11-28T21:58:05.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d701b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d681fab50 con 0x7f8d68158420 2023-11-28T21:58:05.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6df54640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d680c4000 0x7f8d68154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6df54640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d680c4000 0x7f8d68154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37814/0 (socket says 172.21.15.12:37814) 2023-11-28T21:58:05.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6df54640 1 -- 172.21.15.12:0/2337704318 learned_addr learned my addr 172.21.15.12:0/2337704318 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:05.302 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6d753640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d68158420 0x7f8d681efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6e755640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d68161600 0x7f8d681f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6df54640 1 -- 172.21.15.12:0/2337704318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d68158420 msgr2=0x7f8d681efec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6df54640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d68158420 0x7f8d681efec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6df54640 1 -- 172.21.15.12:0/2337704318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d68161600 msgr2=0x7f8d681f0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6df54640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d68161600 0x7f8d681f0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6df54640 1 -- 172.21.15.12:0/2337704318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d60009a70 con 0x7f8d680c4000 2023-11-28T21:58:05.303 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6e755640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d68161600 0x7f8d681f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:58:05.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6d753640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d68158420 0x7f8d681efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:58:05.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d6df54640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d680c4000 0x7f8d68154740 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400c500 tx=0x7f8d6400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:05.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d5e7fc640 1 -- 172.21.15.12:0/2337704318 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64005cf0 con 0x7f8d680c4000 2023-11-28T21:58:05.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d5e7fc640 1 -- 172.21.15.12:0/2337704318 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d64005e50 con 0x7f8d680c4000 2023-11-28T21:58:05.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d701b8640 1 -- 172.21.15.12:0/2337704318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6814e800 con 0x7f8d680c4000 2023-11-28T21:58:05.305 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d5e7fc640 1 -- 172.21.15.12:0/2337704318 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640072f0 con 0x7f8d680c4000 2023-11-28T21:58:05.306 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.296+0000 7f8d701b8640 1 -- 172.21.15.12:0/2337704318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6815f420 con 0x7f8d680c4000 2023-11-28T21:58:05.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.300+0000 7f8d701b8640 1 -- 172.21.15.12:0/2337704318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d68074f60 con 0x7f8d680c4000 2023-11-28T21:58:05.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.300+0000 7f8d5e7fc640 1 -- 172.21.15.12:0/2337704318 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8d64007450 con 0x7f8d680c4000 2023-11-28T21:58:05.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.300+0000 7f8d5e7fc640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8d4c066470 0x7f8d4c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.300+0000 7f8d5e7fc640 1 -- 172.21.15.12:0/2337704318 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f8d64083a90 con 0x7f8d680c4000 2023-11-28T21:58:05.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.300+0000 7f8d6d753640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8d4c066470 0x7f8d4c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.307 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.300+0000 7f8d6d753640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8d4c066470 0x7f8d4c068930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8d68073050 tx=0x7f8d58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:05.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.304+0000 7f8d5e7fc640 1 -- 172.21.15.12:0/2337704318 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8d64050890 con 0x7f8d680c4000 2023-11-28T21:58:05.469 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.464+0000 7f8d701b8640 1 -- 172.21.15.12:0/2337704318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f8d68154480 con 0x7f8d680c4000 2023-11-28T21:58:05.474 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d5e7fc640 1 -- 172.21.15.12:0/2337704318 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8d640170c0 con 0x7f8d680c4000 2023-11-28T21:58:05.475 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d5e7fc640 1 -- 172.21.15.12:0/2337704318 <== mon.0 v2:172.21.15.12:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8d64054550 con 0x7f8d680c4000 2023-11-28T21:58:05.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 -- 172.21.15.12:0/2337704318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8d4c066470 msgr2=0x7f8d4c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8d4c066470 0x7f8d4c068930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8d68073050 tx=0x7f8d58008040 comp rx=0 tx=0).stop 2023-11-28T21:58:05.477 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 -- 172.21.15.12:0/2337704318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d680c4000 msgr2=0x7f8d68154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d680c4000 0x7f8d68154740 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400c500 tx=0x7f8d6400c9c0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 -- 172.21.15.12:0/2337704318 shutdown_connections 2023-11-28T21:58:05.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f8d4c066470 0x7f8d4c068930 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d68161600 0x7f8d681f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d68158420 0x7f8d681efec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 --2- 172.21.15.12:0/2337704318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8d680c4000 0x7f8d68154740 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 -- 172.21.15.12:0/2337704318 >> 172.21.15.12:0/2337704318 conn(0x7f8d68066360 msgr2=0x7f8d680c38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:05.478 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 -- 172.21.15.12:0/2337704318 shutdown_connections 2023-11-28T21:58:05.479 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.472+0000 7f8d701b8640 1 -- 172.21.15.12:0/2337704318 wait complete. 2023-11-28T21:58:05.497 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T21:58:05.498 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:58:05.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.636+0000 7f5c24279640 1 Processor -- start 2023-11-28T21:58:05.641 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.636+0000 7f5c24279640 1 -- start start 2023-11-28T21:58:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c24279640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c1c151af0 0x7f5c1c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c24279640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5c1c06e300 0x7f5c1c072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c24279640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c157b70 con 0x7f5c1c151ed0 2023-11-28T21:58:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c24279640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c0707f0 con 0x7f5c1c06e300 2023-11-28T21:58:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c24279640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0c7b90 con 0x7f5c1c151af0 2023-11-28T21:58:05.644 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21814640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5c1c151ed0 0x7f5c1c0c49d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60794/0 (socket says 172.21.15.12:60794) 2023-11-28T21:58:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21814640 1 -- 172.21.15.12:0/4004250141 learned_addr learned my addr 172.21.15.12:0/4004250141 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:58:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c22816640 1 --2- 172.21.15.12:0/4004250141 >> v2:172.21.15.144:3300/0 conn(0x7f5c1c06e300 0x7f5c1c072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c22015640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c1c151af0 0x7f5c1c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009659641 0 0) 0x7f5c1c157b70 con 0x7f5c1c151ed0 2023-11-28T21:58:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c08003530 con 0x7f5c1c151ed0 2023-11-28T21:58:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2030901502 0 0) 0x7f5c08003530 con 0x7f5c1c151ed0 2023-11-28T21:58:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c1c151af0 msgr2=0x7f5c1c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:05.645 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c1c151af0 0x7f5c1c0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 >> v2:172.21.15.144:3300/0 conn(0x7f5c1c06e300 msgr2=0x7f5c1c072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 --2- 172.21.15.12:0/4004250141 >> v2:172.21.15.144:3300/0 conn(0x7f5c1c06e300 0x7f5c1c072070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c157e50 con 0x7f5c1c151ed0 2023-11-28T21:58:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f5c0c001680 con 0x7f5c1c151ed0 2023-11-28T21:58:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 >> v1:172.21.15.12:6789/0 conn(0x7f5c1c151ed0 legacy=0x7f5c1c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c080039f0 0x7f5c08003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.646 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c080047d0 0x7f5c08004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c1c151af0 0x7f5c08005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0707f0 con 0x7f5c080039f0 2023-11-28T21:58:05.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0c7b90 con 0x7f5c080047d0 2023-11-28T21:58:05.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0800b990 con 0x7f5c1c151af0 2023-11-28T21:58:05.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c22015640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c080039f0 0x7f5c08003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c21814640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c080047d0 0x7f5c08004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.647 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c22816640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c1c151af0 0x7f5c08005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c22015640 1 -- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c1c151af0 msgr2=0x7f5c08005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c22015640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c1c151af0 0x7f5c08005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c22015640 1 -- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c080047d0 msgr2=0x7f5c08004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c22015640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c080047d0 0x7f5c08004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.648 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.640+0000 7f5c22015640 1 -- 172.21.15.12:0/4004250141 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c08003530 con 0x7f5c080039f0 2023-11-28T21:58:05.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c22015640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c080039f0 0x7f5c08003e10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009b10 tx=0x7f5c1801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92234fce50382d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:05.649 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180a9450 con 0x7f5c080039f0 2023-11-28T21:58:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5c180a95b0 con 0x7f5c080039f0 2023-11-28T21:58:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c21013640 1 -- 172.21.15.12:0/4004250141 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180a9980 con 0x7f5c080039f0 2023-11-28T21:58:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c24279640 1 -- 172.21.15.12:0/4004250141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c080039f0 msgr2=0x7f5c08003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c24279640 1 --2- 172.21.15.12:0/4004250141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c080039f0 0x7f5c08003e10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009b10 tx=0x7f5c1801ca50 comp rx=0 tx=0).stop 2023-11-28T21:58:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c22816640 1 -- 172.21.15.12:0/4004250141 reap_dead start 2023-11-28T21:58:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c24279640 1 -- 172.21.15.12:0/4004250141 shutdown_connections 2023-11-28T21:58:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c24279640 1 -- 172.21.15.12:0/4004250141 >> 172.21.15.12:0/4004250141 conn(0x7f5c1c066360 msgr2=0x7f5c1c066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:05.650 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c24279640 1 -- 172.21.15.12:0/4004250141 shutdown_connections 2023-11-28T21:58:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c24279640 1 -- 172.21.15.12:0/4004250141 wait complete. 2023-11-28T21:58:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c24279640 1 Processor -- start 2023-11-28T21:58:05.651 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.644+0000 7f5c24279640 1 -- start start 2023-11-28T21:58:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c24279640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c080039f0 0x7f5c1c1497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c24279640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c161e10 0x7f5c1c15fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c24279640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c1488b0 0x7f5c1c160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c24279640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c149340 con 0x7f5c1c161e10 2023-11-28T21:58:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c24279640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c160db0 con 0x7f5c1c1488b0 2023-11-28T21:58:05.653 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c24279640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c148d60 con 0x7f5c080039f0 2023-11-28T21:58:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c21814640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c161e10 0x7f5c1c15fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c21814640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c161e10 0x7f5c1c15fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37830/0 (socket says 172.21.15.12:37830) 2023-11-28T21:58:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c21814640 1 -- 172.21.15.12:0/3496433417 learned_addr learned my addr 172.21.15.12:0/3496433417 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c22015640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c080039f0 0x7f5c1c1497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c22816640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c1488b0 0x7f5c1c160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c21814640 1 -- 172.21.15.12:0/3496433417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c080039f0 msgr2=0x7f5c1c1497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c21814640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c080039f0 0x7f5c1c1497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.654 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c21814640 1 -- 172.21.15.12:0/3496433417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c1488b0 msgr2=0x7f5c1c160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c21814640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c1488b0 0x7f5c1c160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c21814640 1 -- 172.21.15.12:0/3496433417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c149010 con 0x7f5c1c161e10 2023-11-28T21:58:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c22816640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c1488b0 0x7f5c1c160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:58:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c21814640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c161e10 0x7f5c1c15fc20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c004f80 tx=0x7f5c0c0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c12ffd640 1 -- 172.21.15.12:0/3496433417 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c00f7b0 con 0x7f5c1c161e10 2023-11-28T21:58:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c12ffd640 1 -- 172.21.15.12:0/3496433417 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5c0c00f910 con 0x7f5c1c161e10 2023-11-28T21:58:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c24279640 1 -- 172.21.15.12:0/3496433417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c161030 con 0x7f5c1c161e10 2023-11-28T21:58:05.655 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c12ffd640 1 -- 172.21.15.12:0/3496433417 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c00fce0 con 0x7f5c1c161e10 2023-11-28T21:58:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c24279640 1 -- 172.21.15.12:0/3496433417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c160590 con 0x7f5c1c161e10 2023-11-28T21:58:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.648+0000 7f5c24279640 1 -- 172.21.15.12:0/3496433417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c074f60 con 0x7f5c1c161e10 2023-11-28T21:58:05.656 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.652+0000 7f5c12ffd640 1 -- 172.21.15.12:0/3496433417 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5c0c01d670 con 0x7f5c1c161e10 2023-11-28T21:58:05.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.652+0000 7f5c12ffd640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c000664a0 0x7f5c00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:05.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.652+0000 7f5c12ffd640 1 -- 172.21.15.12:0/3496433417 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f5c0c084980 con 0x7f5c1c161e10 2023-11-28T21:58:05.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.652+0000 7f5c22015640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c000664a0 0x7f5c00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:05.657 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.652+0000 7f5c22015640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c000664a0 0x7f5c00068960 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5c180ae000 tx=0x7f5c180b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:05.659 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.656+0000 7f5c12ffd640 1 -- 172.21.15.12:0/3496433417 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c0c0515b0 con 0x7f5c1c161e10 2023-11-28T21:58:05.848 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.844+0000 7f5c24279640 1 -- 172.21.15.12:0/3496433417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c1c0c47b0 con 0x7f5c1c161e10 2023-11-28T21:58:05.849 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.844+0000 7f5c12ffd640 1 -- 172.21.15.12:0/3496433417 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+6518 (secure 0 0 0) 0x7f5c0c054bd0 con 0x7f5c1c161e10 2023-11-28T21:58:05.849 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:58:05.850 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6813,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/35262902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":35262902},{"type":"v1","addr":"172.21.15.12:6839","nonce":35262902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:56:18.921322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:active","state_seq":44,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:58:05.850 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 87 2023-11-28T21:58:05.851 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 -- 172.21.15.12:0/3496433417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c000664a0 msgr2=0x7f5c00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c000664a0 0x7f5c00068960 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5c180ae000 tx=0x7f5c180b3040 comp rx=0 tx=0).stop 2023-11-28T21:58:05.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 -- 172.21.15.12:0/3496433417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c161e10 msgr2=0x7f5c1c15fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:05.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c161e10 0x7f5c1c15fc20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c004f80 tx=0x7f5c0c0074f0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 -- 172.21.15.12:0/3496433417 shutdown_connections 2023-11-28T21:58:05.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f5c000664a0 0x7f5c00068960 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c1c1488b0 0x7f5c1c160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.852 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c1c161e10 0x7f5c1c15fc20 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 --2- 172.21.15.12:0/3496433417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c080039f0 0x7f5c1c1497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:05.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 -- 172.21.15.12:0/3496433417 >> 172.21.15.12:0/3496433417 conn(0x7f5c1c066360 msgr2=0x7f5c1c06d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:05.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 -- 172.21.15.12:0/3496433417 shutdown_connections 2023-11-28T21:58:05.853 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:05.848+0000 7f5c24279640 1 -- 172.21.15.12:0/3496433417 wait complete. 2023-11-28T21:58:05.871 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T21:58:06.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f7f2a640 1 Processor -- start 2023-11-28T21:58:06.015 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f7f2a640 1 -- start start 2023-11-28T21:58:06.016 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f7f2a640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f54f0154060 0x7f54f00cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f7f2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f0066f10 0x7f54f006a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f7f2a640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f015ac70 con 0x7f54f00ca2e0 2023-11-28T21:58:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f7f2a640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f0154440 con 0x7f54f0154060 2023-11-28T21:58:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f7f2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00cb1d0 con 0x7f54f0066f10 2023-11-28T21:58:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f54c5640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f54f00ca2e0 0x7f54f0066360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60804/0 (socket says 172.21.15.12:60804) 2023-11-28T21:58:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f54c5640 1 -- 172.21.15.12:0/1035353964 learned_addr learned my addr 172.21.15.12:0/1035353964 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:58:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f5cc6640 1 --2- 172.21.15.12:0/1035353964 >> v2:172.21.15.144:3300/0 conn(0x7f54f0154060 0x7f54f00cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.017 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f64c7640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f0066f10 0x7f54f006a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691308116 0 0) 0x7f54f015ac70 con 0x7f54f00ca2e0 2023-11-28T21:58:06.019 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54dc003530 con 0x7f54f00ca2e0 2023-11-28T21:58:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1586424773 0 0) 0x7f54dc003530 con 0x7f54f00ca2e0 2023-11-28T21:58:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f0066f10 msgr2=0x7f54f006a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f0066f10 0x7f54f006a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 >> v2:172.21.15.144:3300/0 conn(0x7f54f0154060 msgr2=0x7f54f00cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 --2- 172.21.15.12:0/1035353964 >> v2:172.21.15.144:3300/0 conn(0x7f54f0154060 0x7f54f00cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f0159da0 con 0x7f54f00ca2e0 2023-11-28T21:58:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7f54e0001680 con 0x7f54f00ca2e0 2023-11-28T21:58:06.020 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 >> v1:172.21.15.12:6789/0 conn(0x7f54f00ca2e0 legacy=0x7f54f0066360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f0066f10 0x7f54dc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54dc003f70 0x7f54dc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54dc004d10 0x7f54dc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f0154440 con 0x7f54dc003f70 2023-11-28T21:58:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00cb1d0 con 0x7f54dc004d10 2023-11-28T21:58:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc00ba40 con 0x7f54f0066f10 2023-11-28T21:58:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.012+0000 7f54f5cc6640 1 --2- 172.21.15.12:0/1035353964 >> v2:172.21.15.144:3300/0 conn(0x7f54f0154060 0x7f54f00cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T21:58:06.021 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f54c5640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54dc003f70 0x7f54dc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f64c7640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54dc004d10 0x7f54dc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f5cc6640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f0066f10 0x7f54dc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f54c5640 1 -- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f0066f10 msgr2=0x7f54dc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:06.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f54c5640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54f0066f10 0x7f54dc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f54c5640 1 -- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54dc004d10 msgr2=0x7f54dc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:06.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f54c5640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54dc004d10 0x7f54dc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.022 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f54c5640 1 -- 172.21.15.12:0/1035353964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc003530 con 0x7f54dc003f70 2023-11-28T21:58:06.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f54c5640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54dc003f70 0x7f54dc0047d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c210 tx=0x7f54e001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac9125002819b9f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:06.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e00058e0 con 0x7f54dc003f70 2023-11-28T21:58:06.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f54e0005a40 con 0x7f54dc003f70 2023-11-28T21:58:06.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f4cc4640 1 -- 172.21.15.12:0/1035353964 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e0005e70 con 0x7f54dc003f70 2023-11-28T21:58:06.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f7f2a640 1 -- 172.21.15.12:0/1035353964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54dc003f70 msgr2=0x7f54dc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f7f2a640 1 --2- 172.21.15.12:0/1035353964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54dc003f70 0x7f54dc0047d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c210 tx=0x7f54e001cc60 comp rx=0 tx=0).stop 2023-11-28T21:58:06.024 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.016+0000 7f54f64c7640 1 -- 172.21.15.12:0/1035353964 reap_dead start 2023-11-28T21:58:06.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 -- 172.21.15.12:0/1035353964 shutdown_connections 2023-11-28T21:58:06.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 -- 172.21.15.12:0/1035353964 >> 172.21.15.12:0/1035353964 conn(0x7f54f00cf5b0 msgr2=0x7f54f00cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:06.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 -- 172.21.15.12:0/1035353964 shutdown_connections 2023-11-28T21:58:06.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 -- 172.21.15.12:0/1035353964 wait complete. 2023-11-28T21:58:06.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 Processor -- start 2023-11-28T21:58:06.025 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 -- start start 2023-11-28T21:58:06.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc003f70 0x7f54f01552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f00b3a30 0x7f54f0145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54f01457a0 0x7f54f0156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f0155830 con 0x7f54f01457a0 2023-11-28T21:58:06.027 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f00c9830 con 0x7f54f00b3a30 2023-11-28T21:58:06.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f7f2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f01f2c00 con 0x7f54dc003f70 2023-11-28T21:58:06.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f64c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54f01457a0 0x7f54f0156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f64c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54f01457a0 0x7f54f0156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37844/0 (socket says 172.21.15.12:37844) 2023-11-28T21:58:06.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f64c7640 1 -- 172.21.15.12:0/3814141891 learned_addr learned my addr 172.21.15.12:0/3814141891 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:06.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f54c5640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f00b3a30 0x7f54f0145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f5cc6640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc003f70 0x7f54f01552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.028 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f64c7640 1 -- 172.21.15.12:0/3814141891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc003f70 msgr2=0x7f54f01552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f64c7640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc003f70 0x7f54f01552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f64c7640 1 -- 172.21.15.12:0/3814141891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f00b3a30 msgr2=0x7f54f0145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f64c7640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f00b3a30 0x7f54f0145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f64c7640 1 -- 172.21.15.12:0/3814141891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f0146fe0 con 0x7f54f01457a0 2023-11-28T21:58:06.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.020+0000 7f54f64c7640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54f01457a0 0x7f54f0156bf0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f54f006ce70 tx=0x7f54e80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:06.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54e67fc640 1 -- 172.21.15.12:0/3814141891 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800e070 con 0x7f54f01457a0 2023-11-28T21:58:06.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54e67fc640 1 -- 172.21.15.12:0/3814141891 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f54e8011e10 con 0x7f54f01457a0 2023-11-28T21:58:06.029 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54e67fc640 1 -- 172.21.15.12:0/3814141891 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e80039a0 con 0x7f54f01457a0 2023-11-28T21:58:06.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54f7f2a640 1 -- 172.21.15.12:0/3814141891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f01f2ec0 con 0x7f54f01457a0 2023-11-28T21:58:06.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54f7f2a640 1 -- 172.21.15.12:0/3814141891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f01f3480 con 0x7f54f01457a0 2023-11-28T21:58:06.030 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54f7f2a640 1 -- 172.21.15.12:0/3814141891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f00c48c0 con 0x7f54f01457a0 2023-11-28T21:58:06.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54e67fc640 1 -- 172.21.15.12:0/3814141891 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f54e8003b00 con 0x7f54f01457a0 2023-11-28T21:58:06.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54e67fc640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54d4066470 0x7f54d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54e67fc640 1 -- 172.21.15.12:0/3814141891 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f54e8084d40 con 0x7f54f01457a0 2023-11-28T21:58:06.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54f5cc6640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54d4066470 0x7f54d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.031 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.024+0000 7f54f5cc6640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54d4066470 0x7f54d4068930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009bc0 tx=0x7f54ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:06.032 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.028+0000 7f54e67fc640 1 -- 172.21.15.12:0/3814141891 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f54e8051a00 con 0x7f54f01457a0 2023-11-28T21:58:06.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.220+0000 7f54f7f2a640 1 -- 172.21.15.12:0/3814141891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54f0074f60 con 0x7f54f01457a0 2023-11-28T21:58:06.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.220+0000 7f54e67fc640 1 -- 172.21.15.12:0/3814141891 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+6518 (secure 0 0 0) 0x7f54e8055020 con 0x7f54f01457a0 2023-11-28T21:58:06.225 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:58:06.226 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3101383243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3101383243},{"type":"v1","addr":"172.21.15.144:6836","nonce":3101383243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/4030769266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":4030769266},{"type":"v1","addr":"172.21.15.144:6837","nonce":4030769266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/1717483656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6835","nonce":1717483656},{"type":"v1","addr":"172.21.15.12:6837","nonce":1717483656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6813,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/35262902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":35262902},{"type":"v1","addr":"172.21.15.12:6839","nonce":35262902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T21:34:36.378416+0000","modified":"2023-11-28T21:56:18.921322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":52,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6513,"mds_1":6502},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_6502":{"gid":6502,"name":"f","rank":1,"incarnation":81,"state":"up:active","state_seq":44,"addr":"172.21.15.144:6833/3873278110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3873278110},{"type":"v1","addr":"172.21.15.144:6833","nonce":3873278110}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6513":{"gid":6513,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":33,"addr":"172.21.15.12:6838/2286739028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":2286739028},{"type":"v1","addr":"172.21.15.12:6838","nonce":2286739028}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T21:58:06.226 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 87 2023-11-28T21:58:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 -- 172.21.15.12:0/3814141891 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54d4066470 msgr2=0x7f54d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.227 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54d4066470 0x7f54d4068930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009bc0 tx=0x7f54ec008040 comp rx=0 tx=0).stop 2023-11-28T21:58:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 -- 172.21.15.12:0/3814141891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54f01457a0 msgr2=0x7f54f0156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54f01457a0 0x7f54f0156bf0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f54f006ce70 tx=0x7f54e80115a0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 -- 172.21.15.12:0/3814141891 shutdown_connections 2023-11-28T21:58:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f54d4066470 0x7f54d4068930 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54f01457a0 0x7f54f0156bf0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f00b3a30 0x7f54f0145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 --2- 172.21.15.12:0/3814141891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54dc003f70 0x7f54f01552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 -- 172.21.15.12:0/3814141891 >> 172.21.15.12:0/3814141891 conn(0x7f54f00cf5b0 msgr2=0x7f54f00cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:06.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 -- 172.21.15.12:0/3814141891 shutdown_connections 2023-11-28T21:58:06.229 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.224+0000 7f54f7f2a640 1 -- 172.21.15.12:0/3814141891 wait complete. 2023-11-28T21:58:06.246 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T21:58:06.390 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632c8fe640 1 Processor -- start 2023-11-28T21:58:06.391 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632c8fe640 1 -- start start 2023-11-28T21:58:06.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632c8fe640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f63281500d0 0x7f63280c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632c8fe640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63280c6ce0 con 0x7f6328156af0 2023-11-28T21:58:06.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632c8fe640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6328156f60 con 0x7f632806d7d0 2023-11-28T21:58:06.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632c8fe640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63280c7170 con 0x7f63281500d0 2023-11-28T21:58:06.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f63281500d0 0x7f63280c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.392 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632659c640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f63281500d0 0x7f63280c27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37850/0 (socket says 172.21.15.12:37850) 2023-11-28T21:58:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632659c640 1 -- 172.21.15.12:0/1349794204 learned_addr learned my addr 172.21.15.12:0/1349794204 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:06.393 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f6325d9b640 1 --1- 172.21.15.12:0/1349794204 >> v1:172.21.15.12:6789/0 conn(0x7f6328156af0 0x7f63280c2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60810/0 (socket says 172.21.15.12:60810) 2023-11-28T21:58:06.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632659c640 1 -- 172.21.15.12:0/1349794204 >> v1:172.21.15.144:6789/0 conn(0x7f632806d7d0 legacy=0x7f63281609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:06.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632659c640 1 -- 172.21.15.12:0/1349794204 >> v1:172.21.15.12:6789/0 conn(0x7f6328156af0 legacy=0x7f63280c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632659c640 1 -- 172.21.15.12:0/1349794204 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280c5df0 con 0x7f63281500d0 2023-11-28T21:58:06.394 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.388+0000 7f632559a640 1 -- 172.21.15.12:0/1349794204 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060319621 0 0) 0x7f63280c6ce0 con 0x7f6328156af0 2023-11-28T21:58:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632659c640 1 --2- 172.21.15.12:0/1349794204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63281500d0 0x7f63280c27c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6318009870 tx=0x7f631801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eb6c42f6763f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632559a640 1 -- 172.21.15.12:0/1349794204 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63180aa5f0 con 0x7f63281500d0 2023-11-28T21:58:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632559a640 1 -- 172.21.15.12:0/1349794204 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f63180aa750 con 0x7f63281500d0 2023-11-28T21:58:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632559a640 1 -- 172.21.15.12:0/1349794204 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63180aab00 con 0x7f63281500d0 2023-11-28T21:58:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632c8fe640 1 -- 172.21.15.12:0/1349794204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63281500d0 msgr2=0x7f63280c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.396 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632c8fe640 1 --2- 172.21.15.12:0/1349794204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63281500d0 0x7f63280c27c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6318009870 tx=0x7f631801c510 comp rx=0 tx=0).stop 2023-11-28T21:58:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632c8fe640 1 -- 172.21.15.12:0/1349794204 shutdown_connections 2023-11-28T21:58:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632c8fe640 1 --2- 172.21.15.12:0/1349794204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f63281500d0 0x7f63280c27c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.397 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632c8fe640 1 -- 172.21.15.12:0/1349794204 >> 172.21.15.12:0/1349794204 conn(0x7f6328066360 msgr2=0x7f6328066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632c8fe640 1 -- 172.21.15.12:0/1349794204 shutdown_connections 2023-11-28T21:58:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.392+0000 7f632c8fe640 1 -- 172.21.15.12:0/1349794204 wait complete. 2023-11-28T21:58:06.398 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 Processor -- start 2023-11-28T21:58:06.399 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 -- start start 2023-11-28T21:58:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f632806d7d0 0x7f63281f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63281500d0 0x7f63281513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6328156af0 0x7f63280ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6328070760 con 0x7f6328156af0 2023-11-28T21:58:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63281537e0 con 0x7f632806d7d0 2023-11-28T21:58:06.401 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63280c9e00 con 0x7f63281500d0 2023-11-28T21:58:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6326d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6328156af0 0x7f63280ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f632806d7d0 0x7f63281f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6326d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6328156af0 0x7f63280ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37860/0 (socket says 172.21.15.12:37860) 2023-11-28T21:58:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6326d9d640 1 -- 172.21.15.12:0/2311398023 learned_addr learned my addr 172.21.15.12:0/2311398023 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6325d9b640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63281500d0 0x7f63281513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6326d9d640 1 -- 172.21.15.12:0/2311398023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63281500d0 msgr2=0x7f63281513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.402 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6326d9d640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63281500d0 0x7f63281513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6326d9d640 1 -- 172.21.15.12:0/2311398023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f632806d7d0 msgr2=0x7f63281f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6326d9d640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f632806d7d0 0x7f63281f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6326d9d640 1 -- 172.21.15.12:0/2311398023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63180094d0 con 0x7f6328156af0 2023-11-28T21:58:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6325d9b640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63281500d0 0x7f63281513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:58:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6326d9d640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6328156af0 0x7f63280ca290 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f632000c770 tx=0x7f632000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6316ffd640 1 -- 172.21.15.12:0/2311398023 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320003b80 con 0x7f6328156af0 2023-11-28T21:58:06.403 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6316ffd640 1 -- 172.21.15.12:0/2311398023 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6320003ce0 con 0x7f6328156af0 2023-11-28T21:58:06.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 -- 172.21.15.12:0/2311398023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63281f3c40 con 0x7f6328156af0 2023-11-28T21:58:06.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f6316ffd640 1 -- 172.21.15.12:0/2311398023 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632000aab0 con 0x7f6328156af0 2023-11-28T21:58:06.404 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 -- 172.21.15.12:0/2311398023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63280c97b0 con 0x7f6328156af0 2023-11-28T21:58:06.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.396+0000 7f632c8fe640 1 -- 172.21.15.12:0/2311398023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63280c6930 con 0x7f6328156af0 2023-11-28T21:58:06.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.400+0000 7f6316ffd640 1 -- 172.21.15.12:0/2311398023 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f632000ac10 con 0x7f6328156af0 2023-11-28T21:58:06.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.400+0000 7f6316ffd640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63080664a0 0x7f6308068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.400+0000 7f6316ffd640 1 -- 172.21.15.12:0/2311398023 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7f6320084420 con 0x7f6328156af0 2023-11-28T21:58:06.405 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.400+0000 7f632659c640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63080664a0 0x7f6308068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.406 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.400+0000 7f632659c640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63080664a0 0x7f6308068960 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f63180ae000 tx=0x7f63180b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:06.408 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.404+0000 7f6316ffd640 1 -- 172.21.15.12:0/2311398023 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f63200510e0 con 0x7f6328156af0 2023-11-28T21:58:06.569 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.564+0000 7f632c8fe640 1 -- 172.21.15.12:0/2311398023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63280c9f70 con 0x7f6328156af0 2023-11-28T21:58:06.571 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f6316ffd640 1 -- 172.21.15.12:0/2311398023 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+20091 (secure 0 0 0) 0x7f6320054700 con 0x7f6328156af0 2023-11-28T21:58:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-11-28T21:58:06.571 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":52,"fsid":"2975d7d0-8668-45d5-addd-c7b2ef978be7","created":"2023-11-28T21:34:18.573251+0000","modified":"2023-11-28T21:55:38.673845+0000","last_up_change":"2023-11-28T21:34:29.023059+0000","last_in_change":"2023-11-28T21:34:20.942681+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-28T21:34:29.878190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-28T21:34:34.039032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-28T21:34:35.369379+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T21:34:39.373079+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"62cf2d46-0ca9-441b-9e65-70b06b579b20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6817","nonce":3923148634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6819","nonce":3923148634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6823","nonce":3923148634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3923148634},{"type":"v1","addr":"172.21.15.12:6821","nonce":3923148634}]},"public_addr":"172.21.15.12:6817/3923148634","cluster_addr":"172.21.15.12:6819/3923148634","heartbeat_back_addr":"172.21.15.12:6823/3923148634","heartbeat_front_addr":"172.21.15.12:6821/3923148634","state":["exists","up"]},{"osd":1,"uuid":"df411803-82fc-4bb1-b849-419f56db865d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6805","nonce":1716423303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6811","nonce":1716423303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6815","nonce":1716423303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":1716423303},{"type":"v1","addr":"172.21.15.12:6813","nonce":1716423303}]},"public_addr":"172.21.15.12:6805/1716423303","cluster_addr":"172.21.15.12:6811/1716423303","heartbeat_back_addr":"172.21.15.12:6815/1716423303","heartbeat_front_addr":"172.21.15.12:6813/1716423303","state":["exists","up"]},{"osd":2,"uuid":"5a9299bc-9381-48fe-bf4b-5f105fffd914","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6801","nonce":2573246535}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6803","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6804","nonce":2573246535}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6809","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6810","nonce":2573246535}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2573246535},{"type":"v1","addr":"172.21.15.12:6807","nonce":2573246535}]},"public_addr":"172.21.15.12:6801/2573246535","cluster_addr":"172.21.15.12:6804/2573246535","heartbeat_back_addr":"172.21.15.12:6810/2573246535","heartbeat_front_addr":"172.21.15.12:6807/2573246535","state":["exists","up"]},{"osd":3,"uuid":"ad45d502-66d6-4abb-b218-a53b0518dc04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6825","nonce":1952033258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6827","nonce":1952033258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6831","nonce":1952033258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":1952033258},{"type":"v1","addr":"172.21.15.12:6829","nonce":1952033258}]},"public_addr":"172.21.15.12:6825/1952033258","cluster_addr":"172.21.15.12:6827/1952033258","heartbeat_back_addr":"172.21.15.12:6831/1952033258","heartbeat_front_addr":"172.21.15.12:6829/1952033258","state":["exists","up"]},{"osd":4,"uuid":"3a48db2f-807a-4b49-b01f-6e23b5074a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6801","nonce":3927315436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6803","nonce":3927315436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6807","nonce":3927315436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":3927315436},{"type":"v1","addr":"172.21.15.144:6805","nonce":3927315436}]},"public_addr":"172.21.15.144:6801/3927315436","cluster_addr":"172.21.15.144:6803/3927315436","heartbeat_back_addr":"172.21.15.144:6807/3927315436","heartbeat_front_addr":"172.21.15.144:6805/3927315436","state":["exists","up"]},{"osd":5,"uuid":"338dba1f-d51c-4886-8760-b1f1356b2f15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6817","nonce":2546756774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6819","nonce":2546756774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6823","nonce":2546756774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2546756774},{"type":"v1","addr":"172.21.15.144:6821","nonce":2546756774}]},"public_addr":"172.21.15.144:6817/2546756774","cluster_addr":"172.21.15.144:6819/2546756774","heartbeat_back_addr":"172.21.15.144:6823/2546756774","heartbeat_front_addr":"172.21.15.144:6821/2546756774","state":["exists","up"]},{"osd":6,"uuid":"3baa1580-096d-4c3c-8fbe-4335a0a12b2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6825","nonce":2995192634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6827","nonce":2995192634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6831","nonce":2995192634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2995192634},{"type":"v1","addr":"172.21.15.144:6829","nonce":2995192634}]},"public_addr":"172.21.15.144:6825/2995192634","cluster_addr":"172.21.15.144:6827/2995192634","heartbeat_back_addr":"172.21.15.144:6831/2995192634","heartbeat_front_addr":"172.21.15.144:6829/2995192634","state":["exists","up"]},{"osd":7,"uuid":"dfc81f70-0ce4-43e8-a8e8-bb7dac0fbd27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6809","nonce":3855981118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6811","nonce":3855981118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6815","nonce":3855981118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3855981118},{"type":"v1","addr":"172.21.15.144:6813","nonce":3855981118}]},"public_addr":"172.21.15.144:6809/3855981118","cluster_addr":"172.21.15.144:6811/3855981118","heartbeat_back_addr":"172.21.15.144:6815/3855981118","heartbeat_front_addr":"172.21.15.144:6813/3855981118","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.672018+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.679831+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.637520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.766468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.437227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.488221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.614537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T21:34:26.441819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:6839/754378087":"2023-11-29T21:55:38.673802+0000","172.21.15.12:6836/754378087":"2023-11-29T21:55:38.673802+0000","172.21.15.144:6832/3053317078":"2023-11-29T21:39:58.381227+0000","172.21.15.144:6833/3053317078":"2023-11-29T21:39:58.381227+0000","172.21.15.12:6839/2705421041":"2023-11-29T21:48:53.535998+0000","172.21.15.12:6834/3741155090":"2023-11-29T21:41:03.398724+0000","172.21.15.12:6838/3474424840":"2023-11-29T21:46:48.503169+0000","172.21.15.144:6832/126572048":"2023-11-29T21:52:53.611902+0000","172.21.15.12:6838/3741155090":"2023-11-29T21:41:03.398724+0000","172.21.15.12:6834/3474424840":"2023-11-29T21:46:48.503169+0000","172.21.15.12:6834/1551825785":"2023-11-29T21:50:38.562185+0000","172.21.15.12:6836/3009112102":"2023-11-29T21:42:38.424201+0000","172.21.15.12:6834/291731973":"2023-11-29T21:44:08.460474+0000","172.21.15.144:6833/126572048":"2023-11-29T21:52:53.611902+0000","172.21.15.12:6838/291731973":"2023-11-29T21:44:08.460474+0000","172.21.15.12:6836/3299030262":"2023-11-29T21:45:23.480028+0000","172.21.15.12:6839/3009112102":"2023-11-29T21:42:38.424201+0000","172.21.15.12:6839/3299030262":"2023-11-29T21:45:23.480028+0000","172.21.15.12:6838/1551825785":"2023-11-29T21:50:38.562185+0000","172.21.15.12:6836/2705421041":"2023-11-29T21:48:53.535998+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T21:58:06.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 -- 172.21.15.12:0/2311398023 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63080664a0 msgr2=0x7f6308068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.573 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63080664a0 0x7f6308068960 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f63180ae000 tx=0x7f63180b5040 comp rx=0 tx=0).stop 2023-11-28T21:58:06.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 -- 172.21.15.12:0/2311398023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6328156af0 msgr2=0x7f63280ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6328156af0 0x7f63280ca290 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f632000c770 tx=0x7f632000cc30 comp rx=0 tx=0).stop 2023-11-28T21:58:06.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 -- 172.21.15.12:0/2311398023 shutdown_connections 2023-11-28T21:58:06.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f63080664a0 0x7f6308068960 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.574 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6328156af0 0x7f63280ca290 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63281500d0 0x7f63281513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 --2- 172.21.15.12:0/2311398023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f632806d7d0 0x7f63281f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.575 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 -- 172.21.15.12:0/2311398023 >> 172.21.15.12:0/2311398023 conn(0x7f6328066360 msgr2=0x7f63280c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 -- 172.21.15.12:0/2311398023 shutdown_connections 2023-11-28T21:58:06.576 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.568+0000 7f632c8fe640 1 -- 172.21.15.12:0/2311398023 wait complete. 2023-11-28T21:58:06.593 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T21:58:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.732+0000 7fb547fdf640 1 Processor -- start 2023-11-28T21:58:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.732+0000 7fb547fdf640 1 -- start start 2023-11-28T21:58:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb547fdf640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb540151af0 0x7fb5400c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb547fdf640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb540157b70 con 0x7fb54006e300 2023-11-28T21:58:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb547fdf640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb540157e50 con 0x7fb540151ed0 2023-11-28T21:58:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb547fdf640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5400707f0 con 0x7fb540151af0 2023-11-28T21:58:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb54657c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb54006e300 0x7fb540072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60812/0 (socket says 172.21.15.12:60812) 2023-11-28T21:58:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb54657c640 1 -- 172.21.15.12:0/1284932397 learned_addr learned my addr 172.21.15.12:0/1284932397 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:58:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb545d7b640 1 --2- 172.21.15.12:0/1284932397 >> v2:172.21.15.144:3300/0 conn(0x7fb540151af0 0x7fb5400c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb544d79640 1 -- 172.21.15.12:0/1284932397 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462280715 0 0) 0x7fb540157b70 con 0x7fb54006e300 2023-11-28T21:58:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb544d79640 1 -- 172.21.15.12:0/1284932397 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb52c003530 con 0x7fb54006e300 2023-11-28T21:58:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb545d7b640 1 -- 172.21.15.12:0/1284932397 >> v1:172.21.15.144:6789/0 conn(0x7fb540151ed0 legacy=0x7fb5400c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb545d7b640 1 -- 172.21.15.12:0/1284932397 >> v1:172.21.15.12:6789/0 conn(0x7fb54006e300 legacy=0x7fb540072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.736+0000 7fb545d7b640 1 -- 172.21.15.12:0/1284932397 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540158130 con 0x7fb540151af0 2023-11-28T21:58:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb545d7b640 1 --2- 172.21.15.12:0/1284932397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb540151af0 0x7fb5400c7650 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0097c0 tx=0x7fb53c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238f31820dba7d5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb544d79640 1 -- 172.21.15.12:0/1284932397 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0ab600 con 0x7fb540151af0 2023-11-28T21:58:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb544d79640 1 -- 172.21.15.12:0/1284932397 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb53c0a7050 con 0x7fb540151af0 2023-11-28T21:58:06.743 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb544d79640 1 -- 172.21.15.12:0/1284932397 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0ab960 con 0x7fb540151af0 2023-11-28T21:58:06.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb547fdf640 1 -- 172.21.15.12:0/1284932397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb540151af0 msgr2=0x7fb5400c7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb547fdf640 1 --2- 172.21.15.12:0/1284932397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb540151af0 0x7fb5400c7650 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0097c0 tx=0x7fb53c01c480 comp rx=0 tx=0).stop 2023-11-28T21:58:06.744 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb547fdf640 1 -- 172.21.15.12:0/1284932397 shutdown_connections 2023-11-28T21:58:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb547fdf640 1 --2- 172.21.15.12:0/1284932397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb540151af0 0x7fb5400c7650 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb547fdf640 1 -- 172.21.15.12:0/1284932397 >> 172.21.15.12:0/1284932397 conn(0x7fb540066360 msgr2=0x7fb540066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb547fdf640 1 -- 172.21.15.12:0/1284932397 shutdown_connections 2023-11-28T21:58:06.745 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb547fdf640 1 -- 172.21.15.12:0/1284932397 wait complete. 2023-11-28T21:58:06.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb547fdf640 1 Processor -- start 2023-11-28T21:58:06.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.740+0000 7fb547fdf640 1 -- start start 2023-11-28T21:58:06.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb547fdf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54006e300 0x7fb540156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.747 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb547fdf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb540151af0 0x7fb5401f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb547fdf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb540143bc0 0x7fb5401f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb547fdf640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540143070 con 0x7fb54006e300 2023-11-28T21:58:06.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb547fdf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5401422e0 con 0x7fb540151af0 2023-11-28T21:58:06.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb547fdf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb540144210 con 0x7fb540143bc0 2023-11-28T21:58:06.748 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb545d7b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54006e300 0x7fb540156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb545d7b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54006e300 0x7fb540156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37876/0 (socket says 172.21.15.12:37876) 2023-11-28T21:58:06.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb545d7b640 1 -- 172.21.15.12:0/645081362 learned_addr learned my addr 172.21.15.12:0/645081362 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:06.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb54557a640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb540151af0 0x7fb5401f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.749 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb54657c640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb540143bc0 0x7fb5401f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb545d7b640 1 -- 172.21.15.12:0/645081362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb540143bc0 msgr2=0x7fb5401f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb545d7b640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb540143bc0 0x7fb5401f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.750 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb545d7b640 1 -- 172.21.15.12:0/645081362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb540151af0 msgr2=0x7fb5401f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb545d7b640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb540151af0 0x7fb5401f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb545d7b640 1 -- 172.21.15.12:0/645081362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c009420 con 0x7fb54006e300 2023-11-28T21:58:06.751 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb545d7b640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54006e300 0x7fb540156960 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009710 tx=0x7fb53c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb5367fc640 1 -- 172.21.15.12:0/645081362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c003b70 con 0x7fb54006e300 2023-11-28T21:58:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb5367fc640 1 -- 172.21.15.12:0/645081362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb53c003cd0 con 0x7fb54006e300 2023-11-28T21:58:06.753 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb5367fc640 1 -- 172.21.15.12:0/645081362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0ab600 con 0x7fb54006e300 2023-11-28T21:58:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb547fdf640 1 -- 172.21.15.12:0/645081362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb540157870 con 0x7fb54006e300 2023-11-28T21:58:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb547fdf640 1 -- 172.21.15.12:0/645081362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5401427d0 con 0x7fb54006e300 2023-11-28T21:58:06.754 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb547fdf640 1 -- 172.21.15.12:0/645081362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb540074f60 con 0x7fb54006e300 2023-11-28T21:58:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.744+0000 7fb5367fc640 1 -- 172.21.15.12:0/645081362 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb53c0b2040 con 0x7fb54006e300 2023-11-28T21:58:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.748+0000 7fb5367fc640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb524066470 0x7fb524068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:06.755 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.748+0000 7fb5367fc640 1 -- 172.21.15.12:0/645081362 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7523+0+0 (secure 0 0 0) 0x7fb53c11e470 con 0x7fb54006e300 2023-11-28T21:58:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.748+0000 7fb54557a640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb524066470 0x7fb524068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:06.756 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.748+0000 7fb54557a640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb524066470 0x7fb524068930 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb540157800 tx=0x7fb530009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:06.757 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.752+0000 7fb5367fc640 1 -- 172.21.15.12:0/645081362 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb53c0eb1c0 con 0x7fb54006e300 2023-11-28T21:58:06.945 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.940+0000 7fb547fdf640 1 -- 172.21.15.12:0/645081362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb540072c10 con 0x7fb54006e300 2023-11-28T21:58:06.949 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.944+0000 7fb5367fc640 1 -- 172.21.15.12:0/645081362 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb53c0ee7e0 con 0x7fb54006e300 2023-11-28T21:58:06.949 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T21:58:06.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 -- 172.21.15.12:0/645081362 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb524066470 msgr2=0x7fb524068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.950 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb524066470 0x7fb524068930 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb540157800 tx=0x7fb530009040 comp rx=0 tx=0).stop 2023-11-28T21:58:06.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 -- 172.21.15.12:0/645081362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54006e300 msgr2=0x7fb540156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:06.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54006e300 0x7fb540156960 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009710 tx=0x7fb53c019220 comp rx=0 tx=0).stop 2023-11-28T21:58:06.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 -- 172.21.15.12:0/645081362 shutdown_connections 2023-11-28T21:58:06.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fb524066470 0x7fb524068930 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb540143bc0 0x7fb5401f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb540151af0 0x7fb5401f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 --2- 172.21.15.12:0/645081362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb54006e300 0x7fb540156960 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:06.952 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 -- 172.21.15.12:0/645081362 >> 172.21.15.12:0/645081362 conn(0x7fb540066360 msgr2=0x7fb5400c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 -- 172.21.15.12:0/645081362 shutdown_connections 2023-11-28T21:58:06.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:06.948+0000 7fb547fdf640 1 -- 172.21.15.12:0/645081362 wait complete. 2023-11-28T21:58:06.974 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T21:58:07.026 INFO:tasks.ceph.mds.a.smithi012.stdout:starting mds.a at 2023-11-28T21:58:07.047 INFO:tasks.ceph.mds.f.smithi144.stdout:starting mds.f at 2023-11-28T21:58:07.122 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34becfd640 1 Processor -- start 2023-11-28T21:58:07.123 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34becfd640 1 -- start start 2023-11-28T21:58:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34becfd640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f34b806e300 0x7f34b8156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34becfd640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f34b806cf40 0x7f34b8158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34becfd640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b8067410 con 0x7f34b8150c20 2023-11-28T21:58:07.124 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34becfd640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b806e090 con 0x7f34b806e300 2023-11-28T21:58:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34becfd640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b8157f30 con 0x7f34b806cf40 2023-11-28T21:58:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34b7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f34b806e300 0x7f34b8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34b7fff640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f34b806e300 0x7f34b8156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37898/0 (socket says 172.21.15.12:37898) 2023-11-28T21:58:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34b7fff640 1 -- 172.21.15.12:0/885636339 learned_addr learned my addr 172.21.15.12:0/885636339 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34bd29a640 1 --2- 172.21.15.12:0/885636339 >> v2:172.21.15.144:3300/0 conn(0x7f34b806cf40 0x7f34b8158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34b7fff640 1 -- 172.21.15.12:0/885636339 >> v2:172.21.15.144:3300/0 conn(0x7f34b806cf40 msgr2=0x7f34b8158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34b7fff640 1 --2- 172.21.15.12:0/885636339 >> v2:172.21.15.144:3300/0 conn(0x7f34b806cf40 0x7f34b8158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:07.125 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34b7fff640 1 -- 172.21.15.12:0/885636339 >> v1:172.21.15.144:6789/0 conn(0x7f34b8150c20 legacy=0x7f34b80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:07.126 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.120+0000 7f34b7fff640 1 -- 172.21.15.12:0/885636339 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b8151000 con 0x7f34b806e300 2023-11-28T21:58:07.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34b7fff640 1 --2- 172.21.15.12:0/885636339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 0x7f34b8156910 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f34b806b9d0 tx=0x7f34a401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffee624ef73ab07e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:07.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34b77fe640 1 -- 172.21.15.12:0/885636339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40ab610 con 0x7f34b806e300 2023-11-28T21:58:07.127 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34b77fe640 1 -- 172.21.15.12:0/885636339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f34a40ab770 con 0x7f34b806e300 2023-11-28T21:58:07.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34b77fe640 1 -- 172.21.15.12:0/885636339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40abb20 con 0x7f34b806e300 2023-11-28T21:58:07.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34becfd640 1 -- 172.21.15.12:0/885636339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 msgr2=0x7f34b8156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:07.128 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34becfd640 1 --2- 172.21.15.12:0/885636339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 0x7f34b8156910 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f34b806b9d0 tx=0x7f34a401c500 comp rx=0 tx=0).stop 2023-11-28T21:58:07.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34becfd640 1 -- 172.21.15.12:0/885636339 shutdown_connections 2023-11-28T21:58:07.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34becfd640 1 --2- 172.21.15.12:0/885636339 >> v2:172.21.15.144:3300/0 conn(0x7f34b806cf40 0x7f34b8158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:07.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34becfd640 1 --2- 172.21.15.12:0/885636339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 0x7f34b8156910 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:07.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34becfd640 1 -- 172.21.15.12:0/885636339 >> 172.21.15.12:0/885636339 conn(0x7f34b8066360 msgr2=0x7f34b8066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:07.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34becfd640 1 -- 172.21.15.12:0/885636339 shutdown_connections 2023-11-28T21:58:07.129 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34becfd640 1 -- 172.21.15.12:0/885636339 wait complete. 2023-11-28T21:58:07.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.124+0000 7f34becfd640 1 Processor -- start 2023-11-28T21:58:07.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34becfd640 1 -- start start 2023-11-28T21:58:07.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34becfd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b806cf40 0x7f34b814f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:07.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34becfd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 0x7f34b814f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:07.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34becfd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34b8150c20 0x7f34b815fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:07.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34becfd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81610b0 con 0x7f34b806e300 2023-11-28T21:58:07.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34becfd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b815f1d0 con 0x7f34b806cf40 2023-11-28T21:58:07.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34becfd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b815f790 con 0x7f34b8150c20 2023-11-28T21:58:07.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 0x7f34b814f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:07.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 0x7f34b814f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37914/0 (socket says 172.21.15.12:37914) 2023-11-28T21:58:07.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b7fff640 1 -- 172.21.15.12:0/1549777647 learned_addr learned my addr 172.21.15.12:0/1549777647 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34bd29a640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34b8150c20 0x7f34b815fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34bca99640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b806cf40 0x7f34b814f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b7fff640 1 -- 172.21.15.12:0/1549777647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34b8150c20 msgr2=0x7f34b815fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b7fff640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34b8150c20 0x7f34b815fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b7fff640 1 -- 172.21.15.12:0/1549777647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b806cf40 msgr2=0x7f34b814f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b7fff640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b806cf40 0x7f34b814f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b7fff640 1 -- 172.21.15.12:0/1549777647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a40094d0 con 0x7f34b806e300 2023-11-28T21:58:07.133 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34bca99640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b806cf40 0x7f34b814f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:58:07.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b7fff640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 0x7f34b814f6c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f34a4019110 tx=0x7f34a401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:07.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b57fa640 1 -- 172.21.15.12:0/1549777647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40b4800 con 0x7f34b806e300 2023-11-28T21:58:07.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b57fa640 1 -- 172.21.15.12:0/1549777647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f34a40b4980 con 0x7f34b806e300 2023-11-28T21:58:07.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34becfd640 1 -- 172.21.15.12:0/1549777647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b81f3850 con 0x7f34b806e300 2023-11-28T21:58:07.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34b57fa640 1 -- 172.21.15.12:0/1549777647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40b4d50 con 0x7f34b806e300 2023-11-28T21:58:07.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34becfd640 1 -- 172.21.15.12:0/1549777647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b815f420 con 0x7f34b806e300 2023-11-28T21:58:07.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.128+0000 7f34becfd640 1 -- 172.21.15.12:0/1549777647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b815f900 con 0x7f34b806e300 2023-11-28T21:58:07.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.132+0000 7f34b57fa640 1 -- 172.21.15.12:0/1549777647 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f34a40b3380 con 0x7f34b806e300 2023-11-28T21:58:07.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.132+0000 7f34b57fa640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f349c066470 0x7f349c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:07.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.132+0000 7f34b57fa640 1 -- 172.21.15.12:0/1549777647 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7695+0+0 (secure 0 0 0) 0x7f34a411e650 con 0x7f34b806e300 2023-11-28T21:58:07.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.132+0000 7f34bca99640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f349c066470 0x7f349c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:07.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.132+0000 7f34bca99640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f349c066470 0x7f349c068930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f34ac000f70 tx=0x7f34ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:07.140 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.136+0000 7f34b57fa640 1 -- 172.21.15.12:0/1549777647 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f34a40eb3a0 con 0x7f34b806e300 2023-11-28T21:58:07.332 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.328+0000 7f34becfd640 1 -- 172.21.15.12:0/1549777647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f34b8074f60 con 0x7f34b806e300 2023-11-28T21:58:07.951 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34b57fa640 1 -- 172.21.15.12:0/1549777647 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) v1 ==== 104+0+0 (secure 0 0 0) 0x7f34a40ee9c0 con 0x7f34b806e300 2023-11-28T21:58:07.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 -- 172.21.15.12:0/1549777647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f349c066470 msgr2=0x7f349c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:07.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f349c066470 0x7f349c068930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f34ac000f70 tx=0x7f34ac009040 comp rx=0 tx=0).stop 2023-11-28T21:58:07.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 -- 172.21.15.12:0/1549777647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 msgr2=0x7f34b814f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:07.953 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 0x7f34b814f6c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f34a4019110 tx=0x7f34a401ad40 comp rx=0 tx=0).stop 2023-11-28T21:58:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 -- 172.21.15.12:0/1549777647 shutdown_connections 2023-11-28T21:58:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f349c066470 0x7f349c068930 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:07.954 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34b8150c20 0x7f34b815fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:07.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34b806e300 0x7f34b814f6c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:07.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 --2- 172.21.15.12:0/1549777647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34b806cf40 0x7f34b814f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:07.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 -- 172.21.15.12:0/1549777647 >> 172.21.15.12:0/1549777647 conn(0x7f34b8066360 msgr2=0x7f34b80c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:07.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 -- 172.21.15.12:0/1549777647 shutdown_connections 2023-11-28T21:58:07.955 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:07.948+0000 7f34becfd640 1 -- 172.21.15.12:0/1549777647 wait complete. 2023-11-28T21:58:07.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T21:58:08.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5cc9b7640 1 Processor -- start 2023-11-28T21:58:08.130 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5cc9b7640 1 -- start start 2023-11-28T21:58:08.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5cc9b7640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd5c8151b00 0x7fd5c8157a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:08.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5cc9b7640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c80c67a0 con 0x7fd5c80703e0 2023-11-28T21:58:08.131 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5cc9b7640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c806cbd0 con 0x7fd5c8151ee0 2023-11-28T21:58:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5cc9b7640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c8158260 con 0x7fd5c8151b00 2023-11-28T21:58:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c659c640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd5c80703e0 0x7fd5c80c7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60824/0 (socket says 172.21.15.12:60824) 2023-11-28T21:58:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c659c640 1 -- 172.21.15.12:0/3004498380 learned_addr learned my addr 172.21.15.12:0/3004498380 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:58:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c6d9d640 1 --2- 172.21.15.12:0/3004498380 >> v2:172.21.15.144:3300/0 conn(0x7fd5c8151b00 0x7fd5c8157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081027459 0 0) 0x7fd5c80c67a0 con 0x7fd5c80703e0 2023-11-28T21:58:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5b0003530 con 0x7fd5c80703e0 2023-11-28T21:58:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 860666464 0 0) 0x7fd5b0003530 con 0x7fd5c80703e0 2023-11-28T21:58:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 >> v2:172.21.15.144:3300/0 conn(0x7fd5c8151b00 msgr2=0x7fd5c8157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 --2- 172.21.15.12:0/3004498380 >> v2:172.21.15.144:3300/0 conn(0x7fd5c8151b00 0x7fd5c8157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 >> v1:172.21.15.144:6789/0 conn(0x7fd5c8151ee0 legacy=0x7fd5c80c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c8158520 con 0x7fd5c80703e0 2023-11-28T21:58:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2665822762 0 0) 0x7fd5bc001680 con 0x7fd5c80703e0 2023-11-28T21:58:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 >> v1:172.21.15.12:6789/0 conn(0x7fd5c80703e0 legacy=0x7fd5c80c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5b00039d0 0x7fd5b0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5b0005190 0x7fd5b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c8158260 con 0x7fd5b00047d0 2023-11-28T21:58:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b000c070 con 0x7fd5b0005190 2023-11-28T21:58:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b000c230 con 0x7fd5b00039d0 2023-11-28T21:58:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c659c640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c6d9d640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5b00039d0 0x7fd5b0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c759e640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5b0005190 0x7fd5b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c659c640 1 -- 172.21.15.12:0/3004498380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5b00039d0 msgr2=0x7fd5b0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c659c640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5b00039d0 0x7fd5b0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c659c640 1 -- 172.21.15.12:0/3004498380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5b0005190 msgr2=0x7fd5b000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c659c640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5b0005190 0x7fd5b000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c659c640 1 -- 172.21.15.12:0/3004498380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b0003530 con 0x7fd5b00047d0 2023-11-28T21:58:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.128+0000 7fd5c759e640 1 -- 172.21.15.12:0/3004498380 reap_dead start 2023-11-28T21:58:08.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5c659c640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5b0004c50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00c0e0 tx=0x7fd5bc01cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6906b4d4cdf17c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:08.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc005ae0 con 0x7fd5b00047d0 2023-11-28T21:58:08.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd5bc005c40 con 0x7fd5b00047d0 2023-11-28T21:58:08.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5c5d9b640 1 -- 172.21.15.12:0/3004498380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0a5ac0 con 0x7fd5b00047d0 2023-11-28T21:58:08.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/3004498380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 msgr2=0x7fd5b0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:08.138 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5cc9b7640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5b0004c50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00c0e0 tx=0x7fd5bc01cb60 comp rx=0 tx=0).stop 2023-11-28T21:58:08.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/3004498380 shutdown_connections 2023-11-28T21:58:08.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5cc9b7640 1 --2- 172.21.15.12:0/3004498380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5b0004c50 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/3004498380 >> 172.21.15.12:0/3004498380 conn(0x7fd5c80c7cb0 msgr2=0x7fd5c80c80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:08.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/3004498380 shutdown_connections 2023-11-28T21:58:08.139 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.132+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/3004498380 wait complete. 2023-11-28T21:58:08.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5cc9b7640 1 Processor -- start 2023-11-28T21:58:08.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5cc9b7640 1 -- start start 2023-11-28T21:58:08.141 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5cc9b7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5c81efd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:08.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5cc9b7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5c80c26b0 0x7fd5c8156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:08.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5cc9b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c81570c0 0x7fd5c814c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:08.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5cc9b7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c81f3530 con 0x7fd5b00047d0 2023-11-28T21:58:08.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5cc9b7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c814de60 con 0x7fd5c81570c0 2023-11-28T21:58:08.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5cc9b7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c81f0250 con 0x7fd5c80c26b0 2023-11-28T21:58:08.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5c81efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:08.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5c81efd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37924/0 (socket says 172.21.15.12:37924) 2023-11-28T21:58:08.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c6d9d640 1 -- 172.21.15.12:0/2502072112 learned_addr learned my addr 172.21.15.12:0/2502072112 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:08.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c659c640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5c80c26b0 0x7fd5c8156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:08.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c759e640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c81570c0 0x7fd5c814c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:08.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c6d9d640 1 -- 172.21.15.12:0/2502072112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5c80c26b0 msgr2=0x7fd5c8156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:08.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c6d9d640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5c80c26b0 0x7fd5c8156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c6d9d640 1 -- 172.21.15.12:0/2502072112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c81570c0 msgr2=0x7fd5c814c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:08.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c6d9d640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c81570c0 0x7fd5c814c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c6d9d640 1 -- 172.21.15.12:0/2502072112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c80c2bf0 con 0x7fd5b00047d0 2023-11-28T21:58:08.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c659c640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5c80c26b0 0x7fd5c8156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:58:08.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5c6d9d640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5c81efd10 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8011530 tx=0x7fd5b80119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:08.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5af7fe640 1 -- 172.21.15.12:0/2502072112 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8015870 con 0x7fd5b00047d0 2023-11-28T21:58:08.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5af7fe640 1 -- 172.21.15.12:0/2502072112 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd5b80159d0 con 0x7fd5b00047d0 2023-11-28T21:58:08.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5af7fe640 1 -- 172.21.15.12:0/2502072112 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8015d60 con 0x7fd5b00047d0 2023-11-28T21:58:08.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/2502072112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c80c2eb0 con 0x7fd5b00047d0 2023-11-28T21:58:08.144 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.136+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/2502072112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c81f2c00 con 0x7fd5b00047d0 2023-11-28T21:58:08.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.140+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/2502072112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c8074f60 con 0x7fd5b00047d0 2023-11-28T21:58:08.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.140+0000 7fd5af7fe640 1 -- 172.21.15.12:0/2502072112 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd5b801d400 con 0x7fd5b00047d0 2023-11-28T21:58:08.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.140+0000 7fd5af7fe640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5b4066470 0x7fd5b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:08.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.140+0000 7fd5af7fe640 1 -- 172.21.15.12:0/2502072112 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 7695+0+0 (secure 0 0 0) 0x7fd5b8084eb0 con 0x7fd5b00047d0 2023-11-28T21:58:08.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.140+0000 7fd5c659c640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5b4066470 0x7fd5b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:08.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.140+0000 7fd5c659c640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5b4066470 0x7fd5b4068930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0ab000 tx=0x7fd5bc0b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:08.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.144+0000 7fd5af7fe640 1 -- 172.21.15.12:0/2502072112 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5b8051a40 con 0x7fd5b00047d0 2023-11-28T21:58:08.308 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.304+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/2502072112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd5c815bea0 con 0x7fd5b00047d0 2023-11-28T21:58:08.960 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.956+0000 7fd5af7fe640 1 -- 172.21.15.12:0/2502072112 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd5b8055060 con 0x7fd5b00047d0 2023-11-28T21:58:08.960 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2023-11-28T21:58:08.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/2502072112 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5b4066470 msgr2=0x7fd5b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:08.962 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5b4066470 0x7fd5b4068930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0ab000 tx=0x7fd5bc0b1040 comp rx=0 tx=0).stop 2023-11-28T21:58:08.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/2502072112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 msgr2=0x7fd5c81efd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:08.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5c81efd10 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8011530 tx=0x7fd5b80119f0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/2502072112 shutdown_connections 2023-11-28T21:58:08.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7fd5b4066470 0x7fd5b4068930 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c81570c0 0x7fd5c814c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd5c80c26b0 0x7fd5c8156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 --2- 172.21.15.12:0/2502072112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd5b00047d0 0x7fd5c81efd10 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:08.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/2502072112 >> 172.21.15.12:0/2502072112 conn(0x7fd5c80c7cb0 msgr2=0x7fd5c806aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:08.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/2502072112 shutdown_connections 2023-11-28T21:58:08.964 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:08.960+0000 7fd5cc9b7640 1 -- 172.21.15.12:0/2502072112 wait complete. 2023-11-28T21:58:09.021 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T21:58:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.204+0000 7f08075c4640 1 Processor -- start 2023-11-28T21:58:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f08075c4640 1 -- start start 2023-11-28T21:58:09.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f08075c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080015a420 0x7f08000ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:09.211 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f08075c4640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08000727b0 con 0x7f08000ca0a0 2023-11-28T21:58:09.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f08075c4640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0800154480 con 0x7f08001540a0 2023-11-28T21:58:09.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f08075c4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080006d680 con 0x7f080015a420 2023-11-28T21:58:09.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f0804b5f640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f08000ca0a0 0x7f0800159d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60840/0 (socket says 172.21.15.12:60840) 2023-11-28T21:58:09.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f0804b5f640 1 -- 172.21.15.12:0/2936213984 learned_addr learned my addr 172.21.15.12:0/2936213984 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:58:09.212 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f0805b61640 1 --2- 172.21.15.12:0/2936213984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080015a420 0x7f08000ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:09.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f07f7fff640 1 -- 172.21.15.12:0/2936213984 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599967373 0 0) 0x7f08000727b0 con 0x7f08000ca0a0 2023-11-28T21:58:09.213 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f07f7fff640 1 -- 172.21.15.12:0/2936213984 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07ec003530 con 0x7f08000ca0a0 2023-11-28T21:58:09.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f07f7fff640 1 -- 172.21.15.12:0/2936213984 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336249131 0 0) 0x7f0800154480 con 0x7f08001540a0 2023-11-28T21:58:09.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f07f7fff640 1 -- 172.21.15.12:0/2936213984 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08000727b0 con 0x7f08001540a0 2023-11-28T21:58:09.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f0805b61640 1 -- 172.21.15.12:0/2936213984 >> v1:172.21.15.144:6789/0 conn(0x7f08001540a0 legacy=0x7f08000c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:09.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f0805b61640 1 -- 172.21.15.12:0/2936213984 >> v1:172.21.15.12:6789/0 conn(0x7f08000ca0a0 legacy=0x7f0800159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:09.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f07f7fff640 1 -- 172.21.15.12:0/2936213984 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464172689 0 0) 0x7f07ec003530 con 0x7f08000ca0a0 2023-11-28T21:58:09.214 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.208+0000 7f0805b61640 1 -- 172.21.15.12:0/2936213984 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800072a30 con 0x7f080015a420 2023-11-28T21:58:09.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f0805b61640 1 --2- 172.21.15.12:0/2936213984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080015a420 0x7f08000ca4f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009df0 tx=0x7f07f801f080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5055cd8320fa53d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:09.215 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f07f7fff640 1 -- 172.21.15.12:0/2936213984 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0001680 con 0x7f080015a420 2023-11-28T21:58:09.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f07f7fff640 1 -- 172.21.15.12:0/2936213984 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f07f80a8c50 con 0x7f080015a420 2023-11-28T21:58:09.216 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f07f7fff640 1 -- 172.21.15.12:0/2936213984 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f80b0440 con 0x7f080015a420 2023-11-28T21:58:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f08075c4640 1 -- 172.21.15.12:0/2936213984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080015a420 msgr2=0x7f08000ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f08075c4640 1 --2- 172.21.15.12:0/2936213984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080015a420 0x7f08000ca4f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009df0 tx=0x7f07f801f080 comp rx=0 tx=0).stop 2023-11-28T21:58:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f08075c4640 1 -- 172.21.15.12:0/2936213984 shutdown_connections 2023-11-28T21:58:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f08075c4640 1 --2- 172.21.15.12:0/2936213984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f080015a420 0x7f08000ca4f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f08075c4640 1 -- 172.21.15.12:0/2936213984 >> 172.21.15.12:0/2936213984 conn(0x7f0800066360 msgr2=0x7f0800066770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f08075c4640 1 -- 172.21.15.12:0/2936213984 shutdown_connections 2023-11-28T21:58:09.217 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.212+0000 7f08075c4640 1 -- 172.21.15.12:0/2936213984 wait complete. 2023-11-28T21:58:09.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 Processor -- start 2023-11-28T21:58:09.218 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 -- start start 2023-11-28T21:58:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08000ca0a0 0x7f0800156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08001540a0 0x7f0800159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f080015a420 0x7f080015b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0800157770 con 0x7f08000ca0a0 2023-11-28T21:58:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080015d400 con 0x7f080015a420 2023-11-28T21:58:09.220 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0800156ed0 con 0x7f08001540a0 2023-11-28T21:58:09.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805360640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08000ca0a0 0x7f0800156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:09.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805360640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08000ca0a0 0x7f0800156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37934/0 (socket says 172.21.15.12:37934) 2023-11-28T21:58:09.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805360640 1 -- 172.21.15.12:0/3989657226 learned_addr learned my addr 172.21.15.12:0/3989657226 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:09.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805b61640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f080015a420 0x7f080015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:09.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0804b5f640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08001540a0 0x7f0800159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:09.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805360640 1 -- 172.21.15.12:0/3989657226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08001540a0 msgr2=0x7f0800159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:09.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805360640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08001540a0 0x7f0800159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:09.221 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805360640 1 -- 172.21.15.12:0/3989657226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f080015a420 msgr2=0x7f080015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:09.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805360640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f080015a420 0x7f080015b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:09.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805360640 1 -- 172.21.15.12:0/3989657226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8009a70 con 0x7f08000ca0a0 2023-11-28T21:58:09.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0804b5f640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08001540a0 0x7f0800159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:58:09.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805b61640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f080015a420 0x7f080015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:58:09.222 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f0805360640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08000ca0a0 0x7f0800156620 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c4c0 tx=0x7f07fc00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:09.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f07f5ffb640 1 -- 172.21.15.12:0/3989657226 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0052d0 con 0x7f08000ca0a0 2023-11-28T21:58:09.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f07f5ffb640 1 -- 172.21.15.12:0/3989657226 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f07fc005430 con 0x7f08000ca0a0 2023-11-28T21:58:09.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f07f5ffb640 1 -- 172.21.15.12:0/3989657226 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc012890 con 0x7f08000ca0a0 2023-11-28T21:58:09.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 -- 172.21.15.12:0/3989657226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08001571b0 con 0x7f08000ca0a0 2023-11-28T21:58:09.223 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 -- 172.21.15.12:0/3989657226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080015b010 con 0x7f08000ca0a0 2023-11-28T21:58:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.216+0000 7f08075c4640 1 -- 172.21.15.12:0/3989657226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080015ae70 con 0x7f08000ca0a0 2023-11-28T21:58:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.220+0000 7f07f5ffb640 1 -- 172.21.15.12:0/3989657226 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f07fc0129f0 con 0x7f08000ca0a0 2023-11-28T21:58:09.224 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.220+0000 7f07f5ffb640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f07e4066470 0x7f07e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.220+0000 7f07f5ffb640 1 -- 172.21.15.12:0/3989657226 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7257+0+0 (secure 0 0 0) 0x7f07fc0841c0 con 0x7f08000ca0a0 2023-11-28T21:58:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.220+0000 7f0804b5f640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f07e4066470 0x7f07e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:09.225 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.220+0000 7f0804b5f640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f07e4066470 0x7f07e4068930 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f08001ed520 tx=0x7f07f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:09.228 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.224+0000 7f07f5ffb640 1 -- 172.21.15.12:0/3989657226 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f080015ae70 con 0x7f08000ca0a0 2023-11-28T21:58:09.387 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.384+0000 7f08075c4640 1 -- 172.21.15.12:0/3989657226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f080015ae70 con 0x7f08000ca0a0 2023-11-28T21:58:09.963 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f07f5ffb640 1 -- 172.21.15.12:0/3989657226 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) v1 ==== 169+0+0 (secure 0 0 0) 0x7f080015ae70 con 0x7f08000ca0a0 2023-11-28T21:58:09.963 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2023-11-28T21:58:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 -- 172.21.15.12:0/3989657226 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f07e4066470 msgr2=0x7f07e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f07e4066470 0x7f07e4068930 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f08001ed520 tx=0x7f07f0009040 comp rx=0 tx=0).stop 2023-11-28T21:58:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 -- 172.21.15.12:0/3989657226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08000ca0a0 msgr2=0x7f0800156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:09.965 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08000ca0a0 0x7f0800156620 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c4c0 tx=0x7f07fc00c980 comp rx=0 tx=0).stop 2023-11-28T21:58:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 -- 172.21.15.12:0/3989657226 shutdown_connections 2023-11-28T21:58:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f07e4066470 0x7f07e4068930 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f080015a420 0x7f080015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:09.967 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08001540a0 0x7f0800159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:09.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 --2- 172.21.15.12:0/3989657226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08000ca0a0 0x7f0800156620 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:09.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 -- 172.21.15.12:0/3989657226 >> 172.21.15.12:0/3989657226 conn(0x7f0800066360 msgr2=0x7f080006fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:09.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 -- 172.21.15.12:0/3989657226 shutdown_connections 2023-11-28T21:58:09.968 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:09.960+0000 7f08075c4640 1 -- 172.21.15.12:0/3989657226 wait complete. 2023-11-28T21:58:09.993 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-28T21:58:10.142 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042111e640 1 Processor -- start 2023-11-28T21:58:10.143 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042111e640 1 -- start start 2023-11-28T21:58:10.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042111e640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f041c0c9260 0x7f041c0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:10.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042111e640 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f041c159c30 0x7f041c0c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:10.145 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042111e640 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041c0c86b0 con 0x7f041c153ea0 2023-11-28T21:58:10.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042111e640 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c0c8950 con 0x7f041c159c30 2023-11-28T21:58:10.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042111e640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041c0c8c30 con 0x7f041c0c9260 2023-11-28T21:58:10.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f041bfff640 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f041c153ea0 0x7f041c0ca4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:60852/0 (socket says 172.21.15.12:60852) 2023-11-28T21:58:10.146 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f041bfff640 1 -- 172.21.15.12:0/1325400603 learned_addr learned my addr 172.21.15.12:0/1325400603 (peer_addr_for_me v1:172.21.15.12:0/0) 2023-11-28T21:58:10.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042091d640 1 --2- 172.21.15.12:0/1325400603 >> v2:172.21.15.12:3300/0 conn(0x7f041c159c30 0x7f041c0c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:10.147 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f041b7fe640 1 --2- 172.21.15.12:0/1325400603 >> v2:172.21.15.144:3300/0 conn(0x7f041c0c9260 0x7f041c0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:10.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f041affd640 1 -- 172.21.15.12:0/1325400603 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799374445 0 0) 0x7f041c0c86b0 con 0x7f041c153ea0 2023-11-28T21:58:10.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f041affd640 1 -- 172.21.15.12:0/1325400603 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0404003530 con 0x7f041c153ea0 2023-11-28T21:58:10.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042091d640 1 -- 172.21.15.12:0/1325400603 >> v2:172.21.15.144:3300/0 conn(0x7f041c0c9260 msgr2=0x7f041c0c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T21:58:10.148 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042091d640 1 --2- 172.21.15.12:0/1325400603 >> v2:172.21.15.144:3300/0 conn(0x7f041c0c9260 0x7f041c0c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:10.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042091d640 1 -- 172.21.15.12:0/1325400603 >> v1:172.21.15.12:6789/0 conn(0x7f041c153ea0 legacy=0x7f041c0ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:10.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f042091d640 1 -- 172.21.15.12:0/1325400603 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0c8f10 con 0x7f041c159c30 2023-11-28T21:58:10.149 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.140+0000 7f041b7fe640 1 --2- 172.21.15.12:0/1325400603 >> v2:172.21.15.144:3300/0 conn(0x7f041c0c9260 0x7f041c0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:58:10.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f042091d640 1 --2- 172.21.15.12:0/1325400603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 0x7f041c0c9710 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f041c06e9d0 tx=0x7f041401f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8307b51bd1e9be65 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:10.153 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f041affd640 1 -- 172.21.15.12:0/1325400603 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140b0440 con 0x7f041c159c30 2023-11-28T21:58:10.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f041affd640 1 -- 172.21.15.12:0/1325400603 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f04140b05a0 con 0x7f041c159c30 2023-11-28T21:58:10.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f041affd640 1 -- 172.21.15.12:0/1325400603 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140b0950 con 0x7f041c159c30 2023-11-28T21:58:10.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f042111e640 1 -- 172.21.15.12:0/1325400603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 msgr2=0x7f041c0c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:10.154 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f042111e640 1 --2- 172.21.15.12:0/1325400603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 0x7f041c0c9710 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f041c06e9d0 tx=0x7f041401f0f0 comp rx=0 tx=0).stop 2023-11-28T21:58:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f042111e640 1 -- 172.21.15.12:0/1325400603 shutdown_connections 2023-11-28T21:58:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f042111e640 1 --2- 172.21.15.12:0/1325400603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 0x7f041c0c9710 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:10.155 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f042111e640 1 --2- 172.21.15.12:0/1325400603 >> v2:172.21.15.144:3300/0 conn(0x7f041c0c9260 0x7f041c0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f042111e640 1 -- 172.21.15.12:0/1325400603 >> 172.21.15.12:0/1325400603 conn(0x7f041c071d00 msgr2=0x7f041c072110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f042111e640 1 -- 172.21.15.12:0/1325400603 shutdown_connections 2023-11-28T21:58:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f042111e640 1 -- 172.21.15.12:0/1325400603 wait complete. 2023-11-28T21:58:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.144+0000 7f042111e640 1 Processor -- start 2023-11-28T21:58:10.156 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042111e640 1 -- start start 2023-11-28T21:58:10.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042111e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f041c0c9260 0x7f041c1effd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:10.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042111e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c153ea0 0x7f041c156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:10.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042111e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 0x7f041c14c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:10.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042111e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c154fe0 con 0x7f041c159c30 2023-11-28T21:58:10.157 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042111e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c155ec0 con 0x7f041c153ea0 2023-11-28T21:58:10.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042111e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041c154740 con 0x7f041c0c9260 2023-11-28T21:58:10.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042091d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 0x7f041c14c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:10.158 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042091d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 0x7f041c14c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37952/0 (socket says 172.21.15.12:37952) 2023-11-28T21:58:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042091d640 1 -- 172.21.15.12:0/190764866 learned_addr learned my addr 172.21.15.12:0/190764866 (peer_addr_for_me v2:172.21.15.12:0/0) 2023-11-28T21:58:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f041b7fe640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c153ea0 0x7f041c156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f041bfff640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f041c0c9260 0x7f041c1effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042091d640 1 -- 172.21.15.12:0/190764866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f041c0c9260 msgr2=0x7f041c1effd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:10.159 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042091d640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f041c0c9260 0x7f041c1effd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042091d640 1 -- 172.21.15.12:0/190764866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c153ea0 msgr2=0x7f041c156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042091d640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c153ea0 0x7f041c156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042091d640 1 -- 172.21.15.12:0/190764866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0414009a70 con 0x7f041c159c30 2023-11-28T21:58:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f041bfff640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f041c0c9260 0x7f041c1effd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T21:58:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042091d640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 0x7f041c14c8c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f04140b3040 tx=0x7f041401be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:10.160 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f0418ff9640 1 -- 172.21.15.12:0/190764866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140b0440 con 0x7f041c159c30 2023-11-28T21:58:10.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f0418ff9640 1 -- 172.21.15.12:0/190764866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f04140bc860 con 0x7f041c159c30 2023-11-28T21:58:10.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f0418ff9640 1 -- 172.21.15.12:0/190764866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04140bcc10 con 0x7f041c159c30 2023-11-28T21:58:10.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042111e640 1 -- 172.21.15.12:0/190764866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c154a20 con 0x7f041c159c30 2023-11-28T21:58:10.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042111e640 1 -- 172.21.15.12:0/190764866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c14c3a0 con 0x7f041c159c30 2023-11-28T21:58:10.161 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.148+0000 7f042111e640 1 -- 172.21.15.12:0/190764866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c14c200 con 0x7f041c159c30 2023-11-28T21:58:10.162 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.152+0000 7f0418ff9640 1 -- 172.21.15.12:0/190764866 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f04140c6410 con 0x7f041c159c30 2023-11-28T21:58:10.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.152+0000 7f0418ff9640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f04080664a0 0x7f0408068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T21:58:10.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.152+0000 7f0418ff9640 1 -- 172.21.15.12:0/190764866 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f0414122a50 con 0x7f041c159c30 2023-11-28T21:58:10.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.152+0000 7f041bfff640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f04080664a0 0x7f0408068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T21:58:10.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.152+0000 7f041bfff640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f04080664a0 0x7f0408068960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f040c0011f0 tx=0x7f040c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T21:58:10.163 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.156+0000 7f0418ff9640 1 -- 172.21.15.12:0/190764866 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f041c14c200 con 0x7f041c159c30 2023-11-28T21:58:10.318 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.316+0000 7f042111e640 1 -- 172.21.15.12:0/190764866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f041c14c200 con 0x7f041c159c30 2023-11-28T21:58:10.985 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.980+0000 7f0418ff9640 1 -- 172.21.15.12:0/190764866 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v56) v1 ==== 178+0+0 (secure 0 0 0) 0x7f041c14c200 con 0x7f041c159c30 2023-11-28T21:58:10.985 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data_ec' removed 2023-11-28T21:58:10.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 -- 172.21.15.12:0/190764866 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f04080664a0 msgr2=0x7f0408068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:10.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f04080664a0 0x7f0408068960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f040c0011f0 tx=0x7f040c009040 comp rx=0 tx=0).stop 2023-11-28T21:58:10.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 -- 172.21.15.12:0/190764866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 msgr2=0x7f041c14c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T21:58:10.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 0x7f041c14c8c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f04140b3040 tx=0x7f041401be90 comp rx=0 tx=0).stop 2023-11-28T21:58:10.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 -- 172.21.15.12:0/190764866 shutdown_connections 2023-11-28T21:58:10.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.12:6832/3598850103,v1:172.21.15.12:6833/3598850103] conn(0x7f04080664a0 0x7f0408068960 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:10.988 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f041c159c30 0x7f041c14c8c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:10.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f041c153ea0 0x7f041c156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:10.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 --2- 172.21.15.12:0/190764866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f041c0c9260 0x7f041c1effd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T21:58:10.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 -- 172.21.15.12:0/190764866 >> 172.21.15.12:0/190764866 conn(0x7f041c071d00 msgr2=0x7f041c0720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T21:58:10.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 -- 172.21.15.12:0/190764866 shutdown_connections 2023-11-28T21:58:10.989 INFO:teuthology.orchestra.run.smithi012.stderr:2023-11-28T21:58:10.984+0000 7f042111e640 1 -- 172.21.15.12:0/190764866 wait complete. 2023-11-28T21:58:11.012 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T21:58:11.012 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T21:58:11.012 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:11.024 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T21:58:11.025 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T21:58:11.025 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:11.079 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T21:58:11.079 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-11-28T21:58:11.079 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:11.138 INFO:tasks.ceph.mds.e:Stopped 2023-11-28T21:58:11.138 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T21:58:11.139 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:11.152 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T21:58:11.153 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T21:58:11.153 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:11.210 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T21:58:11.210 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-11-28T21:58:11.211 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:11.268 INFO:tasks.ceph.mds.f:Stopped 2023-11-28T21:58:11.269 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T21:58:11.269 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T21:58:11.269 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:13.581 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-11-28T21:58:17.371 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T21:58:17.371 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T21:58:17.372 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:17.456 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T21:58:17.457 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T21:58:17.457 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:17.837 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:58:17.842 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:58:19.589 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-11-28T21:58:19.590 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-11-28T21:58:23.558 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T21:58:23.559 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T21:58:23.559 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:23.643 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T21:58:23.643 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T21:58:23.644 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:23.738 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T21:58:23.738 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T21:58:23.739 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:25.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-11-28T21:58:25.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-11-28T21:58:29.840 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T21:58:29.841 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T21:58:29.841 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:29.928 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T21:58:29.928 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T21:58:29.928 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:30.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-28T21:58:30.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-11-28T21:58:36.030 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T21:58:36.031 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T21:58:36.032 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T21:58:36.032 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:36.092 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T21:58:36.092 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T21:58:36.092 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:36.122 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T21:58:36.122 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T21:58:36.123 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T21:58:36.123 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:36.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~6s 2023-11-28T21:58:36.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-11-28T21:58:36.146 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T21:58:36.147 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T21:58:36.147 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:36.185 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T21:58:36.185 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T21:58:36.185 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T21:58:36.242 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T21:58:36.243 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T21:58:36.243 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v 'Replacing daemon mds' | head -n 1 2023-11-28T21:58:36.284 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:58:36.284 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T21:58:36.516 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:58:36.517 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T21:58:36.725 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:58:36.725 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T21:58:36.874 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:58:36.875 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T21:58:37.039 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:58:37.039 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T21:58:37.279 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:58:37.279 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T21:58:37.486 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:58:37.486 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T21:58:37.678 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:58:37.678 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T21:58:37.845 INFO:tasks.ceph:Archiving mon data... 2023-11-28T21:58:37.847 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703/data/mon.a.tgz 2023-11-28T21:58:37.847 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2023-11-28T21:58:37.857 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.v0XoDLLSmD 2023-11-28T21:58:37.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.v0XoDLLSmD 2023-11-28T21:58:38.007 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.v0XoDLLSmD 2023-11-28T21:58:38.092 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.v0XoDLLSmD is 366KB 2023-11-28T21:58:38.157 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.v0XoDLLSmD 2023-11-28T21:58:38.166 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703/data/mon.b.tgz 2023-11-28T21:58:38.167 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2023-11-28T21:58:38.173 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.sX9imLko5a 2023-11-28T21:58:38.174 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.sX9imLko5a 2023-11-28T21:58:38.305 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.sX9imLko5a 2023-11-28T21:58:38.369 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.sX9imLko5a is 373KB 2023-11-28T21:58:38.431 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.sX9imLko5a 2023-11-28T21:58:38.437 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703/data/mon.c.tgz 2023-11-28T21:58:38.437 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2023-11-28T21:58:38.486 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.xdEjjUqoqJ 2023-11-28T21:58:38.487 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.xdEjjUqoqJ 2023-11-28T21:58:38.617 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.xdEjjUqoqJ 2023-11-28T21:58:38.676 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.xdEjjUqoqJ is 373KB 2023-11-28T21:58:38.735 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.xdEjjUqoqJ 2023-11-28T21:58:38.740 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T21:58:38.740 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T21:58:38.744 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T21:58:38.794 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T21:58:38.794 DEBUG:teuthology.orchestra.run.smithi012:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T21:58:38.809 DEBUG:teuthology.orchestra.run.smithi144:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T21:58:38.844 INFO:teuthology.orchestra.run.smithi012.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T21:58:38.865 INFO:teuthology.orchestra.run.smithi144.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T21:58:38.869 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T21:58:38.871 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703/remote/smithi012/crash 2023-11-28T21:58:38.872 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T21:58:38.920 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703/remote/smithi144/crash 2023-11-28T21:58:38.921 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T21:58:38.933 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T21:58:38.934 DEBUG:teuthology.orchestra.run.smithi012:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:58:38.973 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T21:58:39.022 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T21:58:39.045 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T21:58:39.068 INFO:tasks.ceph:Compressing logs... 2023-11-28T21:58:39.068 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-28T21:58:39.089 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-28T21:58:39.107 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42880.log 2023-11-28T21:58:39.108 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41993.log 2023-11-28T21:58:39.109 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46169.log 2023-11-28T21:58:39.110 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42880.log.gz 2023-11-28T21:58:39.116 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41993.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41993.log.gz 2023-11-28T21:58:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50292.log 2023-11-28T21:58:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44979.log 2023-11-28T21:58:39.117 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46169.log.gz 2023-11-28T21:58:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50634.log 2023-11-28T21:58:39.117 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50292.log.gz 2023-11-28T21:58:39.117 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54728.log 2023-11-28T21:58:39.117 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44979.log.gz 2023-11-28T21:58:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45135.log 2023-11-28T21:58:39.118 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50634.log.gz 2023-11-28T21:58:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52270.log 2023-11-28T21:58:39.118 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54728.log.gz 2023-11-28T21:58:39.118 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45135.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45135.log.gz 2023-11-28T21:58:39.119 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44953.log 2023-11-28T21:58:39.122 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51814.log 2023-11-28T21:58:39.122 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52270.log.gz 2023-11-28T21:58:39.123 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44953.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.51119.log -- replaced with /var/log/ceph/ceph-client.admin.44953.log.gz 2023-11-28T21:58:39.123 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-11-28T21:58:39.123 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45005.log 2023-11-28T21:58:39.123 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51814.log.gz 2023-11-28T21:58:39.123 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51119.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51119.log.gz 2023-11-28T21:58:39.124 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48311.log 2023-11-28T21:58:39.124 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43901.log 2023-11-28T21:58:39.124 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45005.log.gz 2023-11-28T21:58:39.125 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55954.log 2023-11-28T21:58:39.125 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48311.log.gz 2023-11-28T21:58:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55254.log 2023-11-28T21:58:39.126 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43901.log.gz 2023-11-28T21:58:39.126 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49471.log 2023-11-28T21:58:39.126 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55954.log.gz 2023-11-28T21:58:39.127 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55254.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55254.log.gz 2023-11-28T21:58:39.127 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42802.log 2023-11-28T21:58:39.127 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49471.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-28T21:58:39.127 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49471.log.gz 2023-11-28T21:58:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54676.log 2023-11-28T21:58:39.129 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42802.log.gz 2023-11-28T21:58:39.129 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43089.log 2023-11-28T21:58:39.130 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54676.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.54676.log.gz 2023-11-28T21:58:39.130 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53990.log 2023-11-28T21:58:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49630.log 2023-11-28T21:58:39.131 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43089.log.gz 2023-11-28T21:58:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53990.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53990.log.gz 2023-11-28T21:58:39.132 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46225.log 2023-11-28T21:58:39.133 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43700.log 2023-11-28T21:58:39.134 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49630.log.gz 2023-11-28T21:58:39.135 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52303.log 2023-11-28T21:58:39.136 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46225.log.gz 2023-11-28T21:58:39.136 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54049.log 2023-11-28T21:58:39.136 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43700.log.gz 2023-11-28T21:58:39.153 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38741.log 2023-11-28T21:58:39.153 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-28T21:58:39.153 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.38741.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-28T21:58:39.153 INFO:teuthology.orchestra.run.smithi144.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38741.log.gz 2023-11-28T21:58:39.153 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38805.log 2023-11-28T21:58:39.154 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-28T21:58:39.154 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.38805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38853.log 2023-11-28T21:58:39.154 INFO:teuthology.orchestra.run.smithi144.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38805.log.gz 2023-11-28T21:58:39.154 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38925.log 2023-11-28T21:58:39.155 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.38853.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-28T21:58:39.155 INFO:teuthology.orchestra.run.smithi144.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38853.log.gz 2023-11-28T21:58:39.155 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2023-11-28T21:58:39.155 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.38925.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-28T21:58:39.155 INFO:teuthology.orchestra.run.smithi144.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38925.log.gz 2023-11-28T21:58:39.156 INFO:teuthology.orchestra.run.smithi144.stderr:gzip/var/log/ceph/ceph-mds.f.log: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-28T21:58:39.156 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38877.log 2023-11-28T21:58:39.156 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-28T21:58:39.156 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.38877.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38877.log.gz 2023-11-28T21:58:39.156 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38829.log 2023-11-28T21:58:39.156 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.audit.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-28T21:58:39.156 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38901.log 2023-11-28T21:58:39.157 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.38829.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38829.log.gz 2023-11-28T21:58:39.157 INFO:teuthology.orchestra.run.smithi144.stderr: 92.8% -- replaced with /var/log/ceph/ceph.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-28T21:58:39.157 INFO:teuthology.orchestra.run.smithi144.stderr: 2023-11-28T21:58:39.157 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.38901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38781.log 2023-11-28T21:58:39.158 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mds.d.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38901.log.gz 2023-11-28T21:58:39.158 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-28T21:58:39.158 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.38781.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38781.log.gz 2023-11-28T21:58:39.158 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-28T21:58:39.162 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mds.b.log: 94.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-28T21:58:39.162 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-28T21:58:39.163 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mon.b.log: 94.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-28T21:58:39.163 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.4.log: 94.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-28T21:58:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52303.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52303.log.gz 2023-11-28T21:58:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48583.log 2023-11-28T21:58:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52005.log 2023-11-28T21:58:39.165 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54049.log.gz 2023-11-28T21:58:39.165 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53964.log 2023-11-28T21:58:39.166 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48583.log.gz 2023-11-28T21:58:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50240.log 88.3% 2023-11-28T21:58:39.166 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52005.log.gz 2023-11-28T21:58:39.166 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44547.log 2023-11-28T21:58:39.166 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53964.log.gz 2023-11-28T21:58:39.167 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56030.log 2023-11-28T21:58:39.167 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50240.log.gz 2023-11-28T21:58:39.167 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49443.log 2023-11-28T21:58:39.167 INFO:teuthology.orchestra.run.smithi012.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44547.log.gz 2023-11-28T21:58:39.167 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.56030.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56030.log.gz 2023-11-28T21:58:39.167 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52731.log 2023-11-28T21:58:39.167 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53242.log 2023-11-28T21:58:39.168 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49443.log.gz 2023-11-28T21:58:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52731.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52731.log.gz 2023-11-28T21:58:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55711.log 2023-11-28T21:58:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53242.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53242.log.gz 2023-11-28T21:58:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46949.log 2023-11-28T21:58:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55711.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55711.log.gz 2023-11-28T21:58:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50045.log 2023-11-28T21:58:39.168 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46949.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46949.log.gz 2023-11-28T21:58:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50396.log 2023-11-28T21:58:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50045.log: gzip -5 88.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.50045.log.gz 2023-11-28T21:58:39.169 INFO:teuthology.orchestra.run.smithi012.stderr: -- /var/log/ceph/ceph-client.admin.55347.log 2023-11-28T21:58:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50396.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50396.log.gz 2023-11-28T21:58:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-28T21:58:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55347.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55347.log.gz 2023-11-28T21:58:39.169 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55926.log 2023-11-28T21:58:39.170 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51336.log 2023-11-28T21:58:39.170 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55926.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55926.log.gz 2023-11-28T21:58:39.170 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50123.log 2023-11-28T21:58:39.170 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51336.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51336.log.gz 2023-11-28T21:58:39.170 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50797.log 2023-11-28T21:58:39.170 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50123.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50123.log.gz 2023-11-28T21:58:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46647.log 2023-11-28T21:58:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50797.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50797.log.gz 2023-11-28T21:58:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55737.log 2023-11-28T21:58:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46647.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46647.log.gz 2023-11-28T21:58:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44495.log 2023-11-28T21:58:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55737.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55737.log.gz 2023-11-28T21:58:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44443.log 2023-11-28T21:58:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44495.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44495.log.gz 2023-11-28T21:58:39.171 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50209.log 2023-11-28T21:58:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44443.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44443.log.gz 2023-11-28T21:58:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43485.log 2023-11-28T21:58:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50209.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50209.log.gz 2023-11-28T21:58:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50370.log 2023-11-28T21:58:39.172 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43485.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43485.log.gz 2023-11-28T21:58:39.173 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46905.log 2023-11-28T21:58:39.183 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50370.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50370.log.gz 2023-11-28T21:58:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53400.log 2023-11-28T21:58:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46905.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46905.log.gz 2023-11-28T21:58:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52572.log 2023-11-28T21:58:39.184 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53400.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53400.log.gz 2023-11-28T21:58:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49966.log 2023-11-28T21:58:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52572.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52572.log.gz 2023-11-28T21:58:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50932.log 2023-11-28T21:58:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49966.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49966.log.gz 2023-11-28T21:58:39.185 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54624.log 2023-11-28T21:58:39.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50932.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50932.log.gz 2023-11-28T21:58:39.186 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43146.log 2023-11-28T21:58:39.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54624.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54624.log.gz 2023-11-28T21:58:39.186 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43201.log 2023-11-28T21:58:39.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43146.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43146.log.gz 2023-11-28T21:58:39.187 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55016.log 2023-11-28T21:58:39.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43201.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43201.log.gz 2023-11-28T21:58:39.187 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41957.log 2023-11-28T21:58:39.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55016.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55016.log.gz 2023-11-28T21:58:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51197.log 2023-11-28T21:58:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.41957.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41957.log.gz 2023-11-28T21:58:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48451.log 2023-11-28T21:58:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51197.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51197.log.gz 2023-11-28T21:58:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42362.log 2023-11-28T21:58:39.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48451.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48451.log.gz 2023-11-28T21:58:39.189 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43480.log 2023-11-28T21:58:39.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42362.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42362.log.gz 2023-11-28T21:58:39.190 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56082.log 2023-11-28T21:58:39.190 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43480.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43480.log.gz 2023-11-28T21:58:39.190 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46104.log 2023-11-28T21:58:39.191 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.56082.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56082.log.gz 2023-11-28T21:58:39.191 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54808.log 2023-11-28T21:58:39.197 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46104.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46104.log.gz 2023-11-28T21:58:39.198 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45586.log 2023-11-28T21:58:39.198 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54808.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54808.log.gz 2023-11-28T21:58:39.198 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45083.log 2023-11-28T21:58:39.198 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45586.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45586.log.gz 2023-11-28T21:58:39.199 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51171.log 2023-11-28T21:58:39.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45083.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45083.log.gz 2023-11-28T21:58:39.199 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51918.log 2023-11-28T21:58:39.199 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51171.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51171.log.gz 2023-11-28T21:58:39.199 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47256.log 2023-11-28T21:58:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51918.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51918.log.gz 2023-11-28T21:58:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50071.log 2023-11-28T21:58:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47256.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47256.log.gz 2023-11-28T21:58:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44365.log 2023-11-28T21:58:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50071.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50071.log.gz 2023-11-28T21:58:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54912.log 2023-11-28T21:58:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44365.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44365.log.gz 2023-11-28T21:58:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51013.log 2023-11-28T21:58:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54912.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54912.log.gz 2023-11-28T21:58:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53268.log 2023-11-28T21:58:39.202 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51013.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51013.log.gz 2023-11-28T21:58:39.202 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48609.log 2023-11-28T21:58:39.202 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53268.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53268.log.gz 2023-11-28T21:58:39.202 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-28T21:58:39.202 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48609.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48609.log.gz 2023-11-28T21:58:39.202 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49331.log 2023-11-28T21:58:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43689.log 2023-11-28T21:58:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49331.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49331.log.gz 2023-11-28T21:58:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56108.log 2023-11-28T21:58:39.205 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55685.log 2023-11-28T21:58:39.205 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43689.log.gz 2023-11-28T21:58:39.206 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.56108.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56108.log.gz 2023-11-28T21:58:39.206 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55425.log 2023-11-28T21:58:39.206 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55685.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55685.log.gz 2023-11-28T21:58:39.206 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44647.log 2023-11-28T21:58:39.207 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52837.log 2023-11-28T21:58:39.207 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55425.log.gz 2023-11-28T21:58:39.208 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44849.log 2023-11-28T21:58:39.208 INFO:teuthology.orchestra.run.smithi012.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44647.log.gz 2023-11-28T21:58:39.209 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51225.log 2023-11-28T21:58:39.209 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52837.log.gz 2023-11-28T21:58:39.210 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44849.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44849.log.gz 2023-11-28T21:58:39.210 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55228.log 2023-11-28T21:58:39.212 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51225.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51225.log.gz 2023-11-28T21:58:39.212 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52783.log 2023-11-28T21:58:39.213 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44709.log 2023-11-28T21:58:39.214 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55228.log.gz 2023-11-28T21:58:39.215 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52783.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52783.log.gz 2023-11-28T21:58:39.215 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51362.log 2023-11-28T21:58:39.216 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44709.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44709.log.gz 2023-11-28T21:58:39.216 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2023-11-28T21:58:39.217 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51362.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51362.log.gz 2023-11-28T21:58:39.218 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53936.log 2023-11-28T21:58:39.219 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44206.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2023-11-28T21:58:39.219 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2023-11-28T21:58:39.220 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49552.log 2023-11-28T21:58:39.220 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53936.log.gz 2023-11-28T21:58:39.222 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51840.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2023-11-28T21:58:39.223 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44829.log 2023-11-28T21:58:39.224 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49552.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49552.log.gz 2023-11-28T21:58:39.224 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54780.log 2023-11-28T21:58:39.225 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.0.44829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50019.log 2023-11-28T21:58:39.226 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44829.log.gz 2023-11-28T21:58:39.227 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44927.log 2023-11-28T21:58:39.227 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54780.log.gz 2023-11-28T21:58:39.227 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50019.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50019.log.gz 2023-11-28T21:58:39.228 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52031.log 2023-11-28T21:58:39.228 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44927.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-28T21:58:39.229 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44927.log.gz 2023-11-28T21:58:39.229 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52031.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52031.log.gz 2023-11-28T21:58:39.230 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54101.log 2023-11-28T21:58:39.231 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33295.log 2023-11-28T21:58:39.233 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45650.log 2023-11-28T21:58:39.233 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54101.log.gz 2023-11-28T21:58:39.234 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.tmp-client.admin.33295.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33295.log.gz 2023-11-28T21:58:39.234 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52057.log 2023-11-28T21:58:39.236 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51522.log 88.7% 2023-11-28T21:58:39.236 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45650.log.gz 2023-11-28T21:58:39.237 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51277.log 2023-11-28T21:58:39.238 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52057.log.gz 2023-11-28T21:58:39.239 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50854.log 2023-11-28T21:58:39.239 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51522.log.gz 2023-11-28T21:58:39.241 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51277.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.51277.log.gz --verbose 2023-11-28T21:58:39.241 INFO:teuthology.orchestra.run.smithi012.stderr: -- /var/log/ceph/ceph-client.admin.43902.log 2023-11-28T21:58:39.242 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45770.log 2023-11-28T21:58:39.242 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50854.log.gz 2023-11-28T21:58:39.243 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51685.log 2023-11-28T21:58:39.243 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43902.log.gz 2023-11-28T21:58:39.245 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55202.log 2023-11-28T21:58:39.245 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45770.log.gz 2023-11-28T21:58:39.246 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45161.log 2023-11-28T21:58:39.247 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51685.log.gz 2023-11-28T21:58:39.248 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43912.log 2023-11-28T21:58:39.248 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55202.log.gz 2023-11-28T21:58:39.249 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45161.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45161.log.gz 2023-11-28T21:58:39.249 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52705.log 2023-11-28T21:58:39.251 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52244.log 2023-11-28T21:58:39.251 INFO:teuthology.orchestra.run.smithi012.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43912.log.gz 2023-11-28T21:58:39.252 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42494.log 2023-11-28T21:58:39.252 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52705.log.gz 2023-11-28T21:58:39.254 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55607.log 2023-11-28T21:58:39.254 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52244.log.gz 2023-11-28T21:58:39.255 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42494.log: gzip -5 --verbose 88.6% -- /var/log/ceph/ceph-client.admin.51762.log -- replaced with /var/log/ceph/ceph-client.admin.42494.log.gz 2023-11-28T21:58:39.255 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-11-28T21:58:39.257 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54598.log 2023-11-28T21:58:39.257 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55607.log.gz 2023-11-28T21:58:39.258 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43942.log 2023-11-28T21:58:39.258 INFO:teuthology.orchestra.run.smithi012.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51762.log.gz 2023-11-28T21:58:39.259 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54598.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54598.log.gz 2023-11-28T21:58:39.260 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45057.log 2023-11-28T21:58:39.261 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43942.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.43942.log.gz 2023-11-28T21:58:39.261 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44258.log 2023-11-28T21:58:39.263 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52918.log 2023-11-28T21:58:39.263 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45057.log.gz 2023-11-28T21:58:39.264 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55763.log 2023-11-28T21:58:39.264 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44258.log.gz 2023-11-28T21:58:39.265 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54367.log 2023-11-28T21:58:39.266 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52918.log.gz 2023-11-28T21:58:39.271 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55763.log: gzip 87.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.55763.log.gz -- 2023-11-28T21:58:39.272 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.51788.log 2023-11-28T21:58:39.272 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54367.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54367.log.gz 2023-11-28T21:58:39.272 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50422.log 2023-11-28T21:58:39.272 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43286.log 2023-11-28T21:58:39.272 INFO:teuthology.orchestra.run.smithi012.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51788.log.gz 2023-11-28T21:58:39.272 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43498.log 2023-11-28T21:58:39.273 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50422.log.gz 2023-11-28T21:58:39.273 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46287.log 2023-11-28T21:58:39.274 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43286.log.gz 2023-11-28T21:58:39.274 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43913.log 2023-11-28T21:58:39.275 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43498.log.gz 2023-11-28T21:58:39.276 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43894.log 2023-11-28T21:58:39.276 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46287.log.gz 2023-11-28T21:58:39.277 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43913.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43913.log.gz 2023-11-28T21:58:39.277 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55373.log 2023-11-28T21:58:39.279 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52513.log 2023-11-28T21:58:39.279 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43894.log.gz 2023-11-28T21:58:39.280 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43120.log 2023-11-28T21:58:39.280 INFO:teuthology.orchestra.run.smithi012.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55373.log.gz 2023-11-28T21:58:39.281 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51602.log 2023-11-28T21:58:39.282 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52513.log.gz 2023-11-28T21:58:39.283 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43120.log: gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.43120.log.gz /var/log/ceph/ceph-client.admin.50712.log 2023-11-28T21:58:39.283 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-11-28T21:58:39.284 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46799.log 2023-11-28T21:58:39.285 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51602.log.gz 2023-11-28T21:58:39.286 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42854.log 2023-11-28T21:58:39.286 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50712.log.gz 2023-11-28T21:58:39.287 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51866.log 2023-11-28T21:58:39.287 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46799.log.gz 2023-11-28T21:58:39.289 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48285.log 2023-11-28T21:58:39.289 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42854.log.gz 2023-11-28T21:58:39.290 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56153.log 2023-11-28T21:58:39.291 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51866.log.gz 2023-11-28T21:58:39.292 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48233.log 2023-11-28T21:58:39.292 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48285.log.gz 2023-11-28T21:58:39.293 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.56153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53530.log 2023-11-28T21:58:39.293 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56153.log.gz 2023-11-28T21:58:39.295 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55529.log 2023-11-28T21:58:39.295 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48233.log.gz 2023-11-28T21:58:39.296 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53164.log 2023-11-28T21:58:39.296 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53530.log.gz 2023-11-28T21:58:39.297 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55529.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.47230.log -- replaced with /var/log/ceph/ceph-client.admin.55529.log.gz 2023-11-28T21:58:39.297 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-11-28T21:58:39.299 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54964.log 2023-11-28T21:58:39.299 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53164.log.gz 2023-11-28T21:58:39.300 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46344.log 2023-11-28T21:58:39.300 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47230.log.gz 2023-11-28T21:58:39.302 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48259.log 2023-11-28T21:58:39.302 INFO:teuthology.orchestra.run.smithi012.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54964.log.gz 2023-11-28T21:58:39.303 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47001.log 2023-11-28T21:58:39.304 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46344.log.gz 2023-11-28T21:58:39.305 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48259.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.48259.log.gz -5 2023-11-28T21:58:39.305 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42108.log 2023-11-28T21:58:39.306 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47001.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47001.log.gz 2023-11-28T21:58:39.306 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43037.log 2023-11-28T21:58:39.308 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52192.log 2023-11-28T21:58:39.308 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42108.log.gz 2023-11-28T21:58:39.309 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43491.log 2023-11-28T21:58:39.309 INFO:teuthology.orchestra.run.smithi012.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43037.log.gz 2023-11-28T21:58:39.311 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50474.log 2023-11-28T21:58:39.311 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52192.log.gz 2023-11-28T21:58:39.312 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51654.log 2023-11-28T21:58:39.313 INFO:teuthology.orchestra.run.smithi012.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43491.log.gz 2023-11-28T21:58:39.314 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50474.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50474.log.gzgzip 2023-11-28T21:58:39.314 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45291.log 2023-11-28T21:58:39.315 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43392.log 2023-11-28T21:58:39.315 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51654.log.gz 2023-11-28T21:58:39.317 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50156.log 2023-11-28T21:58:39.317 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45291.log.gz 2023-11-28T21:58:39.318 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50608.log 2023-11-28T21:58:39.318 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43392.log.gz 2023-11-28T21:58:39.319 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53478.log 2023-11-28T21:58:39.320 INFO:teuthology.orchestra.run.smithi012.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50156.log.gz 2023-11-28T21:58:39.321 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54754.log 2023-11-28T21:58:39.322 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50608.log.gz 2023-11-28T21:58:39.322 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50344.log 2023-11-28T21:58:39.323 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53478.log.gz 2023-11-28T21:58:39.324 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43838.log 2023-11-28T21:58:39.324 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54754.log.gz 2023-11-28T21:58:39.325 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44154.log 2023-11-28T21:58:39.326 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50344.log.gz 2023-11-28T21:58:39.327 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54313.log 2023-11-28T21:58:39.328 INFO:teuthology.orchestra.run.smithi012.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43838.log.gz 2023-11-28T21:58:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44154.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-28T21:58:39.331 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44154.log.gz 2023-11-28T21:58:39.331 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53882.log 2023-11-28T21:58:39.331 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54313.log.gz 2023-11-28T21:58:39.332 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52809.log 2023-11-28T21:58:39.333 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55451.log 2023-11-28T21:58:39.333 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53882.log.gz 2023-11-28T21:58:39.334 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51044.log 2023-11-28T21:58:39.335 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52809.log.gz 2023-11-28T21:58:39.336 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55451.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55451.log.gz 2023-11-28T21:58:39.336 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43698.log 2023-11-28T21:58:39.337 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51044.log: 91.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-28T21:58:39.338 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51044.log.gz 2023-11-28T21:58:39.338 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54261.log 2023-11-28T21:58:39.339 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52218.log 2023-11-28T21:58:39.339 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43698.log.gz 2023-11-28T21:58:39.340 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54261.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54261.log.gz 2023-11-28T21:58:39.340 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49384.log 2023-11-28T21:58:39.341 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52218.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52218.log.gz 2023-11-28T21:58:39.342 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50505.log 2023-11-28T21:58:39.343 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49384.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49384.log.gz 2023-11-28T21:58:39.343 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52086.log 2023-11-28T21:58:39.344 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52461.log 2023-11-28T21:58:39.345 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50505.log.gz 2023-11-28T21:58:39.345 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52086.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52086.log.gz 2023-11-28T21:58:39.346 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50959.log 2023-11-28T21:58:39.347 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52461.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52461.log.gz 2023-11-28T21:58:39.347 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43917.log 2023-11-28T21:58:39.349 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50959.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50959.log.gz 2023-11-28T21:58:39.349 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53831.log 2023-11-28T21:58:39.350 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43917.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43917.log.gz 2023-11-28T21:58:39.351 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54208.log 2023-11-28T21:58:39.352 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53831.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53831.log.gz 2023-11-28T21:58:39.352 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55633.log 2023-11-28T21:58:39.353 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54208.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54208.log.gz -5 2023-11-28T21:58:39.354 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43234.log 2023-11-28T21:58:39.355 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50097.log 2023-11-28T21:58:39.355 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55633.log.gz 2023-11-28T21:58:39.357 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56056.log 2023-11-28T21:58:39.357 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43234.log.gz 2023-11-28T21:58:39.358 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50097.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50097.log.gz 2023-11-28T21:58:39.358 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44102.log 2023-11-28T21:58:39.359 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.56056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42280.log 2023-11-28T21:58:39.360 INFO:teuthology.orchestra.run.smithi012.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56056.log.gz 2023-11-28T21:58:39.361 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44102.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44102.log.gz 2023-11-28T21:58:39.361 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47027.log 2023-11-28T21:58:39.362 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53692.log 2023-11-28T21:58:39.362 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42280.log.gz 2023-11-28T21:58:39.363 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47027.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47027.log.gz 2023-11-28T21:58:39.363 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46567.log 2023-11-28T21:58:39.364 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49500.log 2023-11-28T21:58:39.365 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53692.log.gz 2023-11-28T21:58:39.365 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46567.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46567.log.gz 2023-11-28T21:58:39.366 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54450.log 2023-11-28T21:58:39.367 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55978.log 2023-11-28T21:58:39.367 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49500.log.gz 2023-11-28T21:58:39.368 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54450.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54450.log.gz 2023-11-28T21:58:39.368 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55503.log 2023-11-28T21:58:39.369 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51251.log 2023-11-28T21:58:39.370 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55978.log.gz 2023-11-28T21:58:39.371 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55503.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55503.log.gz 2023-11-28T21:58:39.371 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43338.log 2023-11-28T21:58:39.372 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51251.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-28T21:58:39.372 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51251.log.gz 2023-11-28T21:58:39.373 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43338.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43338.log.gz 2023-11-28T21:58:39.373 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47152.log 2023-11-28T21:58:39.374 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55150.log 2023-11-28T21:58:39.375 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47152.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47152.log.gz 2023-11-28T21:58:39.376 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47282.log 2023-11-28T21:58:39.377 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51972.log 2023-11-28T21:58:39.378 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55150.log.gz 2023-11-28T21:58:39.379 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47282.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47282.log.gz 2023-11-28T21:58:39.379 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50582.log 2023-11-28T21:58:39.380 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45974.log 2023-11-28T21:58:39.381 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51972.log.gz 2023-11-28T21:58:39.382 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50582.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50582.log.gz 2023-11-28T21:58:39.382 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49992.log 2023-11-28T21:58:39.384 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45974.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45974.log.gz 2023-11-28T21:58:39.384 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56004.log 2023-11-28T21:58:39.386 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49992.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49992.log.gz 2023-11-28T21:58:39.386 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48337.log 2023-11-28T21:58:39.387 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.56004.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56004.log.gz 2023-11-28T21:58:39.388 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46482.log 2023-11-28T21:58:39.388 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50318.log 2023-11-28T21:58:39.389 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48337.log.gz 2023-11-28T21:58:39.391 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46482.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.46482.log.gz -5 2023-11-28T21:58:39.391 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43172.log 2023-11-28T21:58:39.392 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54182.log 2023-11-28T21:58:39.392 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50318.log.gz 2023-11-28T21:58:39.393 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43172.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43172.log.gz 2023-11-28T21:58:39.393 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47204.log 2023-11-28T21:58:39.394 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52539.log 2023-11-28T21:58:39.394 INFO:teuthology.orchestra.run.smithi012.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54182.log.gz 2023-11-28T21:58:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49604.log 2023-11-28T21:58:39.396 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47204.log.gz 2023-11-28T21:58:39.396 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52539.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52539.log.gz 2023-11-28T21:58:39.397 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55176.log 2023-11-28T21:58:39.398 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55042.log 2023-11-28T21:58:39.399 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49604.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49604.log.gz 2023-11-28T21:58:39.400 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55176.log: gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.55176.log.gz -- 2023-11-28T21:58:39.400 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.48635.log 2023-11-28T21:58:39.401 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43011.log 2023-11-28T21:58:39.401 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55042.log.gz 2023-11-28T21:58:39.403 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48635.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.50556.log -- replaced with /var/log/ceph/ceph-client.admin.48635.log.gz 2023-11-28T21:58:39.403 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-11-28T21:58:39.404 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53348.log 2023-11-28T21:58:39.404 INFO:teuthology.orchestra.run.smithi012.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43011.log.gz 2023-11-28T21:58:39.405 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50556.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50556.log.gz 2023-11-28T21:58:39.406 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43260.log 2023-11-28T21:58:39.407 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53348.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.53348.log.gz -5 2023-11-28T21:58:39.407 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48370.log 2023-11-28T21:58:39.410 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45109.log 2023-11-28T21:58:39.410 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43260.log.gz 2023-11-28T21:58:39.410 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48370.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48370.log.gz 2023-11-28T21:58:39.411 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51628.log 2023-11-28T21:58:39.412 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45109.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45109.log.gz 2023-11-28T21:58:39.412 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45265.log 2023-11-28T21:58:39.414 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51628.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51628.log.gz 2023-11-28T21:58:39.415 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42520.log 2023-11-28T21:58:39.415 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45265.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45265.log.gz 2023-11-28T21:58:39.416 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53582.log 2023-11-28T21:58:39.418 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42520.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42520.log.gz 2023-11-28T21:58:39.418 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54886.log 2023-11-28T21:58:39.419 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53582.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53582.log.gz 2023-11-28T21:58:39.419 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52863.log 2023-11-28T21:58:39.421 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54886.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54886.log.gz 2023-11-28T21:58:39.421 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38321.log 2023-11-28T21:58:39.423 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52863.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52863.log.gz 2023-11-28T21:58:39.423 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52349.log 2023-11-28T21:58:39.424 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38321.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38321.log.gz 2023-11-28T21:58:39.424 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44286.log 2023-11-28T21:58:39.425 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52349.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52349.log.gz 2023-11-28T21:58:39.426 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55581.log 2023-11-28T21:58:39.428 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44286.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44286.log.gz 2023-11-28T21:58:39.428 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49686.log 2023-11-28T21:58:39.429 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55581.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55581.log.gz 2023-11-28T21:58:39.429 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44128.log 2023-11-28T21:58:39.431 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49686.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49686.log.gz 2023-11-28T21:58:39.431 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47490.log 2023-11-28T21:58:39.432 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44901.log 2023-11-28T21:58:39.433 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2023-11-28T21:58:39.434 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47490.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.47490.log.gz -5 2023-11-28T21:58:39.434 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55073.log 2023-11-28T21:58:39.436 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43312.log 2023-11-28T21:58:39.436 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44901.log.gz 2023-11-28T21:58:39.437 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55073.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55073.log.gz 2023-11-28T21:58:39.437 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46605.log 2023-11-28T21:58:39.438 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51731.log 2023-11-28T21:58:39.438 INFO:teuthology.orchestra.run.smithi012.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43312.log.gz 2023-11-28T21:58:39.439 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46605.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46605.log.gz 2023-11-28T21:58:39.440 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53216.log 2023-11-28T21:58:39.441 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51731.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51731.log.gz 2023-11-28T21:58:39.441 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48505.log 2023-11-28T21:58:39.443 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42932.log 2023-11-28T21:58:39.443 INFO:teuthology.orchestra.run.smithi012.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53216.log.gz 2023-11-28T21:58:39.444 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43918.log 2023-11-28T21:58:39.444 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48505.log.gz 2023-11-28T21:58:39.445 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42932.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42932.log.gz 2023-11-28T21:58:39.445 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44875.log 2023-11-28T21:58:39.447 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2023-11-28T21:58:39.447 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43918.log.gz 2023-11-28T21:58:39.448 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45676.log 2023-11-28T21:58:39.448 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44875.log.gz 2023-11-28T21:58:39.449 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42390.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2023-11-28T21:58:39.450 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-28T21:58:39.451 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49734.log 2023-11-28T21:58:39.451 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45676.log.gz 2023-11-28T21:58:39.453 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43482.log 2023-11-28T21:58:39.454 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49734.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49734.log.gz 2023-11-28T21:58:39.455 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43482.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43482.log.gz 2023-11-28T21:58:39.456 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52487.log 2023-11-28T21:58:39.456 INFO:teuthology.orchestra.run.smithi012.stderr: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-28T21:58:39.457 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42160.log 2023-11-28T21:58:39.458 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52487.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52487.log.gz 2023-11-28T21:58:39.459 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52598.log 2023-11-28T21:58:39.460 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42160.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42160.log.gz 2023-11-28T21:58:39.461 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45817.log 2023-11-28T21:58:39.462 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52598.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52598.log.gz 2023-11-28T21:58:39.462 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46758.log 2023-11-28T21:58:39.464 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45817.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45817.log.gz 2023-11-28T21:58:39.464 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46975.log 2023-11-28T21:58:39.465 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46758.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46758.log.gz 2023-11-28T21:58:39.465 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43492.log 2023-11-28T21:58:39.466 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46975.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46975.log.gz 2023-11-28T21:58:39.467 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52653.log 2023-11-28T21:58:39.469 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52138.log 2023-11-28T21:58:39.469 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43492.log.gz 2023-11-28T21:58:39.469 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54393.log 2023-11-28T21:58:39.470 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52653.log.gz 2023-11-28T21:58:39.471 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52138.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52138.log.gz -5 2023-11-28T21:58:39.471 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55321.log 2023-11-28T21:58:39.473 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49305.log 2023-11-28T21:58:39.473 INFO:teuthology.orchestra.run.smithi012.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54393.log.gz 2023-11-28T21:58:39.474 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55321.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55321.log.gz 2023-11-28T21:58:39.474 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56179.log 2023-11-28T21:58:39.476 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55841.log 2023-11-28T21:58:39.476 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49305.log.gz 2023-11-28T21:58:39.477 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.56179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55789.log 2023-11-28T21:58:39.477 INFO:teuthology.orchestra.run.smithi012.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56179.log.gz 2023-11-28T21:58:39.478 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55841.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55841.log.gz 2023-11-28T21:58:39.479 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51548.log 2023-11-28T21:58:39.480 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55789.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55789.log.gz 2023-11-28T21:58:39.481 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42468.log 2023-11-28T21:58:39.482 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47072.log 2023-11-28T21:58:39.482 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51548.log.gz 2023-11-28T21:58:39.484 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48422.log 2023-11-28T21:58:39.484 INFO:teuthology.orchestra.run.smithi012.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42468.log.gz 2023-11-28T21:58:39.485 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47072.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47072.log.gz -5 2023-11-28T21:58:39.486 INFO:teuthology.orchestra.run.smithi012.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49526.log 2023-11-28T21:58:39.487 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48477.log 2023-11-28T21:58:39.487 INFO:teuthology.orchestra.run.smithi012.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48422.log.gz 2023-11-28T21:58:39.488 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49526.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49526.log.gz 2023-11-28T21:58:39.488 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51470.log 2023-11-28T21:58:39.490 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55869.log 2023-11-28T21:58:39.490 INFO:teuthology.orchestra.run.smithi012.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48477.log.gz 2023-11-28T21:58:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51470.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51470.log.gz 2023-11-28T21:58:39.491 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42546.log 2023-11-28T21:58:39.492 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55869.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55869.log.gz 2023-11-28T21:58:39.493 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52381.log 2023-11-28T21:58:39.494 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54834.log 2023-11-28T21:58:39.495 INFO:teuthology.orchestra.run.smithi012.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42546.log.gz 2023-11-28T21:58:39.496 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52381.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52381.log.gz 2023-11-28T21:58:39.496 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53634.log 2023-11-28T21:58:39.497 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54834.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54834.log.gz 2023-11-28T21:58:39.498 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54545.log 2023-11-28T21:58:39.499 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53634.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53634.log.gz 2023-11-28T21:58:39.499 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42336.log 2023-11-28T21:58:39.500 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52967.log 2023-11-28T21:58:39.501 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54545.log.gz 2023-11-28T21:58:39.502 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42336.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.42336.log.gz 2023-11-28T21:58:39.502 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53556.log 2023-11-28T21:58:39.504 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46707.log 2023-11-28T21:58:39.504 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52967.log.gz 2023-11-28T21:58:39.504 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53556.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53556.log.gz 2023-11-28T21:58:39.505 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50745.log 2023-11-28T21:58:39.506 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42242.log 2023-11-28T21:58:39.507 INFO:teuthology.orchestra.run.smithi012.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46707.log.gz 2023-11-28T21:58:39.508 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50745.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50745.log.gz 2023-11-28T21:58:39.508 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54493.log 2023-11-28T21:58:39.509 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38102.log 2023-11-28T21:58:39.509 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42242.log.gz 2023-11-28T21:58:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54493.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54493.log.gz 2023-11-28T21:58:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46386.log 2023-11-28T21:58:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49357.log 2023-11-28T21:58:39.513 INFO:teuthology.orchestra.run.smithi012.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.38102.log.gz 2023-11-28T21:58:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46386.log: gzip -5 --verbose 87.8% -- /var/log/ceph/ceph-client.admin.46539.log -- replaced with /var/log/ceph/ceph-client.admin.46386.log.gz 2023-11-28T21:58:39.514 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-11-28T21:58:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49357.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49357.log.gz 2023-11-28T21:58:39.515 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51496.log 2023-11-28T21:58:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54016.log 2023-11-28T21:58:39.517 INFO:teuthology.orchestra.run.smithi012.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46539.log.gz 2023-11-28T21:58:39.518 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51496.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51444.log 2023-11-28T21:58:39.519 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51496.log.gz 2023-11-28T21:58:39.519 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54016.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54016.log.gz 2023-11-28T21:58:39.521 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53504.log 2023-11-28T21:58:39.522 INFO:teuthology.orchestra.run.smithi144.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-28T21:58:39.523 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53023.log 2023-11-28T21:58:39.523 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51444.log.gz 2023-11-28T21:58:39.523 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53504.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45031.log 2023-11-28T21:58:39.524 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53504.log.gz 2023-11-28T21:58:39.524 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53023.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53023.log.gz 2023-11-28T21:58:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50771.log 2023-11-28T21:58:39.525 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54860.log 2023-11-28T21:58:39.526 INFO:teuthology.orchestra.run.smithi012.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45031.log.gz 2023-11-28T21:58:39.527 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50771.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50771.log.gz 2023-11-28T21:58:39.527 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49940.log 2023-11-28T21:58:39.528 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55659.log 2023-11-28T21:58:39.528 INFO:teuthology.orchestra.run.smithi012.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54860.log.gz 2023-11-28T21:58:39.530 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49940.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49940.log.gz 2023-11-28T21:58:39.530 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55280.log 2023-11-28T21:58:39.532 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55659.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55659.log.gz 2023-11-28T21:58:39.532 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44232.log 2023-11-28T21:58:39.532 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55280.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55280.log.gz 2023-11-28T21:58:39.533 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43702.log 2023-11-28T21:58:39.534 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44232.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44232.log.gz 2023-11-28T21:58:39.535 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42416.log 2023-11-28T21:58:39.536 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43702.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43702.log.gz 2023-11-28T21:58:39.537 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44312.log 2023-11-28T21:58:39.537 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42416.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42416.log.gz 2023-11-28T21:58:39.539 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48396.log 2023-11-28T21:58:39.539 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44312.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44312.log.gz 2023-11-28T21:58:39.540 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43444.log 2023-11-28T21:58:39.541 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48396.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48396.log.gz 2023-11-28T21:58:39.541 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48531.log 2023-11-28T21:58:39.543 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.43444.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51418.log 2023-11-28T21:58:39.545 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43444.log.gz/var/log/ceph/ceph-client.admin.48531.log: 2023-11-28T21:58:39.545 INFO:teuthology.orchestra.run.smithi012.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48531.log.gz 2023-11-28T21:58:39.545 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50985.log 2023-11-28T21:58:39.546 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43693.log 2023-11-28T21:58:39.546 INFO:teuthology.orchestra.run.smithi012.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51418.log.gz 2023-11-28T21:58:39.547 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50985.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50985.log.gz 2023-11-28T21:58:39.548 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47098.log 2023-11-28T21:58:39.549 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43693.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43693.log.gz 2023-11-28T21:58:39.550 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-28T21:58:39.550 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47098.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47098.log.gz 2023-11-28T21:58:39.551 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51574.log 2023-11-28T21:58:39.553 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48557.log 2023-11-28T21:58:39.555 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51574.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51574.log.gz 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-28T21:58:39.555 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-11-28T21:58:39.555 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-28T21:58:39.556 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48557.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54234.log 2023-11-28T21:58:39.557 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48557.log.gz 2023-11-28T21:58:39.557 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49578.log 2023-11-28T21:58:39.559 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54234.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54234.log.gz 2023-11-28T21:58:39.560 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49578.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49578.log.gz 2023-11-28T21:58:39.561 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50906.log 2023-11-28T21:58:39.562 INFO:teuthology.orchestra.run.smithi012.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-28T21:58:39.562 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42202.log 2023-11-28T21:58:39.564 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50906.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-28T21:58:39.564 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50906.log.gz 2023-11-28T21:58:39.565 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42202.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42202.log.gz 2023-11-28T21:58:39.566 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50660.log 2023-11-28T21:58:39.567 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51145.log 2023-11-28T21:58:39.569 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50660.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50660.log.gz 2023-11-28T21:58:39.570 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43418.log 2023-11-28T21:58:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51145.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51145.log.gz 2023-11-28T21:58:39.571 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43685.log 2023-11-28T21:58:39.573 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43418.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43418.log.gz 2023-11-28T21:58:39.573 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52679.log 2023-11-28T21:58:39.574 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43685.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43685.log.gz 2023-11-28T21:58:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52679.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52679.log.gz 2023-11-28T21:58:39.576 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42442.log 2023-11-28T21:58:39.578 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42622.log 2023-11-28T21:58:39.579 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42442.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42442.log.gz 2023-11-28T21:58:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42622.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42622.log.gz 2023-11-28T21:58:39.581 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46845.log 2023-11-28T21:58:39.583 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50448.log 2023-11-28T21:58:39.585 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.46845.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46845.log.gz 2023-11-28T21:58:39.586 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52112.log 2023-11-28T21:58:39.587 INFO:teuthology.orchestra.run.smithi012.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50448.log.gz 2023-11-28T21:58:39.588 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52997.log 2023-11-28T21:58:39.589 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52112.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52112.log.gz 2023-11-28T21:58:39.590 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52997.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52997.log.gz 2023-11-28T21:58:39.591 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44521.log 2023-11-28T21:58:39.593 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49201.log 2023-11-28T21:58:39.594 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44521.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44521.log.gz 2023-11-28T21:58:39.595 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49201.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49201.log.gz 2023-11-28T21:58:39.596 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48661.log 2023-11-28T21:58:39.597 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55119.log 2023-11-28T21:58:39.599 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48661.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48661.log.gz 2023-11-28T21:58:39.600 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55119.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55119.log.gz 2023-11-28T21:58:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53608.log 2023-11-28T21:58:39.602 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49227.log 2023-11-28T21:58:39.604 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53608.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53608.log.gz 2023-11-28T21:58:39.605 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49227.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49227.log.gz 2023-11-28T21:58:39.606 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50828.log 2023-11-28T21:58:39.608 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54938.log 2023-11-28T21:58:39.609 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50828.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50828.log.gz 2023-11-28T21:58:39.610 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54938.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54938.log.gz 2023-11-28T21:58:39.611 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43063.log 2023-11-28T21:58:39.612 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53082.log 2023-11-28T21:58:39.614 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43063.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43063.log.gz 2023-11-28T21:58:39.615 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53082.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53082.log.gz 2023-11-28T21:58:39.616 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54341.log 2023-11-28T21:58:39.617 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55399.log 2023-11-28T21:58:39.619 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54341.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54341.log.gz 2023-11-28T21:58:39.620 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55399.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55399.log.gz 2023-11-28T21:58:39.620 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49279.log 2023-11-28T21:58:39.622 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42906.log 2023-11-28T21:58:39.624 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49279.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49279.log.gz 2023-11-28T21:58:39.625 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42906.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42906.log.gz 2023-11-28T21:58:39.626 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44597.log 2023-11-28T21:58:39.627 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52433.log 2023-11-28T21:58:39.629 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44597.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44597.log.gz 2023-11-28T21:58:39.630 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52433.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52433.log.gz 2023-11-28T21:58:39.631 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53108.log 2023-11-28T21:58:39.632 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54650.log 2023-11-28T21:58:39.634 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53108.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53108.log.gz 2023-11-28T21:58:39.635 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54650.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54650.log.gz 2023-11-28T21:58:39.635 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55477.log 2023-11-28T21:58:39.637 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54287.log 2023-11-28T21:58:39.638 INFO:teuthology.orchestra.run.smithi144.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-28T21:58:39.639 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55477.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55477.log.gz 2023-11-28T21:58:39.640 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54287.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54287.log.gz 2023-11-28T21:58:39.640 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53136.log 2023-11-28T21:58:39.642 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47308.log 2023-11-28T21:58:39.643 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53136.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53136.log.gz 2023-11-28T21:58:39.644 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47308.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47308.log.gz 2023-11-28T21:58:39.645 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42985.log 2023-11-28T21:58:39.647 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49417.log 2023-11-28T21:58:39.648 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42985.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42985.log.gz 2023-11-28T21:58:39.649 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49417.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49417.log.gz 2023-11-28T21:58:39.650 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45343.log 2023-11-28T21:58:39.651 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54420.log 2023-11-28T21:58:39.653 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45343.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45343.log.gz 2023-11-28T21:58:39.654 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54420.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54420.log.gz 2023-11-28T21:58:39.655 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54702.log 2023-11-28T21:58:39.656 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54130.log 2023-11-28T21:58:39.658 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54702.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54702.log.gz 2023-11-28T21:58:39.659 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54130.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54130.log.gz 2023-11-28T21:58:39.660 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43493.log 2023-11-28T21:58:39.661 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53056.log 2023-11-28T21:58:39.663 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43493.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43493.log.gz 2023-11-28T21:58:39.664 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53056.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53056.log.gz 2023-11-28T21:58:39.665 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44417.log 2023-11-28T21:58:39.666 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53374.log 2023-11-28T21:58:39.668 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44417.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44417.log.gz 2023-11-28T21:58:39.669 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53374.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53374.log.gz 2023-11-28T21:58:39.669 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45239.log 2023-11-28T21:58:39.671 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50686.log 2023-11-28T21:58:39.673 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45239.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45239.log.gz 2023-11-28T21:58:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50686.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50686.log.gz 2023-11-28T21:58:39.674 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55555.log 2023-11-28T21:58:39.676 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43497.log 2023-11-28T21:58:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55555.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55555.log.gz 2023-11-28T21:58:39.678 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52624.log 2023-11-28T21:58:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43497.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43497.log.gz 2023-11-28T21:58:39.680 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53452.log 2023-11-28T21:58:39.681 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52624.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52624.log.gz 2023-11-28T21:58:39.681 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50182.log 2023-11-28T21:58:39.683 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53452.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53452.log.gz 2023-11-28T21:58:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44391.log 2023-11-28T21:58:39.685 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50182.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50182.log.gz 2023-11-28T21:58:39.686 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44391.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44391.log.gz 2023-11-28T21:58:39.687 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45317.log 2023-11-28T21:58:39.688 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53320.log 2023-11-28T21:58:39.690 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45317.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45317.log.gz 2023-11-28T21:58:39.691 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53320.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53320.log.gz 2023-11-28T21:58:39.692 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42308.log 2023-11-28T21:58:39.693 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54990.log 2023-11-28T21:58:39.695 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42308.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42308.log.gz 2023-11-28T21:58:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54990.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54990.log.gz 2023-11-28T21:58:39.696 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43701.log 2023-11-28T21:58:39.698 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53190.log 2023-11-28T21:58:39.700 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43701.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43701.log.gz 2023-11-28T21:58:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53190.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53190.log.gz 2023-11-28T21:58:39.701 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53910.log 2023-11-28T21:58:39.703 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47126.log 2023-11-28T21:58:39.705 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53910.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53910.log.gz 2023-11-28T21:58:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47126.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47126.log.gz 2023-11-28T21:58:39.706 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51389.log 2023-11-28T21:58:39.708 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50266.log 2023-11-28T21:58:39.709 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51389.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51389.log.gz 2023-11-28T21:58:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49656.log 2023-11-28T21:58:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50266.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50266.log.gz 2023-11-28T21:58:39.712 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56205.log 2023-11-28T21:58:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49656.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49656.log.gz 2023-11-28T21:58:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47378.log 2023-11-28T21:58:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.56205.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56205.log.gz 2023-11-28T21:58:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53426.log 2023-11-28T21:58:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47378.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47378.log.gz 2023-11-28T21:58:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51892.log 2023-11-28T21:58:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53426.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53426.log.gz 2023-11-28T21:58:39.719 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2023-11-28T21:58:39.720 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51892.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51892.log.gz 2023-11-28T21:58:39.722 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44180.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2023-11-28T21:58:39.723 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51090.log 2023-11-28T21:58:39.725 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45213.log 2023-11-28T21:58:39.726 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51090.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51090.log.gz 2023-11-28T21:58:39.727 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45213.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45213.log.gz 2023-11-28T21:58:39.728 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54075.log 2023-11-28T21:58:39.730 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44469.log 2023-11-28T21:58:39.731 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54075.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54075.log.gz 2023-11-28T21:58:39.732 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44469.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44469.log.gz 2023-11-28T21:58:39.733 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53661.log 2023-11-28T21:58:39.734 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42077.log 2023-11-28T21:58:39.736 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53661.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53661.log.gz 2023-11-28T21:58:39.737 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42077.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42077.log.gz 2023-11-28T21:58:39.738 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51944.log 2023-11-28T21:58:39.740 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52166.log 2023-11-28T21:58:39.741 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51944.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51944.log.gz 2023-11-28T21:58:39.742 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52757.log 2023-11-28T21:58:39.743 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52166.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52166.log.gz 2023-11-28T21:58:39.744 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51303.log 2023-11-28T21:58:39.745 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52757.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52757.log.gz 2023-11-28T21:58:39.746 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.51303.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51303.log.gz 2023-11-28T21:58:39.747 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2023-11-28T21:58:39.748 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45187.log 2023-11-28T21:58:39.751 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mds.e.log: /var/log/ceph/ceph-client.admin.45187.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45187.log.gz 2023-11-28T21:58:39.752 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45369.log 2023-11-28T21:58:39.754 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52889.log 2023-11-28T21:58:39.758 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52889.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52889.log.gz 2023-11-28T21:58:39.759 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45369.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45369.log.gz 2023-11-28T21:58:39.759 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38211.log 2023-11-28T21:58:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.38211.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38211.log.gz 2023-11-28T21:58:39.762 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55899.log 2023-11-28T21:58:39.765 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55899.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55899.log.gz 2023-11-28T21:58:39.765 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42959.log 2023-11-28T21:58:39.768 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42959.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42959.log.gz 2023-11-28T21:58:39.769 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54519.log 2023-11-28T21:58:39.772 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54519.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54519.log.gz 2023-11-28T21:58:39.772 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47178.log 2023-11-28T21:58:39.774 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50880.log 2023-11-28T21:58:39.775 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.47178.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47178.log.gz 2023-11-28T21:58:39.777 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45902.log 2023-11-28T21:58:39.779 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.50880.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50880.log.gz 2023-11-28T21:58:39.782 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.45902.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45902.log.gz 2023-11-28T21:58:39.782 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43366.log 2023-11-28T21:58:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.43366.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43366.log.gz 2023-11-28T21:58:39.786 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44338.log 2023-11-28T21:58:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.44338.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44338.log.gz 2023-11-28T21:58:39.789 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42134.log 2023-11-28T21:58:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42134.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42134.log.gz 2023-11-28T21:58:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55815.log 2023-11-28T21:58:39.796 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.55815.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55815.log.gz 2023-11-28T21:58:39.796 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54156.log 2023-11-28T21:58:39.798 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44820.log 2023-11-28T21:58:39.802 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.0.44820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44820.log.gz 2023-11-28T21:58:39.803 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.54156.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54156.log.gz 2023-11-28T21:58:39.803 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42828.log 2023-11-28T21:58:39.805 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42828.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42828.log.gz 2023-11-28T21:58:39.806 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49253.log 2023-11-28T21:58:39.809 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.49253.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49253.log.gz 2023-11-28T21:58:39.809 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52407.log 2023-11-28T21:58:39.812 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.52407.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52407.log.gz 2023-11-28T21:58:39.812 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53294.log 2023-11-28T21:58:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42748.log 2023-11-28T21:58:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.53294.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53294.log.gz 2023-11-28T21:58:39.821 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42748.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.42748.log.gz 2023-11-28T21:58:39.821 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42572.log 2023-11-28T21:58:39.825 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.42572.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42572.log.gz 2023-11-28T21:58:39.825 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48690.log 2023-11-28T21:58:39.827 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.48690.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48690.log.gz 2023-11-28T21:58:40.636 INFO:teuthology.orchestra.run.smithi012.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-28T21:58:41.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-28T21:58:41.125 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-11-28T21:58:51.232 INFO:teuthology.orchestra.run.smithi144.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-28T21:58:51.360 INFO:teuthology.orchestra.run.smithi144.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-28T21:58:52.208 INFO:teuthology.orchestra.run.smithi012.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-28T21:58:52.642 INFO:teuthology.orchestra.run.smithi012.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-28T21:58:53.012 INFO:teuthology.orchestra.run.smithi144.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-28T21:58:54.790 INFO:teuthology.orchestra.run.smithi144.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-28T21:58:57.204 INFO:teuthology.orchestra.run.smithi012.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-28T21:58:58.890 INFO:teuthology.orchestra.run.smithi012.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-28T21:59:05.149 INFO:teuthology.orchestra.run.smithi144.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2023-11-28T21:59:05.150 INFO:teuthology.orchestra.run.smithi144.stderr: 2023-11-28T21:59:05.151 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m26.038s 2023-11-28T21:59:05.151 INFO:teuthology.orchestra.run.smithi144.stderr:user 1m18.258s 2023-11-28T21:59:05.151 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m2.446s 2023-11-28T21:59:16.086 INFO:teuthology.orchestra.run.smithi012.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2023-11-28T21:59:31.645 INFO:teuthology.orchestra.run.smithi012.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-28T21:59:31.647 INFO:teuthology.orchestra.run.smithi012.stderr: 2023-11-28T21:59:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:real 0m52.554s 2023-11-28T21:59:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:user 2m24.087s 2023-11-28T21:59:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m11.046s 2023-11-28T21:59:31.648 INFO:tasks.ceph:Archiving logs... 2023-11-28T21:59:31.649 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703/remote/smithi012/log 2023-11-28T21:59:31.650 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T21:59:47.513 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703/remote/smithi144/log 2023-11-28T21:59:47.514 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T21:59:56.731 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T21:59:56.741 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T21:59:56.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T21:59:56.745 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T21:59:56.822 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T21:59:56.835 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T21:59:56.835 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T21:59:56.839 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T21:59:56.852 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2023-11-28T21:59:56.853 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2023-11-28T21:59:56.853 INFO:teuthology.orchestra.run.smithi012.stdout:*hv01.front.sepi 44.4.53.2 2 u 48 64 377 0.059 +0.258 0.053 2023-11-28T21:59:56.853 INFO:teuthology.orchestra.run.smithi012.stdout:+hv02.front.sepi 63.231.80.2 3 u 38 64 377 0.075 +1.435 0.058 2023-11-28T21:59:56.853 INFO:teuthology.orchestra.run.smithi012.stdout:+hv03.front.sepi 74.6.168.72 3 u 43 64 377 0.118 -1.445 0.188 2023-11-28T21:59:56.853 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2023-11-28T21:59:56.855 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2023-11-28T21:59:56.855 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2023-11-28T21:59:56.855 INFO:teuthology.orchestra.run.smithi144.stdout:*hv01.front.sepi 44.4.53.2 2 u 39 64 377 0.109 -3.728 1.220 2023-11-28T21:59:56.855 INFO:teuthology.orchestra.run.smithi144.stdout:+hv02.front.sepi 63.231.80.2 3 u 33 64 377 0.093 -4.183 1.267 2023-11-28T21:59:56.856 INFO:teuthology.orchestra.run.smithi144.stdout:+hv03.front.sepi 74.6.168.72 3 u 40 64 377 0.079 -5.522 1.343 2023-11-28T21:59:56.856 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2023-11-28T21:59:56.856 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T21:59:56.872 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T21:59:56.873 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T21:59:56.903 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T21:59:56.935 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T21:59:56.970 INFO:teuthology.task.internal:Duration was 2283.683283 seconds 2023-11-28T21:59:56.970 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T21:59:57.003 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T21:59:57.003 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T21:59:57.007 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T21:59:57.112 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T21:59:57.113 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2023-11-28T21:59:57.113 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T21:59:57.136 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2023-11-28T21:59:57.136 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T21:59:57.145 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T21:59:57.145 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T21:59:57.185 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T21:59:57.221 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T21:59:57.221 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T21:59:57.266 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T21:59:57.346 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T21:59:57.356 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T21:59:57.357 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T21:59:57.378 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T21:59:57.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T21:59:57.403 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T21:59:57.425 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T21:59:57.440 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2023-11-28T21:59:57.444 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2023-11-28T21:59:57.541 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T21:59:57.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:59:57.550 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T21:59:57.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T21:59:57.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T21:59:57.601 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T21:59:57.601 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703/remote/smithi012 2023-11-28T21:59:57.601 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T21:59:57.628 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703/remote/smithi144 2023-11-28T21:59:57.628 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T21:59:57.651 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T21:59:57.652 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T21:59:57.669 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T21:59:57.695 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T21:59:57.707 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T21:59:57.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T21:59:57.739 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T21:59:57.739 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T21:59:57.742 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T21:59:57.747 INFO:teuthology.orchestra.run.smithi144.stdout: 524292 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 21:59 /home/ubuntu/cephtest 2023-11-28T21:59:57.748 INFO:teuthology.orchestra.run.smithi012.stdout: 658046 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 21:59 /home/ubuntu/cephtest 2023-11-28T21:59:57.750 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T21:59:57.779 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T21:59:57.846 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T21:59:57.875 DEBUG:teuthology.nuke:shortname: smithi012 2023-11-28T21:59:57.875 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T21:59:57.898 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 21:12:41.173991', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUISU2UNRkZ8KvhahKPJ/Cs+rXF3/sD7bxiYbw/RwxUeVWemGwd9BwvJrvWDwo4pvBt+N1nYfzMl5NONDD2Hro='} 2023-11-28T21:59:57.901 DEBUG:teuthology.nuke:shortname: smithi144 2023-11-28T21:59:57.901 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T21:59:57.926 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-28_06:29:14-fs-wip-vshankar-testing-20231127.102914-testing-default-smithi/7469703', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 21:12:41.172423', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWCPwqcYt1BY50Y5frCiKMdgs43eqUVjQ57TA8TpCWOuXtYVKXsQiKY6HgEqCtuq0al9oexvbteSJacgpNbMgg='} 2023-11-28T21:59:57.956 INFO:teuthology.orchestra.console.smithi012:Power off 2023-11-28T21:59:57.957 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T21:59:57.976 INFO:teuthology.orchestra.console.smithi144:Power off 2023-11-28T21:59:57.977 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T21:59:57.994 DEBUG:teuthology.orchestra.console.smithi012:power off output: Chassis Power Control: Down/Off 2023-11-28T21:59:57.995 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T21:59:58.004 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2023-11-28T21:59:58.004 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T21:59:58.018 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-11-28T21:59:58.035 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-11-28T22:00:02.019 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T22:00:02.036 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T22:00:02.131 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-11-28T22:00:02.147 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-11-28T22:00:06.134 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T22:00:06.148 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T22:00:06.245 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-11-28T22:00:06.259 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-11-28T22:00:06.346 INFO:teuthology.orchestra.console.smithi012:Power off completed 2023-11-28T22:00:06.360 INFO:teuthology.orchestra.console.smithi144:Power off completed 2023-11-28T22:00:06.473 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} duration: 2283.683282852173 failure_reason: 'Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi012 with status 128: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43dd94c190073cf16cff1e711dd7f5a7b1cbc883 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f7eaf334d44b4e81a85f4d0780286075 status: fail success: false 2023-11-28T22:00:06.474 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T22:00:06.568 INFO:teuthology.run:FAIL